Downing was born in Newburgh, New York in 1815. After finishing school at 16, Downing helped his brother manage the family business designing botanic gardens and nurseries. He developed an interest in landscape and domestic architecture, and began projects in his home town. Downing started writing articles on horticulture and landscape design for magazines.

In 1841 he published his first book, A Treatise of the Theory and Practice of Landscape Gardening, in which he advocated for an English, romantic-style design. He followed this with a book of designs of small and medium sized houses, Cottage Residences, published in 1842. More books followed, both his own and reworked editions of foreign books, all on the subjects of plants and domestic architecture. His books were extremely popular and influenced house and garden design throughout the United States. Beginning in 1846 he published a monthly magazine, in which he expressed his opinion on agricultural issues and design.

Downing was also an advocate for public parks, particularly in the new suburbs. He felt that they would encourage healthful recreation and provide a place for community activities.

Downing died at 36 in a fire caused by an explosion on a steamboat on which he was traveling.