Dolley Madison's House on Lafayette Square

On the northeast corner of Lafayette Square sits a distinctive yellow house with an ornamental wrought iron porch. Quaint and domestic as it is, it seems transported from a bygone era, a time when Lafayette Square was where the rich and famous lived and this house on the corner was the epicenter of Washington social life. Dolley Madison (1768-1849) owned the house at one time and was by far its most famous resident. Her presence was so large that some believe it still lingers around the house to this day. But I'm getting ahead of myself.

Postcard view of Madison Place, circa 1910. The Dolley Madison House is the one on the left

The house was built in 1820 by former Massachusetts congressman Richard Cutts (1771-1845), then serving as Comptroller of the Treasury, and his wife Anna (1779-1832), Dolley Madison's younger sister. At the time there were few buildings on the "President's Square," which would later be called Lafayette Square. In fact, it had been a great open common, stretching from 15th Street to 17th Street, that was used for mustering the local militia, according to Benjamin Ogle Tayloe, an early resident. The Cutts house was one of the first to break up the open space and the first to be built on the east side of the square. Opposite it on the west side was Decatur House, which had been built in 1819. Some time after these two houses were completed, roads were cut north-south from H Street to Pennsylvania Avenue to form the smaller park in the center that is now Lafayette Square. Additional houses then began lining either side.

The Cutts house originally faced Lafayette Park and stood out prominently as a sturdy two-story house with a gabled roof and large gardens in the rear (extending along H Street) and on the south side. Unfortunately, though he was a high official of the U.S. Treasury, Cutts was apparently not very good at balancing his own books and was thrown in debtors' prison in 1828. To clear himself, he agreed to sell his house to former President James Madison (1751-1836), his brother-in law, for its assessed value of $5,750. The Madisons allowed the Cutts family to continue living in the house until Anna Cutts died in 1832.

Drawing from 1822 of the nearby St. John's Church and the Cutts house as seen from the Decatur House. Drawing by the Baroness Hyde de Neuville (Source: Library of Congress).

Dolley moved in five years later, after James Madison had died. She was an extraordinary and immensely popular individual whose house on Lafayette Square was known far and wide. It became a New Year's Day tradition for Washingtonians of any social standing to call first at the President's House and then cross Lafayette Park to pay respects to Dolley. Her impact on Washington society, beginning in the Jefferson administration and continuing through her husband's, had been profound. Despite an austere Quaker upbringing, Dolley had spearheaded something of a social awakening in Washington through her gracious hosting of everything from informal "levees" to formal White House dinners. Through her social connections she gained such political influence that in 1808, James Madison's rival for the presidency, Charles C. Pinckney, complained that he "was beaten by Mr. and Mrs. Madison. I might have had a better chance had I faced Mr. Madison alone."

The Dolley Madison House c. 1908.

Dolley Madison may be best known today for her heroism during the War of 1812, specifically her valiant efforts (as she reported in a letter to Anna Cutts) to save the Gilbert Stuart portrait of George Washington from falling into British hands on August 24, 1814, when the British army captured Washington and burned public buildings, including the White House and the Capitol. She also packed important government papers into trunks for safekeeping. More than 30 years later, in May 1848, Dolley similarly saved her husband's important papers, which she kept in an upstairs trunk, from immanent destruction. She was then spending her last years in her house on Lafayette Square, and the house caught fire, perhaps from arson. Dolley refused to be rescued from the top floor unless the trunk of papers was safely removed with her.

Nearly as well known as Dolley's heroism were her personal foibles. She loved to dress well and had a famous predilection for extravagant turbans. When fleeing the White House in 1814, along with the Stuart portrait of Washington, she tellingly saved a set of red velvet drapes from the oval drawing room. Some have argued that her choice of these drapes when so much else had to be abandoned was reasonable given their high cost in those days, but the fact remains that the woman really loved red.

Perhaps most poignant of Dolley's foibles was her unwavering and truly blind devotion to her ever-dissolute son, John Payne Todd (1792-1852), who contributed mightily to her ultimate financial ruination. Payne was Dolley's son by her first marriage to John Todd, who died in a devastating cholera epidemic in 1793. Payne had been coddled by his adoring mother since birth—overly much, it would seem. As described by Richard Côté in his biography of Dolley, Payne was nothing but trouble. At the age of 13, he was sent to Baltimore to attend school and live with Dolley's friend Betsy Patterson Bonaparte, who had married Napoleon's younger brother. The young and attractive Mrs. Bonaparte, who had a habit of scandalizing society with her revealing clothing, taught Payne "French, dancing, etiquette, and self-indulgences," according to Côté. He never looked back. Handsome and adept at society airs, he spent his life gambling and otherwise frittering away the family fortune on women and extravagant entertainment. Plagued by alcoholism, he ultimately dying penniless and shunned by all who knew him.

Throughout her life, Dolley would never see anything but good in her son, despite the calamitous effects of his spending and, after James had died, his destructive meddling in her affairs. The Madisons had owned a beautiful plantation estate in central Virginia, called Montpelier, where they retired at the end of Madison's presidency, but Dolley was forced to sell it and move back to the Cutts house in Washington when she could no longer afford to keep Montpelier. Production problems on the farm contributed to her financial difficulties, but much of the blame must go to her son, whose constant debts she (and James before her) had always reliably paid.

Dolley Madison in 1848 (Source: Library of Congress)

After Dolley's death, Payne sold the Dolley Madison House to Captain (later Admiral) Charles Wilkes (1798-1877), best known for commanding a grand expedition to explore the South Seas from 1838 to 1842. Wilkes was the first to undertake significant alterations to the house, including moving the main entrance to the H Street side, adding a bay window on the south side, cutting the first-floor windows down to the floor, and installing the ornamental wrought-iron porch on the Madison Place side of the house that remains there today.

Wilkes and his heirs owned the house for 35 years and leased it at various times to a number of important persons. One of these was General George B. McClellan (1826-1885), head of the Union Army at the outset of the Civil War. McClellan, or "Little Mac" as he was known, was perhaps the most exasperating of Abraham Lincoln's many commanders-in-chief and reportedly referred to him as "nothing more than a well-meaning baboon." On the evening of November 13, 1861, Lincoln came to call on Little Mac at the Dolley Madison House to discuss the progress of the war. Adding insult to insubordination, McClellan made Lincoln wait for 30 minutes downstairs before being told that the general had gone to bed and could not see him. After that, Lincoln insisted that McClellan visit him at the White House—at least for a few more months until he got completely fed up and fired him altogether.

In the early 1880s, the Wilkes family first leased and then sold the Dolley Madison House to the Cosmos Club, a social club founded by scientists and intellectuals, including John Wesley Powell and Clarence King. The Club embarked on extensive modifications and additions to the already Wilkes-modified house, including building out a full third floor and expanding the house significantly to the south and east at the first floor level to create large assembly rooms. Further additions were made to the east along H Street in 1894, along with plumbing and electrical upgrades, including an elevator. The club then bought and razed the next two houses to the south (known as the Ingersoll and Windom houses) and constructed a rather bland five-story residential building in their place. Finally the club completed its conquest of the northern half of Madison Place by purchasing the historic Benjamin Ogle Tayloe House in 1917. The Tayloe House, which adjoined the club's new residential building to the south, had been built in 1828.

The indignity of all these expansions and modifications to Dolley's house was nothing compared to the threat of razing it altogether, which hung over the house for the first half of the 20th century. As early as 1902, the McMillan Commission had proposed ruthlessly obliterating virtually all of the existing structures around Lafayette Square and replacing them with a suffocating expanse of leaden, neoclassical, white-marble government buildings. And, for many years, everyone was just fine with this. Fortunately, only slight progress was made towards this ham-handed vision: the ponderous Treasury Annex, designed by Cass Gilbert, was completed at the southeast corner of the square in 1917, and the U.S. Chamber of Commerce, also designed by Gilbert, was constructed on H Street just above the park in 1929. Although the focus of government construction shifted to the massive Federal Triangle project in the 1930s, the plan to redevelop Lafayette Square persisted, and the federal government finally bought the Cosmos Club properties, including the Dolley Madison House, for $1 million in 1940. The club was allowed to lease the building back from the government for another 12 years because funds were not yet available to undertake new construction.

After the club moved out, the Dolley Madison House hosted two federal agencies: the National Science Foundation, from 1952 to 1958, and the National Aeronautics and Space Administration, from 1958 until 1964. In April 1959, NASA presented its first class of astronauts, known as the Mercury 7, to the world in the assembly room built by the Cosmos Club on the south side of the house.

By the late 1950s, the concept of historic preservation was starting to gain traction just as the government's plans for replacing Lafayette Square's houses were also finally being readied. the Committee of 100 on the Federal City, among others, opposed a large executive office building planned on the west side of the park.  As this opposition was voiced, Congress delayed funding, but the development process proceeded. Soon designs were being presented to the Commission of Fine Arts for an executive building on the west side of the park and a courts building on the east side, replacing the Dolley Madison House and others. As described by Kurt Helfrich in a 1996 article in Washington History magazine, the commission was split over the advisability of these plans. Some, including chairman David Finley, wanted to preserve the old buildings and put up new, contextually-sensitive ones alongside them, while others wanted to make bold, modernistic architectural statements free of the encumbrance of the old structures.

Much of the debate up until this point had been in "either/or" terms: either save the old buildings or construct new ones. But the Committee of 100 worked to advance a new idea, originally suggested by philanthropist Charles Glover, Jr., and sketched by architect Grosvenor Chapman, of building the larger new structures behind the existing historic ones, connecting to them at the rear, and integrating the entire complex. Chapman made a sketch of how this might look on the west side of the square, and the sketch was passed on to President Kennedy. Soon the Kennedys—particularly Mrs. Kennedy—were making a full-court press to preserve as much of Lafayette Square's heritage as possible. Despite an estimated $4 million that, according to the Washington Post, had already been spent on planning replacement buildings, the project was re-worked. Architect John Carl Warnecke, who had personal connections with the Kennedys, developed new designs based on the Glover/Chapman vision, and these were implemented in the 1960s. The Dolley Madison House is now part of the U.S. Court of Appeals for the Federal Circuit.

While it may seem strange now, Warnecke's work was dismissed by some, and the "meddling" of the First Lady in the plans for the square derided. For example, Ralph Walker, a member of the Commission of Fine Arts, complained in 1962 that "...we live in an age of bigness. We don't live in an age of tiny little things put together—this is one opportunity that we had of making [Lafayette Square] one of the most important squares in the whole world.... What we have done is frivolously piddled it away in the restoration of unimportant buildings." Fortunately, he was out-voted, and the Dolley Madison House is still with us.

As, perhaps, is the spirit of Dolley herself, which is said to still linger around her house. Tour guides like to pass on the story that late at night men leaving the Washington Club, which was several doors down in the late 19th century, would tip their hats to the ghost of Dolley, seen gently rocking in her favorite chair on the porch of her house. John Alexander's Ghosts: Washington Revisited mentions the story and includes a picture of the porch. Only trouble is, the porch wasn't added to the house until after Dolley had died. Actually, another trouble is that this particular ornamental porch could scarcely accommodate a rocking chair; it's too shallow. But I suppose if you're a gentle ghost and you want to sit rocking late at night, you'll just do it wherever you please...

Sources for this article included: John Alexander, Ghosts: Washington Revisited (1998); Michael Bednar, L'Enfant's Legacy: Public Open Spaces in Washington, D.C. (2006); Richard N. Côté, Strength and Honor: The Life of Dolley Madison (2005); Harold D. Eberlein and Cortlandt V. Hubbard, Historic Houses of George-town & Washington City (1958); Kurt Helfrich, "Modernism for Washington: The Kennedys and the Redesign of Lafayette Square" in Washington History (Vol. 8, No. 1, 1996); Historic American Buildings Survey, Richard Cutts House (1958); Charles Moore, Washington Past and Present (1929); Thomas M. Spaulding, The Cosmos Club on Lafayette Square (1949); Benjamin Ogle Tayloe, Our Neighbors on Lafayette Square (1872, reprinted 1982); United States Court of Appeals for the Federal Circuit, Dolley Madison House and The Tayloe House (undated pamphlets), and numerous newspaper articles.