Jesse Stone was born in Lincolnshire, England, in 1836 before moving to Watertown, Wisconsin, as a young man and becoming one of its most prominent citizens. He was a partner in Woodard & Stone Bakery and Confectionery and served as a state legislator and later lieutenant governor from 1899 until his death in 1902.

Designed by Joseph L. Kern, The Jesse Stone House is an elaborate Second Empire styled residence built in 1876. Upon its completion, The Stone House became a gathering place for many years for Watertown society and visiting notables. The high ceilings, three fireplaces, library, servants' quarters, summer kitchen, open staircase, six stained glass windows and gas-electric chandeliers reflect an era of gracious, leisurely living. The spacious grounds are still surrounded by the original iron picket fence, adding much to its charm. Complete with mansard roof, bay windows, ornamented porch and brick exterior, the main house, coach house and grounds are all maintained to the highest preservation standards.