The novel is set mainly in the mid-19th century, with glimpses into the history of the house, which was built in the late 17th century. The primary interest of this book is in the subtle and involved descriptions of character and motive.
The house of the title is a gloomy New England mansion, haunted from its foundation by fraudulent dealings, accusations of witchcraft, and sudden death. The current resident, the dignified but desperately poor Hepzibah Pyncheon, opens a shop in a side room to support her brother Clifford, who is about to leave prison after serving thirty years for murder. She refuses all assistance from her unpleasant wealthy cousin Judge Jaffrey Pyncheon. A distant relative, the lively and pretty young Phoebe, turns up and quickly becomes invaluable, charming customers and rousing Clifford from depression. A delicate romance grows between Phoebe and the mysterious lodger Holgrave, who is writing a history of the Pyncheon family.
An organ grinder's visit disturbs the ex-convict's fragile grasp on reality. Judge Pyncheon threatens to have Clifford committed, but his true purpose is to gain access to the house to search for a lost land deed. Hepzibah and Clifford escape on a train (then a very modern form of transport) after the judge dies unexpectedly in the house. However, they soon return, to Phoebe's relief. Events from past and present throw light on the circumstances which sent Clifford to prison, proving his innocence. The novel ends with the characters leaving the old house to start a new life, free of the burdens of the past.
