The building at 67 High Street was built between 1842 and 1846 as the Yale College Library, the university's first freestanding library. Henry Austin designed it in the Gothic Revival style, with a two-story reading room under a hammer-beam ceiling.
A different Dwight Hall was built on the Old Campus in 1886 for the Yale YMCA. It was a brownstone building with a round tower, and it was torn down in 1926 so people could see the new Harkness Tower without anything in the way.
After that, the Dwight Hall organization moved into the old library and the name came with it. In 1930 and 1931 the reading room was turned into Dwight Chapel.