The Quakers group seems to have originated around 1658 in Henley and was meeting in a building at Northfield End as early as 1668 on the site of the present Victorian building. In the early 1660s a Quaker group, which included craftsmen and maltsters with London connections, met at a cottage at Northfield End.
When built, the present building was described as “commodious” and comprised a meeting-room, library and class rooms replacing the western-most end of a range of cottages acquired by the Friends in the late C17 and early C18.