There has been a prison at Coldbath Fields since the 17th century. It got its name because in the early days the prison was surrounded by fields near an important well. Coldbath Fields Prison was rebuilt in 1794 and was enlarged during the 19th century. By the time the building was completed, the jail could house over 1,000 prisoners.

Coldbath Fields Prison was a House of Correction which meant it was a jail run by local magistrates and that most of the prisoners served short term sentences. The prison contained men, women and children until 1850 when it was decided to restrict it to male offenders over 17.

In the 19th century Coldbath Fields Prison, like other jails, adopted the Silent System whereby prisoners were forbidden to communicate with each other. Hard Labour was also introduced. The idea being that prisoners should be forced to carry out unproductive work. The illustration below shows two prisoners working a crank to pump water. Those on the tread-wheel for six hours climbed the equivalent of 8,640 feet. Hard Labour was not officially abolished until 1948. However, for many years prisoners had been given more productive work to do in less severe conditions than those experienced in the 19th century.