The area of land where Yalding is situated was originally a heavily-forested marsh. In the 9th century, Saxons began cutting down the trees in the area.
They found that the land was very fertile and with its excellent water supply from the River Beult, it was a good place to site a village. The Saxons called the village Ealding which meant the "land belonging to Ealda". Over a period of time the spelling of Ealding was changed to Yalding.
This part of Kent was taken over by the Normans in 1067. William the Conqueror gave Yalding and several other villages in Kent to Richard de Clare, one of his most important military commanders. In 1314 Gilbert de Clare, (the tenth Earl of Clare) was killed at the Battle of Bannockburn. Gilbert did not have any children and so his death brought an end to the male line of the Clare family. The family estates were now divided between Gilbert's three sisters, Eleanor, Margaret and Elizabeth.
Margaret, received Yalding and most of the other Clare estates in Kent. Her husband, Hugh de Audley, became Yalding's new Lord of the Manor.