Handsacre Hall existed before the Domesday Book in the 11th Century. It formed the centre of the village before Armitage, and was originally the residence of Robert de Handsacre.

Up to the Middle Ages, the whole district was a vast forest cultivated only in small clearings. The Lord of the manor of Handsacre had control over tenants who worked on unenclosed strips of arable land. Crops and pastures were protected only by ditches or rough fences.The medieval village was a straggling collection of hovels and cottages, built of timber and wattle and daub.

The manor was the centre of social and economic life and in its day, Handsacre Hall would have been very impressive. It is thought that the original building fell into disrepair by the 16th Century and was restructured in the 17th Century. 

It passed to Lady Robert Bertie and then to Sir George Chetwynd of Brocton, Stafford in the 17th and 18th centuries. By 1912 it was owned by Francis Villiers Forster, but he lived at Longdon Grange, and so was farmed by Mrs Boycott and Miss Harvey.

In 1963 the Hall was bought by Robert Boston of Brereton Cross Farm. In the subsequent years the building was subject to vandalism, and in need of repair. By 1973, the Hall was practically a ruin and permission was sought by Mr Boston to have it pulled down. Nothing remains of the Hall except for the protected moat off Shropshire Brook Road.