The land thereabouts was granted to the de Brus (Bruce) family in the years after the Norman Conquest, and a castle was soon built. Added to and altered many times over the years, and passing between various families, it seems to have spent much of 14th-18th centuries slowly disintegrating, until it was unceremoniously flattened in 1788.
At which point I shall let The History & Antiquities of Cleveland by John Walker Ord (1846) take over the story…
Extract from the same letter:— “I have no doubt but that the round tower in the sketch, which had been converted into a pigeon-cote, and the large square tower introduced in the sketch, were there before the Conquest*; and I well remember viewing their destruction with tears in my eyes. The person he employed, and who urged him on to the work of destruction, was a Mr. Mickle, whom I afterwards met at Farnley Hall, and let him feel the lash of my tongue for the mischief he had done in my native vale.”