In 1684 the Lord Lieutenant of North and South Wales, the Duke of Beaufort, then Lord President of the Council of the Marches, stayed at Ruperra with a large retinue while inspecting the militia during his 'official progress' through Glamorgan. Thomas Dineley, the artist, in the service of the Duke, made a famous sketch of the south elevation and mentioned the 'majesty of the old oaks' and the 'proud park of deer'.

A century later Ruperra Castle was destroyed by fire. Thomas Hardwicke was employed to rebuild it and the earlier gables were replaced by flat battlements, depicted in an engraving by J P Neale in 1820. Benjamin Malkin, the antiquary, collecting material for his new book published in 1803 described as 'singularly beautiful' the effect of the harvest moon shining on the Bristol Channel as he walked across the park.