the castle was in use for military purposes from the mid C16th to the early C19th, became derelict after the guns were removed and was 'found' by Edwin Hudson and Edward Lutyens in the early C20th. It was then converted into a somewhat austere summer residence (even gloomier than Lutyen's Castle Drogo) and Gertrude Jeckyll was also dragged up (literally for the last bit) to do a garden in a separate enclosure. I think they must have been happier in Sussex.

The upturned boat as shed has become a tourist shack genre, but this is one of the inspirational originals in the harbour area