The 'rock' is the remnant of a massive volcano which was active in the early carboniferous period, around 340 million years ago, when Scotland lay on the equator. Subsequent ice ages have removed the volcano itself leaving only the hard basalt, which had solidified in the base of the spout. The last ice sheet, around 15,000 years ago, moved from west to east, allowing the Royal Mile to be created in the shelter behind the Castle Rock. A perfect 'Crag and Tail' in geological terms.
I took this photo through the window of the taxi! From this angle, the rectangular barracks building is visible. Built in 1769-99 to replace the barracks formerly housed in the Great hall. It provides accommodations for 600 men.