The rocks which are all around in Žingvellir maintain a constant temperature in the air that is in them, because there is not so much exchange of air in them.
Therefor during cold days the warmer air will freeze and form ice around the top of the cracks.