A public gallery is visible to everyone. When this box is checked, the gallery will be listed in your list of galleries and will be visible to everyone.
If you do not want your gallery to be a public gallery, uncheck the box next to "Public Gallery" when editing your gallery; this makes it an non-public gallery. Anyone could still view the gallery, but they would need to know or guess the URL.
*Not public does not mean private. A clever person could find and view your images.