Mary Bay offers a sand and pebble beach from which to enjoy various activities and some beautiful views of Yellowstone Lake. This bay claims the title of the hottest spot (But is not hot at all) in Yellowstone Lake. Underwater geysers, hot springs, and fumaroles spew heated water up from the bottom of the bay.