Baleen whales (mysticetes) use vocalizations to mediate their complex social and reproductive behaviours in vast, opaque marine environments¹. Adapting to an obligate aquatic lifestyle demanded fundamental physiological changes to efficiently produce sound, including laryngeal specializations^2,3,4. Whereas toothed whales (odontocetes) evolved a nasal vocal organ⁵, mysticetes have been thought to use the larynx for sound production^1,6,7,8. However, there has been no direct demonstration that the mysticete larynx can phonate, or if it does, how it produces the great diversity of mysticete sounds⁹. Here we combine experiments on the excised larynx of three mysticete species with detailed anatomy and computational models to show that mysticetes evolved unique laryngeal structures for sound production. These structures allow some of the largest animals that ever lived to efficiently produce frequency-modulated, low-frequency calls. Furthermore, we show that this phonation mechanism is likely to be ancestral to all mysticetes and shares its fundamental physical basis with most terrestrial mammals, including humans¹⁰, birds¹¹, and their closest relatives, odontocetes⁵. However, these laryngeal structures set insurmountable physiological limits to the frequency range and depth of their vocalizations, preventing them from escaping anthropogenic vessel noise^12,13 and communicating at great depths¹⁴, thereby greatly reducing their active communication range.