Right whales (Eubalaena glacialis) were extirpated from the eastern North Atlantic by commercial whaling. Grey whales (Eschrichtius robustus) disappeared from the entire North Atlantic in still-mysterious circumstances. Here, we test the hypotheses that both species previously occurred in the Mediterranean Sea, an area not currently considered part of their historical range. We used ancient DNA barcoding and collagen fingerprinting methods to taxonomically identify a rare set of 10 presumed whale bones from Roman and pre-Roman archaeological sites in the Strait of Gibraltar region, plus an additional bone from the Asturian coast. We identified three right whales, and three grey whales, demonstrating that the ranges of both of these species historically encompassed the Gibraltar region, probably including the Mediterranean Sea as calving grounds. Our results significantly extend the known range of the Atlantic grey whale, and suggest that 2000 years ago, right and grey whales were common when compared with other whale species. The disappearance of right and grey whales from the Mediterranean region is likely to have been accompanied by broader ecosystem impacts, including the disappearance of their predators (killer whales) and a reduction in marine primary productivity. The evidence that these two coastal and highly accessible species were present along the shores of the Roman Empire raises the hypothesis that they may have formed the basis of a forgotten whaling industry.