Understanding the effects of human exploitation on the genetic composition of wild populations is important for predicting species persistence and adaptive potential. We therefore investigated the genetic legacy of large-scale commercial harvesting by reconstructing, on a global scale, the recent demographic history of the Antarctic fur seal (Arctocephalus gazella), a species that was hunted to the brink of extinction by 18th and 19th century sealers. Molecular genetic data from over 2,000 individuals sampled from all eight major breeding locations across the species’ circumpolar geographic distribution, show that at least four relict populations around Antarctica survived commercial hunting. Coalescent simulations suggest that all of these populations experienced severe bottlenecks down to effective population sizes of around 150–200. Nevertheless, comparably high levels of neutral genetic variability were retained as these declines are unlikely to have been strong enough to deplete allelic richness by more than around 15%. These findings suggest that even dramatic short-term declines need not necessarily result in major losses of diversity, and explain the apparent contradiction between the high genetic diversity of this species and its extreme exploitation history.