Among domestic species, pigs experienced the greatest brain size reduction, but the extent and factors of this reduction remain unclear. Here, we used the brain endocast volume collected from 92 adult skulls of wild, captive, feral and domestic Sus scrofa to explore the effects of domestication, feralization and captivity over the brain size variation of this species. We found a constant brain volume increase over 24 months, while body growth slowed down from month 20. We observed an 18% brain size reduction between wild boars and pigs, disagreeing with the 30%–40% reduction previously mentioned. We did not find significant sexual differences in brain volume, refuting the theory of the attenuation of male secondary sexual characteristics through the selection for reduced male aggression. Feralization in Australia led to brain size reduction—probably as an adaptation to food scarcity and drought, refuting the reversal to wild ancestral brain size. Finally, free-born wild boars raised in captivity showed a slight increase in brain size, potentially due to a constant and high-quality food supply as well as new allospecific interactions. These results support the need to further explore the influence of diet, environment and experience on brain size evolution during animal domestication.