Because the expression of pain in babies’ cries is based on universal acoustic features, it is assumed that adult listeners should be able to detect when a crying baby is experiencing pain. We report that detecting that a baby’s cry expresses pain actually requires learning through experience. Our psychoacoustic experiments reveal that adults with no experience of caring for babies are unable to identify whether a baby’s cry is a pain cry induced by vaccination or a mild discomfort cry recorded during a bath, even when they are familiar with the discomfort cries from this particular baby. In contrast, people with prior experience of babies — parents or professional caregivers — identify a familiar baby’s pain cries without having heard these cries before. Parents of very young children are even able to identify the pain cries of a baby who is completely unfamiliar to them. Exposure through caregiving and/or parenting thus shapes the auditory and cognitive abilities involved in decoding the information conveyed by the baby’s communication signals.