Humans are the only species that can speak. Nonhuman primates, however, share some ‘domain-general’ cognitive properties that are essential to language processes. Whether these shared cognitive properties between humans and nonhuman primates are the results of a continuous evolution [homologies] or of a convergent evolution [analogies] remain difficult to demonstrate. However, comparing their respective underlying structure—the brain—to determinate their similarity or their divergence across species is critical to help increase the probability of either of the two hypotheses, respectively. Key areas associated with language processes are the Planum Temporale, Broca’s Area, the Arcuate Fasciculus, Cingulate Sulcus, The Insula, Superior Temporal Sulcus, the Inferior Parietal lobe, and the Central Sulcus. These structures share a fundamental feature: They are functionally and structurally specialised to one hemisphere. Interestingly, several nonhuman primate species, such as chimpanzees and baboons, show human-like structural brain asymmetries for areas homologous to key language regions. The question then arises: for what function did these asymmetries arise in non-linguistic primates, if not for language per se? In an attempt to provide some answers, we review the literature on the lateralisation of the gestural communication system, which may represent the missing behavioural link to brain asymmetries for language area’s homologues in our common ancestor.