Sex differences in the vocal behavior of nonhuman primates can take various forms: sex-specific call types, differential production of shared call types, or sex discrepancy in phonation. Also, a growing literature is evidencing that systematically analyzing the vocal repertoires of primates at the call level might lead to underestimating their communicative abilities. Here, we present an extensive multi-level analysis of the still unknown vocal repertoire of adult red-capped mangabeys (Cercocebus torquatus), with a special emphasis on sex differences. We collected recordings from seven adult males and seven adult females housed in captivity. We present a structurally-based classification of mangabey calls that we cross-validated by an analysis of the associated contexts of emission. We found 12 sound units (including six sex-specific) that were concatenated to form eight call types (including four sex-specific), which were produced either singularly or in sequences composed of one ("repetition") or several ("combination") call types. We extracted organizational principles that ruled call composition and calling patterns. This revealed a high degree of potentially meaningful variability in terms of semantics and syntax. Male-female discrepancy in terms of phonation could be related to morphological dimorphism and would enable listeners to behave appropriately according to the sex of the caller. Sex differences in repertoire size, structural gradation, and call usage could reflect specificities of male-female social roles. We discuss the pertinence of these sex differences according to social system and habitat quality.