For at least 150,000 years, the human body has been culturally modified by the wearing of personal ornaments and probably by painting with red pigment. The present study used functional magnetic resonance imaging to explore the brain networks involved in attributing social status from face decorations. Results showed the fusiform gyrus, orbitofrontal cortex, and salience network were involved in social encoding, categorization, and evaluation. The hippocampus and parahippocampus were activated due to the memory and associative skills required for the task, while the inferior frontal gyrus likely interpreted face ornaments as symbols. Resting-state functional connectivity analysis clarified the interaction between these regions. The study highlights the importance of these neural interactions in the symbolic interpretation of social markers on the human face, which were likely active in early Homo species and intensified with Homo sapiens populations as more complex technologies were developed to culturalize the human face.