Intermediate acoustic-to-semantic representations link behavioral and neural responses to natural sounds - Archive ouverte HAL Accéder directement au contenu
Article Dans Une Revue Nature Neuroscience Année : 2023

Intermediate acoustic-to-semantic representations link behavioral and neural responses to natural sounds

Résumé

Recognizing sounds implicates the cerebral transformation of input waveforms into semantic representations. Although past research identified the superior temporal gyrus (STG) as a crucial cortical region, the computational fingerprint of these cerebral transformations remains poorly characterized. Here, we exploit a model comparison framework and contrasted the ability of acoustic, semantic (continuous and categorical) and sound-to-event deep neural network representation models to predict perceived sound dissimilarity and 7 T human auditory cortex functional magnetic resonance imaging responses. We confirm that spectrotemporal modulations predict early auditory cortex (Heschl’s gyrus) responses, and that auditory dimensions (for example, loudness, periodicity) predict STG responses and perceived dissimilarity. Sound-to-event deep neural networks predict Heschl’s gyrus responses similar to acoustic models but, notably, they outperform all competing models at predicting both STG responses and perceived dissimilarity. Our findings indicate that STG entails intermediate acoustic-to-semantic sound representations that neither acoustic nor semantic models can account for. These representations are compositional in nature and relevant to behavior.

Domaines

Neurosciences
Fichier principal
Vignette du fichier
s41593-023-01285-9.pdf (4.12 Mo) Télécharger le fichier
Origine : Publication financée par une institution

Dates et versions

hal-04065458 , version 1 (11-04-2023)

Licence

Paternité

Identifiants

Citer

Bruno L Giordano, Michele Esposito, Giancarlo Valente, Elia Formisano. Intermediate acoustic-to-semantic representations link behavioral and neural responses to natural sounds. Nature Neuroscience, 2023, 26 (4), pp.664-672. ⟨10.1038/s41593-023-01285-9⟩. ⟨hal-04065458⟩
40 Consultations
28 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More