How are visemes and graphemes integrated with speech sounds during spoken word recognition? ERP evidence for supra-additive responses during audiovisual compared to auditory speech processing - Archive ouverte HAL Access content directly
Journal Articles Brain and Language Year : 2022

How are visemes and graphemes integrated with speech sounds during spoken word recognition? ERP evidence for supra-additive responses during audiovisual compared to auditory speech processing

Abstract

Articulatory gestures and orthography are connected with speech through a natural and an artificial association, respectively. This EEG study investigated whether the integrations between speech and these two visual inputs rely on the same mechanism, despite their different characteristics. A comparison of skilled readers' brain responses elicited by spoken words presented alone versus synchronously with visemes or graphemes showed that while neither visual input induced audiovisual integration on the N1 acoustic component, both led to a supra-additive integration on P2, with a stronger integration between speech and graphemes on left-anterior electrodes. This pattern persisted on the P350 component and generalized to all electrodes. The finding suggests a strong impact of reading acquisition on phonetic processing and lexical access. It also indirectly indicates that the dynamic and predictive cues present in natural lip movements but not in static visemes are critical to the contribution of visual articulatory gestures to speech processing.
Fichier principal
Vignette du fichier
Pattamadilok&Sato_Brain&Language_MS.pdf (1.1 Mo) Télécharger le fichier
Origin : Files produced by the author(s)

Dates and versions

hal-03472191 , version 1 (09-12-2021)
hal-03472191 , version 2 (27-12-2021)

Identifiers

Cite

Chotiga Pattamadilok, Marc Sato. How are visemes and graphemes integrated with speech sounds during spoken word recognition? ERP evidence for supra-additive responses during audiovisual compared to auditory speech processing. Brain and Language, 2022, 225, ⟨10.1016/j.bandl.2021.105058⟩. ⟨hal-03472191v2⟩
217 View
171 Download

Altmetric

Share

Gmail Facebook Twitter LinkedIn More