Overt speech decoding from cortical activity: a comparison of different linear methods - Archive ouverte HAL
Article Dans Une Revue Frontiers in Human Neuroscience Année : 2023

Overt speech decoding from cortical activity: a comparison of different linear methods

Résumé

Introduction Speech BCIs aim at reconstructing speech in real time from ongoing cortical activity. Ideal BCIs would need to reconstruct speech audio signal frame by frame on a millisecond-timescale. Such approaches require fast computation. In this respect, linear decoder are good candidates and have been widely used in motor BCIs. Yet, they have been very seldomly studied for speech reconstruction, and never for reconstruction of articulatory movements from intracranial activity. Here, we compared vanilla linear regression, ridge-regularized linear regressions, and partial least squares regressions for offline decoding of overt speech from cortical activity. Methods Two decoding paradigms were investigated: (1) direct decoding of acoustic vocoder features of speech, and (2) indirect decoding of vocoder features through an intermediate articulatory representation chained with a real-time-compatible DNN-based articulatory-to-acoustic synthesizer. Participant's articulatory trajectories were estimated from an electromagnetic-articulography dataset using dynamic time warping. The accuracy of the decoders was evaluated by computing correlations between original and reconstructed features. Results We found that similar performance was achieved by all linear methods well above chance levels, albeit without reaching intelligibility. Direct and indirect methods achieved comparable performance, with an advantage for direct decoding. Discussion Future work will address the development of an improved neural speech decoder compatible with fast frame-by-frame speech reconstruction from ongoing activity at a millisecond timescale.

Dates et versions

hal-04837553 , version 1 (13-12-2024)

Identifiants

Citer

Gaël Le Godais, Philémon Roussel, Florent Bocquelet, Marc Aubert, Philippe Kahane, et al.. Overt speech decoding from cortical activity: a comparison of different linear methods. Frontiers in Human Neuroscience, 2023, 17 (4), pp.514-523. ⟨10.3389/fnhum.2023.1124065⟩. ⟨hal-04837553⟩
0 Consultations
0 Téléchargements

Altmetric

Partager

More