Real-time Control of a DNN-based Articulatory Synthesizer for Silent Speech Conversion: a pilot study - Archive ouverte HAL Accéder directement au contenu
Communication Dans Un Congrès Année : 2015

Real-time Control of a DNN-based Articulatory Synthesizer for Silent Speech Conversion: a pilot study

Résumé

This article presents a pilot study on the real-time control of an articulatory synthesizer based on deep neural network (DNN), in the context of silent speech interface. The underlying hypothesis is that a silent speaker could benefit from real-time audio feedback to regulate his/her own production. In this study, we use 3D electromagnetic-articulography (EMA) to capture speech articulation, a DNN to convert EMA to spectral trajectories in real-time, and a standard vocoder excited by white noise for audio synthesis. As shown by recent literature on silent speech, adaptation of the articulo-acoustic modeling process is needed to account for possible inconsistencies between the initial training phase and practical usage conditions. In this study, we focus on different sensor setups across sessions (for the same speaker). Model adaptation is performed by cascading another neural network to the DNN used for articulatory-to-acoustic mapping. The intelligibility of the synthetic speech signal converted in real-time is evaluated using both objective and perceptual measurements.
Fichier principal
Vignette du fichier
bocquelet_et_al_interspeech_2015.pdf (292.88 Ko) Télécharger le fichier
Origine : Fichiers éditeurs autorisés sur une archive ouverte
Loading...

Dates et versions

hal-01726265 , version 1 (11-01-2019)

Identifiants

  • HAL Id : hal-01726265 , version 1

Citer

Florent Bocquelet, Thomas Hueber, Laurent Girin, Christophe Savariaux, Blaise Yvert. Real-time Control of a DNN-based Articulatory Synthesizer for Silent Speech Conversion: a pilot study. Interspeech 2015 - 16th Annual Conference of the International Speech Communication Association, Sep 2015, Dresden, Germany. ⟨hal-01726265⟩
110 Consultations
108 Téléchargements

Partager

Gmail Facebook X LinkedIn More