A study of lip movements during spontaneous dialog and its application to voice activity detection - Archive ouverte HAL Accéder directement au contenu
Article Dans Une Revue Journal of the Acoustical Society of America Année : 2009

A study of lip movements during spontaneous dialog and its application to voice activity detection

Résumé

This paper presents a quantitative and comprehensive study of the lip movements of a given speaker in different speech/nonspeech contexts, with a particular focus on silences i.e., when no sound is produced by the speaker . The aim is to characterize the relationship between "lip activity" and "speech activity" and then to use visual speech information as a voice activity detector VAD . To this aim, an original audiovisual corpus was recorded with two speakers involved in a face-to-face spontaneous dialog, although being in separate rooms. Each speaker communicated with the other using a microphone, a camera, a screen, and headphones. This system was used to capture separate audio stimuli for each speaker and to synchronously monitor the speaker's lip movements. A comprehensive analysis was carried out on the lip shapes and lip movements in either silence or nonsilence i.e., speech+nonspeech audible events . A single visual parameter, defined to characterize the lip movements, was shown to be efficient for the detection of silence sections. This results in a visual VAD that can be used in any kind of environment noise, including intricate and highly nonstationary noises, e.g., multiple and/or moving noise sources or competing speech signals.
Fichier principal
Vignette du fichier
Sodoyer_JASA_2008_text_R2.pdf (180.98 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)
Loading...

Dates et versions

hal-00941145 , version 1 (03-02-2014)

Identifiants

Citer

David Sodoyer, Bertrand Rivet, Laurent Girin, Christophe Savariaux, Jean-Luc Schwartz, et al.. A study of lip movements during spontaneous dialog and its application to voice activity detection. Journal of the Acoustical Society of America, 2009, 125 (2), pp.1184-1196. ⟨10.1121/1.3050257⟩. ⟨hal-00941145⟩
339 Consultations
373 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More