Communication Dans Un Congrès Année : 2016

An Articulatory-Based Singing Voice Synthesis Using Tongue and Lips Imaging

Résumé

Ultrasound imaging of the tongue and videos of lips movements can be used to investigate specific articulation in speech or singing voice. In this study, tongue and lips image sequences recorded during singing performance are used to predict vocal tract properties via Line Spectral Frequencies (LSF). We focused our work on traditional Corsican singing " Cantu in paghjella ". A multimodal Deep Autoencoder (DAE) extracts salient descriptors directly from tongue and lips images. Afterwards, LSF values are predicted from the most relevant of these features using a multilayer perceptron. A vocal tract model is derived from the predicted LSF, while a glottal flow model is computed from a synchronized electroglottographic recording. Articulatory-based singing voice synthesis is developed using both models. The quality of the prediction and singing voice synthesis using this method outperforms the state of the art method.

Fichier principal
Vignette du fichier
IS16_AJaumard-Hakoun_revised.pdf (290.75 Ko) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)
Licence
Loading...

Dates et versions

hal-01529630 , version 1 (31-05-2017)

Licence

Identifiants

Citer

Aurore Jaumard-Hakoun, Kele Xu, Clémence Leboullenger, Pierre Roussel-Ragot, Bruce Denby. An Articulatory-Based Singing Voice Synthesis Using Tongue and Lips Imaging. ISCA Interspeech 2016, Sep 2016, San Francisco, United States. pp.1467 - 1471, ⟨10.21437/Interspeech.2016-385⟩. ⟨hal-01529630⟩
504 Consultations
716 Téléchargements

Altmetric

Partager

  • More