Learning and controlling the source-filter representation of speech with a variational autoencoder - Archive ouverte HAL Accéder directement au contenu
Article Dans Une Revue Speech Communication Année : 2023

Learning and controlling the source-filter representation of speech with a variational autoencoder

Résumé

Understanding and controlling latent representations in deep generative models is a challenging yet important problem for analyzing, transforming and generating various types of data. In speech processing, inspiring from the anatomical mechanisms of phonation, the source-filter model considers that speech signals are produced from a few independent and physically meaningful continuous latent factors, among which the fundamental frequency f0 and the formants are of primary importance. In this work, we show that the source-filter model of speech production naturally arises in the latent space of a variational autoencoder (VAE) trained in an unsupervised manner on a dataset of natural speech signals. Using only a few seconds of labeled speech signals generated with an artificial speech synthesizer, we experimentally illustrate that f0 and the formant frequencies are encoded in orthogonal subspaces of the VAE latent space and we develop a weakly-supervised method to accurately and independently control these speech factors of variation within the learned latent subspaces. Without requiring additional information such as text or human-labeled data, this results in a deep generative model of speech spectrograms that is conditioned on f0 and the formant frequencies, and which is applied to the transformation of speech signals.
Fichier principal
Vignette du fichier
sadok2022learning.pdf (9.97 Mo) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-03650569 , version 1 (25-04-2022)
hal-03650569 , version 2 (04-05-2022)
hal-03650569 , version 3 (21-03-2023)

Identifiants

Citer

Samir Sadok, Simon Leglaive, Laurent Girin, Xavier Alameda-Pineda, Renaud Séguier. Learning and controlling the source-filter representation of speech with a variational autoencoder. Speech Communication, 2023, 148, pp.53-65. ⟨10.1016/j.specom.2023.02.005⟩. ⟨hal-03650569v2⟩
381 Consultations
57 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More