Communication Dans Un Congrès Année : 2024

Exploring the Multidimensional Representation of Unidimensional Speech Acoustic Parameters Extracted by Deep Unsupervised Models

Résumé

Understanding the latent representation of speech obtained by a deep unsupervised model is a key to powerful signal analysis, transformation, and generation. A number of studies have identified the directions of variation of individual speech acoustic features such as fundamental frequency or formant frequency in a deep latent space, but it is not well understood why the variation of such one-dimensional feature is often explained by multiple latent dimensions. This paper proposes a methodology for interpreting these dimensions, in the latent space of variational autoencoders trained on multi-speaker datasets. We show that for each acoustic feature, its distribution in the training set is encoded by one dedicated latent space direction. When the distribution is multimodal, different modes of the acoustic feature are encoded in separate dimensions. In that case, we also have identified the directions that explain the variation of the feature within and across modes, which paves the way to a finer control of such models.

Fichier principal
Vignette du fichier
Jacquelin_ICASSP-XAI_2024.pdf (879.22 Ko) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)
Licence

Dates et versions

hal-04683650 , version 1 (02-09-2024)

Licence

Identifiants

Citer

Maxime Jacquelin, Maëva Garnier, Laurent Girin, Rémy Vincent, Olivier Perrotin. Exploring the Multidimensional Representation of Unidimensional Speech Acoustic Parameters Extracted by Deep Unsupervised Models. ICASSPW 2024 - IEEE International Conference on Acoustics, Speech and Signal Processing Workshops, Apr 2024, Seoul (Korea), South Korea. pp.858-862, ⟨10.1109/ICASSPW62465.2024.10669904⟩. ⟨hal-04683650⟩
379 Consultations
264 Téléchargements

Altmetric

Partager

  • More