Exploring the Multidimensional Representation of Unidimensional Speech Acoustic Parameters Extracted by Deep Unsupervised Models
Résumé
Understanding the latent representation of speech obtained by a deep unsupervised model is a key to powerful signal analysis, transformation, and generation. A number of studies have identified the directions of variation of individual speech acoustic features such as fundamental frequency or formant frequency in a deep latent space, but it is not well understood why the variation of such one-dimensional feature is often explained by multiple latent dimensions. This paper proposes a methodology for interpreting these dimensions, in the latent space of variational autoencoders trained on multi-speaker datasets. We show that for each acoustic feature, its distribution in the training set is encoded by one dedicated latent space direction. When the distribution is multimodal, different modes of the acoustic feature are encoded in separate dimensions. In that case, we also have identified the directions that explain the variation of the feature within and across modes, which paves the way to a finer control of such models.
| Origine | Fichiers produits par l'(les) auteur(s) |
|---|---|
| Licence |