Continuous emotion prediction from audio signal with acoustic and linguistic representations - Archive ouverte HAL
Communication Dans Un Congrès Année : 2022

Continuous emotion prediction from audio signal with acoustic and linguistic representations

Résumé

The use of machine learning techniques has been a wide reference in many speech processing tasks. The power of neural networks (NN) allows to solve some very complex tasks such as automatic speech emotion prediction. Our works aim at continuously estimating the degree of satisfaction or frustration of speaker in call-center conversations. More precisely, we extract an acoustic representation directly from the audio signal and a linguistic representation from the automatic textual transcription, which will then be processed by a recurrent NN able to predict the level of satisfaction between 0 and 1 every 0.25s. Self-supervised learning allows to learn general contextualized speech representations with multi-layer convolutional networks from very large amount of unlabeled data. It is then possible to extract such representations (or embeddings) from new specific data as emotional speech. These representations have the advantage of capturing informations from a lot of data, what is not the case of models learnt on specific emotional speech only. We set up a protocol including different types of representation in input of the network: (i) ce-steal coefficients (MFCCs), (ii) expert prosodic descriptors; (iii) words pre-trained embeddings and (iv) signal pre-trained embeddings. Our results confirm the high potential of embedding representations for our task. More surprisingly, we show that the linguistic content seems to bring more emotional information than the single audio signal. A fine grained linguistic analysis will confirm this result.
Fichier principal
Vignette du fichier
CFA2022_Comm161.pdf (315.97 Ko) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-03847806 , version 1 (10-11-2022)

Identifiants

  • HAL Id : hal-03847806 , version 1

Citer

Marie Tahon, Manon Macary, Yannick Estève. Continuous emotion prediction from audio signal with acoustic and linguistic representations. 16ème Congrès Français d'Acoustique, CFA2022, Société Française d'Acoustique; Laboratoire de Mécanique et d'Acoustique, Apr 2022, Marseille, France. ⟨hal-03847806⟩
44 Consultations
106 Téléchargements

Partager

More