On the use of Self-supervised Pre-trained Acoustic and Linguistic Features for Continuous Speech Emotion Recognition - Archive ouverte HAL Accéder directement au contenu
Communication Dans Un Congrès Année : 2021

On the use of Self-supervised Pre-trained Acoustic and Linguistic Features for Continuous Speech Emotion Recognition

Yannick Estève
Anthony Rousseau
  • Fonction : Auteur
  • PersonId : 966756
  • IdRef : 241403871

Résumé

Pre-training for feature extraction is an increasingly studied approach to get better continuous representations of audio and text content. In the present work, we use wav2vec and camemBERT as self-supervised learned models to represent our data in order to perform continuous emotion recognition from speech (SER) on AlloSat, a large French emotional database describing the satisfaction dimension, and on the state of the art corpus SEWA focusing on valence, arousal and liking dimensions. To the authors' knowledge, this paper presents the first study showing that the joint use of wav2vec and BERT-like pre-trained features is very relevant to deal with continuous SER task, usually characterized by a small amount of labeled training data. Evaluated by the well-known concordance correlation coefficient (CCC), our experiments show that we can reach a CCC value of 0.825 instead of 0.592 when using MFCC in conjunction with word2vec word embedding on the AlloSat dataset.
Fichier principal
Vignette du fichier
2011.09212.pdf (506.78 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-03003469 , version 1 (07-12-2020)

Identifiants

Citer

Manon Macary, Marie Tahon, Yannick Estève, Anthony Rousseau. On the use of Self-supervised Pre-trained Acoustic and Linguistic Features for Continuous Speech Emotion Recognition. IEEE Spoken Language Technology Workshop, Jan 2021, Virtual, China. ⟨hal-03003469⟩
290 Consultations
264 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More