Multiscale Contextual Learning for Speech Emotion Recognition in Emergency Call Center Conversations - Archive ouverte HAL Accéder directement au contenu
Communication Dans Un Congrès Année : 2023

Multiscale Contextual Learning for Speech Emotion Recognition in Emergency Call Center Conversations

Apprentissage Contextuel Multi-échelle pour la Reconnaissance des émotions dans les Conversations des Centres d'Appels d'Urgence.

Résumé

Emotion recognition in conversations is essential for ensuring advanced human-machine interactions. However, creating robust and accurate emotion recognition systems in real life is challenging, mainly due to the scarcity of emotion datasets collected in the wild and the inability to take into account the dialogue context. The CEMO dataset, composed of conversations between agents and patients during emergency calls to a French call center, fills this gap. The nature of these interactions highlights the role of the conversation's emotional flow in predicting patient emotions, as context can often make a difference in understanding observed emotional expressions. This paper presents a multi-scale conversational context learning approach for speech emotion recognition, which takes advantage of this hypothesis. We investigated this approach on both speech transcriptions and acoustic segments. Experimentally, our method uses the previous or next information of the targeted segment. In the text domain, we tested the context window using a wide range of tokens (from 10 to 100) and at the speech turns level, considering inputs from both the same and opposing speakers. According to our tests, the context derived from previous tokens has a more significant influence on accurate prediction than the following tokens. Furthermore, taking the last speech turn of the same speaker in the conversation seems useful. In the acoustic domain, we conducted an in-depth analysis of the impact of the surrounding emotions on the prediction. While multi-scale conversational context learning using Transformers can enhance performance in the textual modality for emergency call recordings, incorporating acoustic context is more challenging.
Fichier principal
Vignette du fichier
Multiscale_Contextual_Learning_for_Speech_Emotion_Recognition.pdf (966.88 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)
Licence : CC BY NC - Paternité - Pas d'utilisation commerciale

Dates et versions

hal-04424176 , version 1 (29-01-2024)

Identifiants

Citer

Théo Deschamps-Berger, Lori Lamel, Laurence Devillers. Multiscale Contextual Learning for Speech Emotion Recognition in Emergency Call Center Conversations. ICMI '23: INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION, Oct 2023, Paris France, France. pp.337-343, ⟨10.1145/3610661.3616189⟩. ⟨hal-04424176⟩
7 Consultations
10 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More