Seeing and hearing what has not been said; A multimodal client behavior classifier in Motivational Interviewing with interpretable fusion - Archive ouverte HAL Accéder directement au contenu
Pré-Publication, Document De Travail Année : 2023

Seeing and hearing what has not been said; A multimodal client behavior classifier in Motivational Interviewing with interpretable fusion

Résumé

Motivational Interviewing (MI) is an approach to therapy that emphasizes collaboration and encourages behavioral change. To evaluate the quality of an MI conversation, client utterances can be classified using the MISC code as either change talk, sustain talk, or follow/neutral talk. The proportion of change talk in a MI conversation is positively correlated with therapy outcomes, making accurate classification of client utterances essential. In this paper, we present a classifier that accurately distinguishes between the three MISC classes (change talk, sustain talk, and follow/neutral talk) leveraging multimodal features such as text, prosody, facial expressivity, and body expressivity. To train our model, we perform annotations on the publicly available AnnoMI dataset to collect multimodal information, including text, audio, facial expressivity, and body expressivity. Furthermore, we identify the most important modalities in the decision-making process, providing valuable insights into the interplay of different modalities during a MI conversation.
Fichier principal
Vignette du fichier
ICMI_HAL2_2023.pdf (1.21 Mo) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-04293266 , version 1 (23-11-2023)

Identifiants

Citer

Lucie Galland, Catherine Pelachaud, Florian Pecune. Seeing and hearing what has not been said; A multimodal client behavior classifier in Motivational Interviewing with interpretable fusion. 2023. ⟨hal-04293266⟩
43 Consultations
8 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More