Multimodal classification of interruptions in humans’ interaction - Archive ouverte HAL Accéder directement au contenu
Communication Dans Un Congrès Année : 2022

Multimodal classification of interruptions in humans’ interaction

Résumé

During an interaction interruptions occur frequently. Interruptions may arise to fulfill different goals such as changing the topic of conversation abruptly, asking for clarification, completing the current speaker's turn. Interruptions may be cooperative or competitive depending on the interrupter's intention. Our main goal is to endow a Socially Interactive Agent with the capacity to handle user interruptions in dyadic interaction. It requires the agent to detect an interruption and recognize its type (cooperative/competitive), and then to plan its behaviours to respond appropriately. As a first step towards this goal, we developed a multimodal classification model using acoustic features, facial expression, head movement, and gaze direction from both, the interrupter and the interruptee. The classification model learns from the sequential information to automatically identify interruptions type. We also present studies we conducted to measure the shortest delay needed (0.6s) for our classification model to identify interruption types with a high classification accuracy (81%). On average, most interruption overlaps last longer than 0.6s, so a Socially Interactive Agent has time to detect and recognize an interruption type and can respond in a timely manner to its human interlocutor's interruption. CCS Concepts: • Human-centered computing → Human agent interaction (HAI).
Fichier principal
Vignette du fichier
ICMI_2022_Liu_CR-final.pdf (587.08 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-03845959 , version 1 (09-11-2022)
hal-03845959 , version 2 (04-12-2023)

Identifiants

Citer

Liu Yang, Catherine Achard, Catherine Pelachaud. Multimodal classification of interruptions in humans’ interaction. ACM International Conference on Multimodal Interaction ICMI, 2022, Bangalore, India. ⟨10.1145/3536221.3556604⟩. ⟨hal-03845959v2⟩
25 Consultations
95 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More