Exploiting temporal information to detect conversational groups in videos and predict the next speaker - Archive ouverte HAL Accéder directement au contenu
Article Dans Une Revue Pattern Recognition Letters Année : 2023

Exploiting temporal information to detect conversational groups in videos and predict the next speaker

Résumé

Studies in human-human interaction have introduced the concept of F-formation to describe the spatial arrangement of participants during social interactions. This paper has two objectives. It aims at detecting F-formations in video sequences and at predicting the next speaker in a group conversation. The proposed approach exploits time information and multimodal signals of humans in video sequences. In particular, we rely on measuring the engagement level of people as a feature of group belonging. Our approach makes use of a recursive neural network, the Long Short Term Memory (LSTM), to predict who will take the speaker's turn in a conversation group. Experiments on the MatchNMingle dataset led to 85% true positives in group detection and 98% accuracy in predicting the next speaker.
Fichier principal
Vignette du fichier
ICPRAI2022_Victor-final.pdf (3.5 Mo) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-04293240 , version 1 (18-11-2023)

Identifiants

Citer

Lucrezia Tosato, Victor Fortier, Isabelle Bloch, Catherine Pelachaud. Exploiting temporal information to detect conversational groups in videos and predict the next speaker. Pattern Recognition Letters, In press, ⟨10.1016/j.patrec.2023.10.002⟩. ⟨hal-04293240⟩
34 Consultations
23 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More