Article Dans Une Revue Pattern Recognition Letters Année : 2024

Exploiting temporal information to detect conversational groups in videos and predict the next speaker

Résumé

Studies in human-human interaction have introduced the concept of F-formation to describe the spatial arrangement of participants during social interactions. This paper has two objectives. It aims at detecting F-formations in video sequences and at predicting the next speaker in a group conversation. The proposed approach exploits time information and multimodal signals of humans in video sequences. In particular, we rely on measuring the engagement level of people as a feature of group belonging. Our approach makes use of a recursive neural network, the Long Short Term Memory (LSTM), to predict who will take the speaker's turn in a conversation group. Experiments on the MatchNMingle dataset led to 85% true positives in group detection and 98% accuracy in predicting the next speaker.

Fichier principal
Vignette du fichier
ICPRAI2022_Victor-final.pdf (3.5 Mo) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)
Licence

Dates et versions

hal-04293240 , version 1 (18-11-2023)

Licence

Identifiants

Citer

Lucrezia Tosato, Victor Fortier, Isabelle Bloch, Catherine Pelachaud. Exploiting temporal information to detect conversational groups in videos and predict the next speaker. Pattern Recognition Letters, 2024, 177, pp.164-168. ⟨10.1016/j.patrec.2023.10.002⟩. ⟨hal-04293240⟩
269 Consultations
329 Téléchargements

Altmetric

Partager

  • More