Graphical models for social behavior modeling in face-to face interaction - Archive ouverte HAL Accéder directement au contenu
Article Dans Une Revue Pattern Recognition Letters Année : 2016

Graphical models for social behavior modeling in face-to face interaction

Résumé

The goal of this paper is to model the coverbal behavior of a subject involved in face-to-face social interactions. For this end, we present a multimodal behavioral model based on a Dynamic Bayesian Network (DBN). The model was inferred from multimodal data of interacting dyads in a specific scenario designed to foster mutual attention and multimodal deixis of objects and places in a collaborative task. The challenge for this behavioral model is to generate coverbal actions (gaze, hand gestures) for the subject given his verbal productions, the current phase of the interaction and the perceived actions of the partner. In our work, the structure of the DBN was learned from data, which revealed an interesting causality graph describing precisely how verbal and coverbal human behaviors are coordinated during the studied interactions. Using this structure, DBN exhibits better performances compared to classical baseline models such as Hidden Markov Models (HMMs) and Hidden Semi-Markov Models (HSMMs). We outperform the baseline in both measures of performance, i.e. interaction unit recognition and behavior generation. DBN also reproduces more faithfully the coordination patterns between modalities observed in ground truth compared to the baseline models.
Fichier principal
Vignette du fichier
PRL_R2.pdf (626.39 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)
Loading...

Dates et versions

hal-01279427 , version 1 (21-10-2016)

Identifiants

Citer

Alaeddine Mihoub, Gérard Bailly, Christian Wolf, Frédéric Elisei. Graphical models for social behavior modeling in face-to face interaction. Pattern Recognition Letters, 2016, 74, pp.82-89. ⟨10.1016/j.patrec.2016.02.005⟩. ⟨hal-01279427⟩
421 Consultations
351 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More