SMYLE: A new multimodal resource of talk-in-interaction including neuro-physiological signal - Archive ouverte HAL Accéder directement au contenu
Communication Dans Un Congrès Année : 2023

SMYLE: A new multimodal resource of talk-in-interaction including neuro-physiological signal

Résumé

This article presents the SMYLE corpus, the first multimodal corpus in French (16h) including neuro-physiological data from 60 participants engaged in face-to-face storytelling (8.2h) and free conversation tasks (7.8h). The originality of this corpus lies first in the fact that it bears all modalities, precisely synchronized and second in the addition for the first time at this scale of neuro-physiological modalities. It constitutes the first corpus of this size offering the opportunity to investigate cognitive characteristics of spontaneous conversation including at the brain level. The storytelling task comprises two conditions: a storyteller talking with a "normal" or a "distracted" listener. Contrasting normal and disrupted conversations allows to study at a behavioral, linguistic and cognitive levels the complex characteristics and organization of conversations. In this article, we present first the methodology developed to acquire and synchronize the different sources and types of signal. In a second part, we detail the large set of automatic, semi-automatic and manual annotations of the complete dataset. In a last section, we illustrate one application of the corpus by providing preliminary analyses of the annotated data, that reveal the impact of distracted listener’s on his/her feedbacks and the quality of the narration.
Fichier principal
Vignette du fichier
SMYLE A new multimodal resource of talk-in-interaction.pdf (2.54 Mo) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-04195031 , version 1 (04-09-2023)

Identifiants

Citer

Auriane Boudin, Roxane Bertrand, Stéphane Rauzy, Matthis Houlès, Thierry Legou, et al.. SMYLE: A new multimodal resource of talk-in-interaction including neuro-physiological signal. INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION (ICMI ’23 Companion), Oct 2023, Paris, France. ⟨10.1145/3610661.3616188⟩. ⟨hal-04195031⟩
101 Consultations
83 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More