Studying common ground instantiation using audio, video and brain behaviours: the BrainKT corpus - Archive ouverte HAL Accéder directement au contenu
Communication Dans Un Congrès Année : 2023

Studying common ground instantiation using audio, video and brain behaviours: the BrainKT corpus

Résumé

An increasing amount of multimodal recordings has been paving the way for the development of a more automatic way to study language and conversational interactions. However this data largely comprises of audio and video recordings, leaving aside other modalities that might complement this external view of the conversation but might be more difficult to collect in naturalistic setups, such as participants brain activity. In this context, we present BrainKT, a natural conversational corpus with audio, video and neuro-physiological signals, collected with the aim of studying information exchanges and common ground instantiation in conversation in a new, more indepth way. We recorded conversations from 28 dyads (56 participants) during 30 minutes experiments where subjects were first tasked to collaborate on a joint information game, then freely drifted to the topic of their choice. During each session, audio and video were captured, along with the participants' neural signal (EEG with Biosemi 64) and their electrophysiological activity (with Empatica-E4). The paper situates this new type of resources in the literature, presents the experimental setup and describes the different kinds of annotations considered for the corpus.
Fichier principal
Vignette du fichier
ranlp2023.pdf (2 Mo) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-04174622 , version 1 (01-08-2023)

Identifiants

  • HAL Id : hal-04174622 , version 1

Citer

Eliot Maës, Leonor Becerra-Bonache, Thierry Legou, Philippe Blache. Studying common ground instantiation using audio, video and brain behaviours: the BrainKT corpus. 14th Conference on RECENT ADVANCES IN NATURAL LANGUAGE PROCESSING (RANLP 2023), Sep 2023, Varna, Bulgaria. ⟨hal-04174622⟩
40 Consultations
62 Téléchargements

Partager

Gmail Facebook X LinkedIn More