Development of an Interactive Human/Agent Loop using Multimodal Recurrent Neural Networks - Archive ouverte HAL Access content directly
Conference Papers Year : 2021

Development of an Interactive Human/Agent Loop using Multimodal Recurrent Neural Networks

Abstract

The development of expressive embodied conversational agent (ECA) still remains a big challenge. During an interaction partners continuously adapt their behaviors one to the other [7]. Adaptation mechanisms may take different forms such as the choice of same vocabulary and grammatical form [31], imitation and synchronization [7]. The aim of my PhD project is to improve the interaction between human and agent. The key idea is to create an interactive loop between human and agent which allows the virtual agent to continuously adapt its behavior according to its partner’s behavior. The main idea is to learn how dyad of humans adapt their behaviors and implement it into human-agent interaction. My work, based on recurrent neural network, focuses on nonverbal behavior generation and addresses several scientific locks like the multimodality, the intra-personal temporality of multimodal signals or the temporality between partner’s social cues. We plan to build a model learned in an end-to-end fashion that generates behaviors considering both acoustic and visual modalities.
Fichier principal
Vignette du fichier
[Woo2021]ICMI2021.pdf (872.95 Ko) Télécharger le fichier
Origin Files produced by the author(s)

Dates and versions

hal-03687600 , version 1 (03-06-2024)

Identifiers

Cite

Jieyeon Woo. Development of an Interactive Human/Agent Loop using Multimodal Recurrent Neural Networks. ICMI '21: Proceedings of the 2021 International Conference on Multimodal Interaction, Oct 2021, Montreal, Canada. pp.822-826, ⟨10.1145/3462244.3481275⟩. ⟨hal-03687600⟩
36 View
5 Download

Altmetric

Share

Gmail Mastodon Facebook X LinkedIn More