Development of an Interactive Human/Agent Loop using Multimodal Recurrent Neural Networks - Archive ouverte HAL Accéder directement au contenu
Communication Dans Un Congrès Année : 2021

Development of an Interactive Human/Agent Loop using Multimodal Recurrent Neural Networks

Résumé

The development of expressive embodied conversational agent (ECA) still remains a big challenge. During an interaction partners continuously adapt their behaviors one to the other [7]. Adaptation mechanisms may take different forms such as the choice of same vocabulary and grammatical form [31], imitation and synchronization [7]. The aim of my PhD project is to improve the interaction between human and agent. The key idea is to create an interactive loop between human and agent which allows the virtual agent to continuously adapt its behavior according to its partner’s behavior. The main idea is to learn how dyad of humans adapt their behaviors and implement it into human-agent interaction. My work, based on recurrent neural network, focuses on nonverbal behavior generation and addresses several scientific locks like the multimodality, the intra-personal temporality of multimodal signals or the temporality between partner’s social cues. We plan to build a model learned in an end-to-end fashion that generates behaviors considering both acoustic and visual modalities.
Fichier principal
Vignette du fichier
[Woo2021]ICMI2021.pdf (872.95 Ko) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-03687600 , version 1 (03-06-2024)

Identifiants

Citer

Jieyeon Woo. Development of an Interactive Human/Agent Loop using Multimodal Recurrent Neural Networks. ICMI '21: Proceedings of the 2021 International Conference on Multimodal Interaction, Oct 2021, Montreal, Canada. pp.822-826, ⟨10.1145/3462244.3481275⟩. ⟨hal-03687600⟩
36 Consultations
5 Téléchargements

Altmetric

Partager

Gmail Mastodon Facebook X LinkedIn More