AMII: Adaptive Multimodal Inter-personal and Intra-personal Model for Adapted Behavior Synthesis - Archive ouverte HAL Accéder directement au contenu
Pré-Publication, Document De Travail Année : 2023

AMII: Adaptive Multimodal Inter-personal and Intra-personal Model for Adapted Behavior Synthesis

Résumé

Socially Interactive Agents (SIAs) are physical or virtual embodied agents that display similar behavior as human multimodal behavior. Modeling SIAs' non-verbal behavior, such as speech and facial gestures, has always been a challenging task, given that a SIA can take the role of a speaker or a listener. A SIA must emit appropriate behavior adapted to its own speech, its previous behaviors (intra-personal), and the User's behaviors (inter-personal) for both roles. We propose AMII, a novel approach to synthesize adaptive facial gestures for SIAs while interacting with Users and acting interchangeably as a speaker or as a listener. AMII is characterized by modality memory encoding schema - where modality corresponds to either speech or facial gestures - and makes use of attention mechanisms to capture the intra-personal and inter-personal relationships. We validate our approach by conducting objective evaluations and comparing it with the state-of-the-art approaches.
Fichier principal
Vignette du fichier
2305.11310.pdf (554.65 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-04293258 , version 1 (21-11-2023)

Identifiants

Citer

Jieyeon Woo, Mireille Fares, Catherine Pelachaud, Catherine Achard. AMII: Adaptive Multimodal Inter-personal and Intra-personal Model for Adapted Behavior Synthesis. 2023. ⟨hal-04293258⟩
16 Consultations
4 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More