Zero-Shot Style Transfer for Multimodal Data-Driven Gesture Synthesis - Archive ouverte HAL Access content directly
Conference Papers Year : 2023

Zero-Shot Style Transfer for Multimodal Data-Driven Gesture Synthesis

Abstract

We propose a multimodal speech driven approach to generate 2D upper-body gestures for virtual agents, in the communicative style of different speakers, seen or unseen by our model during training. Upper-body gestures of a source speaker are generated based on the content of his/her multimodal data-speech acoustics and text semantics. The synthesized source speaker's gestures are conditioned on the multimodal style representation of the target speaker. Our approach is zero-shot, and can generalize the style transfer to new unseen speakers, without any additional training. An objective evaluation is conducted to validate our approach.
Fichier principal
Vignette du fichier
SIVA_2023(2).pdf (344.17 Ko) Télécharger le fichier
Origin : Files produced by the author(s)

Dates and versions

hal-03972560 , version 1 (03-02-2023)

Identifiers

  • HAL Id : hal-03972560 , version 1

Cite

Mireille Fares, Nicolas Obin, Catherine Pelachaud. Zero-Shot Style Transfer for Multimodal Data-Driven Gesture Synthesis. International Conference on Automatic Face and Gesture Recognition 2023, Workshop on Socially Interactive Human-like Virtual Agents, Jan 2023, WAIKOLOA (Hawaii), United States. ⟨hal-03972560⟩
61 View
136 Download

Share

Gmail Facebook X LinkedIn More