Improving Reconstruction Fidelity in Generative Face Video Coding using High-Frequency Shuttling - Laboratoire des Signaux et Systèmes
Communication Dans Un Congrès Année : 2024

Improving Reconstruction Fidelity in Generative Face Video Coding using High-Frequency Shuttling

Goluck Konuko
Anthony Trioux

Résumé

Generative face video coding (GFVC) schemes applied to talking head videos have recently demonstrated significant coding gains compared to traditional coding frameworks, particularly at ultra-low bitrates. Despite advancements in the field, these methods still face challenges in handling large pose and facial expression changes, as well as (dis-)occlusions. Recently, a hybrid approach (HDAC+) that combines a lowquality video coded with a conventional codec and animationbased coding has been proposed for standardization and shown to partially address these issues. Although HDAC+ shows promising results, it still struggles with generating accurate images. In this paper, we propose HDAC-HF, an improvement to the reconstruction process in HDAC+. Based on empirical observations that some pose and expression details are lost during animation, we introduce a high-frequency (HF) shuttling mechanism to enhance reconstruction fidelity at the decoder side, inspired by recent advancements in video super-resolution. By enhancing the flow of high-frequency details in the feature domain, we improve the reconstruction of facial expressions and poses. Qualitative and quantitative experiments confirm that the proposed method improves reconstruction without any additional bitstream or signaling cost compared to the baseline HDAC+ codec.

Fichier principal
Vignette du fichier
0271.pdf (2.02 Mo) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-04861049 , version 1 (01-01-2025)

Identifiants

  • HAL Id : hal-04861049 , version 1

Citer

Goluck Konuko, Giuseppe Valenzise, Anthony Trioux. Improving Reconstruction Fidelity in Generative Face Video Coding using High-Frequency Shuttling. IEEE International Conference on Visual Communications and Image Processing, Dec 2024, Tokyo, Japan. ⟨hal-04861049⟩
0 Consultations
0 Téléchargements

Partager

More