Improving self-supervised 3D face reconstruction with few-shot transfer learning - Archive ouverte HAL Access content directly
Poster Communications Year : 2023

Improving self-supervised 3D face reconstruction with few-shot transfer learning

Abstract

While self-supervised models for 3D face reconstruction from single monocular images have improved over the years, because their training loss is mainly based on the photometric loss, they struggle to predict a 3D face with a correct head pose, which can be critical for some applications. On the other hand, supervised methods can predict more accurate head pose but require a lot of annotated data. In this paper we use transfer learning to adapt a pre-trained face autoencoder to predict from a face image its Projected Normalized Coordinate Code (PNCC), which encodes head pose and geometry information into a 2D image. Our PNCC predictor can be trained using only a few annotated training samples. We then improve a self-supervised 3D face reconstruction method by incorporating the predicted PNCC into the architecture. Compared to the original self-supervised architecture, our method predicts better head pose.
Fichier principal
Vignette du fichier
Poster MIG23.pdf (478.63 Ko) Télécharger le fichier
Origin Files produced by the author(s)

Dates and versions

hal-04290214 , version 1 (16-11-2023)

Identifiers

  • HAL Id : hal-04290214 , version 1

Cite

Martin Dornier, Philippe-Henri Gosselin, Yann Ricquebourg, Christian Raymond, Bertrand B. Coüasnon. Improving self-supervised 3D face reconstruction with few-shot transfer learning. ACM MIG 2023, Nov 2023, Rennes, France. The 16th Annual ACM SIGGRAPH Conference on Motion, Interaction and Games. ⟨hal-04290214⟩
40 View
19 Download

Share

Gmail Mastodon Facebook X LinkedIn More