Super-Resolution through StyleGAN Regularized Latent Search - Archive ouverte HAL
Communication Dans Un Congrès Année : 2022

Super-Resolution through StyleGAN Regularized Latent Search

Résumé

This paper addresses the problem of super-resolution: constructing a highly resolved (HR) image from a low resolved (LR) one. Previous methods to solve this task were mainly supervised and consisted of approaching a real HR image from its down-sampled LR version, using a pixel-wise reconstruction loss. This often results in blurred images. Recognizing that the problem is ill-posed, recent selfsupervised alternative approaches proposed to constrain it by searching the latent space of a StyleGAN pre-trained on HR images, for the image that best downscales to the query LR image. Here we demonstrate that such an optimization process can still shift out of the StyleGAN image manifold and produce out-of-domain images containing artifacts. We thus introduce a new regularizer to facilitate the search in the latent space, by leveraging a normalizing flow model for effective density estimation, which serves as a powerful image prior. This prior is incorporated into the reconstruction process, ensuring that the inverted code lies in the original latent space and thus allowing us to achieve a good trade-off between fidelity and realness.
Fichier principal
Vignette du fichier
2022_Gheisari_NeurIPS_SSL.pdf (39.33 Mo) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-04081416 , version 1 (25-04-2023)

Identifiants

  • HAL Id : hal-04081416 , version 1

Citer

Marzieh Gheisari, Auguste Genovesio. Super-Resolution through StyleGAN Regularized Latent Search. 36th Conference on Neural Information Processing Systems (NeurIPS 2022). NeurIPS - Self-Supervised Learning - Theory and Practice, Nov 2022, New Orleans, United States. ⟨hal-04081416⟩
62 Consultations
4 Téléchargements

Partager

More