Learning multi-scale local conditional probability models of images - Archive ouverte HAL Accéder directement au contenu
Communication Dans Un Congrès Année : 2023

Learning multi-scale local conditional probability models of images

Résumé

Deep neural networks can learn powerful prior probability models for images, as evidenced by the high-quality generations obtained with recent score-based diffusion methods. But the means by which these networks capture complex global statistical structure, apparently without suffering from the curse of dimensionality, remain a mystery. To study this, we incorporate diffusion methods into a multi-scale decomposition, reducing dimensionality by assuming a stationary local Markov model for wavelet coefficients conditioned on coarser-scale coefficients. We instantiate this model using convolutional neural networks (CNNs) with local receptive fields, which enforce both the stationarity and Markov properties. Global structures are captured using a CNN with receptive fields covering the entire (but small) low-pass image. We test this model on a dataset of face images, which are highly non-stationary and contain large-scale geometric structures. Remarkably, denoising, super-resolution, and image synthesis results all demonstrate that these structures can be captured with significantly smaller conditioning neighborhoods than required by a Markov model implemented in the pixel domain. Our results show that score estimation for large complex images can be reduced to low-dimensional Markov conditional models across scales, alleviating the curse of dimensionality.
Fichier principal
Vignette du fichier
2303.02984.pdf (8.77 Mo) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)
Licence : CC BY NC ND - Paternité - Pas d'utilisation commerciale - Pas de modification

Dates et versions

hal-04019909 , version 1 (08-03-2023)

Licence

Paternité - Pas d'utilisation commerciale - Pas de modification

Identifiants

Citer

Zahra Kadkhodaie, Florentin Guth, Stéphane Mallat, Eero P Simoncelli. Learning multi-scale local conditional probability models of images. International Conference on Learning Representations, May 2023, Kigali, Rwanda. ⟨hal-04019909⟩
53 Consultations
19 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More