Learning multi-scale local conditional probability models of images - Archive ouverte HAL
Communication Dans Un Congrès Année : 2023

Learning multi-scale local conditional probability models of images

Résumé

Deep neural networks can learn powerful prior probability models for images, as evidenced by the high-quality generations obtained with recent score-based diffusion methods. But the means by which these networks capture complex global statistical structure, apparently without suffering from the curse of dimensionality, remain a mystery. To study this, we incorporate diffusion methods into a multi-scale decomposition, reducing dimensionality by assuming a stationary local Markov model for wavelet coefficients conditioned on coarser-scale coefficients. We instantiate this model using convolutional neural networks (CNNs) with local receptive fields, which enforce both the stationarity and Markov properties. Global structures are captured using a CNN with receptive fields covering the entire (but small) low-pass image. We test this model on a dataset of face images, which are highly non-stationary and contain large-scale geometric structures. Remarkably, denoising, super-resolution, and image synthesis results all demonstrate that these structures can be captured with significantly smaller conditioning neighborhoods than required by a Markov model implemented in the pixel domain. Our results show that score estimation for large complex images can be reduced to low-dimensional Markov conditional models across scales, alleviating the curse of dimensionality.
Fichier principal
Vignette du fichier
2303.02984.pdf (8.77 Mo) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)
Licence

Dates et versions

hal-04019909 , version 1 (08-03-2023)

Licence

Identifiants

Citer

Zahra Kadkhodaie, Florentin Guth, Stéphane Mallat, Eero P Simoncelli. Learning multi-scale local conditional probability models of images. International Conference on Learning Representations, May 2023, Kigali, Rwanda. ⟨hal-04019909⟩
86 Consultations
44 Téléchargements

Altmetric

Partager

More