Improving Training of Likelihood-based Generative Models with Gaussian Homotopy - Archive ouverte HAL Accéder directement au contenu
Communication Dans Un Congrès Année : 2023

Improving Training of Likelihood-based Generative Models with Gaussian Homotopy

Résumé

Generative Models (GMs) have recently gained popularity thanks to their success in various domains. In computer vision, for instance, they are able to generate astonishing realistic-looking images. Likelihood-based GMs are fast at generating new samples, given that they need a single model evaluation per sample, but their sample quality is usually lower than score-based Diffusion Models (DMs). In this work, we verify that the success of score-based DMs is in part due to the process of data smoothing, by incorporating this in the training of likelihood-based GMs. In the literature of optimization, this process of data smoothing is referred to as Gaussian homotopy (GH), and it has strong theoretical grounding. Crucially, GH does not incur computational overheads, and it can be implemented by adding one line of code in any training loop. We report results on various GMs, including Variational Autoencoders and Normalizing Flows, applied to image datasets demonstrating that GH enables significant improvements in sample quality.
Fichier principal
Vignette du fichier
40_improving_training_of_likeliho.pdf (1.39 Mo) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-04188285 , version 1 (25-08-2023)

Identifiants

  • HAL Id : hal-04188285 , version 1

Citer

Ba-Hien Tran, Giulio Franzese, Pietro Michiardi, Maurizio Filippone. Improving Training of Likelihood-based Generative Models with Gaussian Homotopy. SPIGM 2023, 1st Workshop on Structured Probabilistic Inference & Generative Modeling, co-located with ICML 2023, IEEE, Jul 2023, Honolulu, United States. ⟨hal-04188285⟩
75 Consultations
67 Téléchargements

Partager

Gmail Facebook X LinkedIn More