Designing losses for data-free training of normalizing flows on Boltzmann distributions - Laboratoire Interdisciplinaire des Sciences du Numérique Accéder directement au contenu
Pré-Publication, Document De Travail Année : 2023

Designing losses for data-free training of normalizing flows on Boltzmann distributions

Résumé

Generating a Boltzmann distribution in high dimension has recently been achieved with Normalizing Flows, which enable fast and exact computation of the generated density, and thus unbiased estimation of expectations. However, current implementations rely on accurate training data, which typically comes from computationally expensive simulations. There is therefore a clear incentive to train models with incomplete or no data by relying solely on the target density, which can be obtained from a physical energy model (up to a constant factor). For that purpose, we analyze the properties of standard losses based on Kullback-Leibler divergences. We showcase their limitations, in particular a strong propensity for mode collapse during optimization on high-dimensional distributions. We then propose strategies to alleviate these issues, most importantly a new loss function well-grounded in theory and with suitable optimization properties. Using as a benchmark the generation of 3D molecular configurations, we show on several tasks that, for the first time, imperfect pre-trained models can be further optimized in the absence of training data.
Fichier principal
Vignette du fichier
main.pdf (1.41 Mo) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-03936982 , version 1 (12-01-2023)

Licence

Paternité - Pas d'utilisation commerciale - Partage selon les Conditions Initiales

Identifiants

Citer

Loris Felardos, Jérôme Hénin, Guillaume Charpiat. Designing losses for data-free training of normalizing flows on Boltzmann distributions. 2023. ⟨hal-03936982⟩
61 Consultations
22 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More