Proceedings/Recueil Des Communications Année : 2025

Least squares variational inference

Résumé

Variational inference consists in finding the best approximation of a target distribution within a certain family, where `best' means (typically) smallest Kullback-Leiber divergence. We show that, when the approximation family is exponential, the best approximation is the solution of a fixed-point equation. We introduce LSVI (Least-Squares Variational Inference), a Monte Carlo variant of the corresponding fixed-point recursion, where each iteration boils down to ordinary least squares regression and does not require computing gradients. We show that LSVI is equivalent to stochastic mirror descent; we use this insight to derive convergence guarantees. We introduce various ideas to improve LSVI further when the approximation family is Gaussian, leading to a complexity in the dimension of the target in the full-covariance case, and a complexity in the mean-field case. We show that LSVI outperforms state-of-the-art methods in a range of examples, while remaining gradient-free, that is, it does not require computing gradients.

Dates et versions

hal-05414730 , version 1 (13-12-2025)

Identifiants

Citer

Yvann Le Fay, Nicolas Chopin, Simon Barthelme. Least squares variational inference. NeurIPS, Dec 2025, San Diego (CA), United States. 2025. ⟨hal-05414730⟩
51 Consultations
0 Téléchargements

Altmetric

Partager

  • More