Pré-Publication, Document De Travail Année : 2025

Gradient-free optimization via integration

Christophe Andrieu
  • Fonction : Auteur
Ettore Fincato
  • Fonction : Auteur
Mathieu Gerber
  • Fonction : Auteur

Résumé

We develop and analyse an approach to optimize functions $l\colon \mathbb{R}^d \rightarrow \mathbb{R}$ not assumed to be convex, differentiable or even continuous. The algorithm belongs to the class of model-based search methods. The idea is to fit recursively $l$ to a parametric family of distributions, using a Bayesian update followed by a reprojection back onto the chosen family. Remarkably, reprojection in our scenario boils down to computing expectations, which can be simply approximated through Monte Carlo. We show that when the family of distributions is appropriately chosen this approach can be interpreted as an implicit time-inhomogeneous gradient descent algorithm on a sequence of smoothed approximations of $l$, providing a route to establishing convergence. We establish new results for generic inhomogeneous gradient descent algorithms, which we specialise to the model-based search algorithm in the Gaussian scenario. We illustrate the performance of the algorithm on a challenging classification task in machine learning.

Dates et versions

hal-05414734 , version 1 (13-12-2025)

Identifiants

Citer

Christophe Andrieu, Nicolas Chopin, Ettore Fincato, Mathieu Gerber. Gradient-free optimization via integration. 2025. ⟨hal-05414734⟩
46 Consultations
0 Téléchargements

Altmetric

Partager

  • More