Instance-Dependent Bounds for Zeroth-order Lipschitz Optimization with Error Certificates - Archive ouverte HAL Accéder directement au contenu
Communication Dans Un Congrès Année : 2022

Instance-Dependent Bounds for Zeroth-order Lipschitz Optimization with Error Certificates

Résumé

We study the problem of zeroth-order (black-box) optimization of a Lipschitz function $f$ defined on a compact subset $\mathcal X$ of $\mathbb R^d$, with the additional constraint that algorithms must certify the accuracy of their recommendations. We characterize the optimal number of evaluations of any Lipschitz function $f$ to find and certify an approximate maximizer of $f$ at accuracy $\varepsilon$. Under a weak assumption on $\mathcal X$, this optimal sample complexity is shown to be nearly proportional to the integral $\int_{\mathcal X} \mathrm{d}\boldsymbol x/( \max(f) - f(\boldsymbol x) + \varepsilon )^d$. This result, which was only (and partially) known in dimension $d=1$, solves an open problem dating back to 1991. In terms of techniques, our upper bound relies on a packing bound by Bouttier al. (2020) for the Piyavskii-Shubert algorithm that we link to the above integral. We also show that a certified version of the computationally tractable DOO algorithm matches these packing and integral bounds. Our instance-dependent lower bound differs from traditional worst-case lower bounds in the Lipschitz setting and relies on a local worst-case analysis that could likely prove useful for other learning tasks.
Fichier principal
Vignette du fichier
main.pdf (376.99 Ko) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-03129721 , version 1 (03-02-2021)
hal-03129721 , version 2 (09-03-2021)
hal-03129721 , version 3 (09-06-2021)
hal-03129721 , version 4 (21-03-2023)

Identifiants

Citer

François Bachoc, Tommaso R Cesari, Sébastien Gerchinovitz. Instance-Dependent Bounds for Zeroth-order Lipschitz Optimization with Error Certificates. NeurIPS 2021, Dec 2021, Virtual conference, France. 24 p. ⟨hal-03129721v4⟩
322 Consultations
293 Téléchargements

Altmetric

Partager

Gmail Mastodon Facebook X LinkedIn More