Unimodal Bandits with Continuous Arms: Order-optimal Regret without Smoothness - Archive ouverte HAL Accéder directement au contenu
Article Dans Une Revue Proceedings of the ACM on Measurement and Analysis of Computing Systems Année : 2020

Unimodal Bandits with Continuous Arms: Order-optimal Regret without Smoothness

Résumé

We consider stochastic bandit problems with a continuous set of arms and where the expected reward is a continuous and unimodal function of the arm. For these problems, we propose the Stochastic Polychotomy (SP) algorithms, and derive finite-time upper bounds on its regret and optimization error. We show that, for a class of reward functions, the SP algorithm achieves a regret and an optimization error with optimal scalings, i.e., $O(\sqrtT )$ and $O(1/\sqrtT )$ (up to a logarithmic factor), respectively. SP constitutes the first order-optimal algorithm for non-smooth expected reward functions, as well as for smooth functions with unknown smoothness. The algorithm is based on sequential statistical tests used to successively trim an interval that contains the best arm with high probability. These tests exhibit a minimal sample complexity which confers to SP its adaptivity and optimality. Numerical experiments actually reveal that the algorithm even outperforms state-of-the-art algorithms that exploit the knowledge of the smoothness of the reward function. The performance of SP is further illustrated on the problem of setting optimal reserve prices in repeated second-price auctions; there, the algorithm is evaluated on real-world data.
Fichier principal
Vignette du fichier
1406.7447.pdf (527.77 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-03942575 , version 1 (17-01-2023)

Identifiants

Citer

Richard Combes, Alexandre Proutière, Alexandre Fauquette. Unimodal Bandits with Continuous Arms: Order-optimal Regret without Smoothness. Proceedings of the ACM on Measurement and Analysis of Computing Systems , 2020, 4 (1), pp.1-28. ⟨10.1145/3379480⟩. ⟨hal-03942575⟩
18 Consultations
48 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More