On Bayesian index policies for sequential resource allocation - Archive ouverte HAL Access content directly
Journal Articles Annals of Statistics Year : 2018

On Bayesian index policies for sequential resource allocation

Emilie Kaufmann

Abstract

This paper is about index policies for minimizing (frequentist) regret in a stochastic multi-armed bandit model, inspired by a Bayesian view on the problem. Our main contribution is to prove that the Bayes-UCB algorithm, which relies on quantiles of posterior distributions, is asymptotically optimal when the reward distributions belong to a one-dimensional exponential family, for a large class of prior distributions. We also show that the Bayesian literature gives new insight on what kind of exploration rates could be used in frequentist, UCB-type algorithms. Indeed, approximations of the Bayesian optimal solution or the Finite Horizon Gittins indices provide a justification for the kl-UCB+ and kl-UCB-H+ algorithms, whose asymptotic optimality is also established.
Fichier principal
Vignette du fichier
KaufmannSupplement.pdf (449.24 Ko) Télécharger le fichier
Origin : Files produced by the author(s)
Loading...

Dates and versions

hal-01251606 , version 1 (06-01-2016)
hal-01251606 , version 2 (12-09-2016)
hal-01251606 , version 3 (06-11-2017)

Identifiers

Cite

Emilie Kaufmann. On Bayesian index policies for sequential resource allocation. Annals of Statistics, 2018, 46 (2), pp.842-865. ⟨10.1214/17-AOS1569⟩. ⟨hal-01251606v3⟩
397 View
252 Download

Altmetric

Share

Gmail Facebook X LinkedIn More