The Non-stationary Stochastic Multi-armed Bandit Problem - Archive ouverte HAL Access content directly
Journal Articles International Journal of Data Science and Analytics Year : 2017

The Non-stationary Stochastic Multi-armed Bandit Problem

Abstract

We consider a variant of the stochastic multi-armed bandit with K arms where the rewards are not assumed to be identically distributed, but are generated by a non-stationary stochastic process. We first study the unique best arm setting when there exists one unique best arm. Second, we study the general switching best arm setting when a best arm switches at some unknown steps. For both settings, we target problem-dependent bounds, instead of the more conservative problem-free bounds. We consider two classical problems: (1) identify a best arm with high probability (best arm identification), for which the performance measure by the sample complexity (number of samples before finding a near-optimal arm). To this end, we naturally extend the definition of sample complexity so that it makes sense in the switching best arm setting, which may be of independent interest. (2) Achieve the smallest cumulative regret (regret minimization) where the regret is measured with respect to the strategy pulling an arm with the best instantaneous mean at each step.
Fichier principal
Vignette du fichier
NonStaStoMAB_final.pdf (554.85 Ko) Télécharger le fichier
Loading...

Dates and versions

hal-01575000 , version 1 (23-10-2017)

Identifiers

Cite

Robin Allesiardo, Raphaël Féraud, Odalric-Ambrym Maillard. The Non-stationary Stochastic Multi-armed Bandit Problem. International Journal of Data Science and Analytics, 2017, 3 (4), pp.267-283. ⟨10.1007/s41060-017-0050-5⟩. ⟨hal-01575000⟩
529 View
2904 Download

Altmetric

Share

Gmail Facebook X LinkedIn More