The Non-stationary Stochastic Multi-armed Bandit Problem - Archive ouverte HAL Accéder directement au contenu
Article Dans Une Revue International Journal of Data Science and Analytics Année : 2017

The Non-stationary Stochastic Multi-armed Bandit Problem

Résumé

We consider a variant of the stochastic multi-armed bandit with K arms where the rewards are not assumed to be identically distributed, but are generated by a non-stationary stochastic process. We first study the unique best arm setting when there exists one unique best arm. Second, we study the general switching best arm setting when a best arm switches at some unknown steps. For both settings, we target problem-dependent bounds, instead of the more conservative problem-free bounds. We consider two classical problems: (1) identify a best arm with high probability (best arm identification), for which the performance measure by the sample complexity (number of samples before finding a near-optimal arm). To this end, we naturally extend the definition of sample complexity so that it makes sense in the switching best arm setting, which may be of independent interest. (2) Achieve the smallest cumulative regret (regret minimization) where the regret is measured with respect to the strategy pulling an arm with the best instantaneous mean at each step.
Fichier principal
Vignette du fichier
NonStaStoMAB_final.pdf (554.85 Ko) Télécharger le fichier
Loading...

Dates et versions

hal-01575000 , version 1 (23-10-2017)

Identifiants

Citer

Robin Allesiardo, Raphaël Féraud, Odalric-Ambrym Maillard. The Non-stationary Stochastic Multi-armed Bandit Problem. International Journal of Data Science and Analytics, 2017, 3 (4), pp.267-283. ⟨10.1007/s41060-017-0050-5⟩. ⟨hal-01575000⟩
531 Consultations
2939 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More