Sequential Test for the Lowest Mean: From Thompson to Murphy Sampling - Archive ouverte HAL
Communication Dans Un Congrès Année : 2018

Sequential Test for the Lowest Mean: From Thompson to Murphy Sampling

Résumé

Learning the minimum/maximum mean among a finite set of distributions is a fundamental sub-task in planning, game tree search and reinforcement learning. We formalize this learning task as the problem of sequentially testing how the minimum mean among a finite set of distributions compares to a given threshold. We develop refined non-asymptotic lower bounds, which show that optimality mandates very different sampling behavior for a low vs high true minimum. We show that Thompson Sampling and the intuitive Lower Confidence Bounds policy each nail only one of these cases. We develop a novel approach that we call Murphy Sampling. Even though it entertains exclusively low true minima, we prove that MS is optimal for both possibilities. We then design advanced self-normalized deviation inequalities, fueling more aggressive stopping rules. We complement our theoretical guarantees by experiments showing that MS works best in practice.
Fichier principal
Vignette du fichier
arXiv.pdf (191.65 Ko) Télécharger le fichier

Dates et versions

hal-01804581 , version 1 (31-05-2018)

Identifiants

Citer

Emilie Kaufmann, Wouter Koolen, Aurélien Garivier. Sequential Test for the Lowest Mean: From Thompson to Murphy Sampling. Advances in Neural Information Processing Systems (NIPS), Dec 2018, Montréal, Canada. ⟨hal-01804581⟩
233 Consultations
83 Téléchargements

Altmetric

Partager

More