Stochastic bandits with groups of similar arms - Archive ouverte HAL Accéder directement au contenu
Communication Dans Un Congrès Année : 2021

Stochastic bandits with groups of similar arms

Fabien Pesquerel
  • Fonction : Auteur
  • PersonId : 1116626
Hassan Saber
  • Fonction : Auteur
  • PersonId : 1074055
Odalric-Ambrym Maillard

Résumé

We consider a variant of the stochastic multi-armed bandit problem where arms are known to be organized into different groups having the same mean. The groups are unknown but a lower bound q on their size is known. This situation typically appears when each arm can be described with a list of categorical attributes, and the (unknown) mean reward function only depends on a subset of them, the others being redundant. In this case, q is linked naturally to the number of attributes considered redundant, and the number of categories of each attribute. For this structured problem of practical relevance, we first derive the asymptotic regret lower bound and corresponding constrained optimization problem. They reveal the achievable regret can be substantially reduced when compared to the unstructured setup, possibly by a factor q. However, solving exactly the exact constrained optimization problem involves a combinatorial problem. We introduce a lowerbound inspired strategy involving a computationally efficient relaxation that is based on a sorting mechanism. We further prove it achieves a lower bound close to the optimal one up to a controlled factor, and achieves an asymptotic regret q times smaller than the unstructured one. We believe this shows it is a valuable strategy for the practitioner. Last, we illustrate the performance of the considered strategy on numerical experiments involving a large number of arms.
Fichier principal
Vignette du fichier
Neurips_Submission.pdf (1.05 Mo) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-03427597 , version 1 (14-11-2021)

Identifiants

  • HAL Id : hal-03427597 , version 1

Citer

Fabien Pesquerel, Hassan Saber, Odalric-Ambrym Maillard. Stochastic bandits with groups of similar arms. NeurIPS 2021 - Thirty-fifth Conference on Neural Information Processing Systems, Dec 2021, Sydney, Australia. ⟨hal-03427597⟩
53 Consultations
131 Téléchargements

Partager

Gmail Facebook X LinkedIn More