Local and adaptive mirror descents in extensive-form games - Archive ouverte HAL Accéder directement au contenu
Communication Dans Un Congrès Année : 2023

Local and adaptive mirror descents in extensive-form games

Résumé

We study how to learn ε-optimal strategies in zero-sum imperfect information games (IIG) with trajectory feedback. In this setting, players update their policies sequentially based on their observations over a fixed number of episodes, denoted by T. Existing procedures suffer from high variance due to the use of importance sampling over sequences of actions (Steinberger et al., 2020; McAleer et al., 2022). To reduce this variance, we consider a fixed sampling approach, where players still update their policies over time, but with observations obtained through a given fixed sampling policy. Our approach is based on an adaptive Online Mirror Descent (OMD) algorithm that applies OMD locally to each information set, using individually decreasing learning rates and a regularized loss. We show that this approach guarantees a convergence rate of Õ(T-1/2) with high probability and has a near-optimal dependence on the game parameters when applied with the best theoretical choices of learning rates and sampling policies. To achieve these results, we generalize the notion of OMD stabilization, allowing for time-varying regularization with convex increments.
Fichier principal
Vignette du fichier
2309.00656.pdf (643.05 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-04416177 , version 1 (25-01-2024)

Identifiants

Citer

Côme Fiegel, Pierre Ménard, Tadashi Kozuno, Rémi Munos, Vianney Perchet, et al.. Local and adaptive mirror descents in extensive-form games. ICML 2023 - International Conference on Machine Learning, Jul 2023, Hawaii, United States. ⟨hal-04416177⟩
41 Consultations
27 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More