Symphony of experts: orchestration with adversarial insights in reinforcement learning - Archive ouverte HAL Accéder directement au contenu
Pré-Publication, Document De Travail Année : 2023

Symphony of experts: orchestration with adversarial insights in reinforcement learning

Résumé

Structured reinforcement learning leverages policies with advantageous properties to reach better performance, particularly in scenarios where exploration poses challenges. We explore this field through the concept of orchestration, where a (small) set of expert policies guides decision-making; the modeling thereof constitutes our first contribution. We then establish value-functions regret bounds for orchestration in the tabular setting by transferring regret-bound results from adversarial settings. We generalize and extend the analysis of natural policy gradient in Agarwal et al. [2021, Section 5.3] to arbitrary adversarial aggregation strategies. We also extend it to the case of estimated advantage functions, providing insights into sample complexity both in expectation and high probability. A key point of our approach lies in its arguably more transparent proofs compared to existing methods. Finally, we present simulations for a stochastic matching toy model.
Fichier principal
Vignette du fichier
Agreg-RL--JMS.pdf (467.66 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-04256705 , version 1 (24-10-2023)

Identifiants

Citer

Matthieu Jonckheere, Chiara Mignacco, Gilles Stoltz. Symphony of experts: orchestration with adversarial insights in reinforcement learning. 2023. ⟨hal-04256705⟩
85 Consultations
24 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More