TeachMyAgent: a Benchmark for Automatic Curriculum Learning in Deep RL - Archive ouverte HAL Accéder directement au contenu
Communication Dans Un Congrès Année : 2021

TeachMyAgent: a Benchmark for Automatic Curriculum Learning in Deep RL

Résumé

Training autonomous agents able to generalize to multiple tasks is a key target of Deep Reinforcement Learning (DRL) research. In parallel to improving DRL algorithms themselves, Automatic Curriculum Learning (ACL) study how teacher algorithms can train DRL agents more efficiently by adapting task selection to their evolving abilities. While multiple standard benchmarks exist to compare DRL agents, there is currently no such thing for ACL algorithms. Thus, comparing existing approaches is difficult, as too many experimental parameters differ from paper to paper. In this work, we identify several key challenges faced by ACL algorithms. Based on these, we present TeachMyAgent (TA), a benchmark of current ACL algorithms leveraging procedural task generation. It includes 1) challengespecific unit-tests using variants of a procedural Box2D bipedal walker environment, and 2) a new procedural Parkour environment combining most ACL challenges, making it ideal for global performance assessment. We then use TeachMyAgent to conduct a comparative study of representative existing approaches, showcasing the competitiveness of some ACL algorithms that do not use expert knowledge. We also show that the Parkour environment remains an open problem. We open-source our environments, all studied ACL algorithms (collected from open-source code or re-implemented), and DRL students in a Python package available at https://github. com/flowersteam/TeachMyAgent.
Fichier principal
Vignette du fichier
TeachMyAgent.pdf (6.71 Mo) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-03173198 , version 1 (18-03-2021)
hal-03173198 , version 2 (09-06-2021)

Identifiants

Citer

Clément Romac, Rémy Portelas, Katja Hofmann, Pierre-Yves Oudeyer. TeachMyAgent: a Benchmark for Automatic Curriculum Learning in Deep RL. ICML 2021 - Thirty-eighth International Conference on Machine Learning, Jul 2021, Vienna / Virtual, Austria. pp.9052--9063. ⟨hal-03173198v2⟩
141 Consultations
31 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More