Intrinsically Motivated Goal Exploration Processes with Automatic Curriculum Learning - Archive ouverte HAL Accéder directement au contenu
Article Dans Une Revue Journal of Machine Learning Research Année : 2022

Intrinsically Motivated Goal Exploration Processes with Automatic Curriculum Learning

Résumé

Intrinsically motivated spontaneous exploration is a key enabler of autonomous developmental learning in human children. It enables the discovery of skill repertoires through autotelic learning, i.e. the self-generation, self-selection, self-ordering and self-experimentation of learning goals. We present an algorithmic approach called Intrinsically Motivated Goal Exploration Processes (IMGEP) to enable similar properties of autonomous learning in machines. The IMGEP architecture relies on several principles: 1) self-generation of goals, generalized as parameterized fitness functions; 2) selection of goals based on intrinsic rewards; 3) exploration with incremental goal-parameterized policy search and exploitation with a batch learning algorithm; 4) systematic reuse of information acquired when targeting a goal for improving towards other goals. We present a particularly efficient form of IMGEP, called AMB, that uses a population-based policy and an object-centered spatio-temporal modularity. We provide several implementations of this architecture and demonstrate their ability to automatically generate a learning curriculum within several experimental setups. One of these experiments includes a real humanoid robot exploring multiple spaces of goals with several hundred continuous dimensions and with distractors. While no particular target goal is provided to these autotelic agents, this curriculum allows the discovery of diverse skills that act as stepping stones for learning more complex skills, e.g. nested tool use.
Fichier principal
Vignette du fichier
21-0808 (1).pdf (2.81 Mo) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-01651233 , version 1 (28-11-2017)
hal-01651233 , version 2 (24-11-2021)
hal-01651233 , version 3 (09-12-2022)

Identifiants

  • HAL Id : hal-01651233 , version 3

Citer

Sébastien Forestier, Rémy Portelas, Yoan Mollard, Pierre-Yves Oudeyer. Intrinsically Motivated Goal Exploration Processes with Automatic Curriculum Learning. Journal of Machine Learning Research, 2022. ⟨hal-01651233v3⟩

Collections

INRIA INRIA2
343 Consultations
406 Téléchargements

Partager

Gmail Facebook X LinkedIn More