Unsupervised Learning and Exploration of Reachable Outcome Space - Archive ouverte HAL Accéder directement au contenu
Communication Dans Un Congrès IEEE International Conference on Robotics and Automation (ICRA) Année : 2020

Unsupervised Learning and Exploration of Reachable Outcome Space

Giuseppe Paolo
Alban Laflaquière
  • Fonction : Auteur
Alexandre Coninx
Stephane Doncieux

Résumé

Performing Reinforcement Learning in sparse rewards settings, with very little prior knowledge, is a challenging problem since there is no signal to properly guide the learning process. In such situations, a good search strategy is fundamental. At the same time, not having to adapt the algorithm to every single problem is very desirable. Here we introduce TAXONS, a Task Agnostic eXploration of Outcome spaces through Novelty and Surprise algorithm. Based on a population-based divergent-search approach, it learns a set of diverse policies directly from high-dimensional observations, without any task-specific information. TAXONS builds a repertoire of policies while training an autoencoder on the high-dimensional observation of the final state of the system to build a low-dimensional outcome space. The learned outcome space, combined with the reconstruction error, is used to drive the search for new policies. Results show that TAXONS can find a diverse set of controllers, covering a good part of the ground-truth outcome space, while having no information about such space.

Dates et versions

hal-02951255 , version 1 (28-09-2020)

Identifiants

Citer

Giuseppe Paolo, Alban Laflaquière, Alexandre Coninx, Stephane Doncieux. Unsupervised Learning and Exploration of Reachable Outcome Space. IEEE International Conference on Robotics and Automation (ICRA), 2020, Paris, France. ⟨10.1109/icra40945.2020.9196819⟩. ⟨hal-02951255⟩
63 Consultations
0 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More