Developmental Reinforcement Learning through Sensorimotor Space Enlargement - Archive ouverte HAL Access content directly
Conference Papers Year : 2018

Developmental Reinforcement Learning through Sensorimotor Space Enlargement

Abstract

In the framework of model-free deep reinforcement learning with continuous sensorimotor space, we propose a new type of transfer learning, inspired by the child development, where the sensorimotor space of an agent grows while it is learning a policy. To decide how the dimensions grow in our neural network based actor-critic, we add new developmental layers to the neural networks which progressively uncover some dimensions of the sensorimotor space following an Intrinsic Motivation heuristic. To mitigate the catastrophic forgetting problem, we take inspiration from the Elastic Weight Constraint to regulate the learning of the neural controller. We validate our approach using two state-of-the-art algorithms (DDPG and NFAC) on two high-dimensional environment benchmarks (Half-Cheetah and Humanoid). We show that searching first for a suboptimal solution in a subset of the parameter space, and then in the full space, is helpful to bootstrap learning algorithms, and thus reach better performances in fewer episodes.
Fichier principal
Vignette du fichier
main.pdf (1 Mo) Télécharger le fichier
Origin : Files produced by the author(s)
Loading...

Dates and versions

hal-01876995 , version 1 (19-09-2018)
hal-01876995 , version 2 (04-10-2018)

Identifiers

Cite

Matthieu Zimmer, Yann Boniface, Alain Dutech. Developmental Reinforcement Learning through Sensorimotor Space Enlargement. ICDL-EPIROB 2018 - 8th joint IEEE International Conference on Development and Learning and on Epigenetic Robotics, Sep 2018, Tokyo, Japan. pp.1-6, ⟨10.1109/DEVLRN.2018.8761021⟩. ⟨hal-01876995v2⟩
198 View
856 Download

Altmetric

Share

Gmail Facebook X LinkedIn More