Exploration through Covariance Matrix Adaptation Enables Developmental Motor Learning - Archive ouverte HAL Accéder directement au contenu
Article Dans Une Revue Paladyn: Journal of Behavioral Robotics Année : 2012

Exploration through Covariance Matrix Adaptation Enables Developmental Motor Learning

Résumé

The "Policy Improvement with Path Integrals" (PI2) [25] and "Covariance Matrix Adaptation -- Evolutionary Strategy" [8] are considered to be state-of-the-art in direct reinforcement learning and stochastic optimization respectively. We have recently shown that incorporating covariance matrix adaptation into PI2- which yields the PI CMA 2 algorithm -- enables adaptive exploration by continually and autonomously reconsidering the exploration/exploitation trade-off. In this article, we provide an overview of our recent work on covariance matrix adaptation for direct reinforcement learning [22-24], highlight its relevance to developmental robotics, and conduct further experiments to analyze the results. We investigate two complementary phenomena from developmental robotics. First, we demonstrate PI CMA 2 's ability to adapt to slowly or abruptly changing tasks due to its continual and adaptive exploration. This is an important component of life-long skill learning in dynamic environments. Second, we show on a reaching task how PI CMA 2 subsequently releases degrees of freedom from proximal to more distal limbs as learning progresses. A similar effect is observed in human development, where it is known as 'proximodistal maturation'.

Dates et versions

hal-00922125 , version 1 (23-12-2013)

Identifiants

Citer

Freek Stulp, Pierre-Yves Oudeyer. Exploration through Covariance Matrix Adaptation Enables Developmental Motor Learning. Paladyn: Journal of Behavioral Robotics, 2012, 3 (3), pp.128-135. ⟨10.2478/s13230-013-0108-6⟩. ⟨hal-00922125⟩
99 Consultations
0 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More