Adaptive Exploration for Continual Reinforcement Learning - Archive ouverte HAL Access content directly
Conference Papers Year : 2012

Adaptive Exploration for Continual Reinforcement Learning


Most experiments on policy search for robotics focus on isolated tasks, where the experiment is split into two distinct phases: 1)~the learning phase, where the robot learns the task through exploration; 2)~the exploitation phase, where exploration is turned off, and the robot demonstrates its performance on the task it has learned. In this paper, we present an algorithm that enables robots to continually and autonomously alternate between these phases. We do so by combining the 'Policy Improvement with Path Integrals' direct reinforcement learning algorithm with the covariance matrix adaptation rule from the 'Cross-Entropy Method' optimization algorithm. This integration is possible because both algorithms iteratively update parameters with probability-weighted averaging. A practical advantage of the novel algorithm, called PI2-CMA, is that it alleviates the user from having to manually tune the degree of exploration. We evaluate PI2-CMA's ability to continually and autonomously tune exploration on two tasks.
Not file

Dates and versions

hal-00789389 , version 1 (18-02-2013)


  • HAL Id : hal-00789389 , version 1


Freek Stulp. Adaptive Exploration for Continual Reinforcement Learning. International Conference on Intelligent Robots and Systems (IROS), 2012, Portugal. pp.0-0. ⟨hal-00789389⟩
64 View
0 Download


Gmail Facebook Twitter LinkedIn More