Communication Dans Un Congrès Année : 2025

Leveraging Sequentiality in Reinforcement Learning from a Single Demonstration

Résumé

Deep reinforcement learning faces challenges in long-horizon, high-dimensional robotic tasks that provide rewards only upon completion. In this context, while expert demonstrations can help, they are costly. In this work, we leverage a sequential decomposition bias to learn control policies for such tasks from a single demonstration. Our method learns a goal-conditioned policy that guides the system through lowdimensional intermediate goals, offering more flexibility than using states as goals. However, this approach can complicate the task of ensuring compatibility between successive goals. To address these challenges, we propose a formal framework, GCSeqMDPs, and an algorithm, STIL, which properly handles a hindsight goal relabeling mechanism. We first demonstrate the benefits of STIL on a relatively simple, long-horizon, nonholonomic robotic task. Then, we show that STIL achieves unprecedented sample efficiency on more complex simulated tasks, such as humanoid locomotion and fast running on the Cassie robot, marking a significant step toward the resolution of complex robotic tasks with minimal task-specific information.

Fichier non déposé

Dates et versions

hal-05398834 , version 1 (04-12-2025)

Identifiants

Citer

Alexandre Chenu, Olivier Serris, Olivier Sigaud, Nicolas Perrin-Gilbert. Leveraging Sequentiality in Reinforcement Learning from a Single Demonstration. 2025 IEEE-RAS 24th International Conference on Humanoid Robots (Humanoids), Sep 2025, Seoul, France. pp.405-412, ⟨10.1109/Humanoids65713.2025.11203179⟩. ⟨hal-05398834⟩
64 Consultations
0 Téléchargements

Altmetric

Partager

  • More