A Conservative Approach for Few-Shot Transfer in Off-Dynamics Reinforcement Learning - Archive ouverte HAL Accéder directement au contenu
Communication Dans Un Congrès Année : 2024

A Conservative Approach for Few-Shot Transfer in Off-Dynamics Reinforcement Learning

Paul Daoudi
  • Fonction : Auteur
  • PersonId : 1243056
Merwan Barlier
  • Fonction : Auteur
  • PersonId : 1243058
Ludovic Dos Santos
  • Fonction : Auteur

Résumé

Off-dynamics Reinforcement Learning (ODRL) seeks to transfer a policy from a source environment to a target environment characterized by distinct yet similar dynamics. In this context, traditional RL agents depend excessively on the dynamics of the source environment, resulting in the discovery of policies that excel in this environment but fail to provide reasonable performance in the target one. In the few-shot framework, a limited number of transitions from the target environment are introduced to facilitate a more effective transfer. Addressing this challenge, we propose an innovative approach inspired by recent advancements in Imitation Learning and Conservative RL algorithms. This method introduces a penalty to regulate the trajectories generated by the source-trained policy. We evaluate our method across various environments representing diverse off-dynamics conditions, where access to the target environment is extremely limited. These experiments include high-dimensional systems relevant to real-world applications. Across most tested scenarios, our proposed method demonstrates performance improvements compared to existing baselines.
Fichier principal
Vignette du fichier
main.pdf (8.24 Mo) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-04574316 , version 1 (28-08-2024)

Identifiants

  • HAL Id : hal-04574316 , version 1

Citer

Paul Daoudi, Bogdan Robu, Christophe Prieur, Merwan Barlier, Ludovic Dos Santos. A Conservative Approach for Few-Shot Transfer in Off-Dynamics Reinforcement Learning. IJCAI 2024 - International Joint Conference on Artificial Intelligence, Aug 2024, Jeju Island, South Korea. ⟨hal-04574316⟩
118 Consultations
0 Téléchargements

Partager

Gmail Mastodon Facebook X LinkedIn More