Communication Dans Un Congrès Année : 2025

Predictive Safety Shield for Dyna-Q Reinforcement Learning

Résumé

Obtaining safety guarantees for reinforcement learning is a major challenge to achieve applicability for real-world tasks. Safety shields extend standard reinforcement learning and achieve hard safety guarantees. However, existing safety shields commonly use random sampling of safe actions or a fixed fallback controller, therefore disregarding future performance implications of different safe actions. In this work, we propose a predictive safety shield for model-based reinforcement learning agents in discrete space. Our safety shield updates the Q-function locally based on safe predictions, which originate from a safe simulation of the environment model. This shielding approach improves performance while maintaining hard safety guarantees. Our experiments on gridworld environments demonstrate that even short prediction horizons can be sufficient to identify the optimal path. We observe that our approach is robust to distribution shifts, e.g., between simulation and reality, without requiring additional training.

Fichier principal
Vignette du fichier
Predictive Safety Shield for Dyna-Q Reinforcement Learning.pdf (614.72 Ko) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)
Licence

Dates et versions

hal-05033474 , version 1 (14-04-2025)

Licence

Identifiants

  • HAL Id : hal-05033474 , version 1

Citer

Jin Pin, Hanna Krasowski, Elena Vanneaux. Predictive Safety Shield for Dyna-Q Reinforcement Learning. European Control Conference (ECC2025), Jun 2025, Thessaloniki, France. ⟨hal-05033474⟩
185 Consultations
489 Téléchargements

Partager

  • More