Off‐policy model‐based end‐to‐end safe reinforcement learning - Archive ouverte HAL Accéder directement au contenu
Article Dans Une Revue International Journal of Robust and Nonlinear Control Année : 2024

Off‐policy model‐based end‐to‐end safe reinforcement learning

Résumé

Safety and stability considerations play a crucial role in the development of learning based strategies for control design of systems that require high levels of safety. Safe reinforcement learning (RL) based approaches traditionally seek learning of the control laws that are optimal with respect to system performance whilst ensuring system stability and safety. In this article, an off‐policy safe RL based approach is proposed for nonlinear systems affine in control in continuous time. In this novel work, safety and stability are guaranteed during initialization and exploration phases by adjusting the control input with the solution of a quadratic programming problem combining both input to state stable‐control Lyapunov function and robust control barrier function (R‐CBF) conditions. Moreover, the safety of the learned policy is assured by augmenting the cost function with a CBF to maintain safety and optimize performance simultaneously. Novel mathematically rigorous proofs are provided to establish the stability and safety guarantees, offering a sound theoretical foundation for the approach. To demonstrate the effectiveness of the algorithm, two examples are presented: engine surge and stall dynamics, and an unstable nonlinear system.
Fichier principal
Vignette du fichier
Final2_Revisited_Wiley_Off_Policy_Model_Based_End_to_End_Safe_Reinforcement_Learning(4)(2).pdf (1.11 Mo) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-04307002 , version 1 (25-11-2023)

Identifiants

Citer

Soha Kanso, Mayank Shekhar Jha, Didier Theilliol. Off‐policy model‐based end‐to‐end safe reinforcement learning. International Journal of Robust and Nonlinear Control, 2024, 34 (4), pp.2806-2831. ⟨10.1002/rnc.7109⟩. ⟨hal-04307002⟩
22 Consultations
70 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More