Safe Fault Accommodation via Constrained Reinforcement Learning and Zonotopic Reachability
Résumé
Ensuring both safety and fault tolerance in reinforcement learning (RL)-based control systems is critical for their deployment in aerospace systems. In this context, the aim of this work is twofold. First, a new zonotopic reachability algorithm is proposed to address the case of bounded inputs with bounded variations. It is obtained from a reformulation based on bounded delayed exogenous inputs. This formulation contributes to reducing conservatism, while still covering a significant range of physical phenomena that are not arbitrarily fast, like surface control runaway. Second, this algorithm is used for off-line and training-independent safety verification of control loops hybridizing a baseline model-based controller with a constrained reinforcement learning (CRL) agent. The agent is then trained not only to improve given performance metrics under unmodeled environmental dynamics, but also to achieve incipient fault accommodation. To illustrate the main result, the proposed method is firstly applied to a numerical example based on aircraft dynamics [1], and next assessed on the Civilian Aircraft Landing Challenge (CALC) benchmark [2, 3].