Robust stability and near-optimality for policy iteration: for want of recursive feasibility, all is not lost - Archive ouverte HAL
Article Dans Une Revue IEEE Transactions on Automatic Control Année : 2024

Robust stability and near-optimality for policy iteration: for want of recursive feasibility, all is not lost

Mathieu Granzotto
Dragan Nešić
Zhong-Ping Jiang

Résumé

We consider deterministic nonlinear discrete-time systems whose inputs are generated by PI for undiscounted cost functions. We first assume that PI is recursively feasible, in the sense that the optimization problems solved at each iteration admit a solution. In this case, we provide novel conditions to establish recursive robust stability properties for a general attractor, meaning that the policies generated at each iteration ensure a robust ${KL}$-stability property with respect to a general state measure. We then derive novel explicit bounds on the mismatch between the (suboptimal) value function returned by PI at each iteration and the optimal one. However, we show by a counter-example that PI may fail to be recursively feasible, disallowing the mentioned stability and near-optimality guarantees. We therefore also present a modification of PI so that recursive feasibility is guaranteed a priori under mild conditions. This modified algorithm, called PI$^+$, is shown to preserve the recursive robust stability when the attractor is compact. Additionally, PI$^+$ enjoys the same near-optimality properties as its PI counterpart under the same assumptions.
Fichier principal
Vignette du fichier
PIplus__regularizing_PI___proof_submitted_.pdf (2.03 Mo) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)
Licence

Dates et versions

hal-04581472 , version 1 (21-05-2024)

Licence

Identifiants

Citer

Mathieu Granzotto, Olivier Lindamulage De Silva, Romain Postoyan, Dragan Nešić, Zhong-Ping Jiang. Robust stability and near-optimality for policy iteration: for want of recursive feasibility, all is not lost. IEEE Transactions on Automatic Control, 2024, 69 (12), pp.8247-8262. ⟨10.1109/TAC.2024.3402941⟩. ⟨hal-04581472⟩
68 Consultations
223 Téléchargements

Altmetric

Partager

More