Finite-Time Regret Minimization for Linear Quadratic Adaptive Controllers: an experiment design approach
Résumé
We tackle the problem of finite-time regret minimization in linear quadratic adaptive control. Regret minimization is a scientific field in both adaptive control and reinforcement learning research communities which studies the so-called trade-off between exploration and exploitation. Even though a large focus has been on linear quadratic adaptive control with theoretical finitetime bound guarantees on the expected regret growth rate, most of the proposed optimal exploration strategies do not take into account the scaling constant associated to the growth rate. Moreover, the exploration strategies are limited to white noise excitation. Using the tools from experiment design, we propose a computational tractable solution for the design of the external excitation chosen as a white noise filtered by a finite impulse response filter which is adapted on-line. The numerical example shows a reduced regret than available strategies in the literature.
Origine : Fichiers produits par l'(les) auteur(s)