A vector almost-supermartingale convergence theorem and its applications
Résumé
The almost-supermartingale convergence theorem of Robbins and Siegmund (1971) is a fundamental tool for establishing the convergence of various stochastic iterative algorithms including system identification, adaptive control, and reinforcement learning. The theorem is stated for non-negative scalar valued stochastic processes. In this paper, we generalize the theorem to non-negative vector valued stochastic processes and provide two set of sufficient conditions for such processes to converge almost surely. We present several applications of vector almost-supermartingale convergence theorem, including convergence of autoregressive supermartingales, delayed supermartingales, and stochastic approximation with delayed updates.
Fichier principal
Delayed_stochastic_approximation_supermartingale_VF.pdf (228.45 Ko)
Télécharger le fichier
Origine | Fichiers produits par l'(les) auteur(s) |
---|