Stochastic Approximation Beyond Gradient for Signal Processing and Machine Learning - Archive ouverte HAL Accéder directement au contenu
Article Dans Une Revue IEEE Transactions on Signal Processing Année : 2023

Stochastic Approximation Beyond Gradient for Signal Processing and Machine Learning

Résumé

Stochastic Approximation (SA) is a classical algorithm that has had since the early days a huge impact on signal processing, and nowadays on machine learning, due to the necessity to deal with a large amount of data observed with uncertainties. An exemplar special case of SA pertains to the popular stochastic (sub)gradient algorithm which is the working horse behind many important applications. A lesser-known fact is that the SA scheme also extends to non-stochastic-gradient algorithms such as compressed stochastic gradient, stochastic expectationmaximization, and a number of reinforcement learning algorithms. The aim of this article is to overview and introduce the non-stochastic-gradient perspectives of SA to the signal processing and machine learning audiences through presenting a design guideline of SA algorithms backed by theories. Our central theme is to propose a general framework that unifies existing theories of SA, including its non-asymptotic and asymptotic convergence results, and demonstrate their applications on popular nonstochastic-gradient algorithms. We build our analysis framework based on classes of Lyapunov functions that satisfy a variety of mild conditions. We draw connections between non-stochasticgradient algorithms and scenarios when the Lyapunov function is smooth, convex, or strongly convex. Using the said framework, we illustrate the convergence properties of the non-stochasticgradient algorithms using concrete examples. Extensions to the emerging variance reduction techniques for improved sample complexity will also be discussed.
Fichier principal
Vignette du fichier
Overview_TSP_EM-1.pdf (3.29 Mo) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-04174351 , version 1 (31-07-2023)

Identifiants

  • HAL Id : hal-04174351 , version 1

Citer

Aymeric Dieuleveut, Gersende Fort, Éric Moulines, Hoi-To Wai. Stochastic Approximation Beyond Gradient for Signal Processing and Machine Learning. IEEE Transactions on Signal Processing, In press. ⟨hal-04174351⟩
26 Consultations
27 Téléchargements

Partager

Gmail Facebook X LinkedIn More