Stochastic Variable Metric Proximal Gradient with variance reduction for non-convex composite optimization - Archive ouverte HAL
Article Dans Une Revue Statistics and Computing Année : 2023

Stochastic Variable Metric Proximal Gradient with variance reduction for non-convex composite optimization

Résumé

This paper introduces a novel algorithm, the Perturbed Proximal Preconditioned SPIDER algorithm (3P-SPIDER), designed to solve finite sum non-convex composite optimization. It is a stochastic Variable Metric Forward-Backward algorithm, which allows approximate preconditioned forward operator and uses a variable metric proximity operator as the backward operator; it also proposes a mini-batch strategy with variance reduction to address the finite sum setting. We show that 3P-SPIDER extends some Stochastic preconditioned Gradient Descent-based algorithms and some Incremental Expectation Maximization algorithms to composite optimization and to the case the forward operator can not be computed in closed form. We also provide an explicit control of convergence in expectation of 3P-SPIDER, and study its complexity in order to satisfy the epsilon-approximate stationary condition. Our results are the first to combine the composite non-convex optimization setting, a variance reduction technique to tackle the finite sum setting by using a minibatch strategy and, to allow deterministic or random approximations of the preconditioned forward operator. Finally, through an application to inference in a logistic regression model with random effects, we numerically compare 3P-SPIDER to other stochastic forward-backward algorithms and discuss the role of some design parameters of 3P-SPIDER.
Fichier principal
Vignette du fichier
FM2022_HAL_arXiv.pdf (788.77 Ko) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-03781216 , version 1 (20-09-2022)
hal-03781216 , version 2 (31-12-2022)
hal-03781216 , version 3 (02-03-2023)

Identifiants

Citer

Gersende Fort, Eric Moulines. Stochastic Variable Metric Proximal Gradient with variance reduction for non-convex composite optimization. Statistics and Computing, In press, ⟨10.1007/s11222-023-10230-6⟩. ⟨hal-03781216v3⟩
248 Consultations
167 Téléchargements

Altmetric

Partager

More