On the effect of perturbations in first-order optimization methods with inertia and Hessian driven damping - Archive ouverte HAL
Article Dans Une Revue Evolution Equations and Control Theory Année : 2023

On the effect of perturbations in first-order optimization methods with inertia and Hessian driven damping

Résumé

Second-order continuous-time dissipative dynamical systems with viscous and Hessian driven damping have inspired effective first-order algorithms for solving convex optimization problems. While preserving the fast convergence properties of the Nesterov-type acceleration, the Hessian driven damping makes it possible to significantly attenuate the oscillations. To study the stability of these algorithms with respect to perturbations, we analyze the behaviour of the corresponding continuous systems when the gradient computation is subject to exogenous additive errors. We provide a quantitative analysis of the asymptotic behaviour of two types of systems, those with implicit and explicit Hessian driven damping. We consider convex, strongly convex, and non-smooth objective functions defined on a real Hilbert space and show that, depending on the formulation, different integrability conditions on the perturbations are sufficient to maintain the convergence rates of the systems. We highlight the differences between the implicit and explicit Hessian damping, and in particular point out that the assumptions on the objective and perturbations needed in the implicit case are more stringent than in the explicit case.
Fichier principal
Vignette du fichier
2_Hessian-pert-Mar-29-2022_rev.pdf (3.05 Mo) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-03275215 , version 1 (30-06-2021)
hal-03275215 , version 2 (17-03-2022)
hal-03275215 , version 3 (05-04-2022)

Identifiants

Citer

Hedy Attouch, Jalal M. Fadili, Vyacheslav Kungurtsev. On the effect of perturbations in first-order optimization methods with inertia and Hessian driven damping. Evolution Equations and Control Theory, 2023, 12 (1), pp.71-117. ⟨10.3934/eect.2022022⟩. ⟨hal-03275215v3⟩
139 Consultations
97 Téléchargements

Altmetric

Partager

More