From the Ravine method to the Nesterov method and vice versa: a dynamical system perspective - Archive ouverte HAL Accéder directement au contenu
Article Dans Une Revue SIAM Journal on Optimization Année : 2022

From the Ravine method to the Nesterov method and vice versa: a dynamical system perspective

Hedy Attouch
Jalal M. Fadili

Résumé

We revisit the Ravine method of Gelfand and Tsetlin from a dynamical system perspective, study its convergence properties, and highlight its similarities and differences with the Nesterov accelerated gradient method. The two methods are closely related. They can be deduced from each other by reversing the order of the extrapolation and gradient operations in their definitions. They benefit from similar fast convergence of values and convergence of iterates for general convex objective functions. We will also establish the high resolution ODE of the Ravine and Nesterov methods, and reveal an additional geometric damping term driven by the Hessian for both methods. This will allow us to prove fast convergence towards zero of the gradients not only for the Ravine method but also for the Nesterov method for the first time. We also highlight connections to other algorithms stemming from more subtle discretization schemes, and finally describe a Ravine version of the proximal-gradient algorithms for general structured smooth + non-smooth convex optimization problems.
Fichier principal
Vignette du fichier
Ravine-SIOPT-Jan-27-2022_long.pdf (535.29 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-03546187 , version 1 (27-01-2022)
hal-03546187 , version 2 (01-02-2022)

Identifiants

Citer

Hedy Attouch, Jalal M. Fadili. From the Ravine method to the Nesterov method and vice versa: a dynamical system perspective. SIAM Journal on Optimization, 2022, 32 (3), ⟨10.1137/22M1474357⟩. ⟨hal-03546187v2⟩
82 Consultations
134 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More