Yet another fast variant of Newton's method for nonconvex optimization - Archive ouverte HAL Accéder directement au contenu
Pré-Publication, Document De Travail (Preprint/Prepublication) Année : 2023

Yet another fast variant of Newton's method for nonconvex optimization

Résumé

A class of second-order algorithms is proposed for minimizing smooth nonconvex functions that alternates between regularized Newton and negative curvature steps in an iteration-dependent subspace. In most cases, the Hessian matrix is regularized with the square root of the current gradient and an additional term taking moderate negative curvature into account, a negative curvature step being taken only exceptionally. Practical variants have been detailed where the subspaces are chosen to be the full space, or Krylov subspaces. In the first case, the proposed method only requires the solution of a single linear system at nearly all iterations. We establish that at most $\mathcal{O}\left(|\log\epsilon|\,\epsilon^{-3/2}\right)$ evaluations of the problem's objective function and derivatives are needed for algorithms in the new class to obtain an $\epsilon$-approximate first-order minimizer, and at most $\mathcal{O}\left(|\log\epsilon|\,\epsilon^{-3}\right)$ to obtain a second-order one. Encouraging initial numerical experiments with two full-space and two Krylov-subspaces variants are finally presented.
Fichier principal
Vignette du fichier
2302.10065.pdf (532.96 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-04269968 , version 1 (03-11-2023)

Identifiants

Citer

Serge Gratton, Sadok Jerad, Philippe L. Toint. Yet another fast variant of Newton's method for nonconvex optimization. 2023. ⟨hal-04269968⟩
110 Consultations
22 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More