Multilevel Objective-Function-Free Optimization with an Application to Neural Networks Training - Archive ouverte HAL Accéder directement au contenu
Article Dans Une Revue SIAM Journal on Optimization Année : 2023

Multilevel Objective-Function-Free Optimization with an Application to Neural Networks Training

Résumé

A class of multilevel algorithms for unconstrained nonlinear optimization is presented which does not require the evaluation of the objective function. The class contains the momentum-less AdaGrad method as a particular (single-level) instance. The choice of avoiding the evaluation of the objective function is intended to make the algorithms of the class less sensitive to noise, while the multilevel feature aims at reducing their computational cost. The evaluation complexity of these algorithms is analyzed and their behavior in the presence of noise is then illustrated in the context of training deep neural networks for supervised learning applications.

Dates et versions

hal-04436393 , version 1 (03-02-2024)

Licence

Paternité

Identifiants

Citer

Serge Gratton, Alena Kopaničáková, Philippe L. Toint. Multilevel Objective-Function-Free Optimization with an Application to Neural Networks Training. SIAM Journal on Optimization, 2023, 33 (4), pp.2772-2800. ⟨10.1137/23M1553455⟩. ⟨hal-04436393⟩
35 Consultations
0 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More