Using Euler's Method to Prove the Convergence of Neural Networks - Archive ouverte HAL Accéder directement au contenu
Article Dans Une Revue IEEE Control Systems Letters Année : 2022

Using Euler's Method to Prove the Convergence of Neural Networks

Jawher Jerray
Adnane Saoud
Laurent Fribourg
  • Fonction : Auteur
  • PersonId : 1051357

Résumé

It was shown in the literature that, for a fully connected neural network (NN), the gradient descent algorithm converges to zero. Motivated by that work, we provide here general conditions under which we can derive the convergence of the gradient descent algorithm from the convergence of the gradient flow, in the case of NNs, in a systematic way. Our approach is based on an analysis of the error in Euler's method in the case of NNs, and relies on the concept of local strong convexity. Unlike existing approaches in the literature, our approach allows to provide convergence guarantees without making any assumptions on the number of hidden nodes of the NN or the number of training data points. A numerical example is proposed, showing the merits of our approach.
Fichier principal
Vignette du fichier
CDC22_1664_FI.pdf (358.97 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-03816597 , version 1 (17-10-2022)

Identifiants

Citer

Jawher Jerray, Adnane Saoud, Laurent Fribourg. Using Euler's Method to Prove the Convergence of Neural Networks. IEEE Control Systems Letters, 2022, ⟨10.1109/lcsys.2022.3184040⟩. ⟨hal-03816597⟩
46 Consultations
168 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More