Stochastic Gradient Descent Revisited
Résumé
Stochastic gradient descent (SGD) algorithms have been a go-to solution for nonconvex stochastic optimization problems arising in machine learning. Their theory however often requires a strong framework to guarantee convergence properties. We hereby present a full scope convergence study of nonconvex SGD, including weak convergence, functionvalue convergence and global convergence, and also provide subsequent convergence rates and complexities, all under relatively mild conditions in comparison with literature.
Origine | Fichiers produits par l'(les) auteur(s) |
---|