Barcodes as summary of loss function's topology - Archive ouverte HAL Accéder directement au contenu
Pré-Publication, Document De Travail Année : 2019

Barcodes as summary of loss function's topology

Résumé

We apply the canonical forms (barcodes) of Morse complexes to explore topology of loss surfaces. We present a novel algorithm for calculations of the objective function's barcodes of minima. We have conducted experiments for calculating barcodes of local minima for benchmark functions and for loss surfaces of neural networks. Our experiments confirm two principal observations: (1) the barcodes of minima are located in a small lower part of the range of values of loss function of neural networks and (2) increase of the neural network's depth brings down the minima's barcodes. This has natural implications for the neural network learning and the ability to generalize.
Fichier principal
Vignette du fichier
1912.00043.pdf (2.73 Mo) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-02425996 , version 1 (31-12-2019)
hal-02425996 , version 2 (06-01-2021)

Identifiants

  • HAL Id : hal-02425996 , version 2

Citer

Serguei Barannikov, Alexander Korotin, Dmitry Oganesyan, Daniil Emtsev, Evgeny Burnaev. Barcodes as summary of loss function's topology. 2019. ⟨hal-02425996v2⟩
89 Consultations
73 Téléchargements

Partager

Gmail Facebook X LinkedIn More