Hcore-Init: Neural Network Initialization based on Graph Degeneracy - Archive ouverte HAL Accéder directement au contenu
Communication Dans Un Congrès Année : 2021

Hcore-Init: Neural Network Initialization based on Graph Degeneracy

Résumé

Neural networks have become a very popular tool for many machine learning tasks, as in recent years we witnessed many novel architectures, learning and optimization techniques for deep learning. Capitalizing on the fact that neural networks inherently constitute multipartite graphs among neuron layers, we aim to analyze directly their structure to extract meaningful information that can improve the learning process. To our knowledge graph mining techniques for enhancing learning in neural networks have not been thoroughly investigated. In this paper we propose an adapted version of the k-core structure for the complete weighted multipartite graph extracted from a deep learning architecture. As a multipartite graph is a combination of bipartite graphs, that are in turn the incidence graphs of hypergraphs, we design k-hypercore decomposition, the hypergraph analogue of k-core degeneracy. We applied k-hypercore to several neural network architectures, more specifically to convolutional neural networks and multilayer perceptrons for image recognition tasks after a very short pretraining. Then we used the information provided by the hypercore numbers of the neurons to re-initialize the weights of the neural network, thus biasing the gradient optimization scheme. Extensive experiments proved that k-hypercore outperforms the state-of-the-art initialization methods.
Fichier principal
Vignette du fichier
hcore.pdf (1.52 Mo) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-03002744 , version 1 (20-11-2020)
hal-03002744 , version 2 (24-05-2022)

Identifiants

Citer

Stratis Limnios, George Dasoulas, Dimitrios M. Thilikos, Michalis Vazirgiannis. Hcore-Init: Neural Network Initialization based on Graph Degeneracy. ICPR 2020 - 25th International Conference on Pattern Recognition, Jan 2021, Milan (Virtual), Italy. pp.5852-5858, ⟨10.1109/ICPR48806.2021.9412940⟩. ⟨hal-03002744v2⟩
150 Consultations
85 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More