Improve convolutional neural network pruning by maximizing filter variety - Archive ouverte HAL Accéder directement au contenu
Chapitre D'ouvrage Année : 2022

Improve convolutional neural network pruning by maximizing filter variety

Résumé

Neural network pruning is a widely used strategy for reducing model storage and computing requirements. It allows to lower the complexity of the network by introducing sparsity in the weights. Because taking advantage of sparse matrices is still challenging, pruning is often performed in a structured way, i.e. removing entire convolution filters in the case of ConvNets, according to a chosen pruning criteria. Common pruning criteria, such as l1-norm or movement, usually do not consider the individual utility of filters, which may lead to: (1) the removal of filters exhibiting rare, thus important and discriminative behaviour, and (2) the retaining of filters with redundant information. In this paper, we present a technique solving those two issues, and which can be appended to any pruning criteria. This technique ensures that the criteria of selection focuses on redundant filters, while retaining the rare ones, thus maximizing the variety of remaining filters. The experimental results, carried out on different datasets (CIFAR-10, CIFAR-100 and CALTECH-101) and using different architectures (VGG-16 and ResNet-18) demonstrate that it is possible to achieve similar sparsity levels while maintaining a higher performance when appending our filter selection technique to pruning criteria. Moreover, we assess the quality of the found sparse subnetworks by applying the Lottery Ticket Hypothesis and find that the addition of our method allows to discover better performing tickets in most cases.
Fichier principal
Vignette du fichier
2203.05807 (11.7 Mo) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-03930224 , version 1 (05-01-2024)

Identifiants

Citer

Nathan Hubens, Marius Preda, Matei Mancas, Bernard Gosselin, Titus Zaharia. Improve convolutional neural network pruning by maximizing filter variety. Image Analysis and Processing – ICIAP 2022, 13231, Springer International Publishing, pp.379-390, 2022, Lecture Notes in Computer Science, 978-3-031-06427-2. ⟨10.1007/978-3-031-06427-2_32⟩. ⟨hal-03930224⟩
25 Consultations
2 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More