An experimental study of the impact of pre-training on the pruning of a convolutional neural network - Archive ouverte HAL Accéder directement au contenu
Communication Dans Un Congrès Année : 2020

An experimental study of the impact of pre-training on the pruning of a convolutional neural network

Résumé

In recent years, deep neural networks have known a wide success in various application domains. However, they require important computational and memory resources, which severely hinders their deployment, notably on mobile devices or for real-time applications. Neural networks usually involve a large number of parameters, which correspond to the weights of the network. Such parameters, obtained with the help of a training process, are determinant for the performance of the network. However, they are also highly redundant. The pruning methods notably attempt to reduce the size of the parameter set, by identifying and removing the irrelevant weights. In this paper, we examine the impact of the training strategy on the pruning efficiency. Two training modalities are considered and compared: (1) fine-tuned and (2) from scratch. The experimental results obtained on four datasets (CIFAR10, CIFAR100, SVHN and Caltech101) and for two different CNNs (VGG16 and MobileNet) demonstrate that a network that has been pre-trained on a large corpus (e.g. ImageNet) and then fine-tuned on a particular dataset can be pruned much more efficiently (up to 80% of parameter reduction) than the same network trained from scratch.

Dates et versions

hal-04389265 , version 1 (11-01-2024)

Identifiants

Citer

Nathan Hubens, Matei Mancas, Marc Decombas, Marius Preda, Titus Zaharia, et al.. An experimental study of the impact of pre-training on the pruning of a convolutional neural network. 3rd International Conference on Applications of Intelligent Systems (APPIS), Jan 2020, Las Palmas de Gran Canaria Spain, France. pp.1-6, ⟨10.1145/3378184.3378224⟩. ⟨hal-04389265⟩
8 Consultations
0 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More