Are alternatives to backpropagation useful for training Binary Neural Networks? An experimental study in image classification - Archive ouverte HAL Accéder directement au contenu
Communication Dans Un Congrès Année : 2023

Are alternatives to backpropagation useful for training Binary Neural Networks? An experimental study in image classification

Résumé

Current artificial neural networks are trained with parameters encoded as floating point numbers that occupy lots of memory space at inference time. Due to the increase in size of deep learning models, it is becoming very difficult to consider training and using artificial neural networks on edge devices such as smartphones. Binary neural networks promise to reduce the size of deep neural network models as well as increasing inference speed while decreasing energy consumption and so allow the deployment of more powerful models on edge devices. However, binary neural networks are still proven to be difficult to train using the backpropagation based gradient descent scheme. We propose to adapt to binary neural networks two training algorithms considered as promising alternatives to backpropagation but for continuous neural networks. We provide experimental comparative results for image classification including the backpropagation baseline on the MNIST, Fashion MNIST and CIFAR-10 datasets in both continuous and binary settings. The results demonstrate that binary neural networks can not only be trained using alternative algorithms to backpropagation but can also be shown to lead better performance and a higher tolerance to the presence or absence of batch normalization layers. CCS CONCEPTS • Computing methodologies → Neural networks;
Fichier principal
Vignette du fichier
SAC_binary_neural_networks.pdf (1.11 Mo) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-04161529 , version 1 (13-07-2023)

Identifiants

Citer

Ben Crulis, Barthelemy Serres, Cyril de Runz, Gilles Venturini. Are alternatives to backpropagation useful for training Binary Neural Networks? An experimental study in image classification. SAC '23: 38th ACM/SIGAPP Symposium on Applied Computing, 2023, Tallinn, Estonia. pp.1171-1178, ⟨10.1145/3555776.3577674⟩. ⟨hal-04161529⟩
39 Consultations
25 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More