Non-smooth Bayesian learning for artificial neural networks - Archive ouverte HAL Accéder directement au contenu
Article Dans Une Revue Journal of Ambient Intelligence and Humanized Computing Année : 2022

Non-smooth Bayesian learning for artificial neural networks

Résumé

Artificial Neural Networks (ANNs) are being widely used in supervised Machine Learning (ML) to analyze signals or images for many applications. Using an annotated learning database, one of the main challenges is to optimize the network weights. A lot of work on solving optimization problems or improving optimization methods in machine learning has been proposed successively such as gradient-based method, Newton-type method, meta-heuristic method. Moreover, sparse regularizer is designed to zero superfluous weights and hence remove unneeded connections. However, if one wants to promote sparse networks, such as the 1 norm, one need use sparse regularizations, the optimization process becomes challenging since the error to be minimized is no longer differentiable. In this paper, we propose an MCMC-based optimization scheme formulated in a Bayesian framework. The proposed scheme solves the above-mentioned sparse optimization problem using an efficient sampling scheme and Hamiltonian dynamics. The designed optimizer is conducted on four datasets, two COVID-19 and two standard datasets (Fashion-MNIST and CIFAR-10). The results are verified by a comparative study with three different CNNs, one of which is deeper than the others in order to evaluate the effectiveness of our optimizer. Promising results show the usefulness of the proposed method to allow ANNs, even with low complexity levels, reaching high accuracy rates of up to 94% in most of the experiments carried out. The proposed method is also faster and more robust concerning overfitting issues. More importantly, the training step of the proposed method is much faster than all competing algorithms.
Fichier principal
Vignette du fichier
AIHC-2022.pdf (869.98 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-03669388 , version 1 (16-05-2022)

Identifiants

  • HAL Id : hal-03669388 , version 1

Citer

Mohamed Fakhfakh, Lotfi Chaâri, Bassem Bouaziz, Faiez Gargouri. Non-smooth Bayesian learning for artificial neural networks. Journal of Ambient Intelligence and Humanized Computing, In press. ⟨hal-03669388⟩
80 Consultations
54 Téléchargements

Partager

Gmail Facebook X LinkedIn More