On data selection for the energy efficiency of neural networks: Towards a new solution based on a dynamic selectivity ratio - Archive ouverte HAL
Communication Dans Un Congrès Année : 2023

On data selection for the energy efficiency of neural networks: Towards a new solution based on a dynamic selectivity ratio

Résumé

In this paper, we address the energy efficiency of neural networks training through data selection techniques. We first study the impact of a random data selection approach that renews the selected examples periodically during training. We find that random selection should be considered as a serious option as it allows high energy gains with small accuracy losses. Unexpectedly, it even outperforms a more elaborate approach in some cases. Our study of the random approach conducted us to observe that low selectivity ratios allow important energy savings, but also cause a significant accuracy decrease. To mitigate the effect of such ratios on the prediction quality, we propose to use a dynamic selectivity ratio with a decreasing schedule, that can be integrated to any selection approach. Our first results show that using such a schedule provides around 60% energy gains on the CIFAR-10 dataset with less than 1% accuracy decrease. It also improves the convergence when compared to a fixed ratio.
Fichier principal
Vignette du fichier
ictai_paper_221.pdf (286.94 Ko) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-04282114 , version 1 (13-11-2023)

Licence

Identifiants

Citer

Anais Boumendil, Walid Bechkit, Karima Benatchba. On data selection for the energy efficiency of neural networks: Towards a new solution based on a dynamic selectivity ratio. ICTAI 2023 - IEEE 35th International Conference on Tools with Artificial Intelligence, Nov 2023, Atlanta (GA), United States. ⟨10.1109/ICTAI59109.2023.00054⟩. ⟨hal-04282114⟩
146 Consultations
93 Téléchargements

Altmetric

Partager

More