PSM-nets: Compressing Neural Networks with Product of Sparse Matrices - Archive ouverte HAL
Communication Dans Un Congrès Année : 2021

PSM-nets: Compressing Neural Networks with Product of Sparse Matrices

Luc Giffon
  • Fonction : Auteur
  • PersonId : 174103
  • IdHAL : luc-giffon
Stéphane Ayache
Hachem Kadri
Thierry Artières
Ronan Sicre
  • Fonction : Auteur

Résumé

Over-parameterization of neural networks is a well known issue that comes along with their great performance. Among the many approaches proposed to tackle this problem, low-rank tensor decompositions are largely investigated to compress deep neural networks. Such techniques rely on a low-rank assumption of the layer weight tensors that does not always hold in practice. Following this observation, this paper studies sparsity inducing techniques to build new sparse matrix product layers for high-rate neural networks compression. Specifically, we explore recent advances in sparse optimization to replace each layer's weight matrix, either convolutional or fully connected, by a product of sparse matrices. Our experiments validate that our approach provides a better compression-accuracy trade-off than most popular low-rank-based compression techniques.
Fichier principal
Vignette du fichier
Palmnet_IJCNN__Copy_.pdf (577.45 Ko) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-03151539 , version 1 (24-02-2021)

Identifiants

  • HAL Id : hal-03151539 , version 1

Citer

Luc Giffon, Stéphane Ayache, Hachem Kadri, Thierry Artières, Ronan Sicre. PSM-nets: Compressing Neural Networks with Product of Sparse Matrices. IJCNN, Jul 2021, Virtual Event, United States. ⟨hal-03151539⟩
274 Consultations
625 Téléchargements

Partager

More