PSM-nets: Compressing Neural Networks with Product of Sparse Matrices - Archive ouverte HAL
Conference Papers Year : 2021

PSM-nets: Compressing Neural Networks with Product of Sparse Matrices

Luc Giffon
  • Function : Author
  • PersonId : 174103
  • IdHAL : luc-giffon
Stéphane Ayache
Hachem Kadri
Thierry Artières
Ronan Sicre
  • Function : Author

Abstract

Over-parameterization of neural networks is a well known issue that comes along with their great performance. Among the many approaches proposed to tackle this problem, low-rank tensor decompositions are largely investigated to compress deep neural networks. Such techniques rely on a low-rank assumption of the layer weight tensors that does not always hold in practice. Following this observation, this paper studies sparsity inducing techniques to build new sparse matrix product layers for high-rate neural networks compression. Specifically, we explore recent advances in sparse optimization to replace each layer's weight matrix, either convolutional or fully connected, by a product of sparse matrices. Our experiments validate that our approach provides a better compression-accuracy trade-off than most popular low-rank-based compression techniques.
Fichier principal
Vignette du fichier
Palmnet_IJCNN__Copy_.pdf (577.45 Ko) Télécharger le fichier
Origin Files produced by the author(s)

Dates and versions

hal-03151539 , version 1 (24-02-2021)

Identifiers

  • HAL Id : hal-03151539 , version 1

Cite

Luc Giffon, Stéphane Ayache, Hachem Kadri, Thierry Artières, Ronan Sicre. PSM-nets: Compressing Neural Networks with Product of Sparse Matrices. IJCNN, Jul 2021, Virtual Event, United States. ⟨hal-03151539⟩
272 View
616 Download

Share

More