Low-complexity Approximate Convolutional Neural Networks - Archive ouverte HAL Access content directly
Journal Articles IEEE Transactions on Neural Networks and Learning Systems Year : 2018

Low-complexity Approximate Convolutional Neural Networks

Abstract

In this paper, we present an approach for minimizing the computational complexity of trained ConvolutionalNeural Networks (ConvNet). The idea is to approximate allelements of a given ConvNet and replace the original convolutional filters and parameters (pooling and bias coefficients; and activation function) with efficient approximations capable of extreme reductions in computational complexity. Low-complexity convolution filters are obtained through a binary (zero-one) linear programming scheme based on the Frobenius norm over sets of dyadic rationals. The resulting matrices allow for multiplication- free computations requiring only addition and bit-shifting operations. Such low-complexity structures pave the way for low-power, efficient hardware designs. We applied our approach on three use cases of different complexity: (i) a “light” but efficient ConvNet for face detection (with around 1 000 parameters); (ii) another one for hand-written digit classification (with more than 180 000 parameters); and (iii) a significantly larger ConvNet: AlexNet with ≈1.2 million matrices. We evaluated the overall performance on the respective tasks for different levels of approximations. In all considered applications, very low-complexity approximations have been derived maintaining an almost equal classification performance.
Fichier principal
Vignette du fichier
tnnls2018_corrected.pdf (582.2 Ko) Télécharger le fichier
Origin : Files produced by the author(s)
Loading...

Dates and versions

hal-01727219 , version 1 (27-08-2020)

Identifiers

Cite

Renato J. Cintra, Stefan Duffner, Christophe Garcia, André Leite. Low-complexity Approximate Convolutional Neural Networks. IEEE Transactions on Neural Networks and Learning Systems, 2018, ⟨10.1109/TNNLS.2018.2815435⟩. ⟨hal-01727219⟩
495 View
319 Download

Altmetric

Share

Gmail Facebook X LinkedIn More