Technical report: supervised training of convolutional spiking neural networks with PyTorch - Archive ouverte HAL Accéder directement au contenu
Rapport (Rapport Technique) Année : 2019

Technical report: supervised training of convolutional spiking neural networks with PyTorch

Résumé

Recently, it has been shown that spiking neural networks (SNNs) can be trained efficiently, in a supervised manner, using backpropagation through time. Indeed, the most commonly used spiking neuron model, the leaky integrate-and-fire neuron, obeys a differential equation which can be approximated using discrete time steps, leading to a recurrent relation for the potential. The firing threshold causes optimization issues, but they can be overcome using a surrogate gradient. Here, we extend previous approaches in two ways. Firstly, we show that the approach can be used to train convolutional layers. Convolutions can be done in space, time (which simulates conduction delays), or both. Secondly, we include fast horizontal connections \`a la Den\`eve: when a neuron N fires, we subtract to the potentials of all the neurons with the same receptive the dot product between their weight vectors and the one of neuron N. As Den\`eve et al. showed, this is useful to represent a dynamic multidimensional analog signal in a population of spiking neurons. Here we demonstrate that, in addition, such connections also allow implementing a multidimensional send-on-delta coding scheme. We validate our approach on one speech classification benchmarks: the Google speech command dataset. We managed to reach nearly state-of-the-art accuracy (94%) while maintaining low firing rates (about 5Hz). Our code is based on PyTorch and is available in open source at http://github.com/romainzimmer/s2net
Fichier principal
Vignette du fichier
1911.10124.pdf (742.21 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-02870276 , version 1 (18-12-2020)

Identifiants

Citer

Romain Zimmer, Thomas Pellegrini, Srisht Fateh Singh, Timothée Masquelier. Technical report: supervised training of convolutional spiking neural networks with PyTorch. [Technical Report] CERCO UMR 5549, CNRS – Université Toulouse 3, Toulouse, France. 2019. ⟨hal-02870276⟩
106 Consultations
91 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More