2D versus 3D Convolutional Spiking Neural Networks Trained with Unsupervised STDP for Human Action Recognition - Archive ouverte HAL Accéder directement au contenu
Communication Dans Un Congrès Année : 2022

2D versus 3D Convolutional Spiking Neural Networks Trained with Unsupervised STDP for Human Action Recognition

Résumé

Current advances in technology have highlighted the importance of video analysis in the domain of computer vision. However, video analysis has considerably high computational costs with traditional artificial neural networks (ANNs). Spiking neural networks (SNNs) are third generation biologically plausible models that process the information in the form of spikes. Unsupervised learning with SNNs using the spike timing dependent plasticity (STDP) rule has the potential to overcome some bottlenecks of regular artificial neural networks, but STDP-based SNNs are still immature and their performance is far behind that of ANNs. In this work, we study the performance of SNNs when challenged with the task of human action recognition, because this task has many real-time applications in computer vision, such as video surveillance. In this paper we introduce a multilayered 3D convolutional SNN model trained with unsupervised STDP. We compare the performance of this model to those of a 2D STDP-based SNN when challenged with the KTH and Weizmann datasets. We also compare single-layer and multi-layer versions of these models in order to get an accurate assessment of their performance. We show that STDP-based convolutional SNNs can learn motion patterns using 3D kernels, thus enabling motionbased recognition from videos. Finally, we give evidence that 3D convolution is superior to 2D convolution with STDP-based SNNs, especially when dealing with long video sequences.
Fichier principal
Vignette du fichier
main.pdf (773.04 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-03679597 , version 1 (26-05-2022)

Identifiants

  • HAL Id : hal-03679597 , version 1

Citer

Mireille El-Assal, Pierre Tirilly, Ioan Marius Bilasco. 2D versus 3D Convolutional Spiking Neural Networks Trained with Unsupervised STDP for Human Action Recognition. International Joint Conference on Neural Networks (IJCNN) 2022, Jul 2022, Padova, Italy. ⟨hal-03679597⟩
72 Consultations
81 Téléchargements

Partager

Gmail Facebook X LinkedIn More