Investigating current-based and gating approaches for accurate and e-efficient spiking recurrent neural networks
Résumé
Spiking Neural Networks (SNNs) with spike-based computations and communications may be more energy-efficient than Artificial Neural Networks (ANNs) for embedded applications. However, SNNs have mostly been applied to image processing, although audio applications may better fit their temporal dynamics. We evaluate th e accuracy and energy-efficiency of Leaky Integrate-and-Fire (LIF) models on spiking audio datasets compared to ANNs. We demonstrate that, for processing temporal sequences, the Current-based LIF (Cuba-LIF) outperforms the LIF. Moreover, gated recurrent networks have demonstrated superior accuracy than simple recurrent networks for such tasks. Therefore, we introduce SpikGRU, a gated version of the Cuba-LIF. SpikGRU achieves higher accuracy than other recurrent SNNs on the most difficult task studied in this work. The Cuba-LIF and SpikGRU reach state-of-theart accuracy, only <1.1% below the accuracy of the best ANNs, while showing up to a 49x reduction in the number of operations compared to ANNs, due to the high spike sparsity.
Origine | Fichiers produits par l'(les) auteur(s) |
---|