A GPU-based Associative Memory using Sparse Neural Networks - Archive ouverte HAL Accéder directement au contenu
Communication Dans Un Congrès Année : 2014

A GPU-based Associative Memory using Sparse Neural Networks

Résumé

A variety of algorithms, store content in such a way that it can be later retrieved by probing the memory with a small portion of it, rather than with an address as in more traditional memories. Recently, Gripon and Berrou have introduced a novel construction which builds on ideas from the theory of error correcting codes, greatly outperforming the celebrated Hopfield Neural Networks in terms of the number of stored messages per neuron and the number of stored bits per synapse. The work of Gripon and Berrou proposes two retrieval rules, SUM - OF - SUM and SUM - OF - MAX . In this paper, we implement both rules on a general purpose graphical processing unit (GPU). SUM - OF - SUM uses only matrix-vector multiplication and is easily implemented on the GPU, whereas SUM - OF - MAX , which involves non-linear operations, is much less straightforward to fulfill. However, SUM - OF - MAX gives significantly better retrieval error rates. We propose a hybrid scheme tailored for implementation on a GPU which achieves a 880-fold speedup without sacrificing any accuracy.
Fichier non déposé

Dates et versions

hal-01174294 , version 1 (08-07-2015)

Identifiants

Citer

Zhe Yao, Vincent Gripon. A GPU-based Associative Memory using Sparse Neural Networks. HPCS 2014 : the international conference on High Performance Computing and Simulation, Jul 2014, Bologna, Italy. pp.688 - 692, ⟨10.1109/HPCSim.2014.6903755⟩. ⟨hal-01174294⟩
68 Consultations
0 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More