Backprop Diffusion is Biologically Plausible - Archive ouverte HAL Access content directly
Preprints, Working Papers, ... Year : 2020

Backprop Diffusion is Biologically Plausible

Abstract

The Backpropagation algorithm relies on the abstraction of using a neural model that gets rid of the notion of time, since the input is mapped instantaneously to the output. In this paper, we claim that this abstraction of ignoring time, along with the abrupt input changes that occur when feeding the training set, are in fact the reasons why, in some papers, Backprop biological plausibility is regarded as an arguable issue. We show that as soon as a deep feedforward network operates with neurons with time-delayed response, the backprop weight update turns out to be the basic equation of a biologically plausible diffusion process based on forward-backward waves. We also show that such a process very well approximates the gradient for inputs that are not too fast with respect to the depth of the network. These remarks somewhat disclose the diffusion process behind the backprop equation and leads us to interpret the corresponding algorithm as a degeneration of a more general diffusion process that takes place also in neural networks with cyclic connections.
Fichier principal
Vignette du fichier
1912.04635.pdf (459.95 Ko) Télécharger le fichier
Origin : Files produced by the author(s)
Loading...

Dates and versions

hal-02878574 , version 1 (23-06-2020)

Identifiers

  • HAL Id : hal-02878574 , version 1

Cite

Marco Gori, Alessandro Betti. Backprop Diffusion is Biologically Plausible. 2020. ⟨hal-02878574⟩
63 View
112 Download

Share

Gmail Facebook X LinkedIn More