\newpage
\chapter{Deep learning in spiking neural network}

\label{deep}
\begin{abstract}
Deep learning recently is used in several state-of-the-art studies and developments
due to its ability in pattern recognition data classification.
Most of the research work and applications
are in machine learning domain. In this chapter, we introduce a framework
to use the proposed rate based version of Contrastive Divergence
(CD) updating weight rule. Respecting to the rate aspect of neural
coding, we develop a Spike-Based Deep Belief Network (S-DBN)
with Leaky Integrate-and-Fire (LIF) neurons and the model is
evaluated using ORL face detection dataset.
The proposed method provides a suitable framework that
can be used in implementing deep architectures in the
neuromorphic hardware systems.
\end{abstract}

\section{Introduction}
Deep learning is currently very active research area in machine learning and pattern
recognition society due to its potential to classify and predict as a result of processing Big
data and Internet of Things (IoT) related data. The implementation on neuromorphic
hardware platforms emulating large-scale networks of spiking neurons may present important
advantages from the perspectives of scalability, power dissipation and real-time interfacing, and
better recognition performance in neural network learning \cite{neftci_event-driven_2014}.
Considering brain mechanisms, prerequisite for an intelligent system is utilizing multi-
level architecture such that each level of the model provides
extracted features for the next one.
This approach of feature
extraction leads the system to implement complex functions
using higher level of abstraction. Deep Learning is a
set of powerful machine learning methods for training deep
architectures. Considering the inherent inefficiency of learning
methods from traditional Artificial Neural Networks in deep
architectures, Contrastive Divergence (CD) has been proposed
to train Restricted Boltzmann Machines (RBM) as the main
building blocks of deep networks \cite{hinton_training_2002}.
Deep architectures can
be developed by stacking RBMs and training the layers using
Contrastive Divergence (CD) as the most efficient algorithm
in deep learning in a greedy layer-wise approach \cite{hinton_fast_2006}.

Despite the proficiency of deep architectures in machine
learning tasks, implementing these models in common platforms
can be a very time and resource consuming process. The
conventional digital computers because of using
Von Neumann architecture, waste too many energy specifically
through storing results and retrieving data in/from memory
elements \cite{soudry_memristor-based_2015}. The neuro-inspired
computational platform which has memory next to the
computation unit, could be an alternative solution for
efficient computation using CD algorithm.
Spiking Neural Networks (SNN)
 are more close
to biological neurons rather than abstract mathematical models.
The important aspect of spiking model
of neurons is its potential for hardware implementation as a
very specific and power efficient hardware accelerator.
In SNN the neurons communicate using spikes \cite{gerstner2014neuronal}.
Therefore, we have to
design an NN architecture that implement spiking data.
In this chapter, we present a framework for using Contrastive Divergence
to train an SNN using RBM and spiking LIF neurons.
This framework can open a new window toward the neruromorphic
architecture designer to apply the state-of-the-art of machine learning
learning algorithm in SNN architecture.

To overcome the challenge
of using spiking neurons in machine learning domain, one uses
 the abstract model of neuron. OConnor \textit{et al.} \cite{oconnor_real-time_2013}
applied an abstract model of Leaky Integrate-and-Fire (LIF) neuron
called Siegert neuron \cite{siegert_first_1951} to approximate the average of
output firing rates of LIF neurons. Using this abstract model as
a unit of a Deep Belief Network (DBN) and exploiting the standard
Conterastive Divergence (CD) algorithm, they have trained an
DBN and the adjusted weights are transferred to a functionally
equivalent Spiking platform of LIF neurons. Using such an
offline learning approach, all the weights are fixed and no weight adjustments
can be performed in application platform.

Spike-Timing Dependent Plasticity (STDP) as the main
Hebbian-based learning method, is used for updating the synaptic
weights in SNN \cite{gerstner2014neuronal}.
This spike-based learning rule in a time-averaged can be interpreted
as anti-Hebbian and Hebbian correlations between input and
output spikes \cite{kempter_intrinsic_2001}.
In \cite{neftci_event-driven_2014} utilizing an abstract model based on
neural sampling a STDP-based version of CD has been
proposed and a single RBM is trained online successfully.

In this work, regrading the rate aspect of neural coding,
 we proposed a rate coded contrastive divergence as
an online learning rule for spike-based RBM. Consequently, using the
proposed learning approach, an acceptable accuracy of a Spike Based
DBN is demonstrated.

Finally, the remainder of this study is organized as
follows. In Section \ref{RBM}, we discuss the
mathematical backgrounds of the CD and compare the spike
coding with the rate coding in the neural architecture.
In Section \ref{ANNvsSNN}, after
studying the preliminary
principles of an online model, a contrastive divergence adapted to the
spike-based RBMs is proposed. The proposed approach is developed and
verified in Section \ref{DBN__Siegert} for a single spiking RBM and also
for the desired Spike-Based Deep Belief Network. The model is evaluated in
Section \ref{Evaluation_DBN_LIF}. Finally, the conclusion as well as
future possible works have been discussed in Section \ref{conc-deep}.

\section{Restricted Boltzmann Machine and Contrastive Divergence}
\label{RBM}
Restricted Boltzmann machines, as a model of artificial neural networks, have been driven
from Boltzmann machines.
RBM consists of binary stochastic units connected to each other using bidirectional edges.
It can represent a probabilistic distribution to learn the basic features of an unknown
distribution using observed data, which is considered as the training data. Generally,
training in Boltzmann machine, as fully Recurrent Neural Network (RNN), is involved with
a large number of complex computations. Therefore, applying some restrictions to the
Boltzmann machine topology leads to a less complex structure called
Restricted Boltzmann Machine (Figure \ref{fig:BM_RBM}).
\begin{figure}
\centering
\includegraphics[width=0.46\textwidth] {./figures/ch7/BM_RBM.pdf}
\caption{Restricted Boltzmann Machine is a
network of neurons which neurons in one layer are
connected to all neurons in the next layer.}
\label{fig:BM_RBM}
\end{figure}
From a structural point of view, RBM has one visible and one hidden layer,
where all the units in the visible and hidden layers are symmetrically connected,
but there is no visible-visible or hidden-hidden connection. The structure of the
Boltzmann machine is related to the proposed structure in 1982 by John Hopfield \cite{hopfield_neural_1982}.
The Hopfield model is driven from thermodynamic systems and can be quantified through equilibrium energy.
Each state of the Boltzmann machine can be expressed as a value called energy of the state.
Equation \ref{eq:RBM_Energy} presents the energy function of a given RBM as a restricted type of Boltzmann machine.
\begin{equation}
E(V,H)=-\sum_i \sum_j v_ih_jw_{ij}-\sum_ia_iv_i-\sum_jb_jh_j,
\label{eq:RBM_Energy}
\end{equation}
where $E$ is the total energy of the network, $v_i$ is the state of visible $i^{th}$ unit, $h_j$ is the
state of $j^{th}$ hidden unit, $w_{ij}$ is the weight between $v_i$ and $h_j$, $a_i$ and $b_j$ are the biases.
The assigned probability to each configuration of the network states are:
\begin{equation}
p(V,H)=\frac{1}{Z}e^{-E(V,H)},
\label{eq:prob_energy}
\end{equation}
where $Z=\sum_{VH}e^{E(V,H)}$ is a partition function.
RBM, as a generative model,
tries to generate an internal representation of its environment.
Increasing the
log-probability of the generating input data
vector using equation \ref{eq:prob_energy}
leads to contrastive divergence  \cite{CD_RBM_hinton2002training}
updating weight rules for RBM:
\begin{equation}
\Delta w_{ij}=\eta(<v_ih_j>_{data}-<v_ih_j>_{model}).
\label{eq:CD}
\end{equation}
In equation \ref{eq:CD}, $<v_ih_j>_{data}$ represent the expectation under
the distribution specified by input data vector and $<v_ih_j>_{model}$ is
the expectation under the distribution specified by internal representation
of the RBM model. Also the probability of $h_j$ using a given $V$ as the
input data vector, for each $j$ in hidden layer is 1 with probability $p(h_j=1|V)$:
\begin{equation}
p(h_j=1|V)=\sigma(b_j+\Sigma_i v_iw_{ij}),
\end{equation}
when $\sigma(x)$ is the logistic sigmoid function and can be defined as:
\begin{equation}
\sigma(x)=\frac{1}{1+e^x}
\end{equation}
In contrastive divergence using Gibbs updating chain (initiated with
training data) and iteration sampling for a limited time, the
approximated value for $<v_ih_j>_{model}$ can be computed perfectly
\cite{CD_a_new_welling2002new,CD_RBM_hinton2002training}.
A single step of Gibbs sampling using $H$ as a given hidden vector can be expressed as
\begin{equation}
p(v_i=1|H)=\sigma(a_i+\Sigma_j h_jw_{ij}).
\end{equation}
According to \cite{CD_a_new_welling2002new} only one step of contrastive
divergence ($CD_1$), can provide an acceptable approximation of gradient
of the log probability of the training data and iterating the process
for $k$ times provides a more precise value:
\begin{equation}
{(\Delta w_{ij})}^k=\eta(<v_ih_j>^0-<v_ih_j>^k),
\end{equation}
when $<v_ih_j>^0$ is equal to $<v_ih_j>_{data}$ and $<v_ih_j>^k$ means
iterating Gibbs sampling for $k$ times. Also the biased will be updated using these equations:
\begin{equation}
{(\Delta a_i)}^k=\eta ({v_i}^0-{v_i}^k),
\end{equation}
and
\begin{equation}
{(\Delta b_j)}^k=\eta ({h_j}^0-{h_j}^k).
\end{equation}
\section{Deep learning in artificial neural networks versus spiking neural networks}
\label{ANNvsSNN}
Artificial Neural Networks as a tool of artificial
intelligence can demonstrate high level of accuracy
in machine learning problems and specifically in
face recognition applications  \cite{67_mleczko2015rough,65_taylor2006modeling}.
On the other side the brain inspired
models and specifically Spiking Neural Networks are very suitable to be implemented in VLSI.
Therefore, to have a biological model
of neural networks in hardware level SNN
is applied. The SNN can be trained using an
unsupervised Hebbian-based learning rules
such as Spike-Timing Dependent Plasticity (STDP).
However respecting to the deep
architecture of the brain to have a
deep SNN, we have to use efficient algorithms for deep architectures.

Spiking model of neuron uses short pulses (spikes) to coding data \cite{gerstner2014neuronal}.
Since the shape of spikes are generally the same, then the rate and the time of spikes are
the determinant parameters in data transferring in biological models.
Comparing the biological inspired model to machine learning algorithms,
it is obvious that the
time parameter has no role in data coding in machine learning domain.
Consequently,  we have to overcome this gap between ANN and SNN
if we want to use machine learning algorithms (e.g., ANN) in SNN platforms.

In this study, we use the Siegert neuron model that can approximate
the mean firing rate of Leaky Integrate-and-Fire
neurons with Poisson-process inputs. Siegert abstract model
using mathematical equations can estimate the input-output rate
transfer function of Leaky Integrate-and-Fire neurons.
Figure \ref{fig:Siegert} shows the equation of the Siegert neuron
\cite{siegert_first_1951,sig_jug2012spiking}.
\begin{figure}
\centering
\includegraphics[width=0.8\textwidth] {./figures/ch7/SiegertModel.pdf}
\caption{Siegert abstract neuron model \cite{sig_jug2012spiking}}
\label{fig:Siegert}
\end{figure}
SNNs use spike trains with randomly distribution of spike times.
To be able to use Siegert model as a unit in our network,
we assume the spike
trains are Poisson-process with specific firing rates ($\lambda_{in}$).
As it is depicted in Figure \ref{fig:Siegert}, the Siegert model receives
excitatory and inhibitory inputs where $\lambda_e$ is the excitatory
and $\lambda_i$ is the inhibitory rates of incoming spike trains from
pre-synaptic neurons. $w_e$ and $w_i$ are the corresponding synaptic
weights.
Due to more simplicity, we did not categorize the neurons to excitatory and
inhibitory neurons. We assume $\lambda_{in}$ as total input spike rates from
pre-synaptic neurons as well as $w$ which indicate the both of $w_e$ and $w_i$.
By normalizing the values of pixels of an image, we can convert the
density of each pixel to the spike rate of corresponding pixel such that
brighter pixel has higher spike rate and darker one has less firing rate.
LIF neuron parameters can be set by adjusting
the corresponding variables in Siegert equation.
Table \ref{tab:LIF_params} shows the given values
for the membrane time constant ($\tau_m$), the resting potential ($V_{rest}$),
the reset potential ($V_{reset}$), the threshold potential ($V_{th}$)
and the absolute refractory time ($t_{ref}$).

In the following sections stacking the proposed RBM with Siegert units,
we developed a Deep Belief Network. In this network, after training phase,
we transferred
the weight matrix to a DBN with LIF units using Brian
simulator \cite {goodman_brian:_2008} to evaluate
the proposed model. According to the equality between Siegert transfer
function and LIF transfer function, we can use the LIF neurons with
the same parameters (Table \ref{tab:LIF_params}).


\begin{table}
\centering
\caption{LIF parameters}
\label{tab:LIF_params}
\begin{tabular}{|c|l|c|}
\hline
Parameter & Description & value \\
\hline
\hline $\tau_m$ & Membrane time constant & 5sec\\
\hline $V_{rest}$& Resting potential & 0\\
\hline $V_{reset}$& Reset potential & 0\\
\hline $V_{th}$  & Threshold potential & 5mv\\
\hline $t_{ref}$ & Absolute refractory time & 2ms\\
\hline
\end{tabular}
\end{table}

\section{Developing and Training Deep Belief Network with Siegert Units}
\label{DBN__Siegert}
The RBMs with Siegert units can be trained in machine learning domain.
In fact in this model the mean firing rate of input spike trains will be used instead of spike trains.

Hinton \textit{et al.} introduced an efficient Deep Belief Networks (DBN)
for the first time
\cite{hinton_fast_2006}. They have proposed a greedy
layer wised algorithm to train the DBN by training each RBM sequentially.
Figure\footnote{http://deeplearning.net/tutorial/} \ref{fig:Add_RBM} illustrates the stages of stacking RBMs.
\begin{figure}
\centering
\includegraphics[width=0.67\textwidth] {./figures/ch7/Add_RBM_for_DBN.pdf}
\caption{Stacking RBMs as the main building blocks of DBN}
\label{fig:Add_RBM}
\end{figure}
In this work, we use the Olivetti
Research Laboratory (ORL) face detection dataset from Cambridge, UK \footnote{The source of the Database of
Faces: http://www.cl.cam.ac.uk/research/dtg/attarchive/facedatabase.html}.
This dataset originally contains
400 grey scale face images of 40 distinct volunteers. Figure \ref{fig:ORL_samp}
shows some images of ORL dataset.
\begin{figure}
\centering
\includegraphics[width=0.46\textwidth] {./figures/ch7/ORL_samples}
\caption{Some sample images from ORL dataset}
\label{fig:ORL_samp}
\end{figure}
In this chapter, we use a resized version of ORL that has been proposed in \cite{ORL_32_32}.
The input images with 32$\times$32 pixels are used as input vectors with 1024 elements.
According to \cite{rtguide_hinton2012}, in each step of RBM training process,a
mini-batch of training data consisted of a small subset of all training data is used.
The proposed DBN architecture, respecting to the size of training vectors, is
depicted in Figure \ref{fig:DBN}.
The first RBM has 1024 (32$\times$32) units in visible layer and 500 units in
hidden layer. This RBM is trained without any label (unsupervised learning)
to provide abstract features for the second one. After training the first RBM,
the second RBM is trained using the extracted features generated in previous step.
In the recent RBM as a classifier, we use the joint of 500 extracted values and
80 softmax units. The labels of ORL images have been converted to softmax vectors.
For the first class of images the two first bits are one and others are zero.
For the next one, the two first bits are zero, the next
two bits are one, and the others are zero and so on.

We divided the ORL images into two subsets.
The first one is the training set and consisted of 8
images from 10 of each class. These images are used
to train the model and are not used for testing.
The second one is the test images containing 2 of 10 from each class.
\begin{figure}
\centering
\includegraphics[width=0.67\textwidth] {./figures/ch7/DBN.pdf}
\caption{The proposed DBN with Siegert neurons for learning ORL}
\label{fig:DBN}
\end{figure}
Because of the full connections between each visible unit and each hidden unit,
the dimensions of the arriving connections at each hidden unit are the same as
the dimension of input images. Figure \ref{fig:RBM_hidden_visualzing} shows the
corresponding weights of connections between all visible units and 100 randomly
selected hidden units. The weight vectors has been reshaped as 32$\times$32 images.
During the learning process the hidden units have learned some specific features
such that they can be triggered only with those specific features. Visualizing these
weight vectors (Figure \ref{fig:RBM_hidden_visualzing}), displays the learned
features by each hidden unit. Therefore, it is a appropriate monitoring method to study
the network learning process \cite{rtguide_hinton2012}.
\begin{figure}
\centering
\includegraphics[width=0.44\textwidth] {./figures/ch7/Learned_Features_10_by_10_imagesc_cmd-eps-converted-to}
\caption{Visualizing the learned features by hidden units }
\label{fig:RBM_hidden_visualzing}
\end{figure}
As we can see in Figure \ref{fig:res_dbn_sieg1}, the training process needs too
many iterations to reach a proper result. For this model which is implemented in Matlab,
after about 800 iterations, the results are close to 90\%. The maximum value, 93.2\%,
corresponds to iteration $1910^{th}$.
\begin{figure}
\centering
\includegraphics[width=0.46\textwidth] {./figures/ch7/res_dbn_sieg1-eps-converted-to}
\caption{Accuracy of the proposed DBN with Siegert neurons in face recognition on ORL dataset}
\label{fig:res_dbn_sieg1}
\end{figure}
In this study, we use the free energy function (equation \ref{eq:Free_En})
to find the predicted label. Using this function, each possible label
has been tested to find the configuration with the lowest energy. The corresponding
label to the recent configuration is assumed as the predicted label \cite{rtguide_hinton2012}.
 \begin{equation}
F(V)=-\sum_i v_ia_i - \sum_j {\log ({1+e^{x_j}}) }
\label{eq:Free_En}
\end{equation}
where $x_j=b_j+\sum_i v_i w_{ij}$.

The accuracy of the model depends on the learning  parameters \cite{bengio2012practical,rtguide_hinton2012}.
For example Figure \ref{fig:res_dbn_sieg1} shows the results when the mini-batch size is 4. The effect of
changing the parameters of model using less iterations and using various mini-batch sizes is depicted in
Figure \ref{fig:res_dbn_seig_1000_mini_batch} . Obviously, less iteration and larger mini-batch size
lead to less accuracy. Eventually, if one is interested in more precise accuracy, it can be possible
through adjusting the learning parameters and also the Siegert neuron parameters (Table \ref{tab:LIF_params}).
\begin{figure}
\centering
\includegraphics[width=0.44\textwidth] {./figures/ch7/mini-batch-effect_with_epoch_1000-eps-converted-to}
\caption{Decreasing the number of epochs and increasing the mini-batch size can reduce the model accuracy }
\label{fig:res_dbn_seig_1000_mini_batch}
\end{figure}

\section{Evaluating the model}
\label{Evaluation_DBN_LIF}
Thanks to the equality between the Siegert neuron's transfer function and the LIF
transfer function, without any adjustments, the trained weight matrix in the
previous section can be copied to a network with the same topology consisted of
LIF neurons with the same parameters (Tabel \ref{tab:LIF_params}).
To develop such a network with LIF neurons, we use the Brian simulator.
Brian is a simulator for Spiking Neural Networks. This simulator is
written in the Python programming language. Because of using development
tools such as  SciPy module, Python provides very fast routines for
mathematical operations and specifically matrix operations \cite{goodman_brian:_2008}.
We have developed a Deep Belief Network in Brian simulator with same topology as the
one that has been implemented in Matlab. In this model since we want to test the
accuracy of the model in a more realistic situation, the equation of LIF neuron
(Equation \ref{eq:LIF}) is applied besides the described parameters in Table \ref{tab:LIF_params}.
\begin{equation}
\tau_m \frac{dv}{dt}=-(v(t)-v_{rest})+RI(t)
\label{eq:LIF}
\end{equation}
We know the spiking model uses spike trains instead of the real numbers.
In Section \ref{DBN__Siegert}, we discussed the
assumption to use the normalized value of the pixels to produce the related
firing rate. Consequently in this section to test the proposed spiking model,
we have to convert the firing rate to the spike trains. In \cite{fatahi_evt_mnist:_2016},
we have presented more in details the converting approach for MNIST handwritten digits.
In Brian simulator, using \textit{PoissonGroup} function, we can generate spike
trains with specified firing rates. Therefore, the network can be tested with
spike trains corresponding to the density of pixels of the test images. Having
the trained weights matrix, despite of the Matlab model, the labels are not used
in the second RBM for training however, they are used as the outputs of the model for
classifying the input images \cite{Disc_RBM__larochelle2008classification}.
Respecting to the generative characteristic of DBNs, the model not only can
reconstruct the input images as the internal representations of the given images
(Figure \ref{fig:recons_image_trian_brian}), but also can generate the
corresponding learned labels as the predictions of the model.
\begin{figure}
\centering
\includegraphics[width=0.44\textwidth] {./figures/ch7/detected_faces_3for_trained-eps-converted-to}
\caption{The upper row shows 10 of the training images and the lower one
illustrate the corresponding reconstructed images }
\label{fig:recons_image_trian_brian}
\end{figure}
To evaluate the model in a spiking framework after transferring the weights
matrix, the generated spike trains of test images are passed through the entire
of the network. Furthermore, comparing the predicted labels with the original labels, we
compute the model accuracy. Figure \ref{fig:recons_image_test_brian} shows
predicted images respecting to the input test images. In addition, as it was
predictable, the results of the model with LIF neurons have not changed
considerably. However because of the difference between floating number
precision in Matlab and Brian simulator reloading the weights matrix in
Python can cause a small decreasing in accuracy. The accuracy of the
model in Brian simulator is reduced to 92.4$\%$.
\begin{figure}
\centering
\includegraphics[width=0.44\textwidth] {./figures/ch7/detected_faces_1for_test-eps-converted-to}
\caption{The upper row shows 10 of the test images and the lower one illustrate the predicted images }
\label{fig:recons_image_test_brian}
\end{figure}
\section{Conclusion and future works}
\label {conc-deep}
The ability of brain-like computing has motivated us to evaluate a model
with biological structure in face recognition application. Regarding the
brain as a deep neural network, we have proposed a deep neural network with
spiking neurons to understand if the brain-like models are suitable for
face recognition applications.
The proposed model is different from a traditional neural network.
Indeed this model is a Spike-Based deep model with biological inspired neurons
(Leaky Integrate-and-Fire neurons). Considering the results of Deep Belief
Networks in various tasks in Machine learning domain, a DBN with two RBMs has
been developed in Matlab with Siegert units. Consequently, the trained weights matrix
is transferred to a Spiking DBN with LIF neurons. The recent model is simulated
in the Brian simulator and the results show the capability of the Spiking Deep
Belief Networks in a simple face recognition application.

For future works, we will
take into account using other types of deep models such as Deep
Autoencoders or Convolutional Neural Networks.
In this work, we had to train the model in one
platform and using the trained weights in another one. Indeed
our model is an offline training model. Using the same platform
for training and utilizing the model leads us to an online model
of training, which can be considered as the next work.
Additionally, we are looking for
a way to perform the same work in a single platform more suitable for hardware
implementation by using the emerging nanodevices such as memristor to realize the synapse
model.
