
\chapter{Evaluation methodology and parameter exploration to
  improve performance of memristor-based spiking neural network architecture}
\label{evaluation}
\begin{abstract}
The brain-inspired spiking neural network neuromorphic architecture
offers a promising solution for a wide set of cognitive computation
tasks at a very low power consumption. Due to the practical feasibility
of hardware implementation, we present a memristor-based model of
hardware spiking neural networks which we simulate with N2S3 (Neural
Network Scalable Spiking Simulator), our open source neuromorphic
architecture simulator. Although Spiking Neural Networks (SNN) are widely
used in the community of computational neuroscience and neuromorphic
computation, there is still a need for research on the methods to choose
the optimum parameters for better recognition efficiency. With the help of
our simulator, we analyze and evaluate the impact of different parameters
such as number of neurons, STDP window, neuron threshold, distribution
of input spikes and memristor model parameters on the MNIST hand-
written digit recognition problem. We show that a careful choice of a
few parameters can significantly improve the recognition rate on this
benchmark.
\end{abstract}

\section{Introduction}
Neuroinspired computing has the potential to carry very low
power computation to future computer architectures and embedded
systems~\cite{merolla_million_2014}. Indeed parallel neuromorphic computation, by
performing computation and storage in the same devices can overcome the Von-Neumann
bottleneck. There are two broad types of brain-inspired cognitive computations:
abstract artificial neural networks (ANNs) and closer to biology spiking neural
networks (SNNs)~\cite{eryilmaz_neuromorphic_2016}. Machine learning algorithms
such as classical ANNs and more recently deep belief
networks~\cite{hinton_fast_2006,lecun_deep_2015} are widely used for data
classification and clustering. However, ANNs are a highly abstract mathematical
model of neurons that are designed to be executed on digital traditional von
Neumann processing platforms (more and more using accelerating units such as
GPUs). ANNs only use rate coding to represent neuronal activity and are not
capable of taking into account the precise relative timing of spikes that are
significant when dealing with natural signals that are more and more significant
in the internet of things.
% Therefore, thinking to hardware
% implementation of intelligent and low-power computing platform, leads to
% necessity of using brain-inspired Spiking Neural Network (SNN).
The data should be coded to spikes to be processed in SNN, which is known as
spike coding (versus rate coding in ANN). SNNs offer online real time unsupervised learning
through continuous weight updating which is performed on local synaptic weights.
This temporal and spatial locality is important in hardware implementations of neural
networks because it frees this architecture of the memory bottleneck of Van Neumann architectures.
% Moreover, the memory bandwidth required by high-performance computing
% paradigm has already grown beyond the conventional memory architectures
% efficiently known as \textit{memory wall} of \textit{memory bottleneck}.
% To overcome these challenges, a brain-like computing platform such as SNNs that
% locates computing units next to synaptic memories is offered as an alternative
% computing paradigm \cite{paugam-moisy_computing_2012,vanarse_review_2016}.
% Therefore, the innovation in both device technology and computing architecture
% is required to step further to design a new efficient cognitive and low-power
% architecture.
Recent advances in nanotechnology have provided neuromorphic computing
architecture with novel memristive devices which have the capability of
mimicking synaptic plasticity, such as resistive switching memory
(RRAM)~\cite{HP_2008-switch,RRAM,prezioso_training_2015}, phase change memory
(PCM)~\cite{driscoll_phase-transition_2009,eryilmaz_brain-like_2014,ambrogio_unsupervised_2016},
Conductive Bridge memory
(CBRAM)~\cite{liu_controllable_2010,bichler_design_2013,suri_bio-inspired_2013},
and ferroelectric memory (FeRAM)~\cite{ferro,nishitani_artificial_2014}. The
advantages of using these memristive nanodevices to model the behavior of
synapses are their unique properties, such as scalability, flexibility because
of their analog behavior, manufacturability on top of CMOS technology to make a
crossbar array (shown in Figure~\ref{crossbar}) and ability to remember the last
state in a SNN~\cite{jo_nanoscale_2010}. Fortunately due to close collaboration
with the nano-electronics research center in the University of Lille (IEMN), we
have the opportunity to study the appropriateness of various classes of
memristors (e.g.,TiO$_{2}$, NOMFET, Magnetoelectric) to build a SNN hardware
platform by using real parameters. The non-volatile resistive switch could be
any solid-state RRAM. We have chosen here the resistive memory presented
in~\cite{querlioz_immunity_2013} as non-volatile memory because it is
representative of such devices.
% remember the last state in Spiking Neural Network (SNN)~\cite{jo_nanoscale_2010}.
% Fortunately due to close collaboration with the nano-electronics research center in the
% University of Lille (IEMN), we have the opportunity to study the approperiability
% of various classes of memristors (e.g.,TiO$_{2}$, NOMFET, Magnetoelectric) to
% build a spiking neural network hardware platform by using real practically used parameters.
%  Moreover, We use real parameters of fabricated devices to
% ensure that our simulations are meaningful with respect to the technology.
% Due to
% the demonstrated potential of NOMFET (Nanoparticle Organic Memory Field-Effect
% Transistor)~\cite{alibart_organic_2010, alibart_memristive_2012} to play the role of a synapse, we use it as a
% volatile synapse in neuromorphic accelerator.
% The non-volatile resistive switch could be
% any solid-state RRAM. We have chosen here the resistive memory presented
% in~\cite{querlioz_immunity_2013} as non-volatile memory.

\begin{figure}[t]
\centering
  \includegraphics[scale=1.5]{./figures/ch5/netdif}
 \caption{Neuromorphic vs SNN, \textbf{a)} The memristive synapse connects the spiking neurons in
 configurable crossbar array suitable
 for stdp unsupervised learning, the presynaptic neurons are considered as inputs and postsynaptic
 neurons play output rolls. \textbf{b)} The spiking neural network two layers of this three layers
 could similarly operates as crossbar array.
 }\label{crossbar}
\end{figure}
It is widely believed that plasticity is the key of learning in the biological
brain~\cite{Gerstner_Why_1993}. Consequently, with the latest proposals to use
the memristive nano-devices as synapses,  we implement an efficient and
well-studied unsupervised learning rule known as spike timing dependent
plasticity (STDP)~\cite{Markram_regulation_1997,babadi_stability_2016}.
In this study, we show that the memristive synapse is adapted to unsupervised
STDP learning in SNNs. We explain the required technology and architecture with
system-level simulation. For implementation and results, we use the MNIST
dataset of handwritten digits~\cite{MNIST} to test the performance of neural
networks. Although, there are various research works using STDP learning in SNN
architecture and neuromorphic VLSI implementations, none of them evaluates and
analyzes the key parameters that improve learning and SNN recognition
performance in those architectures. Our main contribution is to evaluate and
explore the impact of several parameters on the learning performance of SNNs for
the MNIST benchmark: the number of neurons in the output layer, the duration of
the STDP window, various thresholds for adaptive threshold LIF neurons,
different distributions of spikes to code the input images and the memristive
synapse fitting parameter.

%
% To be able to perform energy efficient computing, we use an event-driven method for
% computing, processing only occurs at reception of a spike.
%  Our key architectural
% abstraction is a network of spiking neurons that process spikes in an efficient,
% scalable, and flexible way.
% It is widely believed that plasticity is the key of learning in biological
% brain~\cite{Gerstner_Why_1993}. Consequently, with the latest proposals to use the memristive nano-devices as
% synapses,  we implement an efficient and well-studied unsupervised learning rule known as Spike Timing
% Dependent Plasticity (STDP)~\cite{Markram_regulation_1997,babadi_stability_2016}.
% Considering the train of spikes for the sequence of pre and postsynaptic spikes, there are various
% types of STDP \cite{babadi_stability_2016} methods.
% The simple and fast method which is called \textit{pair-based} is used for the STDP learning beside
% applying lateral inhibition to converge to the best feature in pattern learning.
% The model of spiking neuron is used for the first simulation results is
% \textit{Leaky-Integrate-and-Fire (LIF)}~\cite{LIF1999}. More significantly, the fully-connected
% network architecture is used which is called Restricted Boltzmann Machine
% (RBM) network topology in neuroscience or crossbar array architecture in neuromorphic
% architecture community as it is shown in Figure \ref{crossbar}.
%
% A lot of efforts have been
% put into developing appropriate simulation tools and
% techniques \cite{goodman_brian:_2008,bichler_design_2013}. In this paper,
% for implementation we use our simulator; N2S3 (Neural Network Scalable Spiking Simulator)
% an event-driven simulator, that is developed dedicated to
% the architecture exploration of hardware SNNs. The N2S3 has extensibility
% and not only flexible to analyze verify the various classes
% of neuron and synapse (memristors) models,
% different network topologies and learning algorithms but also also it is scalable to distribute the
% simulation on several computers. Indeed, because of the lack of a proper
% platform for hardware architecture exploration in neuromorphic era,
% we target implementation and evaluation of spiking neural network and neuromorphic architecture
% as the main purpose of designing N2S3.
%
% In this study, we show the memristive synapse is adapted to unsupervised STDP learning
% in spiking neural network
% paradigm. we explain the required technology and architecture with running system-level simulation.
% For implementation and results, the MNIST training dataset of handwritten digits \cite{MNIST} is used to
% train and test the performance of neural networks.
% Although, there are various research works using STDP learning in SNN architecture and neuromorphic
% VLSI implementations, non of them
% evaluate and analyze the key parameters to improve learning and SNN recognition performance
% in those architectures.
% However in the main contribution, we evaluate and explore the impact of neuron number in output layer,
% the optimum value
% for stdp windows, various thresholds for adaptive threshold LIF neurons, different distributions and
% their effects on learning as well as the synapse parameter.

\section{Experimental evaluation of the influence of four parameters on the
classification of handwritten digits}
\begin{figure}[t]
 \centering
 \includegraphics[scale=.7]{./figures/ch5/MNIST4.pdf}
 % dist.pdf: 539x547 pixel, 72dpi, 19.01x19.30 cm, bb=0 0 539 547
 \caption{Sample heat maps of the synaptic weights after network training with
 four different numbers of output neurons (20, 30, 50 and 100).
}

\label{MNISTOUTS}
\end{figure}
In this section, we evaluate and explore the effects of four parameters on the
performance of the network architecture namely the distribution of input spike
trains, the STDP window duration, the neuron thershold, and the synapse $\beta$
parameter. These parameters will be fully described in the following. The full
MNIST dataset is presented twice to the networks with four different
numbers of output neurons.  A sample of output results is shown in
Figure~\ref{MNISTOUTS} for 20, 30, 50 and 100 output neurons.

% Generally there are two possibilities for processing the data in a computation platform.
% The first approach is synchronous or clock-based computation which the conventional computers
% using this method for receiving, transferring as well as processing data. The second
% approach is asynchronous or \textit{event-based (event-driven)} computation that all data
% receiving/transferring and processing
% will be performed only by arrival an event or spike.
% This is inspired from the fact that in the
% brain, the neurons and connections are not elements that
% limit the processing time, since all units operate in parallel,
% and only the arrival of spike events can trigger the data processing, consequently
% the neural circuits can adapt the processing speed to the rate of input events.
% Event-driven not only is suitable for LIF units which in the states of LIF neurons are only
% updated upon the arrival of input spikes, but also the system performs energy efficient computation.
%
% To train the network and test the network performance, we used the MNIST
% (Mixed National Institute of Standards and Technology database)
% training dataset of handwritten digits \cite{MNIST}. MNIST dataset has been widely used
% as a benchmark to test and verify different classification algorithms. The
% training set consists of 60000 digits between 0 and 9 and each handwritten
% number is a 28 $\times$ 28 pixel image. In this simulation, we present the full
% dataset (60000 images) and full images. Each pixel is connected to one input
% buffer neuron.
%
% In this section, we evaluate and explore the effects of four parameters on the performance
% of the network architecture namely
% input spike distribution trains, STDP window, neuron thershold, and synapse variability.
% The full set of MNIST dataset is presented to the network with four different number of
% output neurons that one sample of each output results for 20, 30, 50 and 100
% number of output neurons is shown in Figure \ref{MNISTOUTS}.

Simulation has been done using our event-based Neural Network Scalable Spiking Simulator (N2S3).
N2S3 has been developed to implement, evaluate and simulate spiking neuromorphic hardwares.
We vary each parameter independently to asses its influence on the recognition
rate. The default value for these parameters are taken from the values listed in
the literature. We call this set of parameters the baseline.
\begin{itemize}
 \item Input spike train distribution: Poisson.
 \item STDP window duration: 25~ms.
 \item Neuron threshold: 15~mV.
 \item Synapse $\beta$ parameter: 1.5.
\end{itemize}

At the end of this section, in section~\ref{sec:discussion}, we compare the
baseline with the best parameters obtained separately and discuss the results.

In all the simulations, as the weight initialization is random, each simulation
has been repeated 10 times and the results are shown in box plots with the
default parameters of the box plot function of the R statistical tool.

% The box plot is efficient way to
% present the plot for a group or set of data. It displays the variety of results (here for ten times runs)
% the minimum and maximum first and third quartiles and median of data set. The simplest box plot
% contains the central rectangle between the first quartile to the third quartile (the interquartile range or IQR).
% A segment inside the rectangle displays the median or typical value.
% The whiskers above and below the box display the locations of the minimum and maximum values.
% In the some plots there are unfilled circles that represent
% either 1.5$\times$IQR or more above the third quartile or 1.5$\times$IQR or more below the first quartile.
% The output result of the
% simulator has been analyzed using R statistical computing tools.
\begin{figure}[t]
 \centering
 \includegraphics[scale=.7]{./figures/ch5/dist.pdf}
 % dist.pdf: 539x547 pixel, 72dpi, 19.01x19.30 cm, bb=0 0 539 547
 \caption{The comparison of three different distributions for generating spike
 train by transferring MNIST database pixels to spike train. These
 distributions are tested with different number
 of neurons=20, 30, 50, 100.
}
 \label{dist}
\end{figure}

\subsection{Effect of spike distribution}

As the platform is working using spikes for processing the data,
we need to generate spikes train
of the images of MNIST dataset to extract a spike-based dataset
from a frame-based dataset respecting to intensity of each pixel
of images. The pixel intensity is between 0 to 255 and should be transfer to the spike train
to be readable by SNN. For instance, Diehl and Cook \cite{diehl_unsupervised_2015} use a method which the
maximum pixel intensity of 255 is divided by 4, resulting in input
firing rates between 0 and 63.75 Hz. Yet there is a question that with which
interval time between two spikes we generate the spikes? Therefore, statistical distribution rules
can help to generate appropriate spike train when encode the pixels.

\subsubsection{Poisson input spike train distribution}
Here we explain a particular class of random process called a Poisson spike train process.
Define $\rho (t)$ response function to the input stimuli:
\begin{equation}
 \rho (t)=\sum_{i=1}^{k}\delta (t-t_i)
\end{equation}
where $k$ is the total number of spikes in the spike train, and $t_i$ defines the time each spikes occurs.
The unit impulse signal is defined as:
\begin{equation}
 \delta (t)=\left\{\begin{matrix}
 1& \text{if} & t=0\\
 0& \text{otherwise}
\end{matrix}\right.
\end{equation}
The instantaneous firing rate (e.g., of a sensory neuron) can now be formally defined to be the
expectation of the sensory input response function which is $r(t)=\left \langle \rho (t) \right \rangle$.
The average spike count between times $t_1$ and $t_2$  can then be defined from
the instantaneous firing rate:
\begin{equation}
 \left \langle n \right \rangle=\int_{t_1}^{t_2} r(t)dt,
 \label {exp1}
\end{equation}
the probability of a
spike occurring during a given brief time interval is equal to the value of the instantaneous firing
rate during that time interval times the length of the interval:
\begin{equation}
P_{(\text{one spike in } (t-\Delta t,t + \Delta t))} = r(t)\Delta t.
\label {exp2}
\end{equation}
We assume the instantaneous firing rate $r$ is constant over time.
This is called a homogeneous Poisson process.
From Equation \ref{exp1} it is derived $\left \langle n \right \rangle=r\Delta t$ for any interval
$\Delta t=t_2-t_1$. Equation \ref{exp2} can be used to generate a Poisson spike train by first
subdividing time into a bunch of
short intervals, each of duration $\Delta t$.
Then generate a sequence of random numbers $x[i]$ uniformly
distributed between 0 and 1. For each interval, if $x[i]\leq r\Delta t$,
we generate a spike otherwise no spike. This procedure is appropriate only when $\Delta t$ is small
enough for e.g., millisecond range. Using Poisson distribution, we made an event-based version of
MNIST \cite{fatahi_evt_mnist:_2016} and it is available open-source online
(https://github.com/MazdakFatahi/evt-MNIST). We refer to intensity of pixels as the probability that a spike
occurs within an interval.

\subsubsection{Other distributions of the input spike trains}
% In our simulation, MNSIT pixel intensity between 0 to 255 and is transfered to 0 to 22 Hertz spiking
% frequency using different distributions during a 350 ms presentation window.
% Based on previous similar work \cite{diehl_unsupervised_2015}, we have chosen a
% delay of 150 ms between two images. Therefore, there is sufficient time for
% membrane potentials of all neurons to reset back to initial values. The network
% connection weights are between 0 and 1 initialized using a Gaussian
% distribution. We compare uniform, Gaussian and Poisson distribution using different
% output neurons in Figure \ref{dist}.
In our simulation, the pixel intensity between 0 and 255 is transferred to 0 to
22~Hz spiking frequency using different probability distributions during a
350~ms presentation window.  Based on previous similar work
\cite{diehl_unsupervised_2015}, we have chosen a delay of 150~ms between two
images. Therefore, there is enough time for membrane potentials of all neurons
to reset back to their initial value. The network connection weights are between
0 and 1 initialized using a Gaussian distribution. We compare uniform, Gaussian
and Poisson distribution using different numbers of output neurons in
Figure~\ref{dist}. This figure shows that the choice of the input distribution
does not have a significant impact on the recognition rate of the network. That
means that the classification of the inputs is done mainly on the frequency of
the inputs regardless of the precise timing of the spikes.


% The simulation runs with different number of neurons. The MNIST data based
% presented two times to the networks and for more validation of data in each
% distribution case with specific number of neurons, the simulation is run
% 10 times. In addition, we present the 10 iterations of running using
% box plot and the data has been analyzed by R statistical tools.
% Although using different distributions does not have remarkable difference for
% recognition rate of digits however the Poisson distribution demonstrate
% more stable performance to produce the spike trains.

\begin{figure}
\centering
\begin{tikzpicture}[scale=0.8]
  \begin{axis}
    [
    axis x line=bottom,
    axis y line = left,
    xlabel={Number of output neuron},
    ylabel={Recognition rate (\%)},
    xmin = 0,
    %xmax = 16.0,
    ymin = 65,
    %ymax = 120.0,
   legend entries={Poisson,Gaussian, Uniform},
   % legend entries={NS (3 iterations), VNVS (3 iterations)},
   legend style={at={(1,0.5)}},x
    ]
    \addplot table[y index=1,x index=0]{Rrate.txt};
    \addplot table[y index=2,x index=0]{Rrate.txt};
        \addplot table[y index=3,x index=0]{Rrate.txt};
           % \addplot table[y index=4,x index=0, color=red]{Rrate.txt};
    %\addplot table[y index=1,x index=0, red]{Rrate.txt};
\end{axis}
\end{tikzpicture}
\caption{The recognition rate of the network using different number of neurons and three spike train
distributions.}
\label{dist-num}
\end{figure}

Figure \ref{dist-num} depicts recognition rate for different number of neurons
using three various spike train distribution. This figure demonstrate that
increasing the number of neuron increase the recognition rate. We expected such a
behavior because increasing the number of neurons will increase the number of synapse in
the network therefore the chance of learning is increased due to more precise coding.
It is worth to note that in general the relation
between the number of neuron and learning performance is not always increasing ratio while we increase the
number of neurons. The reason is that the random selection
of number of neurons might cause either overfitting or underfitting problems.
% As the number of
% input neurons is the same as the number of pixels in one digit in MNIST (28 $\times$ 28), choosing the small
% number of neurons in output layer may lead to underfitting as well as choosing the numbers more than
% 784 may lead to overfitting. Additionally we must consider the role of the synapses.
% The input patterns are decode to the synaptic weights using STDP between pre- and post synaptic neurons.
% In case of choosing small number of synapses, the network can not decode it properly in output layer.
% This improper coding can decrease the learning ability of network if the number of synapses are too high.
\subsection{Effect of STDP window duration}
% Activity-dependent modification of synaptic weights because of spike-timing-dependent plasticity (STDP)
% is depended on the correlations between pre- and postsynaptic firing over timescales
% of tens of milliseconds \cite{drew_extending_2006}.
% The synaptic weights that can be adjusted by STDP in a way that depends on the temporal ordering
% of events separated by several seconds however the underlying plasticity is introduced to have a much smaller
% temporal window. Indeed, in simulation
% we can manage the time scale with more degree of freedom. Although in neuroscience
% the synaptic conductance is depended on the activities of the two connected neurons to
% the synapse as pre- and postsynaptic neurons however the time-window that we trace the two neuron
% activities is important. We define a STDP-window in our simulation with millisecond
% scale and test the simulation with different number of stdp-windows to analyze the influences on the
% network performance as well as to converge to the best choices.
Activity-dependent modification of synaptic weights because of STDP depends on
the correlations between pre- and postsynaptic firing over timescales of tens of
milliseconds~\cite{drew_extending_2006}. Contrarily to a biological simulation,
as we simulate hardware SNNs, we can choose some parameters. The way STDP is
implemented in hardware is a very important parameter. We use a simplified form
of STDP (from~\cite{querlioz_simulation_2011}) where the weights of the synapses
are always slightly decreased except when there is a temporal correlation
between a presynaptic firing and a postsynaptic firing during a time window of
duration STDP window. We look here at the influence of this STDP window duration
on the learnig capabilities of our networks.

\begin{figure}[t]
 \centering
 \includegraphics[scale=.8]{./figures/ch5/stdp-window.pdf}

 \caption{The comparison of different STDP-window duration for
using different number
 of neurons=20, 30, 50, 100. The MNIST digits dataset after converting to
 the corresponding spikes to the pixels densities, are presenting to the
 network for 350 ms for (each frame). The 150 ms pause between each digit presenting
 are considered. This figure illustrates the performance of neural network
 using four various number of neurons and different STDP-windows.
}
 \label{stdp-num}
\end{figure}

% We read each digit of MNIST inputs in 350 ms, after which the input neurons present
% spikes corresponding to another digit. In addition, based on the pixel density,
% we can compute maximum and minimum number of spikes for each pixel.
% However, depending on the pixel intensity between black and white for each pixel,
% there are different number of input spikes.
% In addition to tracing these calculations for presynaptic activities in input layer,
% for adjusting the weight in STDP, we have to trace the postsynaptic neuron activities too.
% Consequently, a time-window should be defined to modify the synaptic weight based
% on any train sequence of pre- and postsynaptic spikes.
% To be able to evaluate the optimum STDP window, We started using 15 ms time-window and testing various time-window as it
% is depicted in Figure \ref{stdp-num}.
% Here again to be more confident about the results validation,
% the simulator is run 10 times with presenting MNIST data set
% two times during each running.
%
% The results shows a low performance using 15 ms regarding to other time-windows. We have a remarkable improvement
% from 15 ms to 25 ms of STDT-window time. However the optimum results are obtained using the range between 35 and 55 ms.
% Although choosing STDP-window=65 ms, the reducing starts for less number of neurons however for larger number
% of neurons still illustrates better recognition rate. The decreasing is begun for the time-window larger than
% 65 ms even for networks with more output neurons. Figure \ref{stdp-neuron} illustrates the
% recognition rate of  neural networks using various number of neurons.
The duration of the STDP window is related to the frequency of the input spike
trains. As the maximum frequency is 63.75~Hz, the STDP window duration should be
of approximately the same duration as the corresponding period, 15.7~ms, or
higher.  To be able to evaluate the optimum STDP window duration, we have
started using a 15~ms duration and increasing to 65~ms by increments of 10~ms as
it is depicted in Figure~\ref{stdp-num}.

The results show a low performance using 15~ms regarding to other STDP window
durations.  We have a remarkable improvement from 15~ms to 25~ms and the best
results are obtained using the range between 35~ms and 55~ms.  At 65 ms, the
recognition rate starts to decrease. Our interpretation is that a too short
duration does not allow to capture all the high intensity pixels in the input,
and a too long duration is not specific enough as it captures too many mid range
pixels and thus is not specific enough. One would need to do additional
experiments to check how the STDP window influences the learning speed of the
network. Indeed, here we just check the final network state after two
presentations of the 60000 images of the MNIST dataset, but the convergence
speed of the SNN may well depend also on the STDP window duration.
Figure \ref{stdp-neuron} illustrates the average recognition rate of  neural
networks using various number of neurons.

\begin{figure}
\centering
\begin{tikzpicture}[scale=0.8]
  \begin{axis}
    [
    axis x line=bottom,
    axis y line = left,
    xlabel={Number of output neuron},
    ylabel={Recognition rate (\%)},
    xmin = 0,
    %xmax = 16.0,
    ymin = 55,
    %ymax = 120.0,
   legend entries={STDP-Window=15 ms,STDP-Window=25 ms,STDP-Window=35 ms,STDP-Window=45 ms,STDP-Window=55 ms,STDP-Window=65 ms},
   % legend entries={NS (3 iterations), VNVS (3 iterations)},
   legend style={at={(1,0.5)}},x
    ]
    \addplot table[y index=1,x index=0]{Rratestdp.txt};
    \addplot table[y index=2,x index=0]{Rratestdp.txt};
        \addplot table[y index=3,x index=0]{Rratestdp.txt};
            \addplot table[y index=4,x index=0]{Rratestdp.txt};
            \addplot table[y index=5,x index=0]{Rratestdp.txt};
            \addplot table[y index=6,x index=0]{Rratestdp.txt};

\end{axis}
\end{tikzpicture}
\caption{The recognition rate of the network using different number of neuron and
six different STDP-wnidows.}
\label{stdp-neuron}
\end{figure}


\subsection{Effect of neuron threshold}
% In our hardware simulation, we use a well-studied model of the neuron namely Leaky-Integrate-and-Fire (LIF) neuron model.
% This type of neuron mode is fit for SNN architectures \cite{LIF1999} due to several properties such as
% avalablility of low-power CMOS
% design using subthreshold regime transistor \cite{indiveri_neuromorphic_2011}, fast to simulate, and particularly efficient for
% large-scale network simulations~\cite{brette_simulation_2007}.
% The LIF model is described in Chapter \ref{ch_neuromorphic} Equation \ref{eq1} and \ref{eq2}.
% If injected currents from synapses are big enough, cause action potential to pass the threshold
% voltage, and accordingly neuron fires. It means there are enough input spikes in a short time-window.
% When there is no or only a small number of spikes in a time-window, the neuron is in
% the leaky phase and the state variable decreases exponentially. The duration of
% this time window depends on $\tau_n=RC$.
%
% To simulate the LIF neuron, we have to define a threshold for the neuron.
% Furthermore, we used homeostasis technique to improve the network stability
% and performance. It means if one or group of neurons are too active during reading the input data,
% we slightly
% increase the threshold and vice versa if one or some neurons are inactive during the training
% process the threshold is decreased slightly. In this section, we verify and analyze the spiking
% neural network performance while using different thresholds for the same neurons
% to reach the best threshold value.
% To be able to trace only the impact of different thresholds on the system performance,
% we use the same
% homeostasis for all neurons with the same rate of increasing and reduction.
% The range of threshold values have been chosen to cover the minimum and maximum action
% potential in
% the network. Additionally, to validate our evaluation, we have repeated the training and testing of MNIST
% by ten times running the simulator for each threshold and similar to the previous
% experiences the MNIST digit is presented two times for each run.
% The Figure \ref{Nth-box} shows comparison of
% different thresholds using four various number of neurons. The results depict the
% neuron thresholds between 25 and 35 mV have the optimized recognition rate for the
% networks. However, for the larger number of neurons the choosing threshold=45 mV
% also represent the acceptable performance, in contrast, for the networks with the
% smaller number of neurons
% the lower amount of thresholds represent good recognition rate in the network. It
% must be logic as in the networks with more neurons, each neurons is connected to
% the larger number of other neurons and may reach sooner to the firing, therefore
% the neurons with bigger thresholds may cause a better performance in the
% network architecture.
In our hardware simulation, we use the
Leaky-Integrate-and-Fire (LIF) neuron model.  This type of neuron model is fit
for SNN architectures~\cite{LIF1999} due to several properties such as
avalablility of low-power CMOS design using subthreshold regime transistor
\cite{indiveri_neuromorphic_2011}, fast to simulate, and particularly efficient
for large-scale network simulations~\cite{brette_simulation_2007}.

The LIF model is described in Chapter~\ref{ch_neuromorphic}, Equations~\ref{eq1}
and~\ref{eq2}.  If injected currents from synapses are large enough, they cause
the action potential to pass the threshold voltage, and the neuron fires. It
means there are enough input spikes in a short time-window.  When there is no or
only a small number of spikes in a time-window, the neuron is in the leaky phase
and the state variable decreases exponentially. The duration of this time window
depends on $\tau_n=RC$.

To simulate the LIF neuron, we have to define a threshold for the neuron.
Furthermore, we used the homeostasis technique to improve the network stability
and performance. It means that if one or a group of neurons are too active when
reading the input data, we slightly increase the threshold and vice versa if one
or some neurons are inactive during the training process the threshold is
decreased slightly. In this section, we verify and analyze the spiking neural
network performance while using different thresholds for the same neurons to
reach the best threshold value.  To be able to trace only the impact of
different thresholds on the system performance, we use the same homeostasis for
all neurons with the same rate of increasing and reduction.  The range of
threshold values have been chosen to cover the minimum and maximum action
potential in the network. Figures~\ref{Nth-box} and~\ref{Th-neuron} shows the
effect of the different thresholds on the recognition rate. The results show
that the neuron thresholds between 25 and 35~mV lead to the best recognition
rates.  However, for the larger numbers of neurons choosing a threshold of 45~mV
also leads to an acceptable performance. In contrast, for the networks with the
smaller numbers of neurons the lower thresholds lead to a good recognition rate
in the network.  One could think that it is easily explained because in the
networks with more neurons, each neuron is connected to a larger number of other
neurons and thus receives more incoming spikes and thus reach its threshold
sooner. But here, the number of inputs of the neurons is constant and equal to
$28\times 28=784$, the number of pixels in the image. Actually, the
winner-takes-all rule increases the connectivity of the neurons inside the
layer, but for inhibitory purposes.  In general, the optimal value of the neuron
threshold depends on the network architecture, and more generally on the
activity of the network. In our case, the differences are not large enough to
conclude and we would need to run a much larger number of simulations to check
if this shift on the optimal value is real or just a random effect.
\begin{figure}
 \centering
 \includegraphics[scale=.6]{./figures/ch5/Nth.pdf}

 \caption{The comparison of various threshold (15, 25, 35, 45 mV) for
using different number
 of neurons=20, 30, 50, 100. The threshold between 25 and 35 mV demonstrate better
 performance in the network, however, in the network with smaller number of neurons
 the neuron with less threshold have still acceptable performance and on the contrary
 in larger neural networks the neuron with more firing threshold voltage (such as 45 mV
 using for the 100 output neurons) have demonstrated acceptable performance too.
}
 \label{Nth-box}
\end{figure}
In Figure \ref{Th-neuron}, we compared all four threshold values for having a better comparison.
The mean of the ten times running values is considered for this figure. As it is obvious from Figure \ref{Nth-box}
small threshold value has better performance with small number of neurons and larger threshold value has better performance with
only the larger number of neurons. However if we want to choose one threshold for neurons in network, definitely the average values
(35 and 25 mV) demonstrate better performance as it is more obvious in Figure \ref{Th-neuron}.
\begin{figure}
\centering
\begin{tikzpicture}[scale=0.8]
  \begin{axis}
    [
    axis x line=bottom,
    axis y line = left,
    xlabel={Number of output neuron},
    ylabel={Recognition rate (\%)},
    xmin = 0,
    %xmax = 16.0,
    ymin = 67,
    %ymax = 120.0,
   legend entries={ Threshold=15 mV,Threshold=25 mV, Threshold=35 mV, Threshold=45 mV},
   % legend entries={NS (3 iterations), VNVS (3 iterations)},
   legend style={at={(1,0.35)}},x
    ]
    \addplot table[y index=1,x index=0]{Rrateth.txt};
    \addplot table[y index=2,x index=0]{Rrateth.txt};
        \addplot table[y index=3,x index=0]{Rrateth.txt};
            \addplot table[y index=4,x index=0]{Rrateth.txt};


\end{axis}
\end{tikzpicture}
\caption{comparing network performance using various number of neuron with different threshold}
\label{Th-neuron}
\end{figure}

\subsection{Effect of synapse ${\beta}$ parameter}
% The quality synapse and the rate conductance changing of synapse causes
% different learning capabilities in the neural architecture.
% For instance using binary synapses which switches from zero to one
% will cause very less performance in learning rather than using an analog synapse.
% Therefore the range of changing after updating the weights are important in
% learning. The conductance changing not only is depended on the STDP rule for
% modification but also it is depend on the characteristics of the synapse itself.
% Here, we evaluate the network performance by changing parameters in the synapse
% to increase and decrease the synaptic weight changing rate.
%
% For the model of memristor as an artificial synapse, we used the similar model
% introduced in~\cite{querlioz_immunity_2013, querlioz_learning_2011} which is inspired
% from experimental memristive devices measurements \cite{Wei-syn-mem}. The
% increasing and decreasing of conductance is presented by Equation \ref{meminc} and
% Equation \ref{memdec} respectively.
% \begin{equation}
%  \Delta g_{inc}=\alpha_{inc} e^{{-\beta \frac{g-g_{min}}{g_{max}-g_{min}}}}
%  \label{meminc}
% \end{equation}
%
% \begin{equation}
%  \Delta g_{dec}=\alpha_{dec} e^{{-\beta \frac{g_{max}-g}{g_{max}-g_{min}}}}
%  \label{memdec}
% \end{equation}
% $g_{max}$ and $g_{min}$ are the maximum and minimum of
% conductances of the memristor. $\alpha_{inc}$ and $\alpha_{dec}$ characterize the
% conductance step and $\beta$ is a fitting parameter.
%
% To evaluate the impact of synaptic conductance on learning in the neural network,
% because different conductance steps
% are evaluated in \cite{querlioz_immunity_2013}, in this study,
% the various fitting parameters are traced. The other parameters in the simulation
% are fixed to $g_{max}=1$, $g_{min}=0.0001$, $\alpha_{inc}=0.01$ and $\alpha_{dec}=0.005$.
% The initial conductance of nanodevice before starting train the network
% is chosen randomly around the mid-range.
% We observed not remarkable effective impact on the network performance, which is a prove of
% device immunity and robustness to variations as
% it is reported in \cite{querlioz_simulation_2011} too.
% The results for four different number of neurons in the output are presented in
% Figure \ref{beta-box}.
The quality of the synapse and the rate of conductance change of synapses causes
influence on the learning capabilitiesof SNNs.  For instance using
binary synapses which switch from zero to one will cause much lower performance
in learning than using an analog synapse.  Therefore the amount of change after updating the weights is important. The conductance
change does not only depend on the STDP rule for modification but also on the characteristics of the synapse itself.  Here, we evaluate the
network learning performance when changing parameters in the synapse to modulate the synaptic weight change rate.

For the model of memristor as an artificial synapse, we use the model
introduced in~\cite{querlioz_immunity_2013, querlioz_learning_2011} which is
inspired from experimental memristive devices measurements~\cite{Wei-syn-mem}.
The increasing and decreasing of conductance is presented by
Equation~\ref{meminc} and Equation~\ref{memdec} respectively.


\begin{equation}
 \Delta g_{inc}=\alpha_{inc} e^{{-\beta \frac{g-g_{min}}{g_{max}-g_{min}}}}
 \label{meminc}
\end{equation}

\begin{equation}
 \Delta g_{dec}=\alpha_{dec} e^{{-\beta \frac{g_{max}-g}{g_{max}-g_{min}}}}
 \label{memdec}
\end{equation}

$g_{max}$ and $g_{min}$ are the maximum and minimum of conductances of the
memristor. $\alpha_{inc}$ and $\alpha_{dec}$ characterize the conductance step
and $\beta$ is a fitting parameter.  To evaluate the impact of synaptic
conductance on learning in the neural network, we vary the $\beta$ fitting
parameter because it directly affects the amplitude of the weight modifications.
The other parameters in the simulation are fixed to $g_{max}=1$,
$g_{min}=0.0001$, $\alpha_{inc}=0.01$ and $\alpha_{dec}=0.005$.  The initial
conductance of the nanodevices before starting to train the network are chosen
randomly around the mid-range.  We observed no remarkable effective impact on
the network performance, which is a proof of neural network robustness to
variations as is reported in \cite{querlioz_simulation_2011}.  The results for
four different number of neurons in the output are presented in
Figure~\ref{beta-box} and~\ref{beta-neuron}.
\begin{figure}
 \centering
 \includegraphics[scale=.6]{./figures/ch5/betaa.pdf}

 \caption{The comparison of various fitting parameters ($\beta$) for
using different number
 of neurons=20, 30, 50, 100.
The results demonstrate better performance using $\beta$ between 1.8 and 2.
However, the differences are not distinguishable which is a prove of
memristor devices robustness to the variations.
}
 \label{beta-box}
\end{figure}
Although the network performance variation using different $\beta$ parameters is
not remarkable, the results demonstrate slightly better performance using $\beta$
between 1.8 and 2.

\begin{figure}
\centering
\begin{tikzpicture}[scale=0.8]
  \begin{axis}
    [
    axis x line=bottom,
    axis y line = left,
    xlabel={Different beta for artificial synapse},
    ylabel={Recognition rate (\%)},
    xmin = 0,
    %xmax = 16.0,
    ymin = 65,
    %ymax = 120.0,
   legend entries={$\beta_1$ =1.5, $\beta_2$=1.8, $\beta_3$=2, $\beta_4$=2.5},
   % legend entries={NS (3 iterations), VNVS (3 iterations)},
   legend style={at={(1,0.35)}},x
    ]
    \addplot table[y index=1,x index=0]{Rratebeta.txt};
    \addplot table[y index=2,x index=0]{Rratebeta.txt};
        \addplot table[y index=3,x index=0]{Rratebeta.txt};
            \addplot table[y index=4,x index=0]{Rratebeta.txt};


\end{axis}
\end{tikzpicture}
\caption{comparing network performance using various number of neurons with different
fitting parameter ($\beta$) for synapse model.}
\label{beta-neuron}
\end{figure}
% Although the network performance using different $\beta$ parameters is not
% remarkable however, the results demonstrate better performance using $\beta$ between 1.8 and 2.
% The mimristive synapse robustness to the variations can be observed more
% obviously in Figure \ref{beta-neuron}.

\subsection{Discussion}
\label{sec:discussion}

As a final experiment we compare the baseline (values inspired by the
litterature) with the best values for all the evaluated parameters, including
the volatile/non volatile synapse of Chapter~\ref{synapse-box}.  The parameters
are listed in table~\ref{tab:1}. Other network elements and parameters that are
not listed in table~\ref{tab:1} are the same for both models.

\begin{table}\small
 \begin{center}
  %\rowcolors{1}{brown}{gray}
  \begin{tabular}{lccccc}
   \toprule
            & Synapse & Distrib. & STDP w. & Thres. & $\beta$ \\
   \midrule
   Best     & VNV     & Poisson  & 55 ms   & 35 mV  & 2       \\
   Baseline & NV      & Poisson & 25 ms   & 15 mV  & 1.5     \\
   \bottomrule
  \end{tabular}
 \end{center}
 \caption{Best parameters vs. baseline parameters}
 \label{tab:1}
\end{table}

\begin{figure}
 \centering
 \includegraphics{./figures/ch5/final-results.pdf}
 \caption{Using the best parameters significantly improves the recognition rate.}
 \label{best-num}
\end{figure}

We have evaluated both models using the same number of output neurons that were
used in the parameter evaluation sections with the addition of 300 neurons for
the output layer.  The results in Figure~\ref{best-num} demonstrate enhanced
performance using the best obtained parameters rather than baseline parameters.
These results, as most simulations shown in this study, show that the variation of
recognition rate due to the random initialization of the synaptic weights is
around 6 to 8 points with half of the results in a 3 to 4 point range. The
ranges of variation of the best version and of the baseline overlap, so one
does not have the guarantee that using theoretically better parameters will
lead to a better performing network, it is just statistically better.

As a summary, we can conclude from this study that:
\begin{itemize}
  \item The parameters that have a significant influence on the learning
  rate are the number of neurons in the SNN, the kind of synapse
  (volatile or synapse box), the duration of the STDP window and
  the neuron threshold value.
  \item The distribution of the spikes in the input spike trains
  and the $\beta$ fitting parameter of the volatile synapse do
  not have a significant impact on the recognition rate.
  \item One can significantly improve the recognition rate of
  a SNN by choosing better parameters.
  \item There is a relatively large spread of the recognition
  rate due to the random initialization of the synaptic weights in the simulations.
\end{itemize}

Thus a design space exploration should concentrate on the parameters that
influence significantly the recognition rate, first of all the architecture of
the network (number of neurons, topology), and then tune the other parameters.
This design space exploration should be based on a statistical analysis of the
performance of the network by running several simulations for each combination
of parameter values. Finally, researchers should explain clearly what they mean
when they give a single number as the recognition rate of a given SNN, is it a
mean of several rates, the best rate or just a single rate.
\section{Conclusions}
We have presented an empirical study of the influence of several parameters on
the recognition rate of memristor based hardware SNNs on the MNIST benchmark.
This study is based on simulations run with N2S3, an open source simulator we
have developped to help design neuromorphic circuits.

This study has shown that not all parameters have a significant influence on the
learning rate of SNNs and thus that a design space exploration should
concentrate first on the architecture of the network; then, the kind of device
used as a synapse, the STDP mechanism used and its parameters, and the threshold
of the neuron. This study is only valid on the MNIST benchmark and should be
complemented by similar studies on other test cases to confirm these findings,
especially test cases using natural data in spiking form where the precise
relative timings of the input spikes would necessitate more precise STDP
mechanisms than the simplified one used in this study that is only sensitive to
the average frequency of the input spike trains.

In the future, we will explore different models of synapses and neurons, more
complex network topologies and STDP mechanisms, and enhance the N2S3 simulator
with automatic design space exploration facilities that will concentrate on
optimizing the most significant parameters discovered in this study. In addition
to the recognition rate (or classification capabilities), we will also evaluate
other performance measures such as the power consumption of the circuit or its
convergence speed.
