\chapter{Introduction}
\label{intro}
\section{Introduction}
The two most important demands of humans using ICT devices and technologies are becoming two hottest topic of
research in computer science and engineering namely Big Data and Internet of Things (IoT).
In both domains, the way of processing data is quite important. In Big Data the clustering, classification, and
prediction are not avoidable to use and process the data efficiently.
In IoT, the smart devices are connected to other smart devices using different sensors.
Machine learning recently proposed promising solution for processing data in these two domains.
By 2020, there will be 50 to
100 billion devices connected to the Internet, ranging from
smartphones, PCs, and ATMs (Automated Teller Machine) to
manufacturing equipment in factories and products in shipping
containers \cite{perera_sensing_2013}.
For the Big Data or sensory data
not only an efficient processing algorithm is necessary but also finding
a new fast, parallel and power-efficient hardware architecture is unavoidable.

Machine learning algorithms are widely used for data classification
in software domain and using conventional hardware platform.
These algorithms on nowadays computers consume a remarkable time and
energy. The reason is that in conventional computing,
the way of communicating between memory
and central processing unit (CPU) is not efficient.
The memory wall or Von Neumann memory bottelneck is the growing disharmony of
communication speed between the CPU and memory outside the CPU chip.
An important reason for this disharmony is the restricted communication bandwidth
beyond chip boundaries, which is referred to as bandwidth wall as well.
The CPUs access both
data and program in memory using the same shared resources. Finally, 
CPUs spend most of their time idle.

Using emerging technologies such as memristors \cite{strukov_missing_2008},
there is possibility of performing both information processing and storing the computational
results on the same physical platform \cite{Di-Ventra}. Memristors have the potential to be a
promising device for novel paradigms of computation as well as new generation of memory.
The characteristics of memristor are promising to design a processing unit with
local access memory rather than non-local and shared memory. The new high performance
architecture
for next generation of computation regarding to emerging technologies could
be in logic or analog domain. Memristors have potential for both digital and analog
paradigms of computations.

Another novel alternative architecture suitable for neural network and machine learning
algorithms is proposed as Spiking Neural Network (SNN) system
which is known as Neuromorphic architecture too.
SNN is the known way to realize the neural network software abilities on a hardware architecture.
The mammalian nervous system is a network of extreme complexity which is able to perform cognitive
computation in a parallel and power-efficient manner. Understanding the principles of brain processing
for computational modeling is one of the biggest challenges of the 21\lowercase{st} century that led to the new
branch of research e.g., neuromorphic computing. Neuromorphic engineering represents one of the
promising fields for developing new computing paradigms complementing or even replacing current
Von Neumann architecture \cite{indiveri_memory_2015}. The requirements for implementing a
SNN architecture (neuromorphic) are providing electronic devices that can mimic the
biological neural network components such as neurons and synapses.
The Spiking neural model is an electrical model of physiological neuron that has been implemented
on the circuit using state-of-the-art technologies e.g., CMOS transistors or on low-power CMOS
design using subthreshold regime transistor \cite{indiveri_neuromorphic_2011}.
The synapse in biological neural network reacts as a plastic connection controller
between two neurons.
Recently, emerging devices in nano-scale have demonstrated novel properties for
making new memories and Artificial synapse. One of those is the
memristor that was hypothetically presented by Leon Chua in
1971~\cite{chua_memristor-missing_1971} and after a few decades, HP was the first to announce the
successful memristor fabrication~\cite{strukov_missing_2008}. The unique properties  in
memristor nano-devices such as, extreme scalability, flexibility because of analog behavior, and ability to
remember the last state make the memristor a very promising  candidate to apply
it as a synapse in Spiking Neural Network (SNN)~\cite{jo_nanoscale_2010}.

\section{Part I:Motivation, state-of-the-art and
application of using emerging nanodevices
for unconventional computing}
The first part of this dissertation contains the mathematic and
physical model of memristor, memristive nanodevice technologies,
as well as different applications of this emerging
technology. Memristor can remember its last state after the last power
plugging and has a simple physical structure, high-density integration, and
low-power consumption. These features make the memristor an attractive candidate
for building the next generation of memories \cite{Ho:2009,Borghetti,Hamdioui-mem2}, an artificial synapse
in Spiking Neural Network architectures \cite{jo_nanoscale_2010,rose10}, and as a switch in
crossbar array configurations \cite{imp-nature}. Different device structures are still
being developed to determine which
memristor device can be presented as the best choice
for commercial use in memory/flash manufacturing
or in neuromorphic platforms.
This is based on different
factors such as size, switching speed, power consumption, switching longevity, and
CMOS compatibility. A comprehensive survey particularly on recent
research results and recent development of memristive devices
seems be quite useful for future research work and developments.
To better understanding how memristor can restore the data and how it could be
flexible to modify the conductances to be replaced as a synapse, we have modeled
the behavior of this device.
In addition, we perform a classification of the devices based on the manufacturing technology.
In this classification, we discuss 
different characteristics of various
devices. We study the potential applications of a memristor built using specific technology. 
The advantages and disadvantages of each class of memristor with 
various type of device materials are discussed. 
Furthermore, we discuss
potential applications of memristor nanodevice in nonvolatile memories such as RRAM, PCM, CBRAM,
FeRAM and MRAM, digital computation, analog and neuromorphic domains.

In the second chapter, we discuss two methods for unconventional digital computing by memristive
two-terminal devices \cite{shahsavari_unconventional_2015}. Two main digital computation approaches, namely
material implication (IMP) or stateful logic \cite{Leveraging,LehtonenPL12} and
programmable crossbar architecture \cite{Crossbar-mem,crossbar}
are studied in this chapter. By applying memristor as a digital switch, a high memristance
 (memristor resistance) is considered as logic `0' and a low memristance is
 considered as logic `1'. First and foremost, based on the previous research works on IMP, we establish a functionally complete
Boolean operation to be able to build all standard logic gates. Subsequently, building the digital gates has been performed
using programmable crossbar architectures.
Programmable crossbar architectures have been proposed as a
promising approach for future computing architectures because of their
simplicity of fabrication and high density, which support
defect tolerance. At the end of Chapter \ref{imp-crossbar}, the comparison of two methods
in digital computation is presented.

In the last chapter of part I, the basic definition of neuromorphic or Spiking Neural Network (SNN) architecture
have been introduced. Neuromorphic computing has the potential to bring very low power computation to
future computer architectures and embedded systems~\cite{merolla_million_2014}.
The main remarkable difference between conventional Von Neumann architecture and
neuromorphic systems is in their use of memory structures.
The way of communication between memory
and central processing unit (CPU) in conventional computing is not very efficient which is
known as Von Neumann memory bottelneck. CPUs spend most of their time idle
because the speed difference between the CPU and memory. The solution that has been
applied in Von Neumann architecture is memory hierarchy. In other words, a
limited amount of fast but costly memory sit closer to the processing unit, while most of the data is
stored in a cheaper but larger memory. By proposing computing unit next to the
local memory, neuromorphic
brain-inspired computing paradigms offer an attractive solution
for implementing alternative non von Neumann architectures,
using advanced and emerging technologies \cite{rajendran_neuromorphic_2016,indiveri_memory_2015}.
Artificial neural network (ANN) is a mathematical model of the network of neurons in mammalian
brain though SNN is an electronic hardware neuromorphic model of biological brain.
SNNs provide powerful tools to emulate data processing in the brain, including
neural information processing, plasticity and learning. Consequently, spiking networks offer
solutions to a broad range of specific problems in applied engineering image detection, event detection,
classification, speech recognition and many cognitive computation domain applications.

Neurons communicate together using spikes. In Chapter \ref{ch_neuromorphic}, we review
different ways of coding data to spikes known as spike information coding methods.
Furthermore, we study the network topologies and configurations.
The interconnection among units can be structured in
numerous ways resulting in numerous possible topologies.
Two basic classes are defined: Feed-Forward Neural Networks (FFNN) and Recurrent
(or feedback) Neural Networks (RNN) depicted in Figure \ref{twotopol}. 
We add a discussion in
more modern neural networks such as Convolutional Neural 
Networks (CNN) \cite{lecun_gradient-based_1998},
and Deep Belief Networks (DBNs) \cite{hinton_fast_2006}.
Moreover, various spiking model of neurons as dynamic elements and processing units
are reviewed. We discuss which model we have used in our platform and why
we choose this model. Thanks to the plasticity property of synapse,
in neural network system, we can basically say the synapse is where the learning
happens. Therefore, in this chapter both synapse and learning are studied.
Additionally, we discussed various classes of learning algorithms as well as a comprehensive study of
Spike-Timing Dependent Plasticity (STDP) \cite{kempter_hebbian_1999,song_competitive_2000} is presented.
This comprehensive study includes presenting different models for STDP learning based on
single or multiple pre- or postsynaptic spikes occurring across a synapse in an interval of time.
Finally in Chapter \ref{ch_neuromorphic}, we applied lateral inhibition which is winner-take-all (WTA)
\cite{jin_fast_2002,oster_winner-take-all_2004}
in our platform as well as homeostasis as a method of neuron adaptation of learning.


\section{Part II:Our contribution in spiking neural network architecture: Simulator,
New synapse box, Parameter exploration and Spiking deep learning}
The second part of the thesis consists of our contributions to neuromorphic computing and SNN architecture.
In Chapter \ref{N2S3-ch}, we present and develop N2S3 (for Neural Network Scalable Spiking Simulator),
an open-source event-driven simulator that is built to help design spiking neuromorphic circuits based on nanoelectronics.
It is dedicated to intermediate modeling level, between low-level hardware description
languages and high-level neural network simulators used primarily in
neurosciences as well as the integration of synaptic memristive device modeling,
hardware constraints and any custom features required for the target
application. N2S3 has been developed from
the ground up for extensibility, allowing to model various kinds of neurons and
synapses, network topologies, learning procedures, reporting facilities,
and to be user-friendly, with a domain specific language to easily express and
run new experiments. For experimental set up,
N2S3 is distributed with the implementation of two ``classical''
experiments: handwritten digit recognition on the MNIST
dataset~\cite{lecun_gradient-based_1998,lecun_convolutional_1995} and the
highway vehicle counting experiment~\cite{bichler_extraction_2012}.

In Chapter \ref{synapse-box}, with the combination of a volatile and a nonvolatile
memristor we introduce and design a new artificial synapse box that can improve learning in spiking neural
networks architectures \cite{shahsavari_combining_2016}. This novel synapse
box is able to forget and remember by inspiration from biological synapses.
The nonvolatility is a unique property in memristor nanodevice to
introduce it as a promising candidate in building next generation of
non-volatile memory. However, by inspiring of physiological synapse,
a synapse that can forget unimportant data (non-frequent spikes) and remember
significant data (frequent spike trains) can support network to improve learning process.

Thanks to close collaboration with the nano-electronics research center in the University of Lille
(IEMN), we have had the opportunity of studying the suitability of different types of memristors (TiO$_{2}$ ,
NOMFET, magnetoresistive, magnetoelectric) to build a spiking neural network hardware platform.
To add the forgetting property to the synapse box, we have used a volatile memristor named NOMFET
(Nanoparticle Organic Memory Field-Effect Transistor). We have merged NOMFET with
a nonvolatile solid-state meristor nanodevice. At the end of this chapter, we evaluate the synapse box
proposal by comparing it with a single non-volatile memory synapse
by simulation on the MNIST handwritten digit recognition benchmark.

Specific application domains such as Big Data classification, visual
processing, pattern recognition and in general sensory input data,
require information processing systems which are able to classify
the data and to learn from the patterns in the data. Such systems
should be power-efficient. Thus researchers have developed brain-
inspired architectures such as spiking neural networks.
In Chapter \ref{evaluation}, we surveyed brain-
inspired architectures approaches with studying well-known
project and architecture in this neuromorphic domain.
For large scale brain-like computing on neuromorphic hardware
we have recognized four approaches:
\begin{itemize}
 \item Microprocessor based approaches where the system can read the codes to
       execute and model the behavior of neural systems and cognitive computation such
       as the SpiNNaker machine~\cite{furber_spinnaker_2014}.
 \item Fully digital custom circuits where the neural system components are
       modeled in circuit using sate-of-the-art CMOS technology e.g., IBM TrueNorth
       machine~\cite{merolla_million_2014}.
 \item Analog/digital mixed-signal systems that model the behavior of
       biological neural systems, e.g. the NeuroGrid \cite{benjamin_neurogrid:_2014}
       and BrainScales \cite{schemmel_wafer-scale_2010} projects.
 \item Memristor crossbar array based systems where the analog behavior of the
       memristors emulate the synapses of a spiking neural network \cite{bichler_design_2013}.
\end{itemize}
We have studied large scale brain-like architectures such as SpiNNaker, IBM TrueNorth, NeuroGrid, and BrainScales
to recognize the cons and pros of these projects to have our optimized evaluation
and exploration of required items and parameters for neuromorphic and brain-like computation platforms.

Spiking neural networks are widely
used in the community of computational neuroscience and neuromorphic
computation, there is still a need for research on the methods to choose
the optimum parameters for better recognition efficiency. Our main
contribution in Chapter \ref{evaluation}
is to evaluate and
explore the impact of several parameters on the learning performance of SNNs for
the MNIST benchmark: the number of neurons in the output layer, the duration of
the STDP window, various thresholds for adaptive threshold neurons,
different distributions of spikes to code the input images and the memristive
synapse fitting parameter.
This evaluation has shown that a careful choice of a
few parameters can significantly improve the recognition rate on this
benchmark.

Deep learning is currently a very active research area in machine learning and pattern
recognition society due to its potential to classify and predict as a result of processing Big
data and Internet of Things (IoT) related data.
Deep Learning is a
set of powerful machine learning methods for training deep
architectures. Considering the inherent inefficiency of learning
methods from traditional Artificial Neural Networks in deep
architectures, Contrastive Divergence (CD) has been proposed
to train Restricted Boltzmann Machines (RBM) as the main
building blocks of deep networks \cite{hinton_training_2002}.

In Chapter \ref{deep} deep learning architectures are introduced in SNN systems.
In SNN, neurons communicate using spikes \cite{gerstner2014neuronal}.
Therefore, we have to
design an architecture of neurons that are able to implement spiking data.
Consequently, we present a framework for using Contrastive Divergence
to train an SNN using RBM and spiking neurons. The obtained recognition rate or network
accuracy of the architecture using CD algorithms for learning was 92.4$\%$.
This framework can open a new window toward the neruromorphic
architecture designer to apply the state-of-the-art of machine learning
learning algorithm in SNN architecture.
\section{Manuscript outline}
This manuscript presents a spiking neural network
platform suitable to hardware implementation with a focus on learning
in SNN. Therefore in the first two chapters, we review cons and pros of
memristor and memristive-based computation. The different models and
technologies of memristor beside the applications is presented
in chapter 2. The third chapter consists of using memristors
in the unconventional digital manner.
In the chapter 4, the principals of neuromorphic and
spiking neural network architecture is described.
Our contribution in designing and presenting spiking
neural network architecture is presented in the
second part of this manuscript.
First of all, in the contribution part we present our neuromorphic hardware simulator.
Second, we propose an artificial synapse box to improve learning
by combining volatile and nonvolatile types of memristor devices.
Furthermore, in Chapter \ref{evaluation}, we discuss different
parameters in SNN architecture and evaluate the effect of each parameters in
learning. In Chapter \ref{deep}, deep learning in spiking neural network
is presented. Finally the conclusion and potential future work are
explained. A general overview of this dissertation is shown
in Figure \ref{fig:overview}.



%Diagram
\begin{figure}[t]
\centering
\begin{tikzpicture}[
  level 1/.style={sibling distance=80mm,minimum width=5cm,
  node distance=2cm,text width=10em,fill={rgb:red,1;green,1;yellow,1}},
  edge from parent/.style={->,draw,fill={rgb:cyan,3;yellow,1;blue,1}},
  >=latex]

% root of the the initial tree, level 1
\node[root] {CH1. Introduction}
% The first level, as children of the initial tree
  child {node[level 2] (c1) {Part I. Using nanodvices for unconventional computing}}
  child {node[level 2] (c2) {Part II. Our contribution in spiking neural network architecture}};
%    child {node[level 2] (c3) {Analog domain applications}};

% The second level, relatively positioned nodes
\begin{scope}[every node/.style={level 3,minimum width=4.5cm,
node distance=2.2cm,text width=10em, fill={rgb:cyan,3;yellow,1;blue,1}}]
\node [below of = c1, xshift=17pt] (c11) {Ch2. Memristor new emerging device:review and applications};
\node [below of = c11] (c12) {Ch3. Unconventional digital computing using memristors};
\node [below of = c12] (c13) {Ch4. Neuromorphic computing in Spiking Neural Network Architecture};
% \node [below of = c13] (c14) {FeRAM};
% \node [below of = c14] (c15) {MRAM};

\node [below of = c2, xshift=20pt] (c21) {Ch5. N2S3, scalable neuromorphic hardware simulator};
\node [below of = c21] (c22) {Ch6. Combining a Volatile and Nonvolatile Memristor in synapse box };
\node [below of = c22] (c23) {Ch7. Evaluation and parameter exploration in SNN};
\node [below of = c23] (c24) {Ch8. Deep learning in spiking neural network};
%
% \node [below of = c3, xshift=15pt] (c31) {Neuromorphic};
% \node [below of = c31] (c32) {Oscillators};
% \node [below of = c32] (c33) {Chaotic circuits};
% \node [below of = c33] (c34) {Amplifiers};
% \node [below of = c34] (c35) {Filters};
\end{scope}

% lines from each level 1 node to every one of its "children"
\foreach \value in {1,2,3}
  \draw[->] (c1.195) |- (c1\value.west);

\foreach \value in {1,...,4}
  \draw[->] (c2.195) |- (c2\value.west);

% \foreach \value in {1,...,5}
%   \draw[->] (c3.195) |- (c3\value.west);
\end{tikzpicture}
\label{diagram:1}
\caption{General overview of the manuscript.}
\label{fig:overview}
\end{figure}
