\chapter{Conclusion}
\label{conc}
The growing use of ICT devices and technologies in our daily life
increases the demand for high performance, fast (parallel) and
power-efficient computations. In present computing architecture, the way of communicating between memory
and central processing unit (CPU) is not efficient  particularly while we are facing the problem of Big Data processing.
The reason of this inefficiency is called the memory wall issue or Von Neumann memory bottelneck.
This memory bottelneck issue is because of
speed disparity between the CPU and memory outside the CPU chip.

As we are reaching to the end of Moore's law, the possibility of increasing the pulse clock of
CPUs in order to increase the speed of system is facing serious problems. The multi-core
processor was a solution for the power dissipation problem. However, memory hierarchy and memory
management would be more significant and difficult in multi-core processors. Therefore, the memory
wall problem is actually worse in multicores.

New unconventional computing techniques using the emerging nano-devices such as memristors have been
proposed as solutions to Von Neumann architecture problem. Here in this dissertation, we
study the different possibilities of using this nano-device both as a digital logic and analog memristive switch.
Therefore, we began our research with a feasibility study of using memristive devices in different domains.
We briefly review the literature about memristors from mathematic model to the physical
realization and its applications. We discuss different classes of memristors based on the material
used to build the devices. Four general
memristive devices are studied in the first chapter: resistive, spintronic, organic (polymeric) and ferroelectric
memristors. The potential application as well as advantages versus disadvantages of using
each one are presented too. The resistive memristor has been used more than others in different
research works from memory to artificial synapse. The spintronic and ferroelectric devices show
promising properties to make new nonvolatile memories. The organic memristor is more appropriate
to make artificial synapse in Spiking Neural Networks.

In the digital domain, using the capability of memristors to have the processor and the memory in the same unit,
we propose two different architectures using non-volatile memristors. The first approach
is based on material implication and the
second one is based on crossbar arrays. A library of logic gates are provided using both methods.
We compare both methods and the reconfigurability is the most significant advantage of using the
crossbar array architecture.

Neuromorphic computation using Spiking Neural Networks (SNNs) has been proposed as another alternative solution
for the future of high performance computation. Neurons communicate together using spikes.
Therefore different spike codings
have been discussed to improve data transferring and data processing in neuro-inspired computation paradigms.
Choosing the appropriate neural network topology could result in
better performance of computation, recognition and classification.
The model of the neuron is another
important factor to design and implement SNN systems.
The speed of simulation and implementation, the
ability of integration to the other elements of network, and the suitability for scalable networks are the factors
to select an optimized neuron model. The sequential order of pre- or postsynaptic spikes occurring across a synapse in an interval of time leads to
define different learning methods. In chapter \ref{ch_neuromorphic}, we studied the differences
among the methods to help to
choose the most adapted STDP method to a given network.

We present N2S3 (for Neural Network Scalable Spiking Simulator), an open-source simulator that is
built to help design spiking neuromorphic circuits based on nanoelectronics. N2S3 is an event-based
simulator and its main properties are flexibility, extensibility, and scalability. N2S3 has been developed from
the ground up to model various kinds of neuron and
synapse models, various network topologies, various learning procedures, various reporting facilities,
and to be user-friendly, with a domain specific language to easily express and
run new experiments. Experimental set up for two standard
tasks are provided in N2S3: handwritten digit recognition on the MNIST dataset
and car counting on the Freeway dataset.

Furthermore, we have introduced a novel synapse box with the possibility to
forget and remember inspired from biological synapse properties. This synapse
box is composed of a volatile memristor (NOMFET) followed by a nonvolatile
resistive RAM. The volatile memristor acts like a high-pass filter to enhance
short term plasticity  and the nonvolatile resistive RAM enables long term
potentiation. To evaluate and verify the new synapse
box as well as the functionality of the N2S3 simulator, we have used the MNIST
handwritten digit dataset. The first results demonstrate an improvement in
recognition rate by using the synapse box over a single nonvolatile memristor
synapse.

We reviewed the most important existing platform in neuromorphic computing domain. Additionally
We have presented an empirical study of the influence of several parameters on
the recognition rate of memristor based hardware SNNs on the MNIST benchmark.
This study has shown that not all parameters have a significant influence on the
learning rate of SNNs and thus that a design space exploration should
concentrate first on the architecture of the network; then, the kind of device
used as a synapse, the STDP mechanism used and its parameters, and the threshold
of the neuron. This study is only valid on the MNIST benchmark and should be
complemented by similar studies on other test cases to confirm these findings,
especially test cases using natural data in spiking form where the precise
relative timings of the input spikes would necessitate more precise STDP
mechanisms than the simplified one used in this study that is only sensitive to
the average frequency of the input spike trains.

Deep learning recently is used in several state-of-the-art studies and developments
due to its ability in pattern recognition data classification.
Finally, we have proposed a deep neural network with
spiking neurons to verify the functionality of the brain-like models
in SNN for face recognition applications.
We introduced a framework
to use the rate based version of Contrastive Divergence
(CD) updating weight rule.
Indeed this model is a spike-based deep model with biological inspired leaky integrate-and-fire neurons.
Considering the results of Deep Belief
Networks (DBNs) in various tasks in machine learning domain, a DBN with two restricted boltzmann machines has
been developed in Matlab with Siegert units. Then the trained weights matrix
is transferred to a spiking DBN with LIF neurons. The recent model is simulated
in the Brian simulator and the results show the capability of the Spiking Deep
Belief Networks in a simple face recognition application is comparable with
traditional machine learning performance.

To introduce the potential of the future works, there are several possibilities
not only in unconventional computing domain but also in using emerging technologies such as memristor.
The mix combination of memristive devices using different materials propose new research studies
on the capability of both devices such as what we have proposed in Chapter \ref{synapse-box}.

As a proposition for the next study in digital domain, one can organize a
Programmable Logic Device (PLA) platform in such a
way that the crossbar array operates as a programmable AND array beside material implication as a fixed connection
for the OR array. Therefore new generation of FPGA could be the next target of such a study.

For future work in neuromorphic and SNN field, as they are not well-studied yet,
there are plenty of spaces for research in this domain.
 Studying different neural network topologies
such as deep belief, recurrent and convolutional neural networks to evaluate the
different memristive synapses considering the costs (area, energy, manufacturability, variability).
We can explore different models of synapse and neurons, more
complex network topologies and STDP mechanisms, and enhance the N2S3 simulator
with automatic design space exploration facilities that will concentrate on
optimizing the most significant parameters discovered in this study. In addition
to the recognition rate (or classification capabilities), we can also evaluate
other performance measures such as the power consumption of the circuit or its
convergence speed.
