SpinDrop: Dropout-Based Bayesian Binary Neural Networks With Spintronic Implementation
Résumé
Neural Networks (NNs) provide an effective solution
in numerous application domains, including autonomous driving
and medical applications. Nevertheless, NN predictions can be
incorrect if the input sample is outside of the training distribution or contaminated by noise. Consequently, quantifying the
uncertainty of the NN prediction allows the system to make more
insightful decisions by avoiding blind predictions. Therefore,
uncertainty quantification is crucial for a variety of applications,
including safety-critical applications. Bayesian NN (BayNN) using
Dropout-based approximation provides a systematic approach
for estimating the uncertainty of predictions. Despite such merit,
BayNNs are not suitable for implementation in an embedded
device or able to meet high-performance demands for certain
applications. Computation in-memory (CiM) architecture with
emerging non-volatile memories (NVMs) is a great candidate
for high-performance and low-power acceleration BayNNs in
hardware. Among NVMs, Magnetic Tunnel Junction (MTJ) offer
many benefits, but they also suffer from various non-idealities
and limited bit-level resolution. As a result, binarizing BayNNs
is an attractive option that can directly implement BayNN into
a CiM architecture and able to achieve benefits of both CiM
architecture and BayNNs at the same time. Conventional inmemory hardware implementations emphasize conventional NNs,
which can only make predictions, and do not account for both
device and input uncertainty, thus, reducing both reliability and
performance. In this paper, we propose for the first time Binary
Bayesian NNs (BinBayNN) with an end-to-end approach (from
algorithmic level to device level) for their implementation. Our
approach takes the inherent stochastic properties of MTJs as a
feature to implement Dropout-based Bayesian Neural Networks.
We provide an extensive evaluation of our approach from the
device level up to the algorithmic level on various benchmark
datasets.