Nuromorphic computing

1. Jawar Singh, Professor
Electrical Engineering Department
Indian Institute of Technology Patna, Bihar
jawar@iitp.ac.in
Emerging and Futuristic Energy Efficient
Computing Devices and Architectures

2. Natural Intelligence Vs Artificial Intelligence
~15KW ~200KW ~300KW
1997 2011 2016
IBM Deep Blue Vs. Kasparov IBM Watson Vs Ken and Brad Google AlphaGo Vs Lee Sedol
~20W

4. Energy Consumption of Google (in gigawatt
hours)
~3X in Five Years

5. Performance/Energy for Training Different AI
Models Releases as much
CO2 as 1,300 cars
do in 2 weeks
Is AI harming
/helping more to
the environment?

7. Today’s Computing Architecture
Intel’s Technology Brief
https://www.intel.com/content/www/us/en/research/neuromorphic-computing-loihi-2-technology-brief.html

8. Why Brain-inspired Computing?
Nuclear Reactor
 Human brain with less than
20W of power consumption
offers a processing capability
that exceeds the petaflops
mark.
 Thus, outperforms state-of-
the-art supercomputers by
several orders of magnitude
in terms of energy efficiency
and volume.
P. A. Merrolla et. al., Science 345 (2014)

9. Traditional Computing Vs Alternative Computing
Abu Sebastian, et. al., Nature Nanotechnology, 2020
For example, in AI and ML an important
computation is a MAC operation ( y = Σ wi
xi, where xi represents an input pixel value
of an image, and wi represents a learned
filter weight).
A single-integer MAC operation might
require just ~3.2 pJ of energy. However, if
a weight value wi is stored in off-chip
DRAM and brought to the processor for
computation, ~640 pJ of energy is required
just to fetch the wi, and the energy of the
memory request overwhelms that of the

10. Multiply and Accumulate MVM Operation
Analog vector and matrix operations. Using a bitline to perform an analog sum of products operation.
The total current pulled from the supply voltage represents the result of
the computation.

11. [Horowitz, ISSCC 2014]

12. Brain Inspired Vs Von-Neumann
Computing
Von-Neumann
Computing
V
s

13. TrueNorth from IBM emulates the neuronal function comprises of 5.4x109 transistors, occupying 4.3 cm2area in Samsung’s 28-
nm process technology that consumes less than 70mW. It shows that the conventional computing architectures lack energy
efficiency and there is a demand for alternatives computing architectures. The TrueNorth has 4096 cores and each having
1.2x106transistors for 265 neurons, hence, to realize a biological neuron approximately 104 transistors are needed.
IBM’s TrueNorth

14. Intel’s Loihi-2
Recently Loihi has evaluated for a wide
range of applications:
 Adaptive robot arm control
 Visual-tactile sensory perception
 Learning and recognizing new odors
and gestures
 Fast database similarity research
 Solving hard optimization problems
such as railway scheduling
All these applications consumes less than 1W compared to tens of KW that standard CPU & GUP solutions

15.  Electrical circuit representation of the Hodgkin-Huxley model.
 Specific voltage dependent ion-channels that controls the flow of ions
through the membrane.
 Ion-channels are voltage dependent and their opening and closing
depends on voltage across the membrane.
Simplified RC Model
L Hodgkin and A F Huxley, “A quantitative description of membrane current and its application to conduction and excitation in nerve”, The Journal of physiology, 117(4):500–544, 1952.
A Kamal and J Singh, “Simulation based Ultra-Low Energy and High Speed LIF Neuron using Silicon Bipolar Impact Ionization MOSFET for Spiking Neural Networks,” IEEE Transactions on Electron
Devices, 2020.
N Kamal and J Singh, “A Highly Scalable Junctionless FET Leaky Integrate-and-fire Neuron for Spiking Neural Networks”, IEEE Transactions on Electron Devices, 2021.
(a)
(b)
(c)
HH and Simplified RC Model

16. A Kamal and J Singh, “Simulation based Ultra-Low Energy and High Speed LIF Neuron using Silicon Bipolar Impact Ionization
MOSFET for Spiking Neural Networks,” IEEE Transactions on Electron Devices, 2020.
Silicon Neuron Model

17. Silicon Synapse Model
 Depression and Facilitation
analogues to biological synapse-
learning behavior
 Synaptic efficacy or synaptic
weight is the modulated channel
transconductance of the device

18. A. Brain-inspired Computing Architecture
 Applied pulses with fixed pulse width and different polarity leading to
increase/decrease the synaptic weight (trapping/de-trapping of holes that
modulates conductance of channel).
 In addition, these synaptic learning properties are highly rely on the
interval between input pulses (spatio-temporal nature).
 These properties allow us to conclude that they offer very similar learning
characteristics with a biological synapse and paves the way for designing a
neuromorphic computing hardware

19. B. Brain-inspired Computing Architecture
(a) Generalized schematic of an artificial neural
network (ANN).
(b) Crossbar based realization of ANN using
proposed device. Here, W+ and W- denote
the rows representing positive and negative
input weights, respectively, while B+ and B-
denotes the row representing the positive
and negative bias weights, respectively.
(c) Circuit for converting final weighted
sum of current to output voltage.
Kumar, A., Beeraka, S.M., Singh, J. et al. An On-Chip Trainable and Scalable In-Memory ANN Architecture for AI/ML Applications. Circuits Syst

20. Summary
 Brain-inspired computing approach is motivated by the belief that the
brain's ability to process and analyze complex information, learn from
experience, and make decisions based on sensory inputs, provides a
blueprint for more efficient and effective computing systems.
 The goal is to create machines that can perform tasks such as
perception, recognition, and decision-making more naturally and
efficiently than traditional computing systems.
 For implementation of brain-inspired approach, simple, elegant, and
energy efficient models for neuron and synapse are highly essential.

21. Thanks !