The document discusses the Neurogrid system, which was the first hardware system capable of simulating a million neurons in real time. It describes Neurogrid's key design choices of emulating neural elements with shared circuits, implementing circuits in analog form, and connecting arrays in a tree network. The Neurogrid system consists of Neurocore chips containing 256x256 silicon neuron arrays interconnected in a tree network. Spikes are routed between chips using an address event representation scheme and multicast routing allows modeling axonal branching.

1. Introduction
Neurogrid's Design Choices
Neurogrid System
Neurogrid Neuron
Data Transfer
Comparison
Neurogrid: A Mixed-Analog-Digital Multichip System for
Large-Scale Neural Simulations
Group 7
Ashlesha Patil
Nithin Nethipudi
Titto Thomas
EE 746 : Literature Review 1/29

2. Introduction
Neurogrid's Design Choices
Neurogrid System
Neurogrid Neuron
Data Transfer
Comparison
Neurogrid
Neurogrid : Introduction
) First hardware system with the capability of performing biological real-time
simulations of a million neurons and their synaptic connections.
) It's a system for simulating large-scale neural models in real time.
) Four neural elements are axonal arbor, synapse, dendritic tree and soma are realized
by emulating their structure.
) 3 Major design choices for such a system
Emulate the four neural elements with dedicated or shared electronic circuits
Implement these electronic circuits in an analog or digital manner
Interconnect arrays of these neurons with mesh or tree network
EE 746 : Literature Review 2/29

3. Introduction
Neurogrid's Design Choices
Neurogrid System
Neurogrid Neuron
Data Transfer
Comparison
Neurogrid
Neurogrid : Features
) Emulated all neural elements except soma with share electronic circuits to maximize
the number of synaptic connections.
) All electronic circuits except axonal arbors were realized in an analog to maximize
energy eciency.
) Interconnected neural arrays in a tree network to maximize throughput.
EE 746 : Literature Review 3/29

4. Introduction
Neurogrid's Design Choices
Neurogrid System
Neurogrid Neuron
Data Transfer
Comparison
Hardware Realization
Architecture
Realizations
Spike routing networks
Analog or Digital
) Axonal arbors, synapses, dendritic trees, and somas may be implemented in analog
or digital.
) Fully analog and fully digital implementations will be,
EE 746 : Literature Review 4/29

5. Introduction
Neurogrid's Design Choices
Neurogrid System
Neurogrid Neuron
Data Transfer
Comparison
Hardware Realization
Architecture
Realizations
Spike routing networks
Four Architectures
) Fully Dedicated : A fully connected network of N neurons requires N2 synapse
elements
) Shared Axon : Realized using Address event representation (AER), each neuron is
assigned a unique address. Also requires N2 synapse elements for N neurons.
) Shared Synapse : N electronic circuits are required to fully connect N neurons. It
uses RAM to realize axonal branching instead of dedicated wire.
) Shared Dendrite : Also requires only N electronic circuits to fully connect N
neurons. Each shared-synapse circuit feeds its neighbouring neurons as well.
EE 746 : Literature Review 5/29

6. Introduction
Neurogrid's Design Choices
Neurogrid System
Neurogrid Neuron
Data Transfer
Comparison
Hardware Realization
Architecture
Realizations
Spike routing networks
Four Architectures (Contd.)
) (a) Fully dedicated, (b) Shared axon, (c) Shared synapse, (d) Shared dendrite
EE 746 : Literature Review 6/29

7. Introduction
Neurogrid's Design Choices
Neurogrid System
Neurogrid Neuron
Data Transfer
Comparison
Hardware Realization
Architecture
Realizations
Spike routing networks
Realization Comparison
) The hybrid realizations are all analog except for their axonal arbors.
) Available choices are,
Fully dedicated analog (FDA)
Shared axon hybrid (SAH)
Shared axon digital (SAD)
Shared synapse hybrid (SSH)
Shared dendrite hybrid (SDH)
) FDA is faster than real time and have arbitrary connectivity, while SDH has lots of
neurons with mostly local connectivity.
) FDA achieves its cost-eectiveness by minimizing T, while SDH does it by
minimizing A.
EE 746 : Literature Review 7/29

8. Introduction
Neurogrid's Design Choices
Neurogrid System
Neurogrid Neuron
Data Transfer
Comparison
Hardware Realization
Architecture
Realizations
Spike routing networks
Realization Comparison (Contd.)
EE 746 : Literature Review 8/29

9. Introduction
Neurogrid's Design Choices
Neurogrid System
Neurogrid Neuron
Data Transfer
Comparison
Hardware Realization
Architecture
Realizations
Spike routing networks
RAM Costs
EE 746 : Literature Review 9/29

10. Introduction
Neurogrid's Design Choices
Neurogrid System
Neurogrid Neuron
Data Transfer
Comparison
Hardware Realization
Architecture
Realizations
Spike routing networks
Spike routing networks
) Meshes and trees have been explored for routing spikes between neural-element
arrays.
) Meshes oer high bandwidth, but have long latency.
) Trees oer short latency, but have low bandwidth
) Unlike meshes, however, trees support deadlock-free multicast communication,
enabling them to utilize their limited bandwidth eciently, and thereby maximize
throughput.
) Neurogrid targets can use multicast to realize secondary axon-branching, so they
chose tree.
EE 746 : Literature Review 10/29

11. Introduction
Neurogrid's Design Choices
Neurogrid System
Neurogrid Neuron
Data Transfer
Comparison
The system
Neurogrid System
) Two main components
Software to perform interactive visualization
Hardware to perform real-time simulation
) Neurocore packet is a sequence of 12-b words that specify a route, an address, an
arbitrarily long payload, and a tailword. They are mapped on to Neurocores over
USB using driver programs.
) A Neurocore has a 256 256 silicon-neuron array, a transmitter, a receiver, a
router, and two RAMs. A neuron has a soma, a dendrite, four gating-variable and
four synapse-population (i.e.,shared synapse and dendrite) circuits.
EE 746 : Literature Review 11/29

12. Introduction
Neurogrid's Design Choices
Neurogrid System
Neurogrid Neuron
Data Transfer
Comparison
The system
Neurogrid System (Contd.)
EE 746 : Literature Review 12/29

13. Introduction
Neurogrid's Design Choices
Neurogrid System
Neurogrid Neuron
Data Transfer
Comparison
The system
Neurocore
) RAM0 provides 256 locations for target synapse types (or no connection). RAM1
stores 18 con

14. guration bits and 61 analog biases, common to all the Neurocores
silicon neurons.
) DACs produce the analog biases. RstMB provides

15. ve resets and generates DACs
reference current. ADCs digitize four analog signals from a selected neuron.
) Ti02, To02, Li02, L002, Ri02, and Ro02 communicate with parent or either
child.
) The 12 14 mm 2 die, with 23 M transistors and 180 pads, fabricated in a 180nm
complementary CMOS process.
EE 746 : Literature Review 13/29

16. Introduction
Neurogrid's Design Choices
Neurogrid System
Neurogrid Neuron
Data Transfer
Comparison
Dimensionless Models
Soma and Dendrite
Synapse and Ion channel population
Circuit Realization
Circuit Realization
Passive membrane model
) Models composed of conductors, capacitors, voltage, and current sources may be
converted to dimensionless form.
) A passive membrane model is given by,
C _V
= Gleak (V Eleak ) + Iin
) Change the reference voltage to Eleak and normalize with GleakVn to give
v_ = v + u
EE 746 : Literature Review 14/29

17. Introduction
Neurogrid's Design Choices
Neurogrid System
Neurogrid Neuron
Data Transfer
Comparison
Dimensionless Models
Soma and Dendrite
Synapse and Ion channel population
Circuit Realization
Circuit Realization
Soma model
) Soma model is given by,
sv_s = vs + isin +
1
2
v2
s gkvs gresvspres (t) + vd
) s - membrane time constant, isin - input current, vd - dendritic input.
2 v2
) Quadratic positive feedback 1
s models the spike-generating sodium current
) The reset conductance gres , active for the duration tres of a unit-amplitude pulse pres ,
models the refractory period
) High-threshold potassium conductance gK models spike-frequency adaptation.
K g_K = gK + gK1pres (t)
K - decay time constant, gK1 - saturation value
EE 746 : Literature Review 15/29

18. Introduction
Neurogrid's Design Choices
Neurogrid System
Neurogrid Neuron
Data Transfer
Comparison
Dimensionless Models
Soma and Dendrite
Synapse and Ion channel population
Circuit Realization
Circuit Realization
Dendrite model
) Dendrite model is given by,
dv_d = vd + idin ibppres (t) + gch(ech vd )
) vd - membrane time constant, idin - input current, ibp - back propagating input, and
gch - channel populations conductance, ech - reversal potential
) Soma and dendrite could receive synaptic inputs.
EE 746 : Literature Review 16/29

19. Introduction
Neurogrid's Design Choices
Neurogrid System
Neurogrid Neuron
Data Transfer
Comparison
Dimensionless Models
Soma and Dendrite
Synapse and Ion channel population
Circuit Realization
Circuit Realization
Synapse population
) Soma model is given by,
syng_syn = gsyn + gsatprise (t)
) syn - synaptic time constant, gsat - saturation conductance for the population
) The unit-amplitude pulse prise (t) is triggered by an input spike; its width trise models
the duration for which neurotransmitter is available in the cleft.
) gsyn decays spatially in the shared dendritic tree and provides an input current to the
soma or dendrite.
EE 746 : Literature Review 17/29

20. Introduction
Neurogrid's Design Choices
Neurogrid System
Neurogrid Neuron
Data Transfer
Comparison
Dimensionless Models
Soma and Dendrite
Synapse and Ion channel population
Circuit Realization
Circuit Realization
Ion channel population
) Ion-channel populations conductance gch is obtained by scaling a maximum
conductance gmax with a gating variable c ( i.e, gch = cgmax )
) c is modelled as,
gv _ c = c + css
css - steady-state activation or inactivation, gv - its time constant
) css is given by,
css =
+

21. or

22. +

23. ;

24. - model a channels opening and closing rates
EE 746 : Literature Review 18/29

25. Introduction
Neurogrid's Design Choices
Neurogrid System
Neurogrid Neuron
Data Transfer
Comparison
Dimensionless Models
Soma and Dendrite
Synapse and Ion channel population
Circuit Realization
Circuit Realization
Log domain realization
) Subthreshold PMOS current is given by,
Id = I0e
vgbvsb
VT (1 e
vds
VT )
) For vds 4VT and Vsb = 0
Id = I0e
vgb
VT
) Taking natural logarithm on both sides
lnId lnI0 ln =
vgb
VT
) The

26. nal equation will be
_Id
Id
=
v_gb
VT
EE 746 : Literature Review 19/29

27. Introduction
Neurogrid's Design Choices
Neurogrid System
Neurogrid Neuron
Data Transfer
Comparison
Dimensionless Models
Soma and Dendrite
Synapse and Ion channel population
Circuit Realization
Circuit Realization
Passive membranes circuit
) Im ! membrane potential, Ilk ! membrane's leak, Iback ! membrane's input.
C _V
m = Ilk Iback
Vleak + Vin = Vback + Vm )
Ileak
1
Iin
2
=
Iback
4
Im
5
EE 746 : Literature Review 20/29

28. Introduction
Neurogrid's Design Choices
Neurogrid System
Neurogrid Neuron
Data Transfer
Comparison
Dimensionless Models
Soma and Dendrite
Synapse and Ion channel population
Circuit Realization
Circuit Realization
Soma circuit
) Circuit realization operates according to,
) The high-threshold potassium conductances circuit realization operates according to
EE 746 : Literature Review 21/29

29. Introduction
Neurogrid's Design Choices
Neurogrid System
Neurogrid Neuron
Data Transfer
Comparison
Dimensionless Models
Soma and Dendrite
Synapse and Ion channel population
Circuit Realization
Circuit Realization
Circuit realizations
) Dendrite circuit,
) Soma circuit,
EE 746 : Literature Review 22/29

30. Introduction
Neurogrid's Design Choices
Neurogrid System
Neurogrid Neuron
Data Transfer
Comparison
Dimensionless Models
Soma and Dendrite
Synapse and Ion channel population
Circuit Realization
Circuit Realization
Other realizations
) Dendrite,
) Synapse population,
) Ion-channel population,
EE 746 : Literature Review 23/29

31. Introduction
Neurogrid's Design Choices
Neurogrid System
Neurogrid Neuron
Data Transfer
Comparison
Transmitter and Receiver
Router
Transmitter and Receiver
) AER : Realize multiple virtual point-to-point channels with a single physical link by
representing a communication on any channel with its input-ports address.
) Encodes the spikes row and column addresses, conveying these addresses to another
chip, and decoding them to recreate the spike at its destination.
) Two M-way arbiters, built with M-1 two-way arbiters connected in a binary tree
generate a log2 (M)-bit address for each row or column selected.
) Latches allow the next packets addresses to be decoded while the current ones
spikes are being delivered to the array.
EE 746 : Literature Review 24/29

32. Introduction
Neurogrid's Design Choices
Neurogrid System
Neurogrid Neuron
Data Transfer
Comparison
Transmitter and Receiver
Router
Transmitter and Receiver (Contd.)
EE 746 : Literature Review 25/29

33. Introduction
Neurogrid's Design Choices
Neurogrid System
Neurogrid Neuron
Data Transfer
Comparison
Transmitter and Receiver
Router
Transmitter and Receiver (Contd.)
) 256 256 version of the transmitter and a 2048 256 version of the receiver - its
eight lines per row select one of four shared-synapses to activate, one of three sets
of analog signals to sample, or a neuron to disable
) 86 ns is needed to transfer a rows spikes to the arrays periphery and 23 ns to encode
each spikes column address.
) Pipelining and parallelism enable the transmitter to sustain a maximum transmission
rate of 43.4 Mspike/s, or 663 spike/s per neuron. receiver decodes an additional
column address every 16 ns, sustaining a maximum rate of 62.5 Mspike/s, or 956
spike/s per neuron
EE 746 : Literature Review 26/29

34. Introduction
Neurogrid's Design Choices
Neurogrid System
Neurogrid Neuron
Data Transfer
Comparison
Transmitter and Receiver
Router
Router
) Realizes the primary and secondary axon branching using the multicast capability
and the Neurocores embedded memory.
) Two distinct phases: a point-to-point phase and a branching phase.
) Point-to-point phase : the router steers the packet up/down or left/right, based on
a bit in the packets

35. rst word. Branching phase : the router copies the packet to
both left and right ports.
) If
ood bit (F) set, the packet recursively branches to all descendants (
ood mode),
Otherwise, delivered to exclusively (target mode).
) In F mode Depending on how the SRAM is programmed, the packet is either

36. ltered
or delivered. If the packet is delivered, current system appends two additional bits
retrieved from the SRAM to its row address.
) The obtained rates were 234 Mspike/s in normal mode and up to 1.17 Gspike/s in
burst mode.
EE 746 : Literature Review 27/29

37. Introduction
Neurogrid's Design Choices
Neurogrid System
Neurogrid Neuron
Data Transfer
Comparison
Transmitter and Receiver
Router
Router (Contd.)
EE 746 : Literature Review 28/29

38. Introduction
Neurogrid's Design Choices
Neurogrid System
Neurogrid Neuron
Data Transfer
Comparison
Energy and it's comparison
Energy consumption it's comparison
) The energy expended to activate a silicon axons synapses is given below. Dividing it
with total connecting neurons ( ncol np nsyn )
) Comparing the energy consumption with the other networks.
EE 746 : Literature Review 29/29