SpiNNaker – Simulating the Brain with a
Massively Parallel Neuromorphic Platform
Dr Oliver Rhodes, Prof Steve Furber StudentHack VII

200 Years Ago…
• Ada Lovelace, b. 10 Dec. 1815
"I have my hopes, and very distinct
ones too, of one day getting cerebral
phenomena such that I can put them
into mathematical equations--in
short, a law or laws for the mutual
actions of the molecules of brain. …. I
hope to bequeath to the generations
a calculus of the nervous system.”

69 Years Ago…

Bio-inspiration
• Can massively-parallel computing resources accelerate our
understanding of brain function?
• Can our growing understanding of brain function point the way to
more efficient parallel, fault-tolerant computation?

Convolutional Neural Nets
• Dense convolution kernels
• Abstract neurons
• Only feed-forward connections
• Trained through backpropagation

Neo Cortex
• Spiking neurons
• Two-dimensional structure
• Sparse connectivity
Feed-forward
input
Feedback input
Feed-forward
output
Feedback output

CNNs & GPUs
• Dense matrix multiplications
• 3.2kW
• Programmable via mature APIs

Cortical Models & Supercomputers
• Sparse matrix operations, communication via spikes
• MPI parallelisation for synchronised transfer of large data
• 2.3MW

Cortical Models – Neuromorphic HW
• Memory local to computation
• Low-power - 62mW
• Efficient spike communication
• Real time

SpiNNaker Project
• A million mobile phone
processors in one
computer
• Able to model about 1%
of the human brain…
• …or 10 mice!

SpiNNaker Chip
Multi-chip
packaging by
UNISEM Europe

Neural Modelling
• Each core simulates a group of neurons
• Communicate via action potentials: ’spikes’
• Received at synapses via neurotransmitters
• Neuron state dynamics captured in
mathematical models
• Hodgkin and Huxley (1952)
• Leaky Integrate and Fire

Realtime Neural Simulation
Neurons simulated in software
• Leaky integrate and fire neuron
!"
!#
= −
" & '()*+, #
-+
, if 𝑉 > 𝑉2, 𝑉 = 𝑉456#
𝑑𝐼69:
𝑑𝑡
= −
𝐼69:
𝜏69:
+ 𝛿 𝑡 − 𝑡?
• Run in realtime
• Simulation processing matches
biological time
• Spikes transmitted as AER packets
𝑉(𝑡)
𝐼69:(𝑡)

SpiNNaker Chip

Neural Network Modelling

SpiNNaker Router
• Hardware router on each
node
• Packets have a routing key
• Router has a look-up table
of {key, mask, data}
triplets
• If address matches a key-
mask pair, the associated
data tells router what to
do with the packet

Neural Simulation

Neural Simulation

Spike Transmission

Spike Transmission

Spike Transmission

Spike Transmission

SpiNNaker Machines
SpiNNaker racks
(1M ARM cores)
• HBP platform
– 1M cores
– 11 cabinets (including server)
• Launched 2 Nov 2018
– 78 remote users
– >5k SpiNNaker jobs run
SpiNNaker chip
(18 ARM cores)
SpiNNaker board
(864 ARM cores)

Million Core Machine

https://collab.humanbrainproject.eu/

Programming

Model Building
PyNN Script

Host-Based Pre-Processing
PyNN Script
sPyNNaker
SpiNNFront
EndCommon
Synaptic Matrix
Neuron
Parameters
Routing Tables
Neural
Application
Data
Generation
Assign Compiled
Instruction Code
Partitioning
Mapping
Load Data & Run

Runtime Execution
Asynchronous execution
Event-based operation
No global memory
Neural applications:
• Periodic neuron state updates
• Pipelined spike processing
Rhodes, O., et al. sPyNNaker: A Software Package for Running PyNN Simulations on SpiNNaker. Frontiers in Neuroscience. 2018

Cortical Microcolumn
30
1st full-scale simulation of 1mm2 cortex on neuromorphic & HPC systems
• 77,169 neurons, 285M synapses, 2,901 cores
S.J. van Albada, A.G. Rowley, A. Stokes, J. Senk, M. Hopkins, M. Schmidt, D.R. Lester, M. Diesmann, S.B. Furber, “Performance comparison of the digital
neuromorphic hardware SpiNNaker and the Neural network simulation software NEST for a full-scale cortical microcircuit model”, Frontiers 2018.

Computational Neuroscience
Celada, P., et al. Serotonin modulation of cortical neurons and networks. Frontiers in Neuroscience. 2013
• Serotonin modulates Pre Frontal Cortex
• Neurons express range of serotonin
receptors
• Respond at different timescales
• Dorsal Raphe Nucleus stimulation
modulates brain rhythms
• Releases serotonin
• Computational model to simulation neural
circuitry
• Monitor local effect
• Understand global effect on connected brain
regions

Computational Neuroscience
Joshi, A., & Rhodes, O,. et al. Serotonergic modulation of cortical columnar dynamics: A
large-scale neuronal network simulation study using SpiNNaker. In prep.
• Serotonin modulates Pre Frontal Cortex
• Neurons express range of serotonin
receptors
• Respond at different timescales
• Dorsal Raphe Nucleus stimulation
modulates brain rhythms
• Releases serotonin
• Computational model to simulate
serotonergic modulation
• Monitor local effects – firing rates
• Understand global effect on connected brain
regions – oscillation in local field potential

Biological Decision Making
• Basal Ganglia – biological decision
making and action selection
• Single channel model inspired by
biology: neuron dynamics; numbers;
and topology
• Dopamine is central to network
function
• Expressed via two receptor types
Sen-Bhattacharya, B., et al. Building a Spiking Neural Network Model of the Basal Ganglia on
SpiNNaker. IEEE Transaction on Cognitive and Developmental Systems. 2018

Biological Decision Making
Sen-Bhattacharya, B., et al. Building a Spiking Neural Network Model of the Basal Ganglia on
SpiNNaker. IEEE Transaction on Cognitive and Developmental Systems. 2018
• Basal Ganglia – biological decision
making and action selection
• Single channel model inspired by
biology: neuron dynamics; numbers;
and topology
• Dopamine is central to network
function
• Expressed via two receptor types
• Explore how modulation relates to
scale and disease

Synaptic Rewiring
Topographic map formation
• Synapse creation/deletion:
• Source activity
• Proximity
• Weight
• Weights updated via STDP
• SpiNNaker challenges:
• Spike routing via new synapses
• Real-time execution
Petrut A. Bogdan, Andrew G. D. Rowley, Oliver Rhodes and Steve B. Furber. “Structural Plasticity on
the SpiNNaker Many-Core Neuromorphic System”. Frontiers 2018

Synaptic Rewiring

Constraint Satisfaction Problems
G. A. Fonseca Guerra and S. B. Furber,
“Using Stochastic Spiking Neural
Networks on SpiNNaker to Solve
Constraint Satisfaction Problems”,
Frontiers 2018.
S. Habenschuss, Z. Jonke, and
W. Maass, “Stochastic computations in
cortical microcircuit models”, PLOS
Computational Biology,
9(11):e1003311, 2013.
Stochastic spiking neural
network:
• Solves CSPs, e.g. Sudoku
• 37k neurons
• 86M synapses
• Also
• Map colouring
• Ising spin systems

Neuroprosthetics
Behrenbeck, J. et al. Classication and Regression of Spatio-Temporal Signals using
NeuCube and its realization on SpiNNaker Neuromorphic Hardware. Journal of
Neural Engineering. 2018
• Classification of electrical signals
• Realtime control of active prosthetics
• Low power
• Record electrical activity of participants
during prescribed hand movements
• Classification with reservoir of spiking
neurons
• Encode signals into spikes
• Train network (unsupervised)
• Readout to classify

Neuroprosthetics
Behrenbeck, J. et al. Classication and Regression of Spatio-Temporal Signals using
NeuCube and its realization on SpiNNaker Neuromorphic Hardware. Journal of
Neural Engineering. 2018
• Classification of electrical signals
• Realtime control of active prosthetics
• Low power
• Record electrical activity of participants
during prescribed hand movements
• Classification with reservoir of spiking
neurons
• Encode signals into spikes
• Train network (unsupervised)
• Readout to classify

Neurorobotics
Exploring vestibulo-ocular adaptation in a closed-loop neuro-robotic experiment using
STDP. A simulation study. Francisco Naveros, Jesús A. Garrido, Angelo Arleo, Eduardo
Ros, Niceto R. Luque.
• Study vestibular ocular reflex in
iCub robot
• SpiNNaker as neural substrate
• Learn control via cerebellum
inspired spiking neural network
• Range of learning kernels based on
relative spike timing + error

Neurorobotics
• Study vestibular ocular reflex in
iCub robot
• SpiNNaker as neural substrate
• Learn control via cerebellum
inspired spiking neural network
• Range of learning kernels based on
relative spike timing + error
• Research embodiment of neural
control systems
Bartolozzi, C., et al. A Cerebellum Inspired Vestibular Occular Reflex in and iCub Robot
with SpiNNaker as the Neural Substrate. In Prep

Conclusions
The SpiNNaker platform:
• 20 years in conception, 10 years in construction
• 1 Million cores, all interconnected
• Realtime SNN simulation
• Active research applications in Robotics, computational
neuroscience, and theoretical neuroscience
Industrial AI uses 2nd Gen. ANN.
• Expectation is that neuromorphics will contribute –
especially energy efficiency
• SpiNNaker is the ideal research platform to explore this
research

SpiNNaker2 Objective
Approach: Neuromorphic Many Core System
• Processor based à flexibility
• Fixed digital functionality as accelerators à performance
• Low voltage (near threshold) operation enabled by 22FDX technology and adaptive body biasing
(ABB) à energy efficiency
• Event driven operation with fine-grained dynamic power management and energy proportional
chip-2-chip links à workload adaptivity
Scaling Target:
• >x10 capacity compared to SpiNNaker1
• Enabled by new hardware features and
modern CMOS process

SpiNNaker2 Chip Overview
LPDDR4 PoP
• 160 ARM-based processing elements (PEs)
• 8 GByte LPDDR4 DRAM
• 7 energy efficient chip-to-chip links

SpiNNaker2 Hardware Innovations
exp exp
ARM
exp
unitFIFO
AHB
Feature Output/Benefit Patents Publications
Dynamic Neuromorphic Power
Management
Reduce power consumption by up to x5
Examples:
• Reward-based learning
• Synfire chain
• Event based vision sensor processing
DE102017128711.6 [Höppner2017]
Exponential Function Accelerator Enhance performance by x2
Examples:
• Reward-based learning
• STDP
• BCPNN
[Partzsch2017]
[Bauer2017]
[Mikaitis2018]
Pseudo and true random number
generation
Enhance performance by >x2 for
pseudo random numbers
Enable true randomness
Examples:
• Synaptic sampling
• Deep-rewiring
EP3147775A1
US20170083289
[Neumärker2017]