PowerPoint slides from a 2015 Guest Lecture in PSYCH-268A: Computational Neuroscience, Prof. Jeff Krichmar, University of California, Irvine (UCI). Corresponding publication: Beyeler*, M., Carlson*, K. D. , Chou*, T-S., Dutt, N., Krichmar, J. L. (2015). CARLsim 3: A user-friendly and highly optimized library for the creation of neurobiologically detailed spiking neural networks. Proceedings of IEEE International Joint Conference on Neural Networks (IJCNN), Killarney, Ireland. (*equal contribution)

1. Cognitive Anteater Robotics Lab (CARL)
University of California, Irvine, CA, USA
Michael Beyeler
CARLsim 3
Concepts, Tools, and Applications

2. December 1, 2016 2
 Brain architecture =/= conventional
computer architecture
 Massive parallelism (1011 neurons)
 Massive connectivity (1015 synapses)
 Excellent power-efficiency
– ~ 20W for 1016 flops
 Probabilistic responses and fault-
tolerant
 Autonomous, on-line learning
 Low-performance components
(~100 Hz)
 Low-speed comm. (~meters/sec)
 Low-precision synaptic connections
Brain Computations
http://www.socsci.uci.edu/~jkrichma/CARLsim

3. December 1, 2016 http://www.socsci.uci.edu/~jkrichma/CARLsim 3
 Spiking Neural Network (SNN) models describe
key aspects of neural function and network
dynamics
– abstract away many molecular and cellular details
– able to capture neuronal, synaptic, and network
dynamics
– can incorporate on-line learning that depends on
millisecond time resolution
 SNN models are composed of:
– spiking point-neurons for computation
• Izhikevich spiking neurons: 20 different neuron types
– dynamic synapses for learning and memory storage
• Synaptic receptors for AMPA, NMDA, and GABA
– variable-delay axons for communication
– neuromodulatory systems to control action selection
and learning
Spiking Neural Networks
Izhikevich, 2003, 2004

4. December 1, 2016 http://www.socsci.uci.edu/~jkrichma/CARLsim 4
Constructing Functional SNN Models
CARLsim 3
neural circuits:
create
tune/optimize
explore
Theoretical
model
Experimental
data
Functional
application
Theoretical
neuroscience
Real-time
applications
Neuromorphic
engineering

5.  GPU-accelerated spiking neural network simulator
 User-friendly, well documented.
– Runs on Linux, Mac OS X, Windows systems with CUDA SDK.
 Scalable, extendable.
 PyNN-like interface.
 Highly optimized for NVIDIA GPUs.
 Capable of simulating biological detailed neural
models.
 Freely available at:
– http://www.socsci.uci.edu/~jkrichma/CARLsim/
December 1, 2016 http://www.socsci.uci.edu/~jkrichma/CARLsim 5
CARLsim – in a Nutshell

6.  Provides a PyNN-like user interface
 Lets you configure networks, apply input
stimuli, and monitor network activity
December 1, 2016 http://www.socsci.uci.edu/~jkrichma/CARLsim 6
CARLsim API

7. December 1, 2016 http://www.socsci.uci.edu/~jkrichma/CARLsim 7
 Easily create complex network
topographies:
– Pre-defined connection types
• “full”, “random”, “one-to-one”, “gaussian”
– Custom connection types
• arbitrary connectivity
– 1D/2D/3D topography:
Connectivity
• Neurons organized
on a grid
• 1D/2D/3D receptive
fields
• arbitrary topographic
connections

8. December 1, 2016 http://www.socsci.uci.edu/~jkrichma/CARLsim 8
 Short-term plasticity
(STP)
 Homeostatic synaptic
scaling
 Spike-timing dependent
plasticity (STDP)
 Dopamine-modulated
STDP
Synaptic Plasticity

9.  Versatile toolbox for the visualization and analysis of
neuronal, synaptic, and network information.
 Generates raster plots, histograms, heat maps, flow fields.
 Plot, record, load, save.
December 1, 2016 http://www.socsci.uci.edu/~jkrichma/CARLsim 9
MATLAB Offline Analysis Toolbox

10. December 1, 2016 http://www.socsci.uci.edu/~jkrichma/CARLsim 10
 capable of quickly and
efficiently tuning large-scale
SNNs
– arbitrary fitness function
– multi-objective optimization
 utilizes Evolutionary
Computations in Java (ECJ)
 fully integrated with CARLsim
 transparent use of a GPU
– up to 60x speedups compared to
single-threaded CPU simulation
 Emily will tell you all about it.
Automated Parameter Tuning Interface

11.  Benchmark: 80-20 network with E-STDP (Vogels & Abbott,
2005)
 CPU: Intel Core i7 CPU 920 @ 2.67 GHz
 GPU: NVIDIA GTX 780 (3 GB of GDDR5, 2304 CUDA cores)
December 1, 2016 http://www.socsci.uci.edu/~jkrichma/CARLsim 11
Performance Benchmarks

12.  PyNN-like user interface
December 1, 2016 http://www.socsci.uci.edu/~jkrichma/CARLsim 12
CARLsim 3 Code Sample

13. Neuron
model
Synapse model Synaptic plasticity Input Tools
Integration
methods
Front-
ends
Back-ends Platforms
Leakyintegrate-and-fire(LIF)
Izhikevich4-param
Hodgkin-Huxley
Current-bsaed(CUBA)
Conductance-based(COBA)
AMPA,NMDA,GABA
Neuromodulation
Short-termplasticity(STP)
E-STDP
I-STDP
DA-STDP
Synapticscaling/homeostasis
Currentinjection
Spikeinjection
Parametertuning
Analysisandvisualization
Regressionsuite
Forward/exponentialEuler
Exactintegration
Runge-Kutta
Python/PyNN
C/C++
Java
Single-threaded
Multi-threaded
distributed
SingleGPU
Multi-GPU
Linux
MacOSX
Windows
CARLsim X X X X / X X X X X X X X X X X X X X / X X X
Brian X X X X X X / X X X / X X X / / X X X X X X X X / X X X
GeNN X X X X X / / X / X / X X X X X X X
NCS X X X X X / X X X X / X X X X X X X X X
NeMo X X / X X X X X X X X X X X X X X X X
Nengo X X X X X X X X / X X X X X X X X X X X X X X
NEST X X X X X X / X X X X X X X X X X X X X X X X
PCSIM X X X X X X / X X X X X X / X X X / X X X X X X /
December 1, 2016 http://www.socsci.uci.edu/~jkrichma/CARLsim 13
Comparison of SNN Simulators

14.  Visual Cortical Processing
– M. Beyeler, N. Oros, N. Dutt, and J.L. Krichmar (2015). A GPU-accelerated cortical neural network
model for visually guided robot navigation. Neural Networks (in press).
– M. Beyeler, M. Richert, N.D. Dutt, and J.L. Krichmar (2014). Efficient spiking neural network model of
pattern motion selectivity in visual cortex. Neuroinformatics 12, 435-454.
– M. Richert, J.M. Nageswaran, N. Dutt, and J.L. Krichmar (2011). An efficient simulation environment
for modeling large-scale cortical processing. Frontiers in Neuroinformatics 5.
 Tactile Sensory Processing
– L.D. Bucci, T.S. Chou, and J.L. Krichmar (2014). Tactile Sensory Decoding in a Neuromorphic
Interactive Robot. IEEE Conference on Robotics & Automation (ICRA).
 Neuromodulation, Attention, and Working Memory
– M.C. Avery, N. Dutt, and J.L. Krichmar (2014). Mechanisms underlying the basal forebrain
enhancement of top-down and bottom-up attention. European Journal of Neuroscience 39.
– M. Avery, N. Dutt, and J.L. Krichmar (2013). A large-scale neural network model of the influence of
neuromodulatory levels on working memory and behavior. Frontiers in Computational Neuroscience.
 Object Categorization and Plasticity
– M. Beyeler, N.D. Dutt, and J.L. Krichmar (2013). Categorization and decision-making in a
neurobiologically plausible spiking network using a STDP-like learning rule. Neural Networks 48.
– K.D. Carlson, M. Richert, N. Dutt, and J.L. Krichmar (2013). Biologically Plausible Models of
Homeostasis and STDP: Stability and Learning in Spiking Neural Networks. Paper presented at:
International Joint Conference on Neural Networks
December 1, 2016 http://www.socsci.uci.edu/~jkrichma/CARLsim 14
Recent CARLsim Models

15. December 1, 2016 15
 Visually guided robot
navigation:
– SNN model of motion
processing in cortical areas
V1 and MT
– 80k neurons and 4 million
synapses
– controlled an autonomous
robot in real-time
– replicated human behavioral
paths for obstacle avoidance
(Fajen & Warren, 2003)
CARLsim in Real-Time Applications
(Beyeler et al., 2015)
http://www.socsci.uci.edu/~jkrichma/CARLsim

16. December 1, 2016 16
 Primary visual cortex (V1)
– tuned to simple attributes of shape,
motion, color, texture, depth
 Middle temporal (MT) area
– tuned to coherent local motion
(retinal flow)
 Posterior parietal cortex (PPC)
– Polysensory areas (VIP, MST, 7a, etc.)
– Tuned to global, complex motion
– Self-motion and object motion
– Spatial reference frames
– Path integration (?)
Visual Motion Pathway
(Britten, 2008)
http://www.socsci.uci.edu/~jkrichma/CARLsim

17. December 1, 2016 17
 Problem: Local-velocity sample
is different from object velocity
 Goal: Disambiguate local-
velocity samples and integrate
them into an accurate estimate
of the global (object) velocity
 Intersection-of-constraints
(IOC):
– Each local velocity sample constrains
the global object velocity
– Find object velocity by integrating
local samples
– There is evidence that MT firing rates
represent the velocity of moving
objects using IOC
Aperture Problem
(Bradley & Goyal, 2008)
http://www.socsci.uci.edu/~jkrichma/CARLsim

18.  Motion is an orientation in space-time
 V1 simple cells: space-time oriented
receptive fields
 Spatiotemporal energy model:
– Linear filtering / motion energy / opponent
energy
– Adelson & Bergen (1985), Simoncelli & Heeger
(1998)
December 1, 2016 18
Primary Visual Cortex (V1)
(Simoncelli & Heeger, 1998)
(Bradley & Goyal, 2008)
http://www.socsci.uci.edu/~jkrichma/CARLsim

19. December 1, 2016 19
Middle Temporal Area (MT)
Excitation
Feedforward and local
Specific Inhibition
Cross-direction inhibition
Unspecific Inhibition
Divisive normalization
MTCDSMTPDS
http://www.socsci.uci.edu/~jkrichma/CARLsim
(Beyeler et al., 2015)

20. Model Response to Motion Patterns
V1 MT CDS MT PDS
(Beyeler et al., 2014)(CDS: component-direction-selective, PDS: pattern-direction-selective)
December 1, 2016 http://www.socsci.uci.edu/~jkrichma/CARLsim 20

21. December 1, 2016 21
 Simple control law for obstacle
avoidance
– used by honeybees for visual
control of flight (Srinivasan &
Zhang, 1997)
– try to balance flow magnitude in
left and right hemisphere
– turn away from the side with
greater flow magnitude
– Humans might make use of similar
strategy (Kountouriotis et al.,
2013)
Posterior Parietal Cortex (PPC)
𝒘 𝑳 𝒘 𝑹
Δ 𝐹𝐿 − 𝐹𝑅 =
𝑤 𝐿 − 𝑤 𝑅
𝑤 𝐿 + 𝑤 𝑅
http://www.socsci.uci.edu/~jkrichma/CARLsim

22. Introducing Le Carl: The French Robot
December 1, 2016 22
Android Based Robotics
http://www.socsci.uci.edu/~jkrichma/CARLsim

23.  Server / client communication
 ABR server:
– monitors video / sensory information of ABR clients
– remotely starts / stops camera, sensors, GPS, and IOIO
(TCP)
– hosts spiking neural network model
December 1, 2016 23
Android Based Robotics (ABR)
(Oros & Krichmar, 2014)
http://www.socsci.uci.edu/~jkrichma/CARLsim

24. December 1, 2016 24
Technical Setup and Typical Workflow
(Beyeler et al., 2015)
http://www.socsci.uci.edu/~jkrichma/CARLsim

25. December 1, 2016 25
Simulated Neuronal Responses During Visual Navigation
(Beyeler et al., 2015)
http://www.socsci.uci.edu/~jkrichma/CARLsim

26. December 1, 2016 26
Behavioral Results
http://www.socsci.uci.edu/~jkrichma/CARLsim

27. Comparison to Psychophysical Data
December 1, 2016 http://www.socsci.uci.edu/~jkrichma/CARLsim 27

28.  CARLsim highlights:
– Runs on Linux, Mac OS X, Windows.
– Highly optimized for NVIDIA GPUs: CUDA ≥ 5.0, device
capability ≥2.0
– User-friendly, scalable, extendable
 Download: www.socsci.uci.edu/~jkrichma/CARLsim
 Presented a large-scale spiking neural network that
– is biologically inspired
– solves the aperture problem via cortical mechanisms
– is integrated with a real-time, real-world robotics platform
 Android Based Robotics combined with CARLsim is
the first step toward a complete robot navigation
system.
December 1, 2016 http://www.socsci.uci.edu/~jkrichma/CARLsim 28
Conclusions

29. December 1, 2016 http://www.socsci.uci.edu/~jkrichma/CARLsim 29
Team CARL (2015)
Front row – Ian Schweer, Emily Rounds, Jeff Krichmar, Alexis Craig, Sean Campbell
Back row – Nik Dutt, Georgina Lean, Ting-Shuo Chou, Kris Carlson, Tiffany Hwu, Michael Beyeler
Supported by the National Science Foundation and Qualcomm Technologies Incorporated.

30. December 1, 2016 http://www.socsci.uci.edu/~jkrichma/CARLsim 30