The document describes CARLSIM 3, a GPU-accelerated simulator for spiking neural networks (SNN) that captures key aspects of neural function and dynamics with a focus on real-time applications in robotics. It emphasizes its user-friendly interface, scalability, and various synaptic models for simulating biological processes, ultimately showcasing its applications in visual navigation and obstacle avoidance for robots. The software is freely available and optimized for NVIDIA GPUs, making it a valuable tool for theoretical neuroscience and neuromorphic engineering.

1. Cognitive Anteater Robotics Lab (CARL)
University of California, Irvine, CA, USA
Michael Beyeler
CARLsim 3
Concepts, Tools, and Applications

2. December 1, 2016 2
 Brain architecture =/= conventional
computer architecture
 Massive parallelism (1011 neurons)
 Massive connectivity (1015 synapses)
 Excellent power-efficiency
– ~ 20W for 1016 flops
 Probabilistic responses and fault-
tolerant
 Autonomous, on-line learning
 Low-performance components
(~100 Hz)
 Low-speed comm. (~meters/sec)
 Low-precision synaptic connections
Brain Computations
http://www.socsci.uci.edu/~jkrichma/CARLsim

3. December 1, 2016 http://www.socsci.uci.edu/~jkrichma/CARLsim 3
 Spiking Neural Network (SNN) models describe
key aspects of neural function and network
dynamics
– abstract away many molecular and cellular details
– able to capture neuronal, synaptic, and network
dynamics
– can incorporate on-line learning that depends on
millisecond time resolution
 SNN models are composed of:
– spiking point-neurons for computation
• Izhikevich spiking neurons: 20 different neuron types
– dynamic synapses for learning and memory storage
• Synaptic receptors for AMPA, NMDA, and GABA
– variable-delay axons for communication
– neuromodulatory systems to control action selection
and learning
Spiking Neural Networks
Izhikevich, 2003, 2004

4. December 1, 2016 http://www.socsci.uci.edu/~jkrichma/CARLsim 4
Constructing Functional SNN Models
CARLsim 3
neural circuits:
create
tune/optimize
explore
Theoretical
model
Experimental
data
Functional
application
Theoretical
neuroscience
Real-time
applications
Neuromorphic
engineering

5.  GPU-accelerated spiking neural network simulator
 User-friendly, well documented.
– Runs on Linux, Mac OS X, Windows systems with CUDA SDK.
 Scalable, extendable.
 PyNN-like interface.
 Highly optimized for NVIDIA GPUs.
 Capable of simulating biological detailed neural
models.
 Freely available at:
– http://www.socsci.uci.edu/~jkrichma/CARLsim/
December 1, 2016 http://www.socsci.uci.edu/~jkrichma/CARLsim 5
CARLsim – in a Nutshell

6.  Provides a PyNN-like user interface
 Lets you configure networks, apply input
stimuli, and monitor network activity
December 1, 2016 http://www.socsci.uci.edu/~jkrichma/CARLsim 6
CARLsim API

7. December 1, 2016 http://www.socsci.uci.edu/~jkrichma/CARLsim 7
 Easily create complex network
topographies:
– Pre-defined connection types
• “full”, “random”, “one-to-one”, “gaussian”
– Custom connection types
• arbitrary connectivity
– 1D/2D/3D topography:
Connectivity
• Neurons organized
on a grid
• 1D/2D/3D receptive
fields
• arbitrary topographic
connections

8. December 1, 2016 http://www.socsci.uci.edu/~jkrichma/CARLsim 8
 Short-term plasticity
(STP)
 Homeostatic synaptic
scaling
 Spike-timing dependent
plasticity (STDP)
 Dopamine-modulated
STDP
Synaptic Plasticity

9.  Versatile toolbox for the visualization and analysis of
neuronal, synaptic, and network information.
 Generates raster plots, histograms, heat maps, flow fields.
 Plot, record, load, save.
December 1, 2016 http://www.socsci.uci.edu/~jkrichma/CARLsim 9
MATLAB Offline Analysis Toolbox

10. December 1, 2016 http://www.socsci.uci.edu/~jkrichma/CARLsim 10
 capable of quickly and
efficiently tuning large-scale
SNNs
– arbitrary fitness function
– multi-objective optimization
 utilizes Evolutionary
Computations in Java (ECJ)
 fully integrated with CARLsim
 transparent use of a GPU
– up to 60x speedups compared to
single-threaded CPU simulation
 Emily will tell you all about it.
Automated Parameter Tuning Interface

11.  Benchmark: 80-20 network with E-STDP (Vogels & Abbott,
2005)
 CPU: Intel Core i7 CPU 920 @ 2.67 GHz
 GPU: NVIDIA GTX 780 (3 GB of GDDR5, 2304 CUDA cores)
December 1, 2016 http://www.socsci.uci.edu/~jkrichma/CARLsim 11
Performance Benchmarks

12.  PyNN-like user interface
December 1, 2016 http://www.socsci.uci.edu/~jkrichma/CARLsim 12
CARLsim 3 Code Sample

13. Neuron
model
Synapse model Synaptic plasticity Input Tools
Integration
methods
Front-
ends
Back-ends Platforms
Leakyintegrate-and-fire(LIF)
Izhikevich4-param
Hodgkin-Huxley
Current-bsaed(CUBA)
Conductance-based(COBA)
AMPA,NMDA,GABA
Neuromodulation
Short-termplasticity(STP)
E-STDP
I-STDP
DA-STDP
Synapticscaling/homeostasis
Currentinjection
Spikeinjection
Parametertuning
Analysisandvisualization
Regressionsuite
Forward/exponentialEuler
Exactintegration
Runge-Kutta
Python/PyNN
C/C++
Java
Single-threaded
Multi-threaded
distributed
SingleGPU
Multi-GPU
Linux
MacOSX
Windows
CARLsim X X X X / X X X X X X X X X X X X X X / X X X
Brian X X X X X X / X X X / X X X / / X X X X X X X X / X X X
GeNN X X X X X / / X / X / X X X X X X X
NCS X X X X X / X X X X / X X X X X X X X X
NeMo X X / X X X X X X X X X X X X X X X X
Nengo X X X X X X X X / X X X X X X X X X X X X X X
NEST X X X X X X / X X X X X X X X X X X X X X X X
PCSIM X X X X X X / X X X X X X / X X X / X X X X X X /
December 1, 2016 http://www.socsci.uci.edu/~jkrichma/CARLsim 13
Comparison of SNN Simulators

14.  Visual Cortical Processing
– M. Beyeler, N. Oros, N. Dutt, and J.L. Krichmar (2015). A GPU-accelerated cortical neural network
model for visually guided robot navigation. Neural Networks (in press).
– M. Beyeler, M. Richert, N.D. Dutt, and J.L. Krichmar (2014). Efficient spiking neural network model of
pattern motion selectivity in visual cortex. Neuroinformatics 12, 435-454.
– M. Richert, J.M. Nageswaran, N. Dutt, and J.L. Krichmar (2011). An efficient simulation environment
for modeling large-scale cortical processing. Frontiers in Neuroinformatics 5.
 Tactile Sensory Processing
– L.D. Bucci, T.S. Chou, and J.L. Krichmar (2014). Tactile Sensory Decoding in a Neuromorphic
Interactive Robot. IEEE Conference on Robotics & Automation (ICRA).
 Neuromodulation, Attention, and Working Memory
– M.C. Avery, N. Dutt, and J.L. Krichmar (2014). Mechanisms underlying the basal forebrain
enhancement of top-down and bottom-up attention. European Journal of Neuroscience 39.
– M. Avery, N. Dutt, and J.L. Krichmar (2013). A large-scale neural network model of the influence of
neuromodulatory levels on working memory and behavior. Frontiers in Computational Neuroscience.
 Object Categorization and Plasticity
– M. Beyeler, N.D. Dutt, and J.L. Krichmar (2013). Categorization and decision-making in a
neurobiologically plausible spiking network using a STDP-like learning rule. Neural Networks 48.
– K.D. Carlson, M. Richert, N. Dutt, and J.L. Krichmar (2013). Biologically Plausible Models of
Homeostasis and STDP: Stability and Learning in Spiking Neural Networks. Paper presented at:
International Joint Conference on Neural Networks
December 1, 2016 http://www.socsci.uci.edu/~jkrichma/CARLsim 14
Recent CARLsim Models

15. December 1, 2016 15
 Visually guided robot
navigation:
– SNN model of motion
processing in cortical areas
V1 and MT
– 80k neurons and 4 million
synapses
– controlled an autonomous
robot in real-time
– replicated human behavioral
paths for obstacle avoidance
(Fajen & Warren, 2003)
CARLsim in Real-Time Applications
(Beyeler et al., 2015)
http://www.socsci.uci.edu/~jkrichma/CARLsim

16. December 1, 2016 16
 Primary visual cortex (V1)
– tuned to simple attributes of shape,
motion, color, texture, depth
 Middle temporal (MT) area
– tuned to coherent local motion
(retinal flow)
 Posterior parietal cortex (PPC)
– Polysensory areas (VIP, MST, 7a, etc.)
– Tuned to global, complex motion
– Self-motion and object motion
– Spatial reference frames
– Path integration (?)
Visual Motion Pathway
(Britten, 2008)
http://www.socsci.uci.edu/~jkrichma/CARLsim

17. December 1, 2016 17
 Problem: Local-velocity sample
is different from object velocity
 Goal: Disambiguate local-
velocity samples and integrate
them into an accurate estimate
of the global (object) velocity
 Intersection-of-constraints
(IOC):
– Each local velocity sample constrains
the global object velocity
– Find object velocity by integrating
local samples
– There is evidence that MT firing rates
represent the velocity of moving
objects using IOC
Aperture Problem
(Bradley & Goyal, 2008)
http://www.socsci.uci.edu/~jkrichma/CARLsim

18.  Motion is an orientation in space-time
 V1 simple cells: space-time oriented
receptive fields
 Spatiotemporal energy model:
– Linear filtering / motion energy / opponent
energy
– Adelson & Bergen (1985), Simoncelli & Heeger
(1998)
December 1, 2016 18
Primary Visual Cortex (V1)
(Simoncelli & Heeger, 1998)
(Bradley & Goyal, 2008)
http://www.socsci.uci.edu/~jkrichma/CARLsim

19. December 1, 2016 19
Middle Temporal Area (MT)
Excitation
Feedforward and local
Specific Inhibition
Cross-direction inhibition
Unspecific Inhibition
Divisive normalization
MTCDSMTPDS
http://www.socsci.uci.edu/~jkrichma/CARLsim
(Beyeler et al., 2015)

20. Model Response to Motion Patterns
V1 MT CDS MT PDS
(Beyeler et al., 2014)(CDS: component-direction-selective, PDS: pattern-direction-selective)
December 1, 2016 http://www.socsci.uci.edu/~jkrichma/CARLsim 20

21. December 1, 2016 21
 Simple control law for obstacle
avoidance
– used by honeybees for visual
control of flight (Srinivasan &
Zhang, 1997)
– try to balance flow magnitude in
left and right hemisphere
– turn away from the side with
greater flow magnitude
– Humans might make use of similar
strategy (Kountouriotis et al.,
2013)
Posterior Parietal Cortex (PPC)
𝒘 𝑳 𝒘 𝑹
Δ 𝐹𝐿 − 𝐹𝑅 =
𝑤 𝐿 − 𝑤 𝑅
𝑤 𝐿 + 𝑤 𝑅
http://www.socsci.uci.edu/~jkrichma/CARLsim

22. Introducing Le Carl: The French Robot
December 1, 2016 22
Android Based Robotics
http://www.socsci.uci.edu/~jkrichma/CARLsim

23.  Server / client communication
 ABR server:
– monitors video / sensory information of ABR clients
– remotely starts / stops camera, sensors, GPS, and IOIO
(TCP)
– hosts spiking neural network model
December 1, 2016 23
Android Based Robotics (ABR)
(Oros & Krichmar, 2014)
http://www.socsci.uci.edu/~jkrichma/CARLsim

24. December 1, 2016 24
Technical Setup and Typical Workflow
(Beyeler et al., 2015)
http://www.socsci.uci.edu/~jkrichma/CARLsim

25. December 1, 2016 25
Simulated Neuronal Responses During Visual Navigation
(Beyeler et al., 2015)
http://www.socsci.uci.edu/~jkrichma/CARLsim

26. December 1, 2016 26
Behavioral Results
http://www.socsci.uci.edu/~jkrichma/CARLsim

27. Comparison to Psychophysical Data
December 1, 2016 http://www.socsci.uci.edu/~jkrichma/CARLsim 27

28.  CARLsim highlights:
– Runs on Linux, Mac OS X, Windows.
– Highly optimized for NVIDIA GPUs: CUDA ≥ 5.0, device
capability ≥2.0
– User-friendly, scalable, extendable
 Download: www.socsci.uci.edu/~jkrichma/CARLsim
 Presented a large-scale spiking neural network that
– is biologically inspired
– solves the aperture problem via cortical mechanisms
– is integrated with a real-time, real-world robotics platform
 Android Based Robotics combined with CARLsim is
the first step toward a complete robot navigation
system.
December 1, 2016 http://www.socsci.uci.edu/~jkrichma/CARLsim 28
Conclusions

29. December 1, 2016 http://www.socsci.uci.edu/~jkrichma/CARLsim 29
Team CARL (2015)
Front row – Ian Schweer, Emily Rounds, Jeff Krichmar, Alexis Craig, Sean Campbell
Back row – Nik Dutt, Georgina Lean, Ting-Shuo Chou, Kris Carlson, Tiffany Hwu, Michael Beyeler
Supported by the National Science Foundation and Qualcomm Technologies Incorporated.

30. December 1, 2016 http://www.socsci.uci.edu/~jkrichma/CARLsim 30