This document summarizes a seminar on the similarities and differences between artificial and natural neural networks. It discusses how artificial networks like GoogleNet have achieved superhuman performance at games like Go. It also discusses neuromorphic computing projects like IBM's TrueNorth and DARPA's SyNAPSE. The document outlines two main differences between artificial and natural networks: structure, with natural networks having more redundancy, and dynamics, with natural networks evolving versus artificial networks being designed. It then discusses challenges to whole brain simulation projects, like not fully understanding small brains like C. elegans. The document concludes by discussing the history and future of connecting artificial and natural networks through techniques like brain simulations and the human dynamic clamp.

1. Whole Brain Simulations and
the Discrepancy/Similarity between
Artificial & Natural Neural Networks
1st Deep Learning Club Seminar
Tuesday, 11th October 2016
Guillaume Dumas, Human Genetics & Cognitive Functions

2. Introduction
2 • Guillaume Dumas • Whole Brain Simulations and the Discrepancy/Similarity between Artificial & Natural Neural Networks • 2016/10/11

3. 3 • Guillaume Dumas • Whole Brain Simulations and the Discrepancy/Similarity between Artificial & Natural Neural Networks • 2016/10/11
GoogLeNet, a 22 layers deep network

4. 4 • Guillaume Dumas • Whole Brain Simulations and the Discrepancy/Similarity between Artificial & Natural Neural Networks • 2016/10/11
“It’s not a human move.
I’ve never seen a human play this move.
So beautiful.”
Fan Hui, Go European champion

5. 5 • Guillaume Dumas • Whole Brain Simulations and the Discrepancy/Similarity between Artificial & Natural Neural Networks • 2016/10/11
IBM Neuromorphic Computer TrueNorth
DARPA SyNAPSE Program Plan

6. 6 • Guillaume Dumas • Whole Brain Simulations and the Discrepancy/Similarity between Artificial & Natural Neural Networks • 2016/10/11

7. 7 • Guillaume Dumas • Whole Brain Simulations and the Discrepancy/Similarity between Artificial & Natural Neural Networks • 2016/10/11
“a nerve cell is more than a single basic
active organ (…) Thus, all the complexities
referred to here may be irrelevant, but they may
also endow the system with a analog
character, or with a ”mixed” character.”
Von Neumann (1958)
The Computer & the Brain

8. 8 • Guillaume Dumas • Whole Brain Simulations and the Discrepancy/Similarity between Artificial & Natural Neural Networks • 2016/10/11
2 main differences:
Structure : redundancy
Dynamics :
Evolution vs. Design

9. 9 • Guillaume Dumas • Whole Brain Simulations and the Discrepancy/Similarity between Artificial & Natural Neural Networks • 2016/10/11
Felleman & Van Essen (1991) asimovinstitute.org/neural-network-zoo/
. . .

10. Part 1
10 • Guillaume Dumas • Whole Brain Simulations and the Discrepancy/Similarity between Artificial & Natural Neural Networks • 2016/10/11

11. 11 • Guillaume Dumas • Whole Brain Simulations and the Discrepancy/Similarity between Artificial & Natural Neural Networks • 2016/10/11
Plasticity — Going beyond backpropagation
Connectivity — From weight sharing to recurrent networks
Astrocytes — Managing multiple time scales
Body — Convenient to get its own training set!
Oscillations — Time, attention, & subthreshold computing
. . .

12. Izhikevich & Edelman, PNAS 2008
12 • Guillaume Dumas • Whole Brain Simulations and the Discrepancy/Similarity between Artificial & Natural Neural Networks • 2016/10/11

13. 13 • Guillaume Dumas • Whole Brain Simulations and the Discrepancy/Similarity between Artificial & Natural Neural Networks • 2016/10/11

14. 14 • Guillaume Dumas • Whole Brain Simulations and the Discrepancy/Similarity between Artificial & Natural Neural Networks • 2016/10/11
“The dirty secret is that we don’t even understand the
nematode C. Elegans, which only has 302 neurons”
Christof Koch, Allen Brain Institute Chief Scientific Officer
“There is a lot of benefits for each
neuroscientist because we have now a new
Atlas, we can use supercomputers, we can
proof our models, a Neurorobotics Platform,
have new simulation tools and so on.”
Katrin Amunts, JULICH SP2 Leader

15. 15 • Guillaume Dumas • Whole Brain Simulations and the Discrepancy/Similarity between Artificial & Natural Neural Networks • 2016/10/11

16. 16 • Guillaume Dumas • Whole Brain Simulations and the Discrepancy/Similarity between Artificial & Natural Neural Networks • 2016/10/11
Unsupervised Learning of Visual Features through Spike Timing
Dependent Plasticity. Masquelier & Thorpe, PLoS Comp Biol 2007

17. Part 2
17 • Guillaume Dumas • Whole Brain Simulations and the Discrepancy/Similarity between Artificial & Natural Neural Networks • 2016/10/11

18. 18 • Guillaume Dumas • Whole Brain Simulations and the Discrepancy/Similarity between Artificial & Natural Neural Networks • 2016/10/11

19. 19 • Guillaume Dumas • Whole Brain Simulations and the Discrepancy/Similarity between Artificial & Natural Neural Networks • 2016/10/11
Dumas et al., PLoS ONE 2010

20. Dumas et al., PLoS ONE 2012
20 • Guillaume Dumas • Whole Brain Simulations and the Discrepancy/Similarity between Artificial & Natural Neural Networks • 2016/10/11
+ x 2 =
Large-scale, anatomically detailed models
of the brain allow to perform experiments
that are impossible (physically or ethically)

21. Dumas et al., PLoS ONE 2012
21 • Guillaume Dumas • Whole Brain Simulations and the Discrepancy/Similarity between Artificial & Natural Neural Networks • 2016/10/11
Normal Shuffle

22. Dumas et al., PLoS ONE 2012
22 • Guillaume Dumas • Whole Brain Simulations and the Discrepancy/Similarity between Artificial & Natural Neural Networks • 2016/10/11
FFTBrainAreaSignals
Cortical Level Scalp Level
Cintra
CintraFFTEEGSignals

23. 23 • Guillaume Dumas • Whole Brain Simulations and the Discrepancy/Similarity between Artificial & Natural Neural Networks • 2016/10/11
Dumas et al., PLoS ONE 2012
Real connectivity facilitate inter-brain synchronization
Residual synchronization
Information exchanged between the two virtual brains
Inter-brainsynchronization

24. 24 • Guillaume Dumas • Whole Brain Simulations and the Discrepancy/Similarity between Artificial & Natural Neural Networks • 2016/10/11
Kelso, Dumas, & Tognoli, Neural Networks 2013
ExperimentalComputational

25. 25 • Guillaume Dumas • Whole Brain Simulations and the Discrepancy/Similarity between Artificial & Natural Neural Networks • 2016/10/11
Dumas et al. « The Human Dynamic Clamp » PNAS 2014

26. 26 • Guillaume Dumas • Whole Brain Simulations and the Discrepancy/Similarity between Artificial & Natural Neural Networks • 2016/10/11
Dumas et al. « The Human Dynamic Clamp » PNAS 2014
”The Turing test implies only that judges are unable to tell if an agent is a
human or a machine, and as such says nothing about the genuineness of
the path toward that decision. Here, the Human Dynamic Clamp is a tool to
test hypotheses and gain understanding about how humans interact with
each other as well as with machines. In the HDC paradigm, exploration of
the machine’s behavior may be viewed as an exploration of us as well.”

27. Conclusion
27 • Guillaume Dumas • Whole Brain Simulations and the Discrepancy/Similarity between Artificial & Natural Neural Networks • 2016/10/11

28. 28 • Guillaume Dumas • Whole Brain Simulations and the Discrepancy/Similarity between Artificial & Natural Neural Networks • 2016/10/11
1936
1950
1952

29. 29 • Guillaume Dumas • Whole Brain Simulations and the Discrepancy/Similarity between Artificial & Natural Neural Networks • 2016/10/11
”Unless our methods can
deal with a simple
processor, how could we
expect it to work on our
own brain?”
Jonas & Kording 2016
Lesion method Spike trains recordings
Local field potential recordings

30. 30 • Guillaume Dumas • Whole Brain Simulations and the Discrepancy/Similarity between Artificial & Natural Neural Networks • 2016/10/11
Top-Down
(SemioticalView)
Bottom-Up
(InformationalView)
Source: lkm.fri.uni-lj.si

31. 31 • Guillaume Dumas • Whole Brain Simulations and the Discrepancy/Similarity between Artificial & Natural Neural Networks • 2016/10/11
Top-Down
(SemioticalView)
Theory
Hypothesis
Experiment
Data
Pattern
Model
Bottom-Up
(InformationalView)

32. 32 • Guillaume Dumas • Whole Brain Simulations and the Discrepancy/Similarity between Artificial & Natural Neural Networks • 2016/10/11
Top-Down
(SemioticalView)
Bottom-Up
(InformationalView)
Models

33. 33 • Guillaume Dumas • Whole Brain Simulations and the Discrepancy/Similarity between Artificial & Natural Neural Networks • 2016/10/11

34. Thanks for your attention
gdumas@pasteur.fr – Extrospection.eu – @introspection