In this presentation we user Marr's three levels of analysis to approach the question of how similar or different are the brain and the machines. By approaching this discussion in a more structured manner we find that the answer is different depending on the level of abstraction we are operating at.

1. The Brain and the Modern AI
Drastic differences and curious similarities
Ilya Kuzovkin

5. We need to structure this conversation
Suggestion
David Marr's
Three levels of analysis

6. We need to structure this conversation
Suggestion
David Marr's
Three levels of analysis
Goal of the computation
What is the purpose of computation?

7. We need to structure this conversation
Suggestion
David Marr's
Three levels of analysis
Goal of the computation
What is the purpose of computation?
Algorithm and representation
What representations does the system use?
What processes are in use to manipulate representations?

8. We need to structure this conversation
Suggestion
David Marr's
Three levels of analysis
Goal of the computation
What is the purpose of computation?
Algorithm and representation
What representations does the system use?
What processes are in use to manipulate representations?
Implementation
How is the system physically realized?

9. Implementation

10. https://medium.com/@jayeshbahire/the-artificial-neural-networks-handbook-part-4-d2087d1f583e
Output
Activation
function
Sum
Bias
Weights
Inputs
https://www.dreamstime.com/pyramidal-cell-cerebral-cortex-neurons-stained-golgi’s-silver-chromate-conic-shaped-soma-large-apical-image117240595
https://www.sciencedirect.com/science/article/pii/S0896627312005727#fig1
https://antranik.org/synaptic-transmission-by-somatic-motorneurons/

11. https://medium.com/@jayeshbahire/the-artificial-neural-networks-handbook-part-4-d2087d1f583e
Output
Activation
function
Sum
Bias
Weights
Inputs
https://www.dreamstime.com/pyramidal-cell-cerebral-cortex-neurons-stained-golgi’s-silver-chromate-conic-shaped-soma-large-apical-image117240595
https://www.sciencedirect.com/science/article/pii/S0896627312005727#fig1
https://antranik.org/synaptic-transmission-by-somatic-motorneurons/

12. https://medium.com/@jayeshbahire/the-artificial-neural-networks-handbook-part-4-d2087d1f583e
Output
Activation
function
Sum
Bias
Weights
Inputs
https://www.dreamstime.com/pyramidal-cell-cerebral-cortex-neurons-stained-golgi’s-silver-chromate-conic-shaped-soma-large-apical-image117240595
Axon terminals
(from other neurons)
Dendrites
https://www.sciencedirect.com/science/article/pii/S0896627312005727#fig1
https://antranik.org/synaptic-transmission-by-somatic-motorneurons/

13. https://medium.com/@jayeshbahire/the-artificial-neural-networks-handbook-part-4-d2087d1f583e
Output
Activation
function
Sum
Bias
Weights
Inputs
https://www.dreamstime.com/pyramidal-cell-cerebral-cortex-neurons-stained-golgi’s-silver-chromate-conic-shaped-soma-large-apical-image117240595
Axon terminals
(from other neurons)
Dendrites
Synaptic
connection
https://www.sciencedirect.com/science/article/pii/S0896627312005727#fig1
https://antranik.org/synaptic-transmission-by-somatic-motorneurons/

14. https://medium.com/@jayeshbahire/the-artificial-neural-networks-handbook-part-4-d2087d1f583e
Output
Activation
function
Sum
Bias
Weights
Inputs
https://www.dreamstime.com/pyramidal-cell-cerebral-cortex-neurons-stained-golgi’s-silver-chromate-conic-shaped-soma-large-apical-image117240595
Axon terminals
(from other neurons)
Dendrites
Synaptic
connection
Cell
depolarization
https://www.sciencedirect.com/science/article/pii/S0896627312005727#fig1
https://antranik.org/synaptic-transmission-by-somatic-motorneurons/

15. https://medium.com/@jayeshbahire/the-artificial-neural-networks-handbook-part-4-d2087d1f583e
Output
Activation
function
Sum
Bias
Weights
Inputs
https://www.dreamstime.com/pyramidal-cell-cerebral-cortex-neurons-stained-golgi’s-silver-chromate-conic-shaped-soma-large-apical-image117240595
Axon terminals
(from other neurons)
Dendrites
Synaptic
connection
Cell
depolarization
Polarization
threshold
https://www.sciencedirect.com/science/article/pii/S0896627312005727#fig1
https://antranik.org/synaptic-transmission-by-somatic-motorneurons/

16. https://medium.com/@jayeshbahire/the-artificial-neural-networks-handbook-part-4-d2087d1f583e
Output
Activation
function
Sum
Bias
Weights
Inputs
https://www.dreamstime.com/pyramidal-cell-cerebral-cortex-neurons-stained-golgi’s-silver-chromate-conic-shaped-soma-large-apical-image117240595
Axon terminals
(from other neurons)
Dendrites
Synaptic
connection
Cell
depolarization
Polarization
threshold
Action
potential
(aka Spike)
https://www.sciencedirect.com/science/article/pii/S0896627312005727#fig1
https://antranik.org/synaptic-transmission-by-somatic-motorneurons/

17. https://medium.com/@jayeshbahire/the-artificial-neural-networks-handbook-part-4-d2087d1f583e
Output
Activation
function
Sum
Bias
Weights
Inputs
https://www.dreamstime.com/pyramidal-cell-cerebral-cortex-neurons-stained-golgi’s-silver-chromate-conic-shaped-soma-large-apical-image117240595
Axon terminals
(from other neurons)
Dendrites
Synaptic
connection
Cell
depolarization
Polarization
threshold
Action
potential
(aka Spike)
https://www.sciencedirect.com/science/article/pii/S0896627312005727#fig1
https://antranik.org/synaptic-transmission-by-somatic-motorneurons/

18. https://medium.com/@jayeshbahire/the-artificial-neural-networks-handbook-part-4-d2087d1f583e
Output
Activation
function
Sum
Bias
Weights
Inputs
https://www.dreamstime.com/pyramidal-cell-cerebral-cortex-neurons-stained-golgi’s-silver-chromate-conic-shaped-soma-large-apical-image117240595
Axon terminals
(from other neurons)
Dendrites
Synaptic
connection
Cell
depolarization
Polarization
threshold
Action
potential
(aka Spike)
https://www.sciencedirect.com/science/article/pii/S0896627312005727#fig1
https://antranik.org/synaptic-transmission-by-somatic-motorneurons/
Molecular neurobiology
- different types of channels
- different receptors
- mechanism of
neurotransmitter release

19. https://medium.com/@jayeshbahire/the-artificial-neural-networks-handbook-part-4-d2087d1f583e
Output
Activation
function
Sum
Bias
Weights
Inputs
https://www.dreamstime.com/pyramidal-cell-cerebral-cortex-neurons-stained-golgi’s-silver-chromate-conic-shaped-soma-large-apical-image117240595
Axon terminals
(from other neurons)
Dendrites
Synaptic
connection
Cell
depolarization
Polarization
threshold
Action
potential
(aka Spike)
https://www.sciencedirect.com/science/article/pii/S0896627312005727#fig1
https://antranik.org/synaptic-transmission-by-somatic-motorneurons/
Molecular neurobiology
- different types of channels
- different receptors
- mechanism of
neurotransmitter release
Dendritic spike
action potential can be
generated in the dendrite

20. https://medium.com/@jayeshbahire/the-artificial-neural-networks-handbook-part-4-d2087d1f583e
Output
Activation
function
Sum
Bias
Weights
Inputs
https://www.dreamstime.com/pyramidal-cell-cerebral-cortex-neurons-stained-golgi’s-silver-chromate-conic-shaped-soma-large-apical-image117240595
Axon terminals
(from other neurons)
Dendrites
Synaptic
connection
Cell
depolarization
Polarization
threshold
Action
potential
(aka Spike)
https://www.sciencedirect.com/science/article/pii/S0896627312005727#fig1
https://antranik.org/synaptic-transmission-by-somatic-motorneurons/
Molecular neurobiology
- different types of channels
- different receptors
- mechanism of
neurotransmitter release
Dendritic spike
action potential can be
generated in the dendrite
Timing
- no clock synchronization
- time of arrival is important

21. https://medium.com/@jayeshbahire/the-artificial-neural-networks-handbook-part-4-d2087d1f583e
Output
Activation
function
Sum
Bias
Weights
Inputs
https://www.dreamstime.com/pyramidal-cell-cerebral-cortex-neurons-stained-golgi’s-silver-chromate-conic-shaped-soma-large-apical-image117240595
Axon terminals
(from other neurons)
Dendrites
Synaptic
connection
Cell
depolarization
Polarization
threshold
Action
potential
(aka Spike)
https://www.sciencedirect.com/science/article/pii/S0896627312005727#fig1
https://antranik.org/synaptic-transmission-by-somatic-motorneurons/
Molecular neurobiology
- different types of channels
- different receptors
- mechanism of
neurotransmitter release
Dendritic spike
action potential can be
generated in the dendrite
Timing
- no clock synchronization
- time of arrival is importantTopology
Artificial networks are neatly
organized into graphs,
biological network is a mess

22. https://medium.com/@jayeshbahire/the-artificial-neural-networks-handbook-part-4-d2087d1f583e
Output
Activation
function
Sum
Bias
Weights
Inputs
https://www.dreamstime.com/pyramidal-cell-cerebral-cortex-neurons-stained-golgi’s-silver-chromate-conic-shaped-soma-large-apical-image117240595
Axon terminals
(from other neurons)
Dendrites
Synaptic
connection
Cell
depolarization
Polarization
threshold
Action
potential
(aka Spike)
https://www.sciencedirect.com/science/article/pii/S0896627312005727#fig1
https://antranik.org/synaptic-transmission-by-somatic-motorneurons/
Molecular neurobiology
- different types of channels
- different receptors
- mechanism of
neurotransmitter release
Dendritic spike
action potential can be
generated in the dendrite
Timing
- no clock synchronization
- time of arrival is importantTopology
Artificial networks are neatly
organized into graphs,
biological network is a mess
Power consumption
Brain uses 20 watt, one
Titan X — 250 watt

23. The brain cannot be doing backpropagation
(at least not the way we do it)
https://mattmazur.com/2015/03/17/a-step-by-step-backpropagation-example/

24. The brain cannot be doing backpropagation
(at least not the way we do it)
Forward pass
https://mattmazur.com/2015/03/17/a-step-by-step-backpropagation-example/

25. The brain cannot be doing backpropagation
(at least not the way we do it)
Forward pass
Backwards pass
https://mattmazur.com/2015/03/17/a-step-by-step-backpropagation-example/

26. The brain cannot be doing backpropagation
(at least not the way we do it)
Forward pass
Backwards pass
https://mattmazur.com/2015/03/17/a-step-by-step-backpropagation-example/
✓
✓

27. The brain cannot be doing backpropagation
(at least not the way we do it)
Forward pass
Backwards pass
https://mattmazur.com/2015/03/17/a-step-by-step-backpropagation-example/
✓
✓

28. The brain cannot be doing backpropagation
(at least not the way we do it)
Forward pass
Backwards pass
https://mattmazur.com/2015/03/17/a-step-by-step-backpropagation-example/
Forward pass
Neurons send spikes,
not real-valued outputs
https://www.cs.toronto.edu/~hinton/backpropincortex2014.pdf
✓
✓

29. The brain cannot be doing backpropagation
(at least not the way we do it)
Forward pass
Backwards pass
https://mattmazur.com/2015/03/17/a-step-by-step-backpropagation-example/
Forward pass
Neurons send spikes,
not real-valued outputs
Backwards pass
https://www.cs.toronto.edu/~hinton/backpropincortex2014.pdf
Neurons would need
bidirectional connections
with the same weight
Neurons would need to
send two types of signal
and estimate derivative
✓
✓

30. The brain cannot be doing backpropagation
(at least not the way we do it)
Forward pass
Backwards pass
https://mattmazur.com/2015/03/17/a-step-by-step-backpropagation-example/
Forward pass
Neurons send spikes,
not real-valued outputs
Backwards pass
https://www.cs.toronto.edu/~hinton/backpropincortex2014.pdf
Neurons would need
bidirectional connections
with the same weight
Neurons would need to
send two types of signal
and estimate derivative
✓
✓
✓'ish
✗

31. Quite different on the
level of implementation
<
✓'ish
✗

32. Algorithm and representation

33. Hippocampus and experience replay
Hippocampus

34. Hippocampus and experience replay
Hippocampus

35. Hippocampus and experience replay
Hippocampus

36. Hippocampus and experience replay
Hippocampus

37. Hippocampus and experience replay
Hippocampus
DQN algorithm has state memory buffer
and experience replay mechanism:
- store experiences while interacting
with the environment
- randomly sample and replay
experiences from memory while learning
Replay buffer

38. Vision: hierarchy of layers
Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream https://www.jneurosci.org/content/35/27/10005
Comparison of deep neural networks to spatio-temporal cortical dynamics of human visual object recognition reveals hierarchical correspondence https://www.nature.com/articles/srep27755
Activations of deep convolutional neural networks are aligned with gamma band activity of human visual cortex https://www.nature.com/articles/s42003-018-0110-y

39. Vision: hierarchy of layers
Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream https://www.jneurosci.org/content/35/27/10005
Comparison of deep neural networks to spatio-temporal cortical dynamics of human visual object recognition reveals hierarchical correspondence https://www.nature.com/articles/srep27755
Activations of deep convolutional neural networks are aligned with gamma band activity of human visual cortex https://www.nature.com/articles/s42003-018-0110-y
& multiple papers since:

40. Vision: hierarchy of layers
Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream https://www.jneurosci.org/content/35/27/10005
Comparison of deep neural networks to spatio-temporal cortical dynamics of human visual object recognition reveals hierarchical correspondence https://www.nature.com/articles/srep27755
Activations of deep convolutional neural networks are aligned with gamma band activity of human visual cortex https://www.nature.com/articles/s42003-018-0110-y
& multiple papers since:
Layered structure of visual cortex
Neurons focus on certain areas of visual input
Lower layers process simple visual features, higher layers -- complex features

41. Vision: hierarchy of layers
Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream https://www.jneurosci.org/content/35/27/10005
Comparison of deep neural networks to spatio-temporal cortical dynamics of human visual object recognition reveals hierarchical correspondence https://www.nature.com/articles/srep27755
Activations of deep convolutional neural networks are aligned with gamma band activity of human visual cortex https://www.nature.com/articles/s42003-018-0110-y
& multiple papers since:
Layered structure of visual cortex
Neurons focus on certain areas of visual input
Lower layers process simple visual features, higher layers -- complex features

42. Vision: hierarchy of layers
Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream https://www.jneurosci.org/content/35/27/10005
Comparison of deep neural networks to spatio-temporal cortical dynamics of human visual object recognition reveals hierarchical correspondence https://www.nature.com/articles/srep27755
Activations of deep convolutional neural networks are aligned with gamma band activity of human visual cortex https://www.nature.com/articles/s42003-018-0110-y
& multiple papers since:
Layered structure of visual cortex
Neurons focus on certain areas of visual input
Lower layers process simple visual features, higher layers -- complex features

43. Vision: hierarchy of layers
Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream https://www.jneurosci.org/content/35/27/10005
Comparison of deep neural networks to spatio-temporal cortical dynamics of human visual object recognition reveals hierarchical correspondence https://www.nature.com/articles/srep27755
Activations of deep convolutional neural networks are aligned with gamma band activity of human visual cortex https://www.nature.com/articles/s42003-018-0110-y
& multiple papers since:
Layered structure of visual cortex
Neurons focus on certain areas of visual input
Lower layers process simple visual features, higher layers -- complex features

44. Vision: hierarchy of layers
Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream https://www.jneurosci.org/content/35/27/10005
Comparison of deep neural networks to spatio-temporal cortical dynamics of human visual object recognition reveals hierarchical correspondence https://www.nature.com/articles/srep27755
Activations of deep convolutional neural networks are aligned with gamma band activity of human visual cortex https://www.nature.com/articles/s42003-018-0110-y
& multiple papers since:
Layered structure of visual cortex
Neurons focus on certain areas of visual input
Lower layers process simple visual features, higher layers -- complex features

45. Model of the world: grid cells
The Nobel Prize in Physiology or Medicine 2014
"for their discoveries of cells
that constitute a positioning
system in the brain"
http://www.scholarpedia.org/article/Grid_cells
O'Keefe Moser Moser
https://deepmind.com/blog/article/grid-cells
https://www.nature.com/articles/s41586-018-0102-6

46. Model of the world: grid cells
Entorhinal cortex
The Nobel Prize in Physiology or Medicine 2014
"for their discoveries of cells
that constitute a positioning
system in the brain"
http://www.scholarpedia.org/article/Grid_cells
O'Keefe Moser Moser
https://deepmind.com/blog/article/grid-cells
https://www.nature.com/articles/s41586-018-0102-6

47. Model of the world: grid cells
Entorhinal cortex
The Nobel Prize in Physiology or Medicine 2014
"for their discoveries of cells
that constitute a positioning
system in the brain"
http://www.scholarpedia.org/article/Grid_cells
O'Keefe Moser Moser
https://deepmind.com/blog/article/grid-cells
https://www.nature.com/articles/s41586-018-0102-6

48. Model of the world: grid cells
Entorhinal cortex
The Nobel Prize in Physiology or Medicine 2014
"for their discoveries of cells
that constitute a positioning
system in the brain"
http://www.scholarpedia.org/article/Grid_cells
O'Keefe Moser Moser
https://deepmind.com/blog/article/grid-cells
https://www.nature.com/articles/s41586-018-0102-6

49. Model of the world: grid cells
Entorhinal cortex
The Nobel Prize in Physiology or Medicine 2014
"for their discoveries of cells
that constitute a positioning
system in the brain"
http://www.scholarpedia.org/article/Grid_cells
O'Keefe Moser Moser
https://deepmind.com/blog/article/grid-cells
https://www.nature.com/articles/s41586-018-0102-6

50. Model of the world: successor representations
https://julien-vitay.net/2019/05/successor-representations/
Model-free RL
+ fast
– inflexible to change
http://gershmanlab.webfactional.com/pubs/Stachenfeld17.pdf

51. Model of the world: successor representations
https://julien-vitay.net/2019/05/successor-representations/
Model-free RL
+ fast
– inflexible to change
Model-based RL
+ adaptable
– hard to learn
transition
probs
reward
probs
http://gershmanlab.webfactional.com/pubs/Stachenfeld17.pdf

52. Model of the world: successor representations
https://julien-vitay.net/2019/05/successor-representations/
Model-free RL
+ fast
– inflexible to change
Model-based RL
+ adaptable
– hard to learn
transition
probs
reward
probs
Successor representation for RL
discounted future
state occupancy
matrix
expected
state
reward
Why learn the full model when we only
care about the chance of getting into a state
http://gershmanlab.webfactional.com/pubs/Stachenfeld17.pdf

53. Model of the world: successor representations
https://julien-vitay.net/2019/05/successor-representations/
Model-free RL
+ fast
– inflexible to change
Model-based RL
+ adaptable
– hard to learn
transition
probs
reward
probs
Successor representation for RL
discounted future
state occupancy
matrix
expected
state
reward
Why learn the full model when we only
care about the chance of getting into a state
http://gershmanlab.webfactional.com/pubs/Stachenfeld17.pdf
Hippocampus Place cells
Gaussian distance to the location?

54. Model of the world: successor representations
https://julien-vitay.net/2019/05/successor-representations/
Model-free RL
+ fast
– inflexible to change
Model-based RL
+ adaptable
– hard to learn
transition
probs
reward
probs
Successor representation for RL
discounted future
state occupancy
matrix
expected
state
reward
Why learn the full model when we only
care about the chance of getting into a state
http://gershmanlab.webfactional.com/pubs/Stachenfeld17.pdf
Hippocampus Place cells
Gaussian distance to the location?
Adding constraints to the environment modifies the shape of the
place fields.
SR hypothesis is in agreement with the observation that rewarded
locations are represented by a higher number of place cells.

55. Dual DQN network
MERLIN Architecture
Predictive coding
Visual attention
...
Generative Adversarial Networks
Predictive coding
Successor representations
Attention in CNNs
Consolidation of experience
Novel vs. routine circuits
& &
& &

56. Replay buffer
Curious similarities on the level
of algorithm and representation
discounted future
state occupancy
matrix
expected
state
reward

57. Goal of computation

58. Same goal, different strengths... merge!
Fetz Lab
University of Washington
NeuroChip Neuralink
Elon Musk
San Franciso, CA
http://www.csne-erc.org/sites/default/files/Neurochip.pdf https://depts.washington.edu/fetzweb/closed-loop-bci.html https://neuralink.com/

59. Same goal, different strengths... merge!
Fetz Lab
University of Washington
NeuroChip Neuralink
Elon Musk
San Franciso, CA
http://www.csne-erc.org/sites/default/files/Neurochip.pdf https://depts.washington.edu/fetzweb/closed-loop-bci.html https://neuralink.com/

60. Same goal, different strengths... merge!
Fetz Lab
University of Washington
NeuroChip Neuralink
Elon Musk
San Franciso, CA
http://www.csne-erc.org/sites/default/files/Neurochip.pdf https://depts.washington.edu/fetzweb/closed-loop-bci.html https://neuralink.com/

61. Goal of the computation
What is the purpose of computation?
Algorithm and representation
What representations does the system use?
What processes are in use to manipulate representations?
Implementation
How is the system physically realized?
The Brain and the Modern AI
Almost the same
Quite different
Comparable
here and there
Ilya Kuzovkin
ilya.kuzovkin@gmail.com
www.ikuz.eu
Neurotech
Sydney
meetup group
Podcast by
Paul Middlebrooks