Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
The Brain and the Modern AI

Drastic differences and curious similarities
Ilya Kuzovkin
We need to structure this conversation
Suggestion

David Marr's

Three levels of analysis
We need to structure this conversation
Suggestion

David Marr's

Three levels of analysis
Goal of the computation

What is...
We need to structure this conversation
Suggestion

David Marr's

Three levels of analysis
Goal of the computation

What is...
We need to structure this conversation
Suggestion

David Marr's

Three levels of analysis
Goal of the computation

What is...
Implementation
https://medium.com/@jayeshbahire/the-artificial-neural-networks-handbook-part-4-d2087d1f583e
Output
Activation

function
Su...
https://medium.com/@jayeshbahire/the-artificial-neural-networks-handbook-part-4-d2087d1f583e
Output
Activation

function
Su...
https://medium.com/@jayeshbahire/the-artificial-neural-networks-handbook-part-4-d2087d1f583e
Output
Activation

function
Su...
https://medium.com/@jayeshbahire/the-artificial-neural-networks-handbook-part-4-d2087d1f583e
Output
Activation

function
Su...
https://medium.com/@jayeshbahire/the-artificial-neural-networks-handbook-part-4-d2087d1f583e
Output
Activation

function
Su...
https://medium.com/@jayeshbahire/the-artificial-neural-networks-handbook-part-4-d2087d1f583e
Output
Activation

function
Su...
https://medium.com/@jayeshbahire/the-artificial-neural-networks-handbook-part-4-d2087d1f583e
Output
Activation

function
Su...
https://medium.com/@jayeshbahire/the-artificial-neural-networks-handbook-part-4-d2087d1f583e
Output
Activation

function
Su...
https://medium.com/@jayeshbahire/the-artificial-neural-networks-handbook-part-4-d2087d1f583e
Output
Activation

function
Su...
https://medium.com/@jayeshbahire/the-artificial-neural-networks-handbook-part-4-d2087d1f583e
Output
Activation

function
Su...
https://medium.com/@jayeshbahire/the-artificial-neural-networks-handbook-part-4-d2087d1f583e
Output
Activation

function
Su...
https://medium.com/@jayeshbahire/the-artificial-neural-networks-handbook-part-4-d2087d1f583e
Output
Activation

function
Su...
https://medium.com/@jayeshbahire/the-artificial-neural-networks-handbook-part-4-d2087d1f583e
Output
Activation

function
Su...
The brain cannot be doing backpropagation

(at least not the way we do it)
https://mattmazur.com/2015/03/17/a-step-by-step...
The brain cannot be doing backpropagation

(at least not the way we do it)
Forward pass
https://mattmazur.com/2015/03/17/a...
The brain cannot be doing backpropagation

(at least not the way we do it)
Forward pass
Backwards pass
https://mattmazur.c...
The brain cannot be doing backpropagation

(at least not the way we do it)
Forward pass
Backwards pass
https://mattmazur.c...
The brain cannot be doing backpropagation

(at least not the way we do it)
Forward pass
Backwards pass
https://mattmazur.c...
The brain cannot be doing backpropagation

(at least not the way we do it)
Forward pass
Backwards pass
https://mattmazur.c...
The brain cannot be doing backpropagation

(at least not the way we do it)
Forward pass
Backwards pass
https://mattmazur.c...
The brain cannot be doing backpropagation

(at least not the way we do it)
Forward pass
Backwards pass
https://mattmazur.c...
Quite different on the
level of implementation
<
✓'ish
✗
Algorithm and representation
Hippocampus and experience replay
Hippocampus
Hippocampus and experience replay
Hippocampus
Hippocampus and experience replay
Hippocampus
Hippocampus and experience replay
Hippocampus
Hippocampus and experience replay
Hippocampus
DQN algorithm has state memory buffer
and experience replay mechanism:



- ...
Vision: hierarchy of layers
Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the ...
Vision: hierarchy of layers
Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the ...
Vision: hierarchy of layers
Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the ...
Vision: hierarchy of layers
Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the ...
Vision: hierarchy of layers
Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the ...
Vision: hierarchy of layers
Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the ...
Vision: hierarchy of layers
Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the ...
Model of the world: grid cells
The Nobel Prize in Physiology or Medicine 2014
"for their discoveries of cells
that constit...
Model of the world: grid cells
Entorhinal cortex
The Nobel Prize in Physiology or Medicine 2014
"for their discoveries of ...
Model of the world: grid cells
Entorhinal cortex
The Nobel Prize in Physiology or Medicine 2014
"for their discoveries of ...
Model of the world: grid cells
Entorhinal cortex
The Nobel Prize in Physiology or Medicine 2014
"for their discoveries of ...
Model of the world: grid cells
Entorhinal cortex
The Nobel Prize in Physiology or Medicine 2014
"for their discoveries of ...
Model of the world: successor representations
https://julien-vitay.net/2019/05/successor-representations/
Model-free RL
+ ...
Model of the world: successor representations
https://julien-vitay.net/2019/05/successor-representations/
Model-free RL
+ ...
Model of the world: successor representations
https://julien-vitay.net/2019/05/successor-representations/
Model-free RL
+ ...
Model of the world: successor representations
https://julien-vitay.net/2019/05/successor-representations/
Model-free RL
+ ...
Model of the world: successor representations
https://julien-vitay.net/2019/05/successor-representations/
Model-free RL
+ ...
Dual DQN network
MERLIN Architecture
Predictive coding
Visual attention
...
Generative Adversarial Networks
Predictive cod...
Replay buffer
Curious similarities on the level
of algorithm and representation
discounted future

state occupancy

matrix...
Goal of computation
Same goal, different strengths... merge!
Fetz Lab

University of Washington
NeuroChip Neuralink
Elon Musk

San Franciso, C...
Same goal, different strengths... merge!
Fetz Lab

University of Washington
NeuroChip Neuralink
Elon Musk

San Franciso, C...
Same goal, different strengths... merge!
Fetz Lab

University of Washington
NeuroChip Neuralink
Elon Musk

San Franciso, C...
Goal of the computation

What is the purpose of computation?
Algorithm and representation

What representations does the s...
The Brain and the Modern AI: Drastic Differences and Curious Similarities
The Brain and the Modern AI: Drastic Differences and Curious Similarities
The Brain and the Modern AI: Drastic Differences and Curious Similarities
Upcoming SlideShare
Loading in …5
×

The Brain and the Modern AI: Drastic Differences and Curious Similarities

62 views

Published on

In this presentation we user Marr's three levels of analysis to approach the question of how similar or different are the brain and the machines. By approaching this discussion in a more structured manner we find that the answer is different depending on the level of abstraction we are operating at.

Published in: Science
  • Be the first to comment

  • Be the first to like this

The Brain and the Modern AI: Drastic Differences and Curious Similarities

  1. 1. The Brain and the Modern AI
 Drastic differences and curious similarities Ilya Kuzovkin
  2. 2. We need to structure this conversation Suggestion
 David Marr's
 Three levels of analysis
  3. 3. We need to structure this conversation Suggestion
 David Marr's
 Three levels of analysis Goal of the computation
 What is the purpose of computation?
  4. 4. We need to structure this conversation Suggestion
 David Marr's
 Three levels of analysis Goal of the computation
 What is the purpose of computation? Algorithm and representation
 What representations does the system use?
 What processes are in use to manipulate representations?
  5. 5. We need to structure this conversation Suggestion
 David Marr's
 Three levels of analysis Goal of the computation
 What is the purpose of computation? Algorithm and representation
 What representations does the system use?
 What processes are in use to manipulate representations? Implementation
 How is the system physically realized?
  6. 6. Implementation
  7. 7. https://medium.com/@jayeshbahire/the-artificial-neural-networks-handbook-part-4-d2087d1f583e Output Activation
 function Sum Bias Weights Inputs https://www.dreamstime.com/pyramidal-cell-cerebral-cortex-neurons-stained-golgi’s-silver-chromate-conic-shaped-soma-large-apical-image117240595 https://www.sciencedirect.com/science/article/pii/S0896627312005727#fig1 https://antranik.org/synaptic-transmission-by-somatic-motorneurons/
  8. 8. https://medium.com/@jayeshbahire/the-artificial-neural-networks-handbook-part-4-d2087d1f583e Output Activation
 function Sum Bias Weights Inputs https://www.dreamstime.com/pyramidal-cell-cerebral-cortex-neurons-stained-golgi’s-silver-chromate-conic-shaped-soma-large-apical-image117240595 https://www.sciencedirect.com/science/article/pii/S0896627312005727#fig1 https://antranik.org/synaptic-transmission-by-somatic-motorneurons/
  9. 9. https://medium.com/@jayeshbahire/the-artificial-neural-networks-handbook-part-4-d2087d1f583e Output Activation
 function Sum Bias Weights Inputs https://www.dreamstime.com/pyramidal-cell-cerebral-cortex-neurons-stained-golgi’s-silver-chromate-conic-shaped-soma-large-apical-image117240595 Axon terminals
 (from other neurons) Dendrites https://www.sciencedirect.com/science/article/pii/S0896627312005727#fig1 https://antranik.org/synaptic-transmission-by-somatic-motorneurons/
  10. 10. https://medium.com/@jayeshbahire/the-artificial-neural-networks-handbook-part-4-d2087d1f583e Output Activation
 function Sum Bias Weights Inputs https://www.dreamstime.com/pyramidal-cell-cerebral-cortex-neurons-stained-golgi’s-silver-chromate-conic-shaped-soma-large-apical-image117240595 Axon terminals
 (from other neurons) Dendrites Synaptic
 connection https://www.sciencedirect.com/science/article/pii/S0896627312005727#fig1 https://antranik.org/synaptic-transmission-by-somatic-motorneurons/
  11. 11. https://medium.com/@jayeshbahire/the-artificial-neural-networks-handbook-part-4-d2087d1f583e Output Activation
 function Sum Bias Weights Inputs https://www.dreamstime.com/pyramidal-cell-cerebral-cortex-neurons-stained-golgi’s-silver-chromate-conic-shaped-soma-large-apical-image117240595 Axon terminals
 (from other neurons) Dendrites Synaptic
 connection Cell
 depolarization https://www.sciencedirect.com/science/article/pii/S0896627312005727#fig1 https://antranik.org/synaptic-transmission-by-somatic-motorneurons/
  12. 12. https://medium.com/@jayeshbahire/the-artificial-neural-networks-handbook-part-4-d2087d1f583e Output Activation
 function Sum Bias Weights Inputs https://www.dreamstime.com/pyramidal-cell-cerebral-cortex-neurons-stained-golgi’s-silver-chromate-conic-shaped-soma-large-apical-image117240595 Axon terminals
 (from other neurons) Dendrites Synaptic
 connection Cell
 depolarization Polarization
 threshold https://www.sciencedirect.com/science/article/pii/S0896627312005727#fig1 https://antranik.org/synaptic-transmission-by-somatic-motorneurons/
  13. 13. https://medium.com/@jayeshbahire/the-artificial-neural-networks-handbook-part-4-d2087d1f583e Output Activation
 function Sum Bias Weights Inputs https://www.dreamstime.com/pyramidal-cell-cerebral-cortex-neurons-stained-golgi’s-silver-chromate-conic-shaped-soma-large-apical-image117240595 Axon terminals
 (from other neurons) Dendrites Synaptic
 connection Cell
 depolarization Polarization
 threshold Action
 potential
 (aka Spike) https://www.sciencedirect.com/science/article/pii/S0896627312005727#fig1 https://antranik.org/synaptic-transmission-by-somatic-motorneurons/
  14. 14. https://medium.com/@jayeshbahire/the-artificial-neural-networks-handbook-part-4-d2087d1f583e Output Activation
 function Sum Bias Weights Inputs https://www.dreamstime.com/pyramidal-cell-cerebral-cortex-neurons-stained-golgi’s-silver-chromate-conic-shaped-soma-large-apical-image117240595 Axon terminals
 (from other neurons) Dendrites Synaptic
 connection Cell
 depolarization Polarization
 threshold Action
 potential
 (aka Spike) https://www.sciencedirect.com/science/article/pii/S0896627312005727#fig1 https://antranik.org/synaptic-transmission-by-somatic-motorneurons/
  15. 15. https://medium.com/@jayeshbahire/the-artificial-neural-networks-handbook-part-4-d2087d1f583e Output Activation
 function Sum Bias Weights Inputs https://www.dreamstime.com/pyramidal-cell-cerebral-cortex-neurons-stained-golgi’s-silver-chromate-conic-shaped-soma-large-apical-image117240595 Axon terminals
 (from other neurons) Dendrites Synaptic
 connection Cell
 depolarization Polarization
 threshold Action
 potential
 (aka Spike) https://www.sciencedirect.com/science/article/pii/S0896627312005727#fig1 https://antranik.org/synaptic-transmission-by-somatic-motorneurons/ Molecular neurobiology - different types of channels
 - different receptors
 - mechanism of 
 neurotransmitter release
  16. 16. https://medium.com/@jayeshbahire/the-artificial-neural-networks-handbook-part-4-d2087d1f583e Output Activation
 function Sum Bias Weights Inputs https://www.dreamstime.com/pyramidal-cell-cerebral-cortex-neurons-stained-golgi’s-silver-chromate-conic-shaped-soma-large-apical-image117240595 Axon terminals
 (from other neurons) Dendrites Synaptic
 connection Cell
 depolarization Polarization
 threshold Action
 potential
 (aka Spike) https://www.sciencedirect.com/science/article/pii/S0896627312005727#fig1 https://antranik.org/synaptic-transmission-by-somatic-motorneurons/ Molecular neurobiology - different types of channels
 - different receptors
 - mechanism of 
 neurotransmitter release Dendritic spike action potential can be generated in the dendrite
  17. 17. https://medium.com/@jayeshbahire/the-artificial-neural-networks-handbook-part-4-d2087d1f583e Output Activation
 function Sum Bias Weights Inputs https://www.dreamstime.com/pyramidal-cell-cerebral-cortex-neurons-stained-golgi’s-silver-chromate-conic-shaped-soma-large-apical-image117240595 Axon terminals
 (from other neurons) Dendrites Synaptic
 connection Cell
 depolarization Polarization
 threshold Action
 potential
 (aka Spike) https://www.sciencedirect.com/science/article/pii/S0896627312005727#fig1 https://antranik.org/synaptic-transmission-by-somatic-motorneurons/ Molecular neurobiology - different types of channels
 - different receptors
 - mechanism of 
 neurotransmitter release Dendritic spike action potential can be generated in the dendrite Timing - no clock synchronization
 - time of arrival is important
  18. 18. https://medium.com/@jayeshbahire/the-artificial-neural-networks-handbook-part-4-d2087d1f583e Output Activation
 function Sum Bias Weights Inputs https://www.dreamstime.com/pyramidal-cell-cerebral-cortex-neurons-stained-golgi’s-silver-chromate-conic-shaped-soma-large-apical-image117240595 Axon terminals
 (from other neurons) Dendrites Synaptic
 connection Cell
 depolarization Polarization
 threshold Action
 potential
 (aka Spike) https://www.sciencedirect.com/science/article/pii/S0896627312005727#fig1 https://antranik.org/synaptic-transmission-by-somatic-motorneurons/ Molecular neurobiology - different types of channels
 - different receptors
 - mechanism of 
 neurotransmitter release Dendritic spike action potential can be generated in the dendrite Timing - no clock synchronization
 - time of arrival is importantTopology Artificial networks are neatly organized into graphs, biological network is a mess
  19. 19. https://medium.com/@jayeshbahire/the-artificial-neural-networks-handbook-part-4-d2087d1f583e Output Activation
 function Sum Bias Weights Inputs https://www.dreamstime.com/pyramidal-cell-cerebral-cortex-neurons-stained-golgi’s-silver-chromate-conic-shaped-soma-large-apical-image117240595 Axon terminals
 (from other neurons) Dendrites Synaptic
 connection Cell
 depolarization Polarization
 threshold Action
 potential
 (aka Spike) https://www.sciencedirect.com/science/article/pii/S0896627312005727#fig1 https://antranik.org/synaptic-transmission-by-somatic-motorneurons/ Molecular neurobiology - different types of channels
 - different receptors
 - mechanism of 
 neurotransmitter release Dendritic spike action potential can be generated in the dendrite Timing - no clock synchronization
 - time of arrival is importantTopology Artificial networks are neatly organized into graphs, biological network is a mess Power consumption Brain uses 20 watt, one
 Titan X — 250 watt
  20. 20. The brain cannot be doing backpropagation
 (at least not the way we do it) https://mattmazur.com/2015/03/17/a-step-by-step-backpropagation-example/
  21. 21. The brain cannot be doing backpropagation
 (at least not the way we do it) Forward pass https://mattmazur.com/2015/03/17/a-step-by-step-backpropagation-example/
  22. 22. The brain cannot be doing backpropagation
 (at least not the way we do it) Forward pass Backwards pass https://mattmazur.com/2015/03/17/a-step-by-step-backpropagation-example/
  23. 23. The brain cannot be doing backpropagation
 (at least not the way we do it) Forward pass Backwards pass https://mattmazur.com/2015/03/17/a-step-by-step-backpropagation-example/ ✓ ✓
  24. 24. The brain cannot be doing backpropagation
 (at least not the way we do it) Forward pass Backwards pass https://mattmazur.com/2015/03/17/a-step-by-step-backpropagation-example/ ✓ ✓
  25. 25. The brain cannot be doing backpropagation
 (at least not the way we do it) Forward pass Backwards pass https://mattmazur.com/2015/03/17/a-step-by-step-backpropagation-example/ Forward pass Neurons send spikes,
 not real-valued outputs https://www.cs.toronto.edu/~hinton/backpropincortex2014.pdf ✓ ✓
  26. 26. The brain cannot be doing backpropagation
 (at least not the way we do it) Forward pass Backwards pass https://mattmazur.com/2015/03/17/a-step-by-step-backpropagation-example/ Forward pass Neurons send spikes,
 not real-valued outputs Backwards pass https://www.cs.toronto.edu/~hinton/backpropincortex2014.pdf Neurons would need bidirectional connections with the same weight Neurons would need to send two types of signal and estimate derivative ✓ ✓
  27. 27. The brain cannot be doing backpropagation
 (at least not the way we do it) Forward pass Backwards pass https://mattmazur.com/2015/03/17/a-step-by-step-backpropagation-example/ Forward pass Neurons send spikes,
 not real-valued outputs Backwards pass https://www.cs.toronto.edu/~hinton/backpropincortex2014.pdf Neurons would need bidirectional connections with the same weight Neurons would need to send two types of signal and estimate derivative ✓ ✓ ✓'ish ✗
  28. 28. Quite different on the level of implementation < ✓'ish ✗
  29. 29. Algorithm and representation
  30. 30. Hippocampus and experience replay Hippocampus
  31. 31. Hippocampus and experience replay Hippocampus
  32. 32. Hippocampus and experience replay Hippocampus
  33. 33. Hippocampus and experience replay Hippocampus
  34. 34. Hippocampus and experience replay Hippocampus DQN algorithm has state memory buffer and experience replay mechanism:
 
 - store experiences while interacting 
 with the environment
 
 - randomly sample and replay 
 experiences from memory while learning Replay buffer
  35. 35. Vision: hierarchy of layers Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream https://www.jneurosci.org/content/35/27/10005 Comparison of deep neural networks to spatio-temporal cortical dynamics of human visual object recognition reveals hierarchical correspondence https://www.nature.com/articles/srep27755 Activations of deep convolutional neural networks are aligned with gamma band activity of human visual cortex https://www.nature.com/articles/s42003-018-0110-y
  36. 36. Vision: hierarchy of layers Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream https://www.jneurosci.org/content/35/27/10005 Comparison of deep neural networks to spatio-temporal cortical dynamics of human visual object recognition reveals hierarchical correspondence https://www.nature.com/articles/srep27755 Activations of deep convolutional neural networks are aligned with gamma band activity of human visual cortex https://www.nature.com/articles/s42003-018-0110-y & multiple papers since:
  37. 37. Vision: hierarchy of layers Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream https://www.jneurosci.org/content/35/27/10005 Comparison of deep neural networks to spatio-temporal cortical dynamics of human visual object recognition reveals hierarchical correspondence https://www.nature.com/articles/srep27755 Activations of deep convolutional neural networks are aligned with gamma band activity of human visual cortex https://www.nature.com/articles/s42003-018-0110-y & multiple papers since: Layered structure of visual cortex Neurons focus on certain areas of visual input Lower layers process simple visual features, higher layers -- complex features
  38. 38. Vision: hierarchy of layers Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream https://www.jneurosci.org/content/35/27/10005 Comparison of deep neural networks to spatio-temporal cortical dynamics of human visual object recognition reveals hierarchical correspondence https://www.nature.com/articles/srep27755 Activations of deep convolutional neural networks are aligned with gamma band activity of human visual cortex https://www.nature.com/articles/s42003-018-0110-y & multiple papers since: Layered structure of visual cortex Neurons focus on certain areas of visual input Lower layers process simple visual features, higher layers -- complex features
  39. 39. Vision: hierarchy of layers Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream https://www.jneurosci.org/content/35/27/10005 Comparison of deep neural networks to spatio-temporal cortical dynamics of human visual object recognition reveals hierarchical correspondence https://www.nature.com/articles/srep27755 Activations of deep convolutional neural networks are aligned with gamma band activity of human visual cortex https://www.nature.com/articles/s42003-018-0110-y & multiple papers since: Layered structure of visual cortex Neurons focus on certain areas of visual input Lower layers process simple visual features, higher layers -- complex features
  40. 40. Vision: hierarchy of layers Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream https://www.jneurosci.org/content/35/27/10005 Comparison of deep neural networks to spatio-temporal cortical dynamics of human visual object recognition reveals hierarchical correspondence https://www.nature.com/articles/srep27755 Activations of deep convolutional neural networks are aligned with gamma band activity of human visual cortex https://www.nature.com/articles/s42003-018-0110-y & multiple papers since: Layered structure of visual cortex Neurons focus on certain areas of visual input Lower layers process simple visual features, higher layers -- complex features
  41. 41. Vision: hierarchy of layers Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream https://www.jneurosci.org/content/35/27/10005 Comparison of deep neural networks to spatio-temporal cortical dynamics of human visual object recognition reveals hierarchical correspondence https://www.nature.com/articles/srep27755 Activations of deep convolutional neural networks are aligned with gamma band activity of human visual cortex https://www.nature.com/articles/s42003-018-0110-y & multiple papers since: Layered structure of visual cortex Neurons focus on certain areas of visual input Lower layers process simple visual features, higher layers -- complex features
  42. 42. Model of the world: grid cells The Nobel Prize in Physiology or Medicine 2014 "for their discoveries of cells that constitute a positioning system in the brain" http://www.scholarpedia.org/article/Grid_cells O'Keefe Moser Moser https://deepmind.com/blog/article/grid-cells https://www.nature.com/articles/s41586-018-0102-6
  43. 43. Model of the world: grid cells Entorhinal cortex The Nobel Prize in Physiology or Medicine 2014 "for their discoveries of cells that constitute a positioning system in the brain" http://www.scholarpedia.org/article/Grid_cells O'Keefe Moser Moser https://deepmind.com/blog/article/grid-cells https://www.nature.com/articles/s41586-018-0102-6
  44. 44. Model of the world: grid cells Entorhinal cortex The Nobel Prize in Physiology or Medicine 2014 "for their discoveries of cells that constitute a positioning system in the brain" http://www.scholarpedia.org/article/Grid_cells O'Keefe Moser Moser https://deepmind.com/blog/article/grid-cells https://www.nature.com/articles/s41586-018-0102-6
  45. 45. Model of the world: grid cells Entorhinal cortex The Nobel Prize in Physiology or Medicine 2014 "for their discoveries of cells that constitute a positioning system in the brain" http://www.scholarpedia.org/article/Grid_cells O'Keefe Moser Moser https://deepmind.com/blog/article/grid-cells https://www.nature.com/articles/s41586-018-0102-6
  46. 46. Model of the world: grid cells Entorhinal cortex The Nobel Prize in Physiology or Medicine 2014 "for their discoveries of cells that constitute a positioning system in the brain" http://www.scholarpedia.org/article/Grid_cells O'Keefe Moser Moser https://deepmind.com/blog/article/grid-cells https://www.nature.com/articles/s41586-018-0102-6
  47. 47. Model of the world: successor representations https://julien-vitay.net/2019/05/successor-representations/ Model-free RL + fast
 – inflexible to change http://gershmanlab.webfactional.com/pubs/Stachenfeld17.pdf
  48. 48. Model of the world: successor representations https://julien-vitay.net/2019/05/successor-representations/ Model-free RL + fast
 – inflexible to change Model-based RL + adaptable
 – hard to learn transition
 probs reward
 probs http://gershmanlab.webfactional.com/pubs/Stachenfeld17.pdf
  49. 49. Model of the world: successor representations https://julien-vitay.net/2019/05/successor-representations/ Model-free RL + fast
 – inflexible to change Model-based RL + adaptable
 – hard to learn transition
 probs reward
 probs Successor representation for RL discounted future
 state occupancy
 matrix expected 
 state
 reward Why learn the full model when we only 
 care about the chance of getting into a state http://gershmanlab.webfactional.com/pubs/Stachenfeld17.pdf
  50. 50. Model of the world: successor representations https://julien-vitay.net/2019/05/successor-representations/ Model-free RL + fast
 – inflexible to change Model-based RL + adaptable
 – hard to learn transition
 probs reward
 probs Successor representation for RL discounted future
 state occupancy
 matrix expected 
 state
 reward Why learn the full model when we only 
 care about the chance of getting into a state http://gershmanlab.webfactional.com/pubs/Stachenfeld17.pdf Hippocampus Place cells Gaussian distance to the location?
  51. 51. Model of the world: successor representations https://julien-vitay.net/2019/05/successor-representations/ Model-free RL + fast
 – inflexible to change Model-based RL + adaptable
 – hard to learn transition
 probs reward
 probs Successor representation for RL discounted future
 state occupancy
 matrix expected 
 state
 reward Why learn the full model when we only 
 care about the chance of getting into a state http://gershmanlab.webfactional.com/pubs/Stachenfeld17.pdf Hippocampus Place cells Gaussian distance to the location? Adding constraints to the environment modifies the shape of the place fields.
 SR hypothesis is in agreement with the observation that rewarded locations are represented by a higher number of place cells.
  52. 52. Dual DQN network MERLIN Architecture Predictive coding Visual attention ... Generative Adversarial Networks Predictive coding Successor representations Attention in CNNs Consolidation of experience
 Novel vs. routine circuits & & & &
  53. 53. Replay buffer Curious similarities on the level of algorithm and representation discounted future
 state occupancy
 matrix expected 
 state
 reward
  54. 54. Goal of computation
  55. 55. Same goal, different strengths... merge! Fetz Lab
 University of Washington NeuroChip Neuralink Elon Musk
 San Franciso, CA http://www.csne-erc.org/sites/default/files/Neurochip.pdf https://depts.washington.edu/fetzweb/closed-loop-bci.html https://neuralink.com/
  56. 56. Same goal, different strengths... merge! Fetz Lab
 University of Washington NeuroChip Neuralink Elon Musk
 San Franciso, CA http://www.csne-erc.org/sites/default/files/Neurochip.pdf https://depts.washington.edu/fetzweb/closed-loop-bci.html https://neuralink.com/
  57. 57. Same goal, different strengths... merge! Fetz Lab
 University of Washington NeuroChip Neuralink Elon Musk
 San Franciso, CA http://www.csne-erc.org/sites/default/files/Neurochip.pdf https://depts.washington.edu/fetzweb/closed-loop-bci.html https://neuralink.com/
  58. 58. Goal of the computation
 What is the purpose of computation? Algorithm and representation
 What representations does the system use?
 What processes are in use to manipulate representations? Implementation
 How is the system physically realized? The Brain and the Modern AI Almost the same Quite different Comparable
 here and there Ilya Kuzovkin
 
 ilya.kuzovkin@gmail.com
 www.ikuz.eu Neurotech Sydney meetup group Podcast by
 Paul Middlebrooks

×