Deep learning in a brain (skolkovo 2016)
•Download as PPTX, PDF•
1 like•791 views
The document discusses how the brain uses deep learning techniques similar to modern neural networks. It argues that the neocortex uses unsupervised learning like deep belief networks for pattern recognition. The basal ganglia use reinforcement learning like LSTM to learn from rewards and punishments via dopamine signals. Together, these deep learning techniques in different brain areas allow for hierarchical feature extraction, goal-directed learning of strategies and plans, and reinforcement of successful behaviors.
More Related Content
Similar to Deep learning in a brain (skolkovo 2016)
Similar to Deep learning in a brain (skolkovo 2016) (20)
More from Skolkovo Robotics Center
More from Skolkovo Robotics Center (20)
Recently uploaded
Recently uploaded (20)
Deep learning in a brain (skolkovo 2016)
- 1. Deep Learning in a Brain What makes our mind deep? serge.shumsky@gmail.com
- 2. Brain is a survival machine Basic tasks What happens? What should I do?
- 3. What makes it deep? New (mammalian) brain Deep unsupervised learning Similar to Deep Belief Nets Old (reptilian) brain Deep reinforcement learning Similar to Long Short Term Memory
- 4. Neocortex Homogeneous 2D tissue Mammals: 2-3 mm Humans: 75% of brain volume Single basic algorithm Pattern recognition Unsupervised learning
- 5. Cortical (hyper)columns 1-3 mm С. Boucsein, …“Beyond the cortical column” (2011) V. Mountcastle, “The columnar organization of neocortex” (1997) 0.3 mm
- 6. Cortex: Self-Organizing Maps Hebb learning rule: “Fire together – wire together” Recognition 0.3 мм T. Kohonen (1982) 1-3 мм
- 7. V1 map 1 mm
- 8. Cortical hierarchy A.R. Luria, “Higher Cortical Functions in Man” (1962)
- 9. Cortical hierarchy A.R. Luria, “Higher Cortical Functions in Man”(1962) RBM
- 10. Deep Neural Networks G. Hinton, “Deep belief networks” (2009)
- 11. Features Organization of behavior Sensors Strategies Plans Acts Muscles Scenes Goal-directed learning? 0,3 сек
- 15. Strategies Scenes Plans Who & how choose strategies? Reinforcement learning
- 16. Subcortical structures Basal Ganglia Thalamus Neocortex
- 17. Thalamus: Attention Cortex Thalamus ~ 40 Gz Positive feedback Gamma-rhythm Self-excitation
- 18. Thalamus: Attention, Binding Cortex Thalamus ~ 40 Gz ~ 40 Gz Sinchronization LaBerge “Triangular Circuit Theory of Attention” (1998)
- 19. Thalamus: Attention, Binding ~ 40 Gz ~ 40 Gz Thalamus ColorShape Word Notion TRIANGLE triangle
- 20. Cortical-subcortical interaction Basal Ganglia Thalamus Neocortex
- 21. Basal ganglia: global inhibition Striatum – value function Pallidus – action control Substantia nigra – reinforcement
- 22. Basal ganglia: action control Cortex Thalamus Striatum Pallidus Substantia nigra Basal ganglia brake No activity without Striatum approval
- 23. Cortex Striatum Pallidus Activity approved by Striatum Basal ganglia Substantia nigra Basal ganglia: action control Thalamus LSTM Hochreiter, Schmidhuber (1997)
- 24. Cortex Thalamus Basal ganglia Striatum Pallidus Weights strengthening by dopamine signal Substantia nigra Basal ganglia: reinforcement learning
- 25. Dopamine reinforcement signal (hormone of joy) From Substantia nigra From cortex D+ D–
- 26. Cortex Basal ganglia Value of action a in state s state s action a Actual reward 𝑟𝑡 Hypothalamus, Amygdala, … Reward prediction error Basal ganglia: reinforcement learning 𝑄𝑡 ± 𝑎, 𝑠 𝛿𝑡 = 𝑟𝑡 − 𝑄𝑡 + + 𝑄𝑡 − Reward prediction 𝑄𝑡 − − 𝑄𝑡 +
- 27. Cortex Basal ganglia Basal ganglia: reinforcement learning 𝑄𝑡 + → 𝑄𝑡−1 + + 𝛿𝑡 = … + 𝛿𝑡−2 + 𝛿𝑡−1 + 𝛿𝑡 𝑄𝑡 − → 𝑄𝑡−1 − − 𝛿𝑡 = 𝛿𝑡+1+ 𝛿𝑡+2 + 𝛿𝑡+3 + ⋯ Accumulated joy Anticipated joy state s action a
- 28. Mammals – dopamine addicts Curious – we like surprises Explorative behavior Motivation for learning Forward thinking Anticipated, not immediate reward Value-based, not reactive behavior
- 29. “Deep control” architecture Motor Execution Motivation
- 30. “Deep control” architecture r s a s maxa s maxa s maxa s maxa Deep Belief networks Recurrent Reinforcement networks Motor Execution Motivation
- 31. Takeaway The Brain is not THAT complex We can HACK IT