The document discusses how the brain uses deep learning techniques similar to modern neural networks. It argues that the neocortex uses unsupervised learning like deep belief networks for pattern recognition. The basal ganglia use reinforcement learning like LSTM to learn from rewards and punishments via dopamine signals. Together, these deep learning techniques in different brain areas allow for hierarchical feature extraction, goal-directed learning of strategies and plans, and reinforcement of successful behaviors.
Tech-Forward - Achieving Business Readiness For Copilot in Microsoft 365
How Deep Learning Works in the Brain
1. Deep Learning in a Brain
What makes
our mind deep?
serge.shumsky@gmail.com
2. Brain is a survival machine
Basic tasks
What happens?
What should I do?
3. What makes it deep?
New (mammalian) brain
Deep unsupervised learning
Similar to Deep Belief Nets
Old (reptilian) brain
Deep reinforcement learning
Similar to Long Short Term
Memory
4. Neocortex
Homogeneous 2D tissue
Mammals: 2-3 mm
Humans: 75% of brain volume
Single basic algorithm
Pattern recognition
Unsupervised learning
5. Cortical (hyper)columns
1-3 mm
С. Boucsein, …“Beyond
the cortical column” (2011)
V. Mountcastle, “The columnar
organization of neocortex”
(1997)
0.3 mm
26. Cortex
Basal ganglia
Value of action a in state s
state s
action a
Actual reward 𝑟𝑡
Hypothalamus,
Amygdala, …
Reward prediction error
Basal ganglia: reinforcement learning
𝑄𝑡
±
𝑎, 𝑠
𝛿𝑡 = 𝑟𝑡 − 𝑄𝑡
+
+ 𝑄𝑡
−
Reward
prediction 𝑄𝑡
−
− 𝑄𝑡
+
27. Cortex
Basal ganglia
Basal ganglia: reinforcement learning
𝑄𝑡
+
→ 𝑄𝑡−1
+
+ 𝛿𝑡 = … + 𝛿𝑡−2 + 𝛿𝑡−1 + 𝛿𝑡
𝑄𝑡
−
→ 𝑄𝑡−1
−
− 𝛿𝑡 = 𝛿𝑡+1+ 𝛿𝑡+2 + 𝛿𝑡+3 + ⋯
Accumulated joy
Anticipated joy
state s
action a
28. Mammals – dopamine addicts
Curious – we like surprises
Explorative behavior
Motivation for learning
Forward thinking
Anticipated, not immediate reward
Value-based, not reactive behavior