The document discusses a model of sequence learning in the neocortex based on the "predictive neuron" concept. The model relies on active dendrites in pyramidal neurons and fast inhibitory networks to learn complex temporal sequences. It has been applied successfully to real-world streaming data applications. Key aspects of the predictive neuron model include identical pyramidal cells that can predict sensorimotor sequences through active dendrites, and feedback signals that provide additional context. The document also provides a detailed list of experimentally testable properties of the model and some early supporting experimental results.

1. Workshop: Insights Gained By
Detailed Dendritic Modeling
July 18, 2018
Subutai Ahmad
sahmad@numenta.com
@SubutaiAhmad
The Predictive Neuron: How Active Dendrites Enable
Spatiotemporal Computation In The Neocortex
Co-authors:
-Jeff Hawkins, Marcus Lewis, Scott Purdy,
Yuwei Cui

2. Observation:
The neocortex is constantly predicting its inputs.
“the most important and also the most neglected problem of
cerebral physiology” (Lashley, 1951)
How can networks of pyramidal neurons learn predictive
models of the world?
Research question:

3. 1) How can neurons learn predictive models of temporal sequences?
3) Experimentally testable predictions
- Impact of NMDA spikes
- Branch specific plasticity
- Sparse correlation structure
- Pyramidal neuron uses active dendrites for prediction
- A single layer network model for complex predictions
- Works on real world applications
- Basic model can be used in very flexible ways
- Sensorimotor sequences and feedback context
“Why Neurons Have Thousands of Synapses, a Theory of Sequence Memory in the Neocortex”
Hawkins and Ahmad, Frontiers in Neural Circuits, 2016/03/30
“A Theory of How Columns in the Neocortex Learn the Structure of the World”
Hawkins, Ahmad, and Cui, Frontiers in Neural Circuits, 2017/10/25
2) The predictive neuron

4. 5K to 30K excitatory synapses
- 10% proximal
- 90% distal
Distal dendrites are pattern detectors
- 8-15 co-active, co-located synapses
generate dendritic NMDA spikes
- sustained depolarization of soma but
does not typically generate AP
Pyramidal Neuron
(Mel, 1992; Branco & Häusser, 2011; Schiller et al, 2000; Losonczy, 2006; Antic
et al, 2010; Major et al, 2013; Spruston, 2008; Milojkovic et al, 2005, etc.)
Prediction Starts in the Neuron

5. Proximal synapses: Cause somatic spikes
Define classic receptive field of neuron
Distal synapses: Cause dendritic spikes
Put the cell into a depolarized, or “predictive” state
Depolarized neurons fire sooner, inhibiting nearby neurons.
A neuron can predict its activity in hundreds of unique contexts.
5K to 30K excitatory synapses
- 10% proximal
- 90% distal
Distal dendrites are pattern detectors
- 8-15 co-active, co-located synapses
generate dendritic NMDA spikes
- sustained depolarization of soma but
does not typically generate AP
HTM Neuron Model
Prediction Starts in the Neuron
Pyramidal Neuron
(Poirazi et al., 2003)
(Hawkins & Ahmad, 2016)

6. A Single Layer Network Model for Sequence Memory
- Neurons in a mini-column learn same FF receptive field.
- Active dendritic segments form connections to nearby cells.
- Depolarized cells fire first, and inhibit other cells within mini-column.
No prediction Predicted input
(Hawkins & Ahmad, 2016)
(Cui et al, 2016)
t=0
t=1
Predicted cells inhibit
neighbors
Next prediction t=2
t=0
t=1

7. Synaptic changes localized to dendritic segments:
(Stuart and Häusser, 2001; Losonczy et al., 2008)
1. If a cell was correctly predicted, positively reinforce the dendritic
segment that caused the prediction.
2. If a cell was incorrectly predicted, slightly negatively reinforce the
corresponding dendritic segment.
3. If no cell was predicted in a mini-column, reinforce the dendritic
segment that best matched the previous input.
Continuous Branch Specific Learning

8. X
A B
B
C
C
D
Y
Before learning
X B’’ C’’
D’
Y’’
After learning
A B’ C’
Same columns,
but only one cell active per column.
High Order (Non-Markovian) Sequences
Two sequences: A-B-C-D
X-B-C-Y

9. C’ predicted
Prediction of next input
A input B’ predicted B input
B input C input D’ AND Y” predictedC’ AND C” predicted
Sequence Prediction
Train on two sequences: A-B-C-D
X-B-C-Y
Surprise and multiple simultaneous predictions
Test without the starting elements:
B-C-?

10. Application To Real World Streaming Data Sources
- Accuracy is comparable to state of the art ML techniques (LSTM, ARIMA, etc.)
- Continuous unsupervised learning - adapts to changes far better than other techniques
- Top benchmark score in detecting anomalies and unusual behavior
- Extremely fault tolerant (tolerant to 40% noise and faults)
- Multiple open source implementations (some commercial)
“Continuous online sequence learning with an unsupervised neural network model”
Cui, Ahmad and Hawkins, Neural Computation, 2016
“Unsupervised real-time anomaly detection for streaming data”
Ahmad, Lavin, Purdy and Zuha, Neurocomputing, 2017
2015-04-20
Monday
2015-04-21
Tuesday
2015-04-22
Wednesday
2015-04-23
Thursday
2015-04-24
Friday
2015-04-25
Saturday
2015-04-26
Sunday
0 k
5 k
10 k
15 k
20 k
25 k
30 k
PassengerCountin30minwindow
A
B C
Shift
AR
IM
A
LSTM
1000
LSTM
3000
LSTM
6000
TM
0.0
0.2
0.4
0.6
0.8
1.0
NRMSE
0.00
0.05
0.10
0.15
0.20
0.25
0.30
0.35
MAPE
0.0
0.5
1.0
1.5
2.0
2.5
NegativeLog-likelihood
Shift
AR
IM
A
LSTM
1000
LSTM
3000
LSTM
6000
TM
LSTM
1000
LSTM
3000
LSTM
6000
TM
D
?
Taxi Demand Prediction Anomaly Detection on Machine Sensor Data

11. 1) How can neurons learn predictive models of temporal sequences?
3) Experimentally testable predictions
- Impact of NMDA spikes
- Branch specific plasticity
- Sparse correlation structure
- Pyramidal neuron uses active dendrites for prediction
- A single layer network model for complex predictions
- Works on real world applications
- Basic model can be used in very flexible ways
- Sensorimotor sequences and feedback context
2) The predictive neuron

12. Can Network Learn Predictive Models of Sensorimotor Sequences?
Sensorimotor sequences
Sensory inputMotor-related context
80 objects, designed for robotics grasping tasks
Model achieved 98.7% recall accuracy (77/78 uniquely classified)
Yale-CMU-Berkeley (YCB) Object Benchmark (Calli et al., 2017)
(Hawkins, Ahmad & Cui, 2017)

13. Can Network Untangle Sensorimotor and Temporal Sequences?
Sensorimotor sequences
A-B-C-D-X-B-C-Y-A-B-C-D-X-B-C-Y
Temporal sequences
Sensory inputMotor-related context
Classifier
Input is mixture of
sensorimotor and
extrinsic sequences
(Ahmad & Hawkins, 2017)

14. Prediction with Apical Dendrites and Feedback
Feedback signal represents
additional context and an
additional source of bias
Apical dendrites
Pooling Layer

15. 1) How can neurons learn predictive models of temporal sequences?
3) Experimentally testable predictions
- Impact of NMDA spikes
- Branch specific plasticity
- Sparse correlation structure
- Pyramidal neuron uses active dendrites for prediction
- A single layer network model for complex predictions
- Works on real world applications
- Basic model can be used in very flexible ways
- Sensorimotor sequences and feedback context
2) The predictive neuron

16. 1) Impact of NMDA spikes:
Dendritic NMDA spikes cause cells to fire faster than they would otherwise.
Fast local inhibitory networks (e.g. minicolumns) inhibit cells that don’t fire early.
Sparser activations during a predictable sensory stream.
For predictable natural stimuli, dendritic spikes will be more frequent than APs.
(Vinje & Gallant, 2002; Smith et al, 2013; Wilmes et al, 2016; Moore et al, 2017)
2) Branch specific plasticity:
Strong LTP in dendritic branch when NMDA spike followed by back action potential (bAP).
Weak LTP (without NMDA spike) if synapse cluster becomes active followed by a bAP.
Weak LTD when an NMDA spike is not followed by an action potential/bAP.
(Holthoff et al, 2004; Losonczy et al, 2008; Yang et al, 2014; Cichon & Gang, 2015)
3) Correlation structure:
Low pair-wise correlations between cells but significant high-order correlations.
High order assembly correlated with specific point in a predictable sequence.
Unanticipated inputs leads to a burst of activity, correlated within minicolumns.
Activity during predicted inputs will be a subset of activity during unpredicted inputs.
Neighboring mini-columns will be uncorrelated.
(Ecker et al, 2010; Smith & Häusser, 2010; Schneidman et al, 2006; Miller et al, 2014; Homann et al, 2017)
Properties And Experimentally Testable Predictions
16

17. Depolarization From NMDA Spike Decreases Somatic Spike Latency
(Weinan Sun, Janelia Labs, personal communication)

18. Correlation Structure With Natural Sequences
CellAssemblyOrder
3 4 5 6
Number
cellas
0
1
2
3
Time(sec)
5 10 15 20 25 300
50
100
150
Time(sec)
5 10 15 20 25 30
0
50
100
150
5 10 15 20 25 30
Neuron#
0
20
40
60
80
100
120
140
160
5 10 15 20 25 30
Neuron#
0
20
40
60
80
100
120
140
160
V1
AL
F
0.1
0.2
0.3
Prob.ofobserving
epeatedcellassembly
(Stirman et al, 2016)
Spencer L. Smith YiYi Yu
20 presentations of a 30-
second natural movie

19. Sparser Activity With Repeated Presentations
Time(sec)
5 10 15 20 25 30
Neuron#
0
20
40
60
80
100
120
140
160
Time(sec)
5 10 15 20 25 30
Neuron#
0
20
40
60
80
100
120
140
160
V1
-1 -0.5 0 0.5 1
0
0.1
0.2
0.3
Timejitter(sec)
Prob.ofobserving
repeatedcellassembly
CellAssemblyOr
3 4 5
Numb
cell
0
1
2
3
Time(sec)
5 10 15 20 25 300
50
100
Time(sec)
5 10 15 20 25 30
Neuron#
40
60
80
100
120
140
160
V1
AL
0.2
0.3
ofobserving
dcellassembly
Similar to (Vinje & Gallant, 2002)

20. Emergence of High Order Cell Assemblies
e(sec)
15 20 25 30
3-o
ass
sin
-1 -0.5 0 0.5 1
0
0.1
0.2
0.3
-1 -0.5 0 0.5 1
0
0.02
0.04
0.06
0.08
Timejitter(sec)
Prob.ofobserving
repeatedcellassembly
Timejitter(sec)
Prob.ofobserving
repeatedcellassembly
Cell assemblies are significantly more likely to
occur in sequences than predicted by a
Poisson model (p<0.001).
ec)
20 25 30
-1 -0.5 0 0.5 1
0
0.1
0.2
0.3
-1 -0.5 0
0
0.02
0.04
0.06
0.08
Timejitter(sec)
Prob.ofobserving
repeatedcellassembly
Timejitter(se
Prob.ofobserving
repeatedcellassembly
Sparse code predicts specific point in a
sequence (single cells don’t).
Similar to (Miller et al, 2014)

21. 1) Impact of NMDA spikes:
Dendritic NMDA spikes cause cells to fire faster than they would otherwise.
Fast local inhibitory networks (e.g. minicolumns) inhibit cells that don’t fire early.
Sparser activations during a predictable sensory stream.
For predictable natural stimuli, dendritic spikes will be more frequent than APs.
(Vinje & Gallant, 2002; Smith et al, 2013; Wilmes et al, 2016; Moore et al, 2017)
2) Branch specific plasticity:
Strong LTP in dendritic branch when NMDA spike followed by back action potential (bAP).
Weak LTP (without NMDA spike) if synapse cluster becomes active followed by a bAP.
Weak LTD when an NMDA spike is not followed by an action potential/bAP.
(Holthoff et al, 2004; Losonczy et al, 2008; Yang et al, 2014; Cichon & Gang, 2015)
3) Correlation structure:
Low pair-wise correlations between cells but significant high-order correlations.
High order assembly correlated with specific point in a predictable sequence.
Unanticipated inputs leads to a burst of activity, correlated within minicolumns.
Activity during predicted inputs will be a subset of activity during unpredicted inputs.
Neighboring mini-columns will be uncorrelated.
(Ecker et al, 2010; Smith & Häusser, 2010; Schneidman et al, 2006; Miller et al, 2014; Homann et al, 2017)
Properties And Experimentally Testable Predictions
21

22. - A model of sequence learning in cortex
- Relies on “predictive neuron” with active dendrites and fast inhibitory networks
- Can learn complex temporal sequences
- Applied to real world streaming applications
- Predictive neuron
- Identical network of pyramidal cells can predict sensorimotor sequences
- Feedback signal can add an additional source of bias
- Detailed list of experimentally testable properties
- Early results on some of these properties
22
Summary

23. Open Issues / Discussion
Are active dendrites necessary? (Yes!)
- Is a two layer network of uniform point neurons sufficient? (No!)
How to integrate calcium spikes, BAC firing, and apical dendrites?
Continuous time model of HTM, including inhibitory networks
Collaborations
We are always interested in hosting visiting scholars and interns.
Co-authors: Jeff Hawkins, Scott Purdy, Marcus Lewis (Numenta)
Contact info: sahmad@numenta.com
@SubutaiAhmad