Jeff Hawkins presented a talk on "The Thousand Brains Theory: A Roadmap to Machine Intelligence" at the Beijing Academy of Artificial Intelligence Conference on 1st June 2021. In this talk, he discussed the key components of The Thousand Brains Theory and Numenta's recent work.

1. The Thousand Brains Theory
A Roadmap for Creating Machine Intelligence
Jeff Hawkins
Research company in California
1) Reverse engineer the neocortex
2) Create machine intelligence
using brain principles
Chinese edition
Cheers Publishing Co.

2. Neocortex
“Older” brain areas
Dozens of specialized brain regions
30% of brain by volume
- Breathing, digestion, reflex behaviors
- Walking, running, chewing
- Emotions
One continuous sheet of neural tissue
70% of brain by volume
- Perception
- Language
- Cognition, thought, planning
(engr., math, science, literature….)
The Neocortex is the organ of
intelligence.
If we understood how it works,
we would know how to build
intelligent machines.

3. The Neocortex Learns a Model of the World.
- How things look, feel, and sound
- Where things are located
- How things change when we interact with them
- Includes tens of thousands of objects, words, and concepts
1) Everything you know is stored in this model.
2) The brain’s model allows us to:
- Recognize objects and where we are
- Predict the consequences of our actions
- Plan and achieve goals

5. x
y
z

6. The Neocortex Learns a Model of the World.
- How things look, feel, and sound
- Where things are located
- How things change when we interact with them
- Includes tens of thousands of objects, words, and concepts
1) Everything you know is stored in the model.
2) The brain’s model allows us to:
How does the neocortex learn a model of the world?
- Recognize where we are
- Predict the consequences of our actions
- Plan and achieve goals
3) Intelligence requires learning a model of the world
and updating it continuously.

7. The Neocortex looks uniform.
But it is divided into dozens of functional regions.
Somatic regions
Visual regions
Auditory regions
Language regions

8. The circuits of the neocortex look similar everywhere.
Common Circuitry
- Types of neurons
- Organized in layers
- Connections between layers
- Sensory input
- Motor output
L3
L4
L6a
L6b
L5a
L5b
L2
Cajal, 1899
2.5
mm
How is it possible that the neocortex looks similar everywhere?
sense motor

9. Vernon Mountcastle’s Big Idea
1) All areas of the neocortex look the same because they perform the same intrinsic function.
What makes one region visual and another auditory is what it is connected to.
2) The human neocortex got large by copying a functional unit, the “cortical column.”
(~1mm2, 150K columns, 100K neurons per column)
What does a cortical column do?
Mountcastle 1997
Completely
Heterogeneous
Completely
Homogeneous
common

10. Thought Experiment

11. The Thousand Brains Theory
1) Columns learn models by integrating sensory input and movement over time.
Location
(Reference Frame)
Sensed feature
Movement
Sensed
feature
Column
Object

12. Location
Reference Frame
Object
Sensed feature
Vision is similar to touch
A patch of the retina is
analogous to a patch of
skin
Our perception is stable
While inputs are changing
Column 1 Column 2 Column 3
Columns vote to reach a consensus
The Thousand Brains Theory
1) Columns learn models by integrating sensory input and movement over time.
2) There are thousands of models for every object

13. Example Simulation

14. Feature
Feature
Feature
Location
Location
Location
Output
Input
Illustration of Inference

15. Feature
Location
Feature
Location
Feature
Location
Column 1 Column 2 Column 3
Output
Input
Illustration of Voting (faster inference)

16. Yale-CMU-Berkeley (YCB) Object Benchmark (Calli et al, 2017)
- 80 objects designed for robotics grasping tasks
- Includes high-resolution 3D CAD files
YCB Object Benchmark
We created a virtual hand using the Unity game engine
Curvature based sensor on each fingertip
4096 neurons per layer per column
98.7% recall accuracy (77/78 uniquely classified)
Convergence time depends on object, sequence of
sensations, number of fingers.
Simulation using YCB Object Benchmark

17. Pairwise confusion between objects after 1
touch
Convergence 1 finger 1 touch

18. Pairwise confusion between objects after 2
touches
Convergence 1 finger 2 touches

19. Pairwise confusion between objects after 6
touches
Convergence 1 finger 6 touches

20. Pairwise confusion between objects after 10
touches
Convergence 1 finger 10 touches

21. Convergence Time vs. Number of Columns
This is why we can infer complex objects in a single grasp or single visual fixation.

22. Location
Reference Frame
Sensed feature
at location
Movement
Sensed
feature
Column
“Grid” cells
“Place” cells
Proposal:
Cortical columns create models using the same mechanisms as
grid cells and place cells use to model environments.
Hawkins et.al. 2017, 2019
Prediction:
Cortical columns will have cells that are equivalent to:
- Grid cells
- Place cells
- Object vector cells
- etc.
=

23. Doeller, C. F., Barry, C., & Burgess, N. (2010). Evidence
for grid cells in a human memory network. Nature
Grid cells exist in pre-frontal cortex, used to model concepts.
Constantinescu, A., O’Reilly, J., Behrens, T. (2016)
Organizing Conceptual Knowledge in Humans with a
Gridlike Code. Science

24. Grid cells, place cells, border cells in Somatosensory cortex
Xiaoyang Long & Sheng-Jia Zhang (2021) A novel
somatosensory spatial navigation system outside the
hippocampal formation. Cell Research

25. AI and Machine Intelligence

26. What Does the Thousand Brains Theory Tell Us About
Machine Intelligence?
1) Intelligent machines need to learn a model of the world.
- Inference, prediction, planning, and motor behavior are
based on the model.
2) The model is distributed among many nearly identical units
that vote to reach a consensus.
- Highly robust
- Scales from small to large systems
- Works with any type and size of sensor array
- Voting solves the binding problem
3) In each unit, knowledge is stored in reference frames and is
learned via sensory-motor interaction.
- Unsupervised learning
- Fast learning
- Motor behavior is integrated (robotic / AI fusion)

27. Point neuron
Sparsity Active dendrites Reference frames Cortical columns
ROADMAP: FROM ANNS TO MACHINE INTELLIGENCE
Robustness and performance
• Sparse activations and weights
• Robust to noise
• Custom sparse processing logic
• 50X to 100X more efficient
• Scale to large models

28. Network
Mean
accuracy
Mean accuracy
with noise
Non-zero
weights
Sparsity
Dense CNN 97.05% 31.08% 1,700,000 0%
Sparse CNN 97.03% 44.45% 160,952 90.6%
Dataset of spoken commands
• One word utterances, thousands of individuals
• State of the art accuracy is 95 - 97.5% for 10 categories
• Tested robustness to white noise
1) Networks used two sparse CNN layers + one sparse linear layer + one softmax output layer.
2) Trained with random static sparse masks
GOOGLE SPEECH COMMANDS DATASET

29. Name of chip Network
type
Throughput
for single
network
Speedup
over
dense
Number of
networks on
chip
Full chip
throughput
Full chip
speedup
Alveo U250 Dense 3,049 - 4 12,195 -
Alveo U250 Sparse 102,564 33.63 20 1,369,863 112.3
ZU3EG Dense 0 - 0 0 -
ZU3EG Sparse 45,455 Infinite 1 45,455 Infinite
SPARSE NETWORKS: MORE THAN 100X FASTER
Overall >100X throughput
Each network is
>30X faster
Dense network does not
even fit on the small chip

30. Point neuron
Sparsity Active dendrites Reference frames Cortical columns
ROADMAP: FROM ANNS TO MACHINE INTELLIGENCE
Robustness and performance
• Sparse activations and weights
• Robust to noise
• Custom sparse processing logic
• 50X to 100X more efficient
• Scale to large models
Continuous self-supervised learning
• Learn new patterns without
disrupting existing patterns
• Learn from prediction errors
• Requires far less labeled data
Invariant representations
• Much smaller training sets
• Compositional structures
• Improved generalization
Common cortical algorithm
• Common repeating
circuit for intelligence
• Integrated sensorimotor
• Highly scalable
• Advanced robotics
Contact:
Jeff: jhawkins@numenta.com
Papers: numenta.com/papers
Collaborators: Subutai Ahmad, Marcus Lewis, Luiz Scheinkman, Lucas
Souza, Kevin Hunter, Michaelangelo Caporale, Karan Grewal, Scott Purdy,
Yuwei Cui.
Basic Books (English)
Cheers Publishing (Chinese)