Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Jeff Hawkins Human Brain Project Summit Keynote: "Location, Location, Location - A Framework for Intelligence and Cortical Computation"

641 views

Published on

Jeff Hawkins delivered this keynote presentation at the 2018 Human Brain Project Summit Open Day in Maastricht, the Netherlands on October 15, 2018. A screencast recording of the slides is also available at: https://numenta.com/resources/videos/jeff-hawkins-human-brain-project-screencast/

Published in: Science
  • Be the first to comment

  • Be the first to like this

Jeff Hawkins Human Brain Project Summit Keynote: "Location, Location, Location - A Framework for Intelligence and Cortical Computation"

  1. 1. Human Brain Project Summit Location, Location, Location A Framework for Intelligence and Cortical Computation October 15, 2018 Maastricht, Netherlands Jeff Hawkins jhawkins@numenta.com
  2. 2. “In spite of the accumulation of detailed knowledge, how the human brain works is still profoundly mysterious” To understand the brain we need “new ways of thinking about it,” more experimental data will not be sufficient “What is conspicuously lacking is a framework of ideas within which to interpret all these different approaches.” “It is not that most neurobiologists do not have some general concept what is going on. The trouble is the concept is not precisely formulated. Touch it and it crumbles.” Crick 1979
  3. 3. The Human Neocortex 75% of brain Organ of intelligence How it works is still a mystery The most important scientific problem of all time Talk Outline 1) Background 2) A Framework for Intelligence and Cortical Computation 3) Implications
  4. 4. Regions and Hierarchy Retina Simple features Complex features ObjectRegion 3 Region 2 Region 1 Somatic regions Visual regions Auditory regions Felleman, van Essen, 1991 Region to region connectivity Complex Not strictly hierarchical (40% of all possible connections exist) Local circuitry Remarkably similar everywhere Macaque monkey Skin Simple features Complex features Object Multi-modal Object
  5. 5. Local Cortical Circuits Dozens of neuron types Organized in layers Prototypical projections across layers Limited horizontal projections All regions have a motor output Similar circuits in all regions L3 L4 L6a L6b L5a L5b Sense Motor L2 Cajal, 1899 2.5mm
  6. 6. Vernon Mountcastle’s Big Idea 1) All areas of the neocortex look the same because they perform the same basic function. 2) What makes one region visual and another auditory is what it is connected to. 3) A small area of cortex, a 1mm2 “cortical column”, is the unit of replication and contains the common cortical algorithm. Mountcastle, 1978
  7. 7. Q. What Does the Neocortex Do? A. The neocortex learns a model of the world - Thousands of objects, how they look, feel, and sound - Where objects are located relative to other objects - How objects behave - Physical and abstract objects - A Predictive model Q. How does the neocortex learn this model?
  8. 8. Talk Outline 1) Background 2) A Framework for Intelligence and Cortical Computation 3) Implications
  9. 9. Thought Experiment
  10. 10. L2/3 L4 L6a Location relative to object Object A single column learns completes models of objects by integrating features and locations over time. “A Theory of How Columns in the Neocortex Enable Learning the Structure of the World” (Hawkins, et. al., 2017) Multiple columns can infer objects in a single sensation by “voting” on object identity. ? Sensed feature
  11. 11. Grid Cells “Grid Cell” neurons in entorhinal cortex represent the location of the body relative to an environment. The Big Idea: Grid cells also exist in the neocortex. Cortical grid cells represent the location of sensory input relative to objects.
  12. 12. How Grid Cells Represent Location
  13. 13. How Grid Cells Represent Location A grid cell fires at multiple locations as the animal moves. A grid cell’s activity is updated by copies of motor commands. Firing locations of two grid cells Grid cells cannot represent a unique location. Grid cell “modules” differ by scale and orientation. Module 1 Module 2 Unique location Representation of location is unique to position in room and to the room. Stensola, Solstad, Frøland, Moser, Moser: 2012
  14. 14. Entorhinal Cortex Learns environments Grid cells represent location of body relative to room Location representation is updated by movement Each room has a unique “location space” Room 1 Room 2 Cortical grid cells represent location of sensor input relative to object Location representation is updated by movement Each object has a unique “location space” Neocortex Learns objects
  15. 15. L2/3 L4 L6a Object Location relative to object Sensed feature ?Grid Cell Modules Lewis et. al., 2018
  16. 16. Our Proposal So Far 1) Grid cells exist in every cortical column. They represent the location of the input to the column relative to the object being sensed. 2) Each column learns complete models of objects. 3) Objects have their own unique “location space”. This defines a location-based framework for understanding the neocortex and suggests solutions other mysteries.
  17. 17. Compositional Structure x z y a c b Cup is a previously learned object. Logo is a previously learned object. How can we rapidly and efficiently learn new object, “cup with logo”, without relearning cup or logo? Objects are composed of other objects, arranged in a particular way. Cup and logo have their own location spaces. Cup with logo can be represented by a single transform (blue arrow) that converts any location in cup space to an equivalent location in logo space.
  18. 18. Displacement Cells (proposed) x z y a c b Location “x” on logot=2 module 1 module 2 module n Location “a” on cupt=1Grid cells Displacement cells “Logo on cup” at a particular position Hawkins et. al., 2018
  19. 19. L2/3 L4 L6a Object Location Sensed feature Grid Cell Modules Displacement Cell Modules L5 Hawkins et. al., 2018
  20. 20. Object Behaviors Object behaviors can be represented and learned as sequences of displacements. Displacement N Displacement A Hawkins et. al., 2016 (sequence memory)
  21. 21. Rethinking Hierarchy All columns learn models of objects. If columns observe the same object then connections between them are useful for resolving ambiguity. (Solves “sensor fusion” problem.) SenseSense Simple features Complex features Object Multi-modal Object Classic view “Thousand Brains Theory of Intelligence” Retina Skin
  22. 22. 1) Cortical Columns: They are far more powerful than currently believed Columns learn complete models of objects Including sub-objects and behaviors Grid cells and displacement cells define location spaces for objects and their relative positions 2) Neocortex as a Whole: Composed of thousands of models Models differ based on their inputs (Mountcastle) Long range connections allow columns to vote Francis Crick We need “new ways of thinking about the brain” We need “a broad framework in which to interpret” experimental results Does this Framework Apply to Concepts and Other Forms of Intelligence?
  23. 23. Object-centric location signal in sensory regions: Cells fire only if feature is at object-centric location on object, even in V1 and V2. (Zhou et al., 2000; Willford & von der Heydt, 2015) Grid cells in neocortex: Human neocortex shows grid cell-like signatures (fMRI and single cell recordings) (Doeller et al., 2010; Jacobs et al. 2013; Constantinescu et al., 2016; ) Sensorimotor prediction in sensory regions: Cells predict their activity before a movement is completed. (Duhamel et al., 1992; Nakamura and Colby, 2002; Li and DiCarlo, 2008) Neocortex evolved from brain areas involved in mapping and navigation: Six-layer neocortex evolved by stacking 3-layer hippocampus and piriform cortex (Jarvis et al., 2005; Luzatti, 2015) Experimental Support for a Location-based Framework 24
  24. 24. Implications 1) Neuroscience 2) Theoretical foundation for - Pedagogy - Belief and false belief - Limits of human intelligence - Diseases of the mind 3) Artificial Intelligence and Robotics
  25. 25. True AI requires - Distributed models (Thousand Brains Theory of Intelligence) - Each model built using: Object-centric locations and location spaces Compositional structure Learning through movement - Embodiment: AI and Robotics are not separable True AI does not have to be human like - Faster, Larger, Smaller - Different sensors - New embodiments, including virtual
  26. 26. Implications of a Neocortical Theory 1) Neuroscience 2) Theoretical foundation for - Pedagogy - Belief and false belief - Limits of human intelligence - Diseases of the mind 3) Artificial Intelligence and Robotics - Purpose-built brains, e.g. for mathematics or physics - Virtual brains, e.g. for cyber-security - Real robotics, for industry, space exploration, and colonization www.Numenta.com
  27. 27. Numenta Team Subutai Ahmad Marcus Lewis Thank You Scott Purdy Luiz Scheinkman Mirko Klukas

×