Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Location, Location, Location - A Framework for Intelligence and Cortical Computation

798 views

Published on

Jeff Hawkins gave this presentation as part of the Johns Hopkins APL Colloquium Series on Septemer 21, 2018.

View the video of the talk here: https://numenta.com/resources/videos/jeff-hawkins-johns-hopkins-apl-talk/

Published in: Science
  • Be the first to comment

Location, Location, Location - A Framework for Intelligence and Cortical Computation

  1. 1. Johns Hopkins APL September 21, 2018 Jeff Hawkins jhawkins@numenta.com Location, Location, Location A Framework for Intelligence and Cortical Computation
  2. 2. The Human Neocortex 75% of brain Organ of intelligence How it works is a mystery Solving this mystery is the most important scientific problem of all time Talk Outline 1) Background 2) Recent Advances 3) Implications
  3. 3. Regions and Hierarchy Retinal array Simple features Complex features Objects Region 3 Region 2 Region 1 Somatosensory Visual regions Auditory Felleman, van Essen, 1991 Hierarchy is complex… 40% of all possible connections between regions exist. However… Local circuitry is remarkably similar in all regions and in all species. Macaque monkey
  4. 4. Local Cortical Circuits Dozens of neuron types Organized in layers Prototypical projections across all layers Limited horizontal projections All regions have a motor output Similar circuitry in all regions L3 L4 L6a L6b L5a L5b Sense Motor L2 Cajal, 1899 2.5mm
  5. 5. Vernon Mountcastle’s Big Idea 1) All areas of the neocortex look the same because they perform the same basic function. 2) What makes one region a visual region and another an auditory region is what it is connected to. 3) A small area of cortex, a 1mm2 “cortical column”, is the unit of replication and contains the common cortical algorithm. Mountcastle, 1978
  6. 6. What Does the Neocortex Do? The neocortex learns a model of the world - Thousands of objects, how they appear on the sensors - Where objects are located relative to other objects - How objects behave - Learned via movement of sensors - Physical and abstract objects Mountcastle corollary: If the neocortex learns models of objects, then each column learns models of objects (including morphology, location relative to other objects, and behaviors)
  7. 7. Talk Outline 1) Background 2) Recent Advances 3) Implications
  8. 8. Thought Experiment
  9. 9. L3 L4 L6a Location relative to object Object A single column learns completes models of objects by integrating features and locations over time. “A Theory of How Columns in the Neocortex Enable Learning the Structure of the World” (Hawkins, et. al., 2017) Multiple columns can infer objects in a single sensation by “voting” on object identity. ? Sensed feature
  10. 10. How Can Neurons Represent Object-centric Location? “Grid Cell” neurons in entorhinal cortex represent the location of the body relative to an environment. The Big Idea: Grid cell mechanisms were preserved, also exist in the neocortex. Cortical grid cells define a location-based framework for understanding how the neocortex functions.
  11. 11. How Grid Cells Represent Location
  12. 12. How Grid Cells Represent Location A grid cell fires at multiple locations as the animal moves. The locations are “anchored” by sensory input, and “updated” by motor commands. Firing locations of two grid cells Grid cells cannot represent a unique location. Grid cell “modules” differ by scale and orientation. Module 1 Module 2 Unique location If grid cell modules anchor independently, then location is unique to position in room and to the room. Stensola, Solstad, Frøland, Moser, Moser: 2012
  13. 13. Grid cells represent location of body in room Representation of locations are unique to each room Location is updated by movement Each room has its own “location space” Cortical grid cells represent location sensor input on object Representation of locations are unique to each object Location is updated by movement Each object has its own “location space” Grid cells in older part of the brain Learns models of environments Room 1 Room 2 Grid cells in the neocortex Learns models of objects
  14. 14. L3 L4 L6a Object Location relative to object Sensed feature ?Grid Cell Modules Lucas et. al., submitted
  15. 15. Our Proposal So Far 1) Grid cells exist throughout the neocortex, in every column. 2) They represent the location of the input to the column relative to the object being sensed. 3) Individual columns learn complete models of objects. (by integrating input+location over movement) 4) Each object has its own “location space”. This creates a framework for reverse engineering the rest of the neocortex……
  16. 16. Compositional Structure x z y a c b Cup is a previously learned object. Logo is a previously learned object. How can we rapidly and efficiently learn new object, “cup with logo”, without relearning cup or logo? Everything in the world is composed of other things, arranged in a particular way. How is this accomplished? Cup and logo have their own location spaces. Cup with logo can be represented by a single transform (blue arrow) that converts any location in cup space to an equivalent location in logo space.
  17. 17. Displacement Cells (hypothesized) x z y a c b Location “x” on logot=2 module 1 module 2 module n Location “a” on cupt=1Grid cells Displacement cells “Logo on cup” at a particular position Hawkins et. al., in process
  18. 18. L3 L4 L6a Object Location Sensed feature Grid Cell Modules Displacement Cell Modules L5 Hawkins et. al., in process
  19. 19. Object Behaviors Object behaviors can be represented and learned as sequences of displacements. Displacement N Displacement A Hawkins et. al., 2016 (sequence memory)
  20. 20. Thousand Brains Theory of Intelligence a new understanding of hierarchy Sense array Objects Objects Objects Sense array Every column learns models of objects. Each model is different depending on its inputs. If two columns learn models of the same object then connections between them are useful. (Solves “sensor fusion” problem.)
  21. 21. 1) Object-centric location signal in sensory cortex: Cells fire only if feature is at object-centric location on object, even in V1 and V2. (Zhou et al., 2000; Willford & von der Heydt, 2015) 2) Grid cells in neocortex: Human neocortex shows grid cell-like signatures (fMRI and single cell recordings) Seen while subjects navigate conceptual object spaces and virtual environments. (Doeller et al., 2010; Jacobs et al. 2013; Constantinescu et al., 2016; ) 3) Sensorimotor prediction in sensory regions: Cells predict their activity before a movement is completed. (Duhamel et al., 1992; Nakamura and Colby, 2002; Li and DiCarlo, 2008) 4) Neocortex evolved from older brain areas involved in mapping and navigation: Six-layer neocortex evolved by stacking 3-layer hippocampus and piriform cortex (Jarvis et al., 2005; Luzatti, 2015) Empirical Support for Theory 22
  22. 22. Implications of Neocortical Theory 1) Neuroscience 2) Theoretical foundation for - epistemology - science of belief and false belief - limits of human intelligence - pedagogy - diseases of the mind 3) Artificial Intelligence and Robotics
  23. 23. True AI requires - Thousand brains model of intelligence - Each model built using: - Object-centric locations and location spaces - Compositional structure - Embodiment and learning through movement True AI does not have to be human like - Faster, Larger or Smaller - Different sensors - Physically distributed - New embodiments, including virtual
  24. 24. Implications of Neocortical Theory 1) Neuroscience 2) Theoretical foundation for - epistemology - science of belief and false belief - limits of human intelligence - pedagogy - diseases of the mind 3) Machine Intelligence and Robotics - purpose-built brains, e.g. for mathematics or physics - virtual brains for cyber-security, cyber-warfare - robotics for industry and military - robotics for space exploration and colonization
  25. 25. Numenta Team Subutai Ahmad Marcus Lewis Thank You Scott Purdy Mirko Klukas Luiz Scheinkman

×