Embodied Cognition with Pproject Intu

133 views

Published on

Grady Booch, IBM Fellow and IBM’s Chief Scientist for Watson, presented “Embodied Cognition with Project Intu” as part of the Cognitive Systems Institute Speaker Series on December 8, 2016

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
133
On SlideShare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
7
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Embodied Cognition with Pproject Intu

  1. 1. © 2016 IBM Corporation 1 Embodied Cognition with Project Intu Grady Booch Chief Architect, Watson/M gbooch@us.ibm.com @grady_booch
  2. 2. © 2016 IBM Corporation 2 Imagine unleashing Watson in the physical world. Give it eyes, ears, and touch, then let it act in that world with hands and feet and a face, not just as an action of force but also as an action of influence. This is embodied cognition: by placing the cognitive power of Watson in a robot, in an avatar, an object in your hand, or even in the walls of an operating room, conference room, or spacecraft, we take Watson's ability to understand and reason and draw it closer to the natural ways in which humans live and work. In so doing, we augment individual human senses and abilities, giving Watson the ability see a patient's complete medical condition, feel the flow of a supply chain, or orchestrate the tasks in a day in the life of an individual.
  3. 3. © 2016 IBM Corporation 3 Embodied Cognition Defined § An embodied cognition must – Be in and of the world – Reason – Learn – Have identity § An embodied cognition is not – Simply an STT -> NLC -> action pipeline
  4. 4. © 2016 IBM Corporation 4 An Agent May Be Embodied In A Robot…
  5. 5. © 2016 IBM Corporation 5 An Avatar…
  6. 6. © 2016 IBM Corporation 6 A Space…
  7. 7. © 2016 IBM Corporation 7 Or An Object
  8. 8. © 2016 IBM Corporation 8 Embodied Cognition 8 Use Cases Concierge (Robot, Avatar, Space) Where is the elevator? Retail (Robot, Avatar, Space) Do you wanna build a snowman? Elder Care (Robot, Avatar, Space) I’ve fallen and I can’t get up! Cobot (Robot) Get me a screwdriver. Manufacturing (Robot) Watch me do this. Transportation (Robot, Avatar, Space) Open the pod bay doors, Watson. Boardroom (Avatar, Space) Help me decide. Companion (Avatar, Device) Let’s play a game.
  9. 9. © 2016 IBM Corporation 9 Points of Technical Confluence Who said that? Sensory Fusion I feel sad. Theory of Mind I need the Phillips head screwdriver. Context Give me that one; no, I mean that one! Embodied Conversation It’s good to see you again, Sandia! Learning I’ve just picked up a fault in the AE35 unit. Devices In what room was Alyssa working yesterday? Knowledge Representation
  10. 10. © 2016 IBM Corporation 10 Self Basic Principles § Augment human capabilities § Learn, don’t program § Theory of mind and social intelligence § Self understanding § Platform agnostic § Embodiment as a robot, an avatar, a device, or a space § Deployment as middleware with microservices in the cloud
  11. 11. © 2016 IBM Corporation 11 Self Significant Design Decisions § Self is a hybrid architecture, encompassing explicit symbolic computation in the center together with neural networks at the edges. § Inspired by Minsky’s Society of Mind, behavior takes place in the context of multiple concurrent agents that communicate opportunistically via blackboards and deterministically via peer to peer connections. § Inspired by Brooks’s subsumption architecture, behavior takes place in a hierarchy of cognition, from involuntary reflexes to voluntary skills to goals and planning. § We maintain a clear separation of concerns among perception, actuating, models, and behavior. § As much as possible, behavior is either taught or is learned, not programmed. § As much as possible – driven by these separation of concerns, the needs of packaging, and performance – all components are made manifest as RESTful microservices. § As much as possible, plans, skills, and reflexes are extensible. § Self is intentionally full of strange loops: components of Self are also parts of the models of itself. § Self is intentionally fractal: an instance of Self may have models of others, which themselves are other instances of Self.
  12. 12. © 2016 IBM Corporation 12 Self 12 Sensors & Percep,on Actuators Agents Goals & Planning Pla5orm Management Infrastructure Meta Management Models Voluntary Behavior & Skills Involuntary Behavior & Reflexes Others World Self Architecture
  13. 13. © 2016 IBM Corporation 1313 Project Intu An Experimental platform for Embodied Cognition Intu is about behavior -- about creating presence; about giving the cognitive mind a body through which it can express itself with intelligent relevance Coordinates cognitive services seamlessly Framework for tying multiple services together with different external sensors, actuators, and services (i.e. Speaker/Microphone with STT/Conversation Services) Connects & manages various devices for scale Ability to extend cognitive services to new devices bringing reasoning and learning in the physical world with minimum coding (i.e. Avatars and Devices) Expands access Capability to connect disparate and trained knowledge sources together and integrate 3rd-party services for robust experience (i.e. Weather API or Nexmo API, etc.) Simplifies and enhances cognitive services Pre-trained capabilities or “Behaviors” that work across various devices and operating systems enrich and create an immersive experience (i.e. Emotion and Gestures) Ibm.biz/ProjectIntu avatars In-the-walls devices spaces wearables robotics
  14. 14. © 2016 IBM Corporation 14 Project Intu An Experimental platform for Embodied Cognition Ibm.biz/ProjectIntu
  15. 15. © 2016 IBM Corporation 15 Project Intu An Experimental platform for Embodied Cognition Ibm.biz/ProjectIntu
  16. 16. © 2016 IBM Corporation 16 Embodied Cognition Core Technology Topic Project Name Lab Goals & Planning Planning Yorktown Embodied Action Almaden Multi-modal Contextual Interaction Yorktown Ethical Embodied Decision Support Systems Yorktown Models of the World, Others, and Self Spatial Intelligence Zurich, Yorktown People Intelligence Haifa Theory of Mind Yorktown Learning Teachable Agents Yorktown Sensors & Perception/Actuators Physical Interaction Tokyo Edge Intelligence Australia Platform & Meta Management Tooling Yorktown Solution Area Project Robots, Avatars, & Spaces Watson Labs Robots & Spaces Watson IoT Robots Industrial Robots Spaces Elder Care Spaces M&A Objects Cognitive Objects
  17. 17. © 2016 IBM Corporation 17 Imagine unleashing Watson in the physical world. Give it eyes, ears, and touch, then let it act in that world with hands and feet and a face, not just as an action of force but also as an action of influence. This is embodied cognition: by placing the cognitive power of Watson in a robot, in an avatar, an object in your hand, or even in the walls of an operating room, conference room, or spacecraft, we take Watson's ability to understand and reason and draw it closer to the natural ways in which humans live and work. In so doing, we augment individual human senses and abilities, giving Watson the ability see a patient's complete medical condition, feel the flow of a supply chain, or orchestrate the tasks in a day in the life of an individual.

×