Cognitive Assistant for the Blind

297 views

Published on

Kris Kitani from Carnegie Mellon University and Chieko Asakawa, IBM Fellow, presented a “Cognitive Assistant for the Blind” as part of the Cognitive Systems Institute Speaker Series.

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
297
On SlideShare
0
From Embeds
0
Number of Embeds
9
Actions
Shares
0
Downloads
19
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Cognitive Assistant for the Blind

  1. 1. Cognitive Assistant for the Blind Chieko Asakawa, IBM Fellow, IBM Research Kris Kitani, Robotics Institute, Carnegie Mellon University 1
  2. 2. © 2016 IBM Corporation 18th  century,  the  mathematician  Euler  authored  a   leading  textbook  on  mathematics  after  he  became   blind. Captions  for  TV  programs  were   invented  for  deaf  people,  but   are  now  widely  used  in  sports   bars  and  to  help  immigrants,   etc.   Based  on  Self-‐‑‒standing   wheelchair. 2011,  voice  dialog   technologies  for   smartphones   TelephoneKeyboard Voice  recognition Voice  dialogTV  Captions Voice   synthesis   and  OCR Modern  Mathematics Automatic  vehicles Segway “Blind  driving  challenges” Early  use  of  keyboards  was   a  to  help  people  with  hand-‐‑‒ and-‐‑‒motor  disabilities  in   writing   First  consumer  product  was  a   reading  assistant  for  the  blind   in  the  1970s. 1876,  Alexander  Graham  Bell  invented  the   phone  after  teaching  hearing  impaired   people. Smart Machines 2 17th  century,   Dom  Perignon.   Champagne Accessibility Needs Ignite Innovation
  3. 3. 1960s Talking Typewriter 1960s Talking Typewriter 1975 1403 Braille Printer 1984 Talking 3270 Terminal 1990 VoiceType™ 2008 Social Accessibility 1998 ViaVoice® 2000 Accessibility Center 1997 Home Page Reader 1988 ScreenReader/DOS 2004 aDesigner 2007 Eclipse Accessibility Tools Framework 2009 Rich Internet Accessibility Technologies 1914 First IBMer with disabilities 2012 Brazil Accessible Vocational Training Project 2011 Senior Workforce Project 1975 1403 Braille Printer IBM History of Accessibility 3 © 2016 IBM Corporation
  4. 4. Progress in Information Accessibility Exponential increase of accessible information (Conceptual chart) 1980 1990 2000 2010 Paper Braille Digital Braille Voice Web access 4 Amountofaccessibleinformation © 2016 IBM Corporation
  5. 5. 5 Walk by myself Finding discount posters Finding restaurants Shopping Many things are difficult… Challenges of Real World Accessibility © 2016 IBM Corporation
  6. 6. Chika 6 Chika from “Esper the Light Speed” © 2016 IBM Corporation
  7. 7. Natural Language Processing Infer from Knowledge Predict the Future Natural Interaction Learning Senses Watson 1.0, 2.0, … Apply Watson Modeling and Simulation Neural Chip SyNAPSE Image Processing Speech Recognition Technology Traffic Simulation Dialog System Question and Answering System 7 Challenges of “Cognitive Computer” © 2016 IBM Corporation
  8. 8. 8 Demonstration Video © 2016 IBM Corporation
  9. 9. Localization Radio-wave Vision Dead reckoning Recognition Objects People Environment Knowledge Personal Social media Encyclopedic Interaction Speech Gesture Sonification Building Blocks of Cognitive Assistant 9 © 2016 IBM Corporation
  10. 10. Understanding the Functionality of the Environment Nick Rhinehart and Kris Kitani, CVPR 2016
  11. 11. © 2016 IBM Corporation When we observe a scene… … we know how we can act in that environment
  12. 12. © 2016 IBM Corporation When we observe a scene… … we know how we can act in that environment Walk Walk
  13. 13. © 2016 IBM Corporation When we observe a scene… … we know how we can act in that environment Sit Sit Sit
  14. 14. © 2016 IBM Corporation When we observe a scene… … we know how we can act in that environment Open Open Open
  15. 15. © 2016 IBM Corporation Can we teach a computer to understand scene functionality by observing people?
  16. 16. © 2016 IBM Corporation Setup: One wearable camera Detect actions using computer vision Learn the function of an environment
  17. 17. © 2016 IBM Corporation
  18. 18. © 2016 IBM Corporation
  19. 19. © 2016 IBM Corporation
  20. 20. © 2016 IBM Corporation
  21. 21. 21 Information Accessibility Progress (revisited) 1980 1990 2000 2010 Paper Braille Digital Braille Voice Web access Amountofaccessibleinformation Exponential increase of accessible information (Conceptual chart) 2020 2030 2016 © 2016 IBM Corporation “Real-world” information access

×