Download as PDF, PPTX
More Related Content
Similar to Cognitive Assistant for the Blind
![Vibe Coding vs. Spec-Driven Development [Free Meetup]](https://cdn.slidesharecdn.com/ss_thumbnails/vibecodingvsspecdrivendevelopment-251209105622-43f455e7-thumbnail.jpg?width=640&height=640&fit=bounds)
Cognitive Assistant for the Blind
- 1. Cognitive Assistant forthe Blind Chieko Asakawa, IBM Fellow, IBM Research Kris Kitani, Robotics Institute, Carnegie Mellon University 1
- 2. © 2016 IBMCorporation 18th century, the mathematician Euler authored a leading textbook on mathematics after he became blind. Captions for TV programs were invented for deaf people, but are now widely used in sports bars and to help immigrants, etc. Based on Self-‐‑‒standing wheelchair. 2011, voice dialog technologies for smartphones TelephoneKeyboard Voice recognition Voice dialogTV Captions Voice synthesis and OCR Modern Mathematics Automatic vehicles Segway “Blind driving challenges” Early use of keyboards was a to help people with hand-‐‑‒ and-‐‑‒motor disabilities in writing First consumer product was a reading assistant for the blind in the 1970s. 1876, Alexander Graham Bell invented the phone after teaching hearing impaired people. Smart Machines 2 17th century, Dom Perignon. Champagne Accessibility Needs Ignite Innovation
- 3. 1960s Talking Typewriter 1960s Talking Typewriter 19751403 Braille Printer 1984 Talking 3270 Terminal 1990 VoiceType™ 2008 Social Accessibility 1998 ViaVoice® 2000 Accessibility Center 1997 Home Page Reader 1988 ScreenReader/DOS 2004 aDesigner 2007 Eclipse Accessibility Tools Framework 2009 Rich Internet Accessibility Technologies 1914 First IBMer with disabilities 2012 Brazil Accessible Vocational Training Project 2011 Senior Workforce Project 1975 1403 Braille Printer IBM History of Accessibility 3 © 2016 IBM Corporation
- 4. Progress in InformationAccessibility Exponential increase of accessible information (Conceptual chart) 1980 1990 2000 2010 Paper Braille Digital Braille Voice Web access 4 Amountofaccessibleinformation © 2016 IBM Corporation
- 5. 5 Walk by myself Finding discount posters Finding restaurants Shopping Many things are difficult… Challenges of Real World Accessibility © 2016 IBM Corporation
- 6. Chika 6 Chika from “Esperthe Light Speed” © 2016 IBM Corporation
- 7. Natural Language Processing Infer from Knowledge Predict the Future Natural Interaction Learning Senses Watson1.0, 2.0, … Apply Watson Modeling and Simulation Neural Chip SyNAPSE Image Processing Speech Recognition Technology Traffic Simulation Dialog System Question and Answering System 7 Challenges of “Cognitive Computer” © 2016 IBM Corporation
- 8. 8 Demonstration Video © 2016IBM Corporation
- 9. Localization Radio-wave Vision Dead reckoning Recognition Objects People Environment Knowledge Personal Social media Encyclopedic Interaction Speech Gesture Sonification BuildingBlocks of Cognitive Assistant 9 © 2016 IBM Corporation
- 10. Understanding the Functionalityof the Environment Nick Rhinehart and Kris Kitani, CVPR 2016
- 11. © 2016 IBMCorporation When we observe a scene… … we know how we can act in that environment
- 12. © 2016 IBMCorporation When we observe a scene… … we know how we can act in that environment Walk Walk
- 13. © 2016 IBMCorporation When we observe a scene… … we know how we can act in that environment Sit Sit Sit
- 14. © 2016 IBMCorporation When we observe a scene… … we know how we can act in that environment Open Open Open
- 15. © 2016 IBMCorporation Can we teach a computer to understand scene functionality by observing people?
- 16. © 2016 IBMCorporation Setup: One wearable camera Detect actions using computer vision Learn the function of an environment
- 17. © 2016 IBMCorporation
- 18. © 2016 IBMCorporation
- 19. © 2016 IBMCorporation
- 20. © 2016 IBMCorporation
- 21. 21 Information Accessibility Progress(revisited) 1980 1990 2000 2010 Paper Braille Digital Braille Voice Web access Amountofaccessibleinformation Exponential increase of accessible information (Conceptual chart) 2020 2030 2016 © 2016 IBM Corporation “Real-world” information access