Successfully reported this slideshow.
Your SlideShare is downloading. ×

Cognitive assistance at work


Check these out next

1 of 30 Ad

More Related Content

Slideshows for you (20)

Viewers also liked (20)


Similar to Cognitive assistance at work (20)

Recently uploaded (20)


Cognitive assistance at work

  1. 1. © 2015 IBM Corporation Hamid R. Motahari-Nezhad IBM Almaden Research Center San Jose, CA Cognitive Assistance at Work Cognitive Assistant for Employees and Citizens AAAI 2015 Fall Symposium
  2. 2. © 2013 IBM Corporation 2 Workers and Work Management Onboarding, Orientation and Growth Communicationand interactions WorkandProject Management Knowledge Worker
  3. 3. © 2013 IBM Corporation The Work Practices of Human Administrative Assistants  Human assistant activities –Calendaring • Scheduling, information formatting and preparation –Task Management –Email Management • Filtering emails, • Email classification  Interruption management –Mediating interruption –Prioritizing interruptions  Taking care of routine tasks –Tracking –Following up –Travel arrangement, and preparation –Reminding, and organizing –Managing work of human • Pre-processing • Filtering • Prioritizing • Compiling information and reports 3 An assistant “will remove much of the burden of administrative chores from its human user and provide guidance, advice, and assistance in problem solving and decision making.” Gutierrze and Hilfdalgo, 1988
  4. 4. © 2013 IBM Corporation4 Credit: Rob Koplowitz, IBM Insight 2015
  5. 5. © 2013 IBM Corporation COGNITIVE ASSISTANCE 5
  6. 6. © 2013 IBM Corporation Cognitive Assistant  A software agent (cog) that – “augments human intelligence” (Engelbart’s definition1 in 1962) – Performs tasks and offer services (assists human in decision making and taking actions) – Complements human by offering capabilities that is beyond the ordinary power and reach of human (intelligence amplification)  A more technical definition – Cognitive Assistant offers computational capabilities typically based on Natural Language Processing (NLP), Machine Learning (ML), and reasoning chains, on large amount of data, which provides cognition powers that augment and scale human intelligence  Getting us closer to the vision painted for human-machine partnership in 1960: – “The hope is that, in not too many years, human brains and computing machines will be coupled together very tightly, and that the resulting partnership will think as no human brain has ever thought and process data in a way not approached by the information handling machines we know today” “Man-Computer Symbiosis , J. C. R. Licklider IRE Transactions on Human Factors in Electronics, volume HFE-1, pages 4-11, March 1960 6 1 Augmenting Human Intellect: A Conceptual Framework, by Douglas C. Engelbart, October 1962
  7. 7. © 2013 IBM Corporation History of Cognitive Assistants from the lens of AI 7 1945 Memex (Bush) 1962 NLS/Augment (Engelbart) 1955/6 Logic Theorist (Newwell, Simon, 1955) Checker Player (Samuel, 1956) Touring Test, 1950 Thinking machines 1966 Eliza (Weizenbaum) 1965-1987 DENDRAL 1974-1984 MYCIN 1987 Cognitive Tutors (Anderson) Apple’s Knowledge Navigator System Expert Systems 1965-1987 1992-1998 Virtual Telephone Assistant Portico, Wildfire, Webley; Speech Recognition Voice Controlled 2002-08 DARPA PAL Program CALO IRIS
  8. 8. © 2013 IBM Corporation Modern Cognitive Assistants: State of the art (2008-present) Commercial  Personal Assistants and Bots – Siri, Google Now, Microsoft Cortana, Amazon Echo, FB M – Braina, Samsung's S Voice, LG's Voice Mate, SILVIA, HTC's Hidi, Nuance’ Vlingo – AIVC, Skyvi, IRIS, Everfriend, Evi (Q&A), Alme (patient assistant) – Viv (Global Brain as a Service) –, Telegram bots  Cognitive and intelligent platforms – IBM Watson – Wolfram Alpha – Saffron 10 – Vicarious (Captcha) Open Source/Research  OAQA  DeepDive  OpenCog  YodaQA  OpenSherlock  OpenIRIS  iCub EU projects  Cougaar  Inquire* (intelligent textbook) 8 * Curated knowledge base
  9. 9. © 2013 IBM Corporation A Society of Interacting Cognitive Agents (Bots) and Humans 9 Cognitive Agent to Agent Human to Human Cognitive Agent to HumanHuman-Cog interaction Cog-Cog interaction Cog-mediated Human Interaction Natural Language Natural Language, or ACL? ACL: Agent Communication Language, KQML, etc. Natural Language-ACL-Natural Language Weather Cog Health Agent Personality Insight Cog. Provider Cogs Travel Cog 1 Travel Cog 2 Planning a Vacation Trip Considering preferences, experience, conditions, cost, Availability, etc. Mediated and facilitated by Cogs
  10. 10. © 2013 IBM Corporation A major challenge towards offering cognitive assistance: Building knowledge models, and knowledge acquisition  “For an artifact, a computational intelligence, to be able to behave with high levels of performance on complex intellectual tasks, perhaps surpassing human level, it must have extensive knowledge of the domain”  The challenge of AI in making progress toward building human-like artifacts: – Knowledge representation, and (especially) knowledge acquisition  Approaches – Build a large knowledge base by reading text – Distilling from the WWW a huge knowledge base  Semantic Web and Linked Data methods over the last decade extensively has explored building models, ontologies and rule-set that contributes to WWW knowledge representation – Manual and semi-automated, focused on curated ontologies – Community participation in building ontologies have resulted in creation of large knowledge bases: DBPedia, Yago, Wikidata, Freebase, MediaWiki, etc. – Ontologies are expensive to build and scale, and are generic in nature 10 EDWARD A. FEIGENBAUM, Some Challenges and Grand Challenges for Computational Intelligence, Journal of the ACM, Vol. 50, No. 1, January 2003, pp. 32–40
  11. 11. © 2013 IBM Corporation Lesson Learned from Watson in Jeopardy  “The Watson program is already a breakthrough technology in AI. For many years it had been largely assumed that for a computer to go beyond search and really be able to perform complex human language tasks it needed to do one of two things: either it would “understand” the texts using some kind of deep “knowledge representation,” or it would have a complex statistical model based on millions of texts.” – James Hendler, Watson goes to college: How the world’s smartest PC will revolutionize AI, GigaOm, 3/2/2013  Breakthrough: – Developing a systematic approach for scalable knowledge model building from large, less reliable data sources, and deploying a large array of individually imperfect techniques to find right answers • Building and curating a robust, and comprehensive knowledge base and ruleset is laborious, time consuming and slow • Watson approach for building on massive, mixed curated and not-curated and less reliable information sources with uncertainty has proved effective 11 Source: Inquire Intelligent Book
  12. 12. © 2013 IBM Corporation Towards Mass Computing as the Shared Characteristic of Recent Computing Advances 12 Scalable Computing over Massive Commodity Hardware Building Stronger Super Computers Cloud Computing Crowd Computing Advanced individual algorithms Mass computing applied to AI Complex array of algorithms applied to make sense of data, and offer cognitive assistance Big Data Complex Analytics
  13. 13. © 2013 IBM Corporation COGNITIVE ASSISTANT FOR WORKERS 13
  14. 14. © 2013 IBM Corporation eAssistant: a cognitive assistant for the enterprise  A mobile intelligent assistant for the enterprise that assist a user (worker) to be more productive by supporting following a methodology of monitor, process, recommend and do actions with the following capabilities – Understands human language – Monitors input channels including email, calendar chat and enterprise information sources – Builds a model of the user and the world, and is situational aware (context) – Offer assistance by • Pre-processing information, and presenting information in desired format • Categorizing and filtering information • Gathering and organizing information • Scheduling meetings • Identifying requests, and organizing to-dos of its human subject • Assists in performing tasks such as organizing events, travel assistant, and learns new tasks • And, suggest taking certain actions to its human subject that supports increasing productivity, and growth 14
  15. 15. © 2013 IBM Corporation eAssistant: Cognitive Assistant Types in Work Environment  Personal (employee) eAssistant – Personal eAssistants have access to the data space (and applications) that the principal has access to with the same level of visibility – While eAssistant is proactive in making suggestions, it takes action under the control and direction of the principal  Assistant’s eAssistant – An assistant to Human Assistants helping them to become more productive, and focus on work that require human judgment  Expert/Process eAssistants – Assistants that are experts in a specific domain such as travel policy, human resources, etc. 15 Cognitive Assistant Platform Individual cognitive agents Assistant’s Cognitive Agents Expert Cognitive Agents Systems of cognitive agents that collaborate effectively with one another to support human activities. Interactions types need to be supported: • cog-to-cog interactions, • human-cog interactions, and • cog-backed human-to-human interactions
  16. 16. © 2013 IBM Corporation Actionable Statement Identification Over Unstructured Conversations 16 Email, Chat, and Calendaring apps are the most used channels for doing work in the enterprise Addressing the work organization and management for Knowledge workers: monitoring communication channels (email, chat), and: - capturing, prioritizing and organizing work of a worker - Identifying actionable statements (requests, commitments, questions) and track them over the course of conversations
  17. 17. © 2013 IBM Corporation17 Inbox - Verse Highlighting actionable statements Recommending fulfilment actions IBM Insight 2015 – The session on “Given your collaboration tools a brain”
  18. 18. © 2013 IBM Corporation18 IBM Insight 2015 – The session on “Given your collaboration tools a brain” Send File Action Archetype Send File Action Archetype Send File Action Archetype
  19. 19. © 2013 IBM Corporation19 IBM Insight 2015 – The session on “Given your collaboration tools a brain” Invite/Calendar Action Archetype Automated Invite Parameters Extraction Calendar Entry Creation
  20. 20. © 2013 IBM Corporation20 IBM Insight 2015 – The session on “Given your collaboration tools a brain” Integration with Watson Health Integration with Watson Health Integration with Watson Health
  21. 21. © 2013 IBM Corporation eAssistant App and APIs 21 Watson (& BigInsight NLP) Apps and Services on BlueMix CollaborationTools Enterprise Repositories, Applications and Data Sources Feeds Repositories Document collections … eAssistant Apps Personal Knowledge Graph Builder Conversation Analytics, Auto-Response, Prioritization Calendar and Scheduling Assistant Context-aware Information Finder To-do, Task and Process Assistant Cognitive Work Assistant APIs Semantic Role Labeling POS tagging Dependency Analysis Co-reference resolution Named Entity Recognition Knowledge Graph Builder
  22. 22. © 2013 IBM Corporation COGNITIVE BPM Cognitive Assistance for Case Management 22
  23. 23. © 2013 IBM Corporation Cognitive assistance in case management for knowledge workers  Knowledge workers are involved in handling cases in the work context (in domains such as social care, legal, government services, citizen services, etc.).  Cognitive case management is about providing cognitive support to knowledge workers in handling customer cases.  Cognitive assistance for employees: Handling and managing cases involves understanding policies, laws, rules, regulations, processes, plans, as well as customers, surrounding world, news, social networks, etc.  Cognitive assistance for customers/agents: Assists citizens by empowering them by knowing their rights and responsibilities, and helping them to expedite the progress of the case 23 Citizens Assistant Business Employees/ agents Plansworkflows Rules Policies Regulations Templates Instructions/ Procedures ApplicationsSchedules Communications such as email, chat, social media, etc. Organization Cog. Agent Unstructured Linked Information
  24. 24. © 2013 IBM Corporation Spectrum of work, and Cognitive Assistance 24 Ref: Motahar-Nezhad, Swanson, 2013, and Sandy Kemsley, Column2 Spectrum of Work Cognitive BPM
  25. 25. © 2013 IBM Corporation Cognitive BPM Systems  A Cognitive BPM system offers the computational capability of a cognitive system to provide support the whole lifecycle of processes over structured and unstructured information sources, and continuously discovers, learns and acts to achieve a desired process outcome – Meets two pressing needs: supporting complex process decisions, and processing large amount of data – Intelligent and integrated process (model) definition, reasoning and adaptation • Process is not assumed apriori defined; discovered, learned and customized based on accumulated knowledge and experience – Intelligence supporting the execution of process • When, What action (how) and whom to contact – The need for revisiting some basic abstractions of BPM • Real-world, and real-time course of actions • New information availability changes course of actions in a plan • Fluid actions/tasks, notion of task completion, and process/plan adaptation 25
  26. 26. © 2013 IBM Corporation Process Definition, Discovery, Learning Process Enactment Process/ Environment Sensing Process Analytics Proactive/ Reactive Process Response/Adaptation Cognitive BPM Lifecycle 26 Environment Sensing Data sources Data Processing/ Analytics Process Composition / Enactment Update Process Monitoring/Analytics IoT
  27. 27. © 2013 IBM Corporation Use Case: Cognitive Case Assistance  Assume an executive admin is managing an event organization process for their department – Step 1: sending invite to an event to employees in their department, through email and requests for RSVP • Cognitive BPM (1): Q&A ability for the admin: How many have confirmed, how many pending, how many not answered • Cognitive BPM (2): Predictive analytics: how many will eventually RSVP? • Cognitive BPM (3): Diagnostic analytics: why some not accepted (customers in case of marketing case)? – Step 2: Ordering place, food, transportation, etc • Cognitive BPM (1): tracking of the process steps, which vendor have replied, which ones pending, have questions, etc. • Cognitive BPM (2): keeping track of synchronization and consistency (dates, amounts, numbers, etc.) among different steps – Step 3: Pre-event steps • Reminding people who have RSVPed • Compiling and sending logistic information (from different steps) 27
  28. 28. © 2013 IBM Corporation Research in Support of Cognitive BPM in Work Assistant Space Task, commitment and task lifecycle extraction from workers interactions over email and chat 28 Anup K. Kalia, Hamid R. Motahari Nezhad, Claudio Bartolini, Munindar P. Singh: Monitoring Commitments in People-Driven Service Engagements. IEEE SCC 2013: 160-167
  29. 29. © 2013 IBM Corporation Research Directions  Abstractions and models for Cognitive Work Assistants and Cognitive Processes  Knowledge representation models, and scalable knowledge acquisition methods from unstructured information (text, image, etc.) and building actionable knowledge graphs  Cognitive Work Assistants –Cognitive augmentation of workers in work environments, and in process management  Cognitive Process Management System –Analytics on unstructured information to support process understanding –Analytics to support process adaptation, customization and configuration –Proactive process adaptation  Learning and teaching tasks and processes to cognitive agents –Interactive learning where cognitive agents ask process questions –Gradual learning through experience, and process improvement 29
  30. 30. © 2013 IBM Corporation QUESTIONS? Thank You! 30