Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Human-like Chatbots: Promises, Challenges, and Implications


Published on

Keynote at ICWSM-2018 workshop on Chatbots: June 25, 2018, Palo Alto, CA

My interest in and vision for Computing for Human Experience is centered on developing and using AI to serve, rather than replace, humans. In order to make technology appear more human-like, it is critical to align technology with human’s multi-sensory capabilities, and correspondingly deal with multimodal information. Two decades ago, Mark Weiser’s ubicomp vision brought welcome advancement in human-computer interaction. Next frontier for more natural interactions between man and machines is voice and language, which I am sure will be followed by integration with other senses, esp. vision, leading to chatbots merging into the broader technology of robots. In the interim, chatbots that interact with humans in more natural ways hold tremendous promise. The bigger challenge in my view is not the “front end” such as speech recognition and transcription, but the “back end” - processing information as a human brain would, making interactions with computing feel more natural to humans. This calls for a further breakthrough in contextualizing and personalizing information exchange between a human and chatbot to better understand the human needs and the actions that can be taken.

In this talk, I will focus on chatbots that are narrow but deep (very good in a well-defined application or domain) to help address humans in the way an expert (e.g., a clinician in a healthcare context) would. I will take examples from the augmented personalized health applications we are pursuing at Kno.e.sis using our kHealth technology for achieving better outcomes for managing asthma, post-surgery care, and depression. I seek to explain what a human-like chatbot would be expected to do, how knowledge-enhanced AI and big data approaches may advance the current state of the technologies in Natural Language Understanding (NLU) and Q/A and if successful, how this can demonstrate the promise of the machines serving vital human needs and wants.

For Augmented Personalized Health and Computing for Human Experiences: See

Published in: Technology
  • Be the first to comment

Human-like Chatbots: Promises, Challenges, and Implications

  1. 1. Human-like Chatbots: Promises, Challenges and Implications Icon source used in the entire presentation - Presentation template by SlidesCarnival Photographs by Unsplash Prof. Amit Sheth LexisNexis Ohio Eminent Scholar Executive Director, Kno.e.sis - Ohio Center of Excellence in Knowledge-enabled Computing & BioHealth Innovation
  2. 2. Machine-centric to Human-centric Computing Artificial Intelligence 2 Ambient Intelligence Augmenting Human Intellect Human-Computer Symbiosis Computing for Human Experience Machine-centric Human-centric John McCarthy Mark Weiser Douglas Engelbart Joseph C.R. Licklider Figure: Views along the spectrum of machine-centric to human-centric computing. At the far right is our work on Computing for Human Experience, which explores paradigms such as Semantic, Cognitive, and Perceptual Computing. Kno.e.sis Center,
  3. 3. Machine Replacing Humans, Possible? 3 Source: 5e1d5812e1e7?token=12BqnkqpquQiaXYw Source: Source:
  4. 4. 4 Current AI is Far from Singularity Source:
  5. 5. Computing for Human Experience 5 Source: “Computing for Human Experience will employ a suite of technologies to nondestructively and unobtrusively complement and enrich normal human activities, with minimal explicit concern or effort on the humans’ part.”
  6. 6. HOW TO MAKE TECHNOLOGY MORE HUMAN-LIKE? Image Source: Critical to align technology with human’s multi-sensory capabilities AND correspondingly deal with multimodal information
  7. 7. Richness of Interactions between Man and Machines Natural Language Voice CHATBOTS Language Source: Media Richness Theory
  8. 8. 9 CHATBOTS, CONVERSATIONAL AGENTS, DIGITAL ASSISTANTS “Messaging is the new browser, and bots are the new websites.” - Mike Roberts, Kik's head of messenger services Messaging Apps are where people spend their time online. Chatbots are a way to reach them. Example Companies Adopting Chatbot 9 Messaging Apps are where people spend their time online. Chatbots are a way to reach them.
  9. 9. 10 CHATBOTS Applications (Health) Capturing data that is otherwise not available to physicians Personal Assistant Engaging Personalized Automation Shallow and Broad Narrow and Deep Types of Chatbot Opens up a higher dimension for human and machine Richness & Expressiveness ExampleChatbotApplicationsonVariousDomains INSURANCE Chatbot Allstate Business Insurance expert (or ABIe) FINANCIAL Chatbot Capital One Financial (Eno) Info on credit card balances, transactions, due dates, and limit HOTEL Chatbot Marriott International’s Book travel in more than 4,700 hotels alofts-chatbotlr-simplify-travel-guests-throughout-journey/ THERAPY & HEALTH Chatbot 10 Benefits over Mobile Applications
  10. 10. 11 General Outline of Conversational AI Techniques We are (artificially) intelligent. We are only as smart as the words you feed me. You don’t have to install any apps to talk to me. Conversation Intent Conversational Datasets, Commonsense Reasoning and Knowledge Ingestion Natural Language Understanding (NLU) Techniques ● Named-entity Recognition (NER) and Disambiguation ● Sentence Completion ● Topic and Domain Detection ● Implicit Entity Recognition ● Relation Extraction ● Text Summarization ● Sentiment, Emotion, and Intent Detection ● Emoji Sense Disambiguation ● Machine Translation ● Ranking and Selection (Open-domain social conversations) Response GenerationConversational Topic Tracker Inappropriate and Offensive Speech Detection
  11. 11. 12 Source:
  12. 12. “ 13 The bigger challenge in my view is not the “front end” such as speech recognition and transcription, but the “back end” - processing information as a human brain would, making interactions with computing feel more natural to humans.’
  13. 13. “Calls for a further breakthrough in contextualizing and personalizing information exchange between a human and chatbot to better understand the human needs and the actions that can be taken. 14
  14. 14. A Deep Learning Based Chatbot implemented using the Seq2Seq model and trained on the Twitter 2017 and Cornell Movie Dialogs Corpus. Resources and References: 15 State-of-the-Art Seq2Seq Chatbot (RNN-LSTM) ❖ As good as the data trained on ❖ But lacks context & personalization VIDEO ON THE NEXT SLIDE
  15. 15. Intelligent Chatbots? ● COGNITIVE UNDERPINNING & EXPLAINABILITY with Deeper Actor Model, Domain Model/Knowledge Graphs and Protocols ● CONTEXTUALIZATION ● PERSONALIZATION ● ABSTRACTION
  16. 16. 17 Contextualization refers to data interpretation in terms of knowledge (context). Without Domain Knowledge With Domain Knowledge Chatbot with domain (drug) knowledge is potentially more natural and able to deal with variations.
  17. 17. Personalization refers to future course of action by taking into account the contextual factors such as user’s health history, physical characteristics, environmental factors, activity, and lifestyle. 18 Without Contextualized Personalization With Contextualized Personalization Chatbot with contextualized (asthma) knowledge is potentially more personalized and engaging.
  18. 18. Abstraction A computational technique that maps and associates raw data to action-related information. 19 With AbstractionWithout Abstraction .
  19. 19. 20 CHATBOTS @ KNO.E.SIS a. ReaCTrack Personalized Adverse Reaction Conversational-based Tracker for Clinical Depression b. kBOT Knowledge-enabled (kHealth) Personalized ChatBot for Asthma
  20. 20. 21 A sample video demo of ReaCTrack. Main objectives are: (i) Monitor patient’s depression severity score (ii) Track medication adherence and mood changes over the course of prescribed medications. (iii) Document ADRs and side-effects from antidepressant medications (iv) Answer domain-specific (depression) questions, specifically the use and side-effects of antidepressant using semantic technologies. A sample video demo of kBOT: Knowledge-enabled (kHealth) Personalized ChatBot for Asthma Contextualized & Personalized Conversations from Multimodal data and IoT sensor data VIDEO ON THE NEXT SLIDE VIDEO ON THE NEXT SLIDE
  21. 21. 22 Sensor, Social, Clinical Datastreams: Informed & Intelligent Questions Weather information (temperature, pollen, humidity, etc) Elasticsearch (ES) Database Query & Rule Abstract raw values into information Asthma Domain Knowledge Patient Data from EMR & PGHD (Compliance score, prescribed medications, asthma control level) IoTs (Foobot & Fitbit) Conversation Rules & Scripts (DialogFlow) Sensor, Social, Clinical Architectural Framework for Intelligent & Informed Conversations: kBOT Asthma Sensor (IoTs) & Cyber Datastream Clinical (Baseline) Datastream Patient Consented Social Data (Facebook, Instagram, Twitter Activity) Social Datastream Knowledge Datastream ★ Smarter & engaging agent ★ Minimize active sensing (Questions to be asked) ★ Ask only informed & intelligent questions ★ Relevant & Contextualized conversations ★ Personalized & Human-Like Human-Like Aspect
  22. 22. Contextualization and Personalization kBOT initiates greeting conversation. Understands the patient’s health condition (allergic reaction to high ragweed pollen level) via the personalized patient’s knowledge graph generated from EMR, PGHD, and prior interactions with the kBOT. Generates predictions or recommended course of actions. Inference based on patient’s historical records and background health knowledge graph containing contextualized (domain-specific) knowledge. Figure: Example kBOT conversation which utilizes background health knowledge graph and patient’s knowledge graph to infer and generate recommendation to patients. ★ Conversing only information relevant to the patient 23
  23. 23. TOWARDS HUMAN-LIKE CHATBOTS A chatbot should have a deeper understanding of real world entities and the person it is interacting with, backed with knowledge and facts. 24
  24. 24. What does it mean to be Human-like? Medication reminder Option 1: Alarm alerts patient to take timely medication. 25 Option 2: Pill sensor bottle detects medications are not taken timely and sounds alarm. Option 3: Chatbot knows you have not taken your medications, sounds alarm and pops up a reminder (optionally, the system even knows you are now in the bedroom which has medicine closet). Option 4: Robot: “You seem to have missed your medications. Shall I get you your pill bottle?”
  25. 25. But how do we “teach” these bots real world entities? Not every rule can be programmed, bots have to learn progressively just like how humans would. 26 Image Source:
  26. 26. “ Progressive Intelligence: A form of intelligence by which one learns progressively through continuous stream of data, understand the Semantic Associations in a given context, distill the knowledge, and synthesize the right decision(s). 27
  27. 27. 30 Value of Knowledge-enhanced AI (Research) Li et al, 2016: Build personalized conversation engine by adding personal information as extra input. Xing et al, 2016: Incorporate topic information into the seq2seq framework to generate informative and interesting responses for chatbots. Mazumder, Ma, and Liu, 2018: Propose a general knowledge learning engine for chatbots to continuously and interactively learn new knowledge during conversations (Lifelong interactive Learning and inference (LiLi)). With Knowledge Sordoni et al, 2015: Represent utterances in previous turns as context vector and incorporated them into response generation. Yao, Zweig, and Peng, 2015: Added an extra RNN between the encoder and decoder of the Seq2Seq model to represent intentions. Gu et al, 2016: Introduce copynet to simulate the repeating behavior of human in conversation. Without Knowledge Sheth, Perera, Wijeratne, and Thirunarayan, 2017: Discuss the indispensable role of knowledge for deeper understanding of complex text and multimodal data. Knowledge will Propel Machine Understanding of Content: Extrapolating from Current Examples Sheth, Thomas, and Mehra, 2010: Continuous Semantics to Analyze Real-Time Data Kno.e.sis Center
  28. 28. Various distributed knowledge sources Update Knowledge Graph Core Services Personalized Health Management Evolving Open Health Knowledge Graph Disease Progression Intervention CDI and CAC 31 Role of Knowledge 31
  29. 29. Semantic Browsing Extraction Data Integration and Interlinking Entity Complex Extraction Aberrant Drug-related Behaviour Neuro-Cognitive Symptoms Adverse Drug Reaction Relation Event Severity Personal Sensor Data De-identified EMR Blog Post Context Representation Relevant Subgraph Selection Semantic Search Disease-specific Chatbot Visualization Health Knowledge Graph 32 Intent Open Health Knowledge Graph 32
  31. 31. 34 Semantic, Cognitive, Perceptual Computing: Paradigms That Shape Human Experience
  32. 32. CHATBOTS IN HEALTHCARE Putting it all together with 3 Pedagogical Examples kHealth Asthma, Depression, Elder health
  33. 33. 36 LIMITED DATA due to episodic visits TIME CONSTRAINT during clinical visits Significant information seeking time is required every time Comprehending clinical notes every time which contains only text is difficult Each individual is DIFFERENT and thus, personalised treatment is needed Insufficient time and data for personalization Image Source: WHY Healthcare? [a technology take] Traditional Healthcare
  34. 34. 37 Solution: Augmented Personalized Health with CHATBOT and IoTs Medical Internet of Things Patient Generated Health Data (PGHD) with Medical Internet-Of-Things (IoTs) Digital footprint representative of patient’s health ★ Episodic to Continuous Monitoring ★ Clinic-centric to Patient-centric ★ Clinician controlled to Patient empowered ★ Disease Focused to beyond Medical intervention ★ Sparse data to 360 Multimodal data 37 But how to turn these “Big Data” into Insights into Actionable Information?
  35. 35. 40 Example 1: kHealth Asthma 40 Data Sources Heterogeneous data and collection method (1852 data points/ patient /day) Semantic, Cognitive, Perceptual Computing Framework Smarter conversations with actionable meaningful information.
  36. 36. “ 41 Modern Healthcare is not just about diagnosing disease and prescribing medication. Patient- doctor communication exerts a placebo “healing” effect on the patient. (Human-like) Chatbot can be a useful vehicle to emulate such relationship.
  37. 37. 42 Example 2: Depression Chatbot that engages the patient on a daily basis understands the emotion and social well-being for better interactions. 42
  38. 38. 43 Example 3: Proposed “Putting-it-all” Together Architecture for Elder Care 43
  39. 39. 44 In Short, ❖ Far more useful in interacting compared to other alternatives (browser) when physician is not available. ❖ Emulate social-humanistic element. Promises ❖ Chatbots that interact in more natural ways hold tremendous promise. ❖ Opportunities in disease-specific health, general fitness, and well-being. Challenges ❖ The biggest challenge is to process information as a human brain would. ❖ Multi-sensory capabilities, multimodal information ❖ Contextualization, Personalization, Abstraction Implications
  40. 40. 46 Special Thanks Hong Yung (Joey) Yip (Graduate Student)