Artificial Intelligence Institute at UofSC, profile picture
Uploaded byArtificial Intelligence Institute at UofSC
PPTX, PDF394 views

KGCTutorial_AIISC_2022.pptx

The document discusses knowledge-infused reinforcement learning (KIRL) and its applications in conversational systems across various domains such as healthcare, education, and food safety. It highlights the development of safe, interpretable conversational agents by integrating process knowledge and user-level explanations, addressing the limitations of traditional statistical AI models. Additionally, it covers techniques for enhancing the performance and safety of AI-driven conversational agents through knowledge infusion methods.

Embed presentation

Downloaded 11 times
Knowledge Infused Reinforcement Learning Use-case: Conversational Systems Spread the word! Tweet, post on LinkedIn your insights, screenshots, and questions #KGC2022 #ProcessKnowledge #KnowledgeInfusedLearning #safeAI #InterpretableML #ExplainbleAI http://aiisc.ai/kirl
About the Tutors 2 Qi Zhang, Professor, CS@UofSC Planning and Reinforcement Learning in Multi-Agent Systems, Knowledge-infused Reinforcement Learning Amit Sheth, Professor, CS@UofSC Director, AI Institute @ UofSC Knowledge-infused Learning, Knowledge Graphs, Semantic Web, Natural Language Understanding, Conversational Systems Manas Gaur, Ph.D. in CS @ UofSC Researcher, AI Institute @ UofSC Knowledge-infused Learning, Natural Language Understanding, Knowledge Graphs, Kaushik Roy, Ph.D. in CS @ UofSC Researcher, AI Institute @ UofSC Knowledge-infused Reinforcement Learning, Conversational Systems
D O M A I N S 3 Conversational AI in [Healthcare] Sheth et al. 2018, Roy et al. 2021, Gaur et al. 2022 [Pandemic/Crisis] Matching Support Seekers with Support Providers (Gaur et al. 2021) [Education] ALLURE Collaborative Rubik’s Cube Solver Agostinelli et al. 2021 (https://tinyurl.com/AIISC-Rubiks) [Food] Allergy-Aware Recipe Understanding (Khandelwal et al. 2022) (https://tinyurl.com/AIISC-Allergies)
How to Build a Conversational Agent 4 Attention Neural Network (Vaswani et al. NIPS’17) Large Amount of Textual Conversational Data in General Domain Trained Generative Model Picture Credit: https://keyreply.com/conversational-ai- for-healthcare/
Agent tasked to Converse with Patient with Mental healthcare Condition 5 Do you feel nervous? More than half the days Do you feel irritated or self destructive? Do you feel something extreme might happen to you? Are you able to relax? The model can generate and ask either of these questions They are either bad questions or irrelevant ( A clinician won’t ask either of these) A model trained to asked questions Risky
6 Unsafe Question Generation Do you feel nervous? More than half the days Do you feel irritated or self destructive? Do you feel something extreme might happen to you? Are you able to relax? The model can generate and ask either of these questions A model trained to asked questions
Large Language Models Hallucinate 7 ❏ Generate Factually Incorrect Response ❏ Blenderbot 1 [Roller et al. 2020] ❏ Blenderbot 2 [https://tinyurl.com/Meta-bbot2] ❏ Generate Questions that aren’t Safe ❏ OpenAI GPT-3 [Brown et al. 2020] ❏ DeepMind’s RETRO [Borgeaud et al. 2021] ❏ Google’s LaMDA [Thoppilan et al. 2022] ❏ Utilize Google’s long list of Safety Guidelines to place constraint on Safety.
8 Do you feel nervous? More than half the days Do you feel irritated or self destructive? Do you feel something extreme might happen to you? Are you able to relax? Do you feel nervous? More than half the days Do you feel Irritated? Are you bothered by becoming easily annoyed or irritable? Are you bothered by any relaxation troubles? Knowledge Infusion using Medical Questionnaire (MedQ) These questions are medically valid and safe. Safety Checks Knowledge fix : Safety Checks Roy and Gaur et al. ACL’22
9 Do you feel nervous? More than half the days T5 (Raffel et. al. ACL’20) KI Attention Model (Ours) 30.6% 17.1% 10.6% 13.3% KI: Knowledge Infusion Do you feel Irritated? Are you bothered by becoming easily annoyed or irritable? Are you bothered by any relaxation troubles? (Ours) Roy and Gaur et al. ACL (under review) Effect of Knowledge fix : Safety Checks
Attention Model I feel bothered by little interest and have least pleasure in doing anything Did you check your dopamine levels? Do you feel your brain is affected? Did you intend to indulge in risky behaviors I feel bothered by little interest and have least pleasure in doing anything What does “lack of pleasure” mean to you? Do you feel little pleasure doing things you used to enjoy? How long have you struggled with lack of interest in things you used to enjoy? KI Attention Model DSM-5 Lexicons for Depression SCID for Depression PHQ-9 Questionnair e Gaur et al. WWW’19; Yazdavar et al. ASONAM’17 0.83 0.78 0.71 0.33 0.21 0.19
Safety Checks 11 If is cause then symptom If is symptom then medication If is medication then treatment Probability next question generation is Process Do you feel nervous? More than half the days Do you feel Irritated? Are you bothered by becoming easily annoyed or irritable? Are you bothered by any relaxation troubles?
Uncertainty and Risk 12 Generation Task Handling Uncertainty or Risk Human Annotation Experience (e.g. History) Web Search Corpus Expert Guidelines Generative Output Classification Output Labeled Dataset
Safety Checks: A method of infuse Knowledge 13 Generation Task Handling Uncertainty or Risk Generative Output Classification Output Labeled Dataset Information graph (KG, Lexicons, etc.) Kashyap and Sheth CIKM’94
Viable forms of knowledge to infuse 14 Generative Output Classification Output Labeled Dataset Interpretability User-level Explainability Information graph (KG, Lexicons, etc.) Kashyap and Sheth CIKM’94 Generative Output Classification Output Labeled Dataset Interpretability User-level Explainability ConceptNet Healthcare lexicons
15 Example KG constructed either from manual effort (A, B, C), automatically (D, E), or semi-automatically (F) (A) is empathi ontology designed to identify concepts in disaster scenarios (Gaur et al. 2019). (B) Chem2Bio2RDF (Chen et al. 2010). (C) ATOMIC (Sap et al. 2019). (D) Education Knowledge Graph by Embibe (Faldu et al. 2020). (E) Event Cascade Graph in WildFire (Jiang et al. 2019). (F) Opioid Drug Knowledge Graph (Kamdar et al. 2019)
Process Knowledge : Suicide Risk 16 Columbia Suicide Severity Rating Scale (C-SSRS) Process Knowledge Structure of C-SSRS Posner et al. 2011 Am. Journal of Psychiatry
Process Knowledge : Anxiety 17 Spitzer et al. 2006 Archives of Internal Medicine Bartolo et al. 2017 SciELO Brasil
Knowledge Infused Learning (KiL) 18 Knowledge-infused Learning is a class of Neuro-Symbolic AI techniques that incorporate broader forms of knowledge (lexical, domain-specific, common-sense, and constraint-based) into addressing limitations of either symbolic or statistical AI approaches, such as model interpretations and user-level explanations. Compared to powerful statistical AI that exploit data, KiL benefit from data as well as knowledge.
19 Shallow Infusion of Process Knowledge Deep Language Model for Classification Semantic Embedding Loss ❏ External Knowledge is utilized in Annotating the Dataset ❏ Patient Health Questionnaire-9 (PHQ-9): an instrument to measure severity of Depression. ❏ Input is matched to Question ❏ To perform user-level classification ❏ Encodes knowledge as either word embeddings (text) or graph embeddings (graphs,rules, trees) ❏ User-level Explainability and Safety: ❏ Model’s classification is checked by the questions answered correctly. ❏ Correctly answered questions are matched to concepts highlighted in the input. Agarwal and Gupta et al. ACL RR Under Review embedding
Semi-Deep Infusion of Process Knowledge 20 Deep Language Model For Representation Bernoulli Loss embedding Decision Tree ❏ Utilizes knowledge in its original form to modify the parameters of the ML model ❏ We employ and adapt interpretable ML models E.g. Decision Trees, Naive Bayes, K-Nearest Neighbors ❏ Knowledge is incorporated in the objective function to achieve parameter tweaking. ❏ User-level Explainability, Safety, & Interpretability: ❏ Model’s outcome is a tree → Interpretability ❏ Predicted Tree is checked with PHQ-9 Questions → User Level Explainability ❏ Bernoulli Loss keep refining the tree and can be tweaked by experts. Roy and Gaur et al. Arxiv (https://tinyurl.com/AIISC-PKiL)
Deep Infusion of Process Knowledge 21 ❏ Every layer within the deep neural network is checked for information divergence using external knowledge ❏ In case of Process Knowledge and in the context of Depression, we use Structured Clinical Interview information for Depression ❏ Knowledge Control: ❏ Checks if current learned representation can answer questions in SCID ❏ If none questions are answered, ❏ Information from SCID replace learned representation ❏ If some questions are answered incorrectly ( ) ❏ Learned representation is altered as follows: Structured Clinical Interview for Depression (SCID) backprop Kursuncu and Gaur et al. AAAI-MAKE’19
22 Context Sensitive Capture: Statistical AI is opinionated based on the text it sees and input is partial representation of the world. Uncertainty and Risk: Statistical AI, fail to establish the connection between input and output User-level Explainable: Statistical AI’s explanations are system-oriented and not rich enough for user-level understanding. Interpretable: A Statistical AI model that you can understand and control Task Transferable: Statistical AI learns the data and not the task Knowledge can highlight the context in input. Knowledge can assess risky prediction Knowledge can influence attention of statistical AI. Knowledge can enable User-level explanations Knowledge can help in generalize across tasks Benefits of Knowledge-infused Learning
Tutorial Overview 23 ❏ Challenges in Conversational Agents in Healthcare ❏ Can a model capture context and handle uncertainty in input? ❏ Can user-level explanations be obtained from the success or failure of an AI model? ❏ Can we control an AI model by making it interpretable? ❏ How can we make an AI model self-explainable? ❏ Tutorial Highlights ❏ Reinforcement Learning for Co-operative Multi-Agent System Interactions ❏ Process Knowledge Integration in Reinforcement Learning ❏ Demonstration of Process Knowledge Infusion
Tutorial Outline 24 ❏ Introduction ❏ Reinforcement Learning ❏ Knowledge-infused Reinforcement Learning ❏ Demo ❏ Summary and Future Work
25 1. Reinforcement Learning
What’s Reinforcement Learning? Learn to make good sequences of decisions 26
What’s Reinforcement Learning? Learn to make good sequences of decisions ⬢ Repeated interactions with environment 27
What’s Reinforcement Learning? Learn to make good sequences of decisions ⬢ Repeated interactions with environment ⬢ Reward to measure the quality of sequence of decisions 28
What’s Reinforcement Learning? Learn to make good sequences of decisions ⬢ Repeated interactions with environment ⬢ Reward to measure the quality of sequence of decisions ⬢ Don’t know in advance how environment works 29
Animal Reinforcements Negative reinforcements ⬢ Pain and hunger Positive reinforcements ⬢ Pleasure and food 30 Reinforcements used to train animals Let’s do the same with machines! ⬢ Here reinforcements are called rewards. ⬢ Machines are called (reinforcement learning) agents.
Rewards A reward r_t is a scalar feedback signal Indicates how well the agent is doing at step t The agent’s job is to maximize cumulative reward 31 Reinforcement learning is based on the reward hypothesis: All goals can be described by the maximization of expected cumulative reward
Examples of Rewards Defeat the world champion at Chess ⬢ +/- reward for winning/losing a game ⬢ No reward (i.e. 0 reward) in the middle of a game Manage an investment portfolio ⬢ + reward for each $ in bank Control a power station ⬢ + reward for producing power ⬢ − reward for exceeding safety thresholds Make a humanoid robot walk ⬢ + reward for forward motion ⬢ − reward for falling over 32
The RL Loop ⬢ Goal: select actions to maximize total future reward ⬢ May require sacrificing immediate reward to gain more long-term reward 33 Environment Agent Observation, Reward Action Examples: game playing (Go, Atari), operations research (pricing, vehicle routing), robotic control, conversational agents, autonomous vehicles, computational finance, etc.
The RL Loop: Blood Pressure Control 34 Environment Agent Observation: Patient’s information Reward: +1 if healthy pressure; - 0.05 for side effects Action: Exercise or medication ⬢ Goal: select actions to maximize total future reward ⬢ May require sacrificing immediate reward to gain more long-term reward
The RL Loop: Conversational Agent 35 Environment Agent Observation: User utterance Reward: Task completion, user satisfaction, etc. Action: Next utterance ⬢ Goal: select actions to maximize total future reward ⬢ May require sacrificing immediate reward to gain more long-term reward
Multi-Agent RL: A One-Slider 36 Environment Observation, Reward Action Agent N Agent 1 . . . Action 1 Action N Obs. & Reward 1 Obs. & Reward N
Multi Agent example: Manufacturing 37
38 2. Knowledge-infused Reinforcement Learning
Knowledge Infused Reinforcement Learning (KiRL) 39 Knowledge-infused Reinforcement Learning is a class of Neuro- Symbolic AI techniques that incorporates knowledge (lexical, domain-specific, common-sense, and constraint-based) into the policy function, resulting in better performance, interpretable models and user-level explanations.
Data Knowledge Infused RL State representation Knowledge States, actions, rewards Knowledge infused Policy Explanation module Knowledge Infused RL architecture (single agent) 40
Example: Adaptive Contagion Control 41 Roy et al. AAAI-MAKE’20
Example: Adaptive Contagion Control 42 Roy et al. AAAI-MAKE’20
43 End to end Chatbot using KiRL
Overall Execution Flow: Mental Health 44 ❏ User Profile Graphs: ❏ Historical Information about patients stored as Graph ❏ User Profile Graph + Clinical guidelines drives the high level decision making of the RL algorithm. ❏ The knowledge modifies the policy to tend towards the knowledge as a strong prior in a Bayesian formulation (We derive a MAP estimate by formulating it as an optimization problem)
Example: Information Gathering 45 ❏ Once the high level task is chosen, it is expanded to low level execution. ❏ For example what information to gather is obtained from a dialogue generation system that takes as input the patient profile ❏ The responses are parsed for new information to be added to the patient knowledge graph.
52 User Level Explainability
User level explanations 53 Definition from DSM 5 KG: SNOMEDCT
Process Knowledge-based User level explanation 54
Process Knowledge-based User level explanation 55 Process Knowledge Structure in C-SSRS I wish I could give a shit about what would make it to the front page. I have been there and got nothing. Same as my life. I do have a gun.’, ’I thought I was talking about it. I am not on a ledge or something, but I do have my gun in my lap.’, ’No. I made sure she got an education and she knows how to get a job. I also have recently bought her clothes to make her more attractive. She has told me she only loves me because I buy her things. 1. Wish to be dead - Yes 2. Non-specific Active Suicidal Thoughts - Yes 3. Active Suicidal Ideation with Some Intent to Act - Yes 4. Label: Suicide Behavior or Attempt Agreement with Experts 47% Process Knowledge (Ours) 70% XLNet Yang et al. NIPS’19 Gaur and Sheth et al. Internet Computing Under Review
57 Process Knowledge-based User level explanation
58 End-to-End Conversational Information Seeking using KiRL
Example: Conversational Information Seeking What is Conversational Information Seeking? Conversational Information Seeking (CIS) is an emerging research area within Conversation AI that attempts to seek information from end-users in order to understand and satisfy user’s needs. A CIS system can assist clinicians in pre-screening or triaging patients in healthcare. Here, the agent is the propeller of a conversation with a user What are Information Seeking Questions (ISQs) ? ISQs differ from other question type (e.g Clarifying questions, Follow-up questions) by having a ● Structure : semantic relations between questions and logical coherence ● Cover Objective Details ● Expand on the breath of topic Gaur et al. AAAI 2022 ® Samsung Research America
Properties of Human's Information Seeking Questions 60 ❏ Task Oriented ❏Seeking information for preparing a delicious cuisine ❏Seeking information about a health condition ❏ Question Styles ❏Contextually, lexically, and syntactically diverse ❏Semantically related and have logical order ❏ Response Shaping ❏Require understanding procedural questions ❏Keep track of entities and actions Gaur et al. AAAI 2022 ® Samsung Research America
ISEEQ: Information SEEking Question generation using Dynamic Meta-Information Retrieval and Knowledge Graphs 61 Gaur et al. AAAI 2022 ® Samsung Research America
Baseline T5 & ISEEQ 62 1. Deep Language Models generate irrelevant questions (blockchain and currencies) in absence of external knowledge 2. Without external knowledge, the model fails to (a) prevent redundancy (b) showcase diversity in question generation ISEEQ​ Baseline T5 Gaur et al. AAAI 2022 ® Samsung Research America
Example 64 Sentence BERT Encoder Sentence BERT Encoder 1. What is gross_domestic_product? 2. What is the measure of gross_domestic product? 3. What is the reason nation income relations gross_domestic_product? 4. What is the influence of inflation to gross_domestic_product? 5. What is the meaning of unemployment in inflation? 6. What is the influence of inflation on cost_of_living? Title: Economy and Employment Statistics Description: Learn Information about key economic concepts including gdp, inflation, and the influence on employment Constituency Parsing Information + { economy, employment statistics, employment, influence employment, inflation influence employment, gdp, gdp influence employment, key economic concepts} economics economy inflation employment gdp gross domestic product unemployment gnp gross national product national income cost of living income personal income income tax ConceptNet Graph for Semantic Query Expansion ISQ by Generative Adversarial Reinforcement Learning Knowledge-guided Passage Retriever Gaur et al. AAAI 2022 ® Samsung Research America Query
How ISEEQ show improvement? (Context: Healthcare) 65 Bothered by feeling hopeless and depressed. Need Advice. Generator Network • Are you feeling bothered? • Are you depressed? • Do you feel depressed? Generated questions are do not (a) Capture User Context (b) Share Semantic Relations (c) Have Logical Order Qcurr Baseline Generator-only Approach (T5) Bothered by feeling hopeless and depressed. Need Advice. Generator Network Neural Passage Retriever Reward (Qcurr , Qtrue) • Are you feeling bothered? • Are you depressed? • Do you feel depressed? • Do you feel like you are depressed sometime? • Do you know depression can make you mentally slow? Generated questions (a) Captures the context (b) Share semantic relations (c) Follow Logical Order Generated questions are unsafe Baseline Generator-only Approach (T5) with Neural Passage Retriever Gaur et al. AAAI 2022 ® Samsung Research America
66 How ISEEQ show improvement? (Context: Healthcare) Bothered by feeling hopeless and depressed. Need Advice. Generator Network Neural Passage Retriever Context Reward (Qcurr , Qtrue) • Do you know what cause depression? • Can you describe hopelessness? • Do you think you feel hopeless and depressed? • What is the cause of sadness? Commonsense Knowledge Graph ISEEQ Variant 1 Generated questions are (a) Captures the context (b) Share semantic relations (c) In logical Order Gaur et al. AAAI 2022 ® Samsung Research America
67 How ISEEQ show improvement? (Context: Healthcare) Bothered by feeling hopeless and depressed. Need Advice. Generator Network Evaluator Network ISEEQ Variant 2 Context Reward (Qcurr , Qtrue) Order (Qcurr, Qprev) Neural Passage Retriever Qcurr: Current Generated Question Qprev: Previous Generated Question Qtrue: Ground truth Question 1. Do you think you feel down most of the time? 2. How often do you feel depressed or hopeless? 3. How long have you struggled with depression? 4. Do you know what cause depression? Generated questions are (a) Captures the context (b) Share semantic relations (c) In logical Order Gaur et al. AAAI 2022 ® Samsung Research America
Summary and Future Directions
System 1 and System 2 Synchrony 69 Sheth and Thirunarayan, Duality of Data and Knowledge, IEEE Computer Society 2021 Low-level Data Sensors, Text, Image, and Collection Neural Network and Deep Learning Knowledge (rules, graphs, process knowledge, ..) Symbolic Reasoning Decisions/Actions System 1 System 2 Neural Network and Deep Learning Decisions/Actions System 1 Low-level Data Sensors, Text, Image, and Collection Daniel Kahneman - 2011 Thinking Fast and Slow
Corpus of Linguistic Acceptability Summarizing Clinical Interviews DSM-5 and PHQ-9 Flesch Reading, Divergence, Theme Overlap Matthew Correlation Rich Evaluation metrics Stanford Sentiment Treebank Assessing Severity in User-generated Content Ordinal Error, Perceived Risk Measure, Ranked Precision/Recall DSM-5 and Drug Abuse Ontology Accuracy Question NLI ConceptNet and WordNet Concept Mover Distance, BLEURT F1-Score and Accuracy User-language Paraphrase Corpus Microsoft Paraphrase Corpus Recognizing Textual Entailment Conversational Information Seeking Process Knowledge NLG Gaur et al. JMIR’21 Gaur et al. Pone’21, WWW’19, ACL’ 22 Roy & Gaur et al. ACL Under Review ConceptNet, WikiNews, Wikipedia , MS-MARCO Logical Coherence, Semantic Relevance, BLEURT Accuracy PHQ-9, GAD-7, C-SSRS Accuracy Avg. # Unsafe Matches, Avg. #KG concept Matches, Avg. Sq. Rank Error Reagle & Gaur FirstMonday’22 General Language Understanding Evaluation (GLUE) (Wang et al. ICLR’19) Knowledge-intensive Language Understanding (KILU) (Sheth and Gaur IEEE IC’21) Gaur et al. AAAI’22 Publicly Available Datasets with Knowledge Sources
71 Multimodal Conversational System for Autism Assessment Using ADOS Autism Diagnostic Observation Schedule
72 Personalized Knowledge Graph in Autism Assessment
73 User-level Explanations for Trust in AI System for Autism Assessment
References 74 ❏ Kaushik, R., Gaur, M., Zhang, Q., & Sheth, A. (2022). Process Knowledge-infused Learning for Suicidality Assessment on Social Media. Scholar Commons. ❏ Sheth, Amit, Manas Gaur, Kaushik Roy, and Keyur Faldu. "Knowledge-intensive language understanding for explainable ai." IEEE Internet Computing 25, 2021. ❏ Manas Gaur, Kalpa Gunaratna, Vijay Srinivasan, and Hongxia Jin. "ISEEQ: Information Seeking Question Generation using Dynamic Meta-Information Retrieval and Knowledge Graphs." arXiv preprint arXiv:2112.07622 (2021). In AAAI 2022 ❏ Kaushik Roy, Qi Zhang, Manas Gaur, and Amit Sheth. "Knowledge Infused Policy Gradients for Adaptive Pandemic Control." (2021). In AAAI 2021 ❏ Kaushik Roy, Qi Zhang, Manas Gaur, and Amit Sheth. "Knowledge infused policy gradients with upper confidence bound for relational bandits." In ECML-PKDD 2021 ❏ Ugur Kursuncu, Manas Gaur, and Amit Sheth. "Knowledge infused learning (k-il): Towards deep incorporation of knowledge in deep learning." In AAAI 2019 ❏ Gaur, Manas, Keyur Faldu, and Amit Sheth. "Semantics of the black-box: Can knowledge graphs help make deep learning systems more interpretable and explainable?." IEEE Internet Computing, 2021 ❏ Sheth, Amit, Manas Gaur, Ugur Kursuncu, and Ruwan Wickramarachchi. "Shades of knowledge-infused learning for enhancing deep learning." IEEE Internet Computing, 2019
Acknowledgement 75 Contribution to Demo on Virtual Assistant for Mental Health We acknowledge partial support from the National Science Foundation (NSF) award # 2133842 “EAGER: Advancing Neuro-symbolic AI with Deep Knowledge- infused Learning,” with PI Dr. Amit Sheth. We also acknowledge partial support from University of South Carolina ASPIRE Award Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation or University of South Carolina. Vedant Khandelwal Ph.D. Student, AIISC
Questions? For further details, please send email to mgaur@email.sc.edu More Project on Knowledge Graphs and Knowledge-infused Learning: http://wiki.aiisc.ai/ 76

More Related Content

PDF
AI in Healthcare a hands on workshop resource
byVaikunthan Rajaratnam
 
PPTX
ManasGaur_PhD_Dissertation_Defense_March25_2022.pptx
byArtificial Intelligence Institute at UofSC
 
PPTX
Knowledge-infused NLU for Addiction and Mental Health Research
byArtificial Intelligence Institute at UofSC
 
PPTX
Knowledge-infused NLU for Addiction and Mental Health Research (Keynote at MA...
byArtificial Intelligence Institute at UofSC
 
PDF
AI in Healthcare Resource forhands on Workshop
byVaikunthan Rajaratnam
 
PDF
AI for Neuroscience and Neuroscience for AI
byMLconf
 
PDF
AI in Healthcare UP Cambodia 29Jan24v2.pdf
byVaikunthan Rajaratnam
 
PDF
台灣人工智慧學校南部智慧醫療專班開學典禮 - 主題演講：邁向智慧醫療新時代（陳昇瑋執行長）
byAI.academy
 
AI in Healthcare a hands on workshop resource
byVaikunthan Rajaratnam
 
ManasGaur_PhD_Dissertation_Defense_March25_2022.pptx
byArtificial Intelligence Institute at UofSC
 
Knowledge-infused NLU for Addiction and Mental Health Research
byArtificial Intelligence Institute at UofSC
 
Knowledge-infused NLU for Addiction and Mental Health Research (Keynote at MA...
byArtificial Intelligence Institute at UofSC
 
AI in Healthcare Resource forhands on Workshop
byVaikunthan Rajaratnam
 
AI for Neuroscience and Neuroscience for AI
byMLconf
 
AI in Healthcare UP Cambodia 29Jan24v2.pdf
byVaikunthan Rajaratnam
 
台灣人工智慧學校南部智慧醫療專班開學典禮 - 主題演講：邁向智慧醫療新時代（陳昇瑋執行長）
byAI.academy
 

Similar to KGCTutorial_AIISC_2022.pptx

PPTX
ppt1 - Copy (1).pptx
bySRINIVASARAOKALUPUKU
 
PDF
Artificial Intelligence in Medicine
bySven Van Poucke, MD, PhD
 
PPTX
Invited_Reyes_BME_Basel_March2025v2.pptx
byMauricio Reyes
 
PDF
AI in Healthcare: From Hype to Impact (updated)
byMei Chen, PhD
 
PPTX
Semantics of the Black-Box: Using knowledge-infused learning approach to make...
byAmit Sheth
 
PPTX
Semantics of the Black-Box: Using knowledge-infused learning approach to make...
byArtificial Intelligence Institute at UofSC
 
PPTX
Module teaching for aft method for data science
byIPMCBIT
 
PDF
Comparing emotion classification: machine learning algorithms and hybrid mode...
byIAESIJAI
 
PDF
The Web We Mix - benevolent AIs for a resilient web
byFabien Gandon
 
PDF
Artificial Intelligence: Connecting the dots and relieving the pain in Health...
byJAI NAHAR, MD MBA
 
PDF
WHC Keynote Learn^JUnlearn and Relearn Sept 2024.pdf
byVaikunthan Rajaratnam
 
PPTX
Irina Rish, Researcher, IBM Watson, at MLconf NYC 2017
byMLconf
 
PDF
Big Data and Artificial Intelligence in Critical Care
byTommaso Scquizzato
 
PDF
Knowledge Representation For Health Care Aime 2015 International Joint Worksh...
byucedakariemu
 
PDF
Knowledge Representation For Health Care Aime 2015 International Joint Worksh...
byucedakariemu
 
PPTX
Artificial Intelligence for Medicine
byTassilo Klein
 
PDF
PMED Transition Workshop - Machine Learning Methods to Learn Improved Electro...
byThe Statistical and Applied Mathematical Sciences Institute
 
PDF
Ethics of Artificial Intelligence in Medicine
byAndrew Doyle
 
PDF
PyData Salamanca knowledge infusion in healthcare
byArtificial Intelligence Institute at UofSC
 
PDF
PyData Salamanca knowledge infusion in healthcare
byManas Gaur
 
ppt1 - Copy (1).pptx
bySRINIVASARAOKALUPUKU
 
Artificial Intelligence in Medicine
bySven Van Poucke, MD, PhD
 
Invited_Reyes_BME_Basel_March2025v2.pptx
byMauricio Reyes
 
AI in Healthcare: From Hype to Impact (updated)
byMei Chen, PhD
 
Semantics of the Black-Box: Using knowledge-infused learning approach to make...
byAmit Sheth
 
Semantics of the Black-Box: Using knowledge-infused learning approach to make...
byArtificial Intelligence Institute at UofSC
 
Module teaching for aft method for data science
byIPMCBIT
 
Comparing emotion classification: machine learning algorithms and hybrid mode...
byIAESIJAI
 
The Web We Mix - benevolent AIs for a resilient web
byFabien Gandon
 
Artificial Intelligence: Connecting the dots and relieving the pain in Health...
byJAI NAHAR, MD MBA
 
WHC Keynote Learn^JUnlearn and Relearn Sept 2024.pdf
byVaikunthan Rajaratnam
 
Irina Rish, Researcher, IBM Watson, at MLconf NYC 2017
byMLconf
 
Big Data and Artificial Intelligence in Critical Care
byTommaso Scquizzato
 
Knowledge Representation For Health Care Aime 2015 International Joint Worksh...
byucedakariemu
 
Knowledge Representation For Health Care Aime 2015 International Joint Worksh...
byucedakariemu
 
Artificial Intelligence for Medicine
byTassilo Klein
 
PMED Transition Workshop - Machine Learning Methods to Learn Improved Electro...
byThe Statistical and Applied Mathematical Sciences Institute
 
Ethics of Artificial Intelligence in Medicine
byAndrew Doyle
 
PyData Salamanca knowledge infusion in healthcare
byArtificial Intelligence Institute at UofSC
 
PyData Salamanca knowledge infusion in healthcare
byManas Gaur
 

Recently uploaded

PDF
Presentation - French Revolution (Student Lecture).pdf
byacdebussy
 
PPTX
Overview-Swine-Production-and-Breeds-of-Swine.pptx
bypanangcadjhonemart2
 
PDF
French Revolution { Class 9 } Presentation .pdf
bymanjusuhag88
 
PDF
Novel training approaches for the public administration in the context of Gre...
byProf. Dr. Fotios Fitsilis
 
PDF
Designing Spaces that Open Science and Foster Creative Activities
bynami575695
 
PPTX
2026-02-08 Hebrews 06 (shared slides).pptx
byDale Wells
 
PDF
AAC2026_Lavrova_ The Second Spear of Agility Why We Need Meaning, Not Metrics...
byAgile Austria Conference
 
PDF
MENTAL HEALTH SESSION BY REHEMA MUTHONI MUCHAI.
by2402232
 
PDF
AAC2026_Tanzer_Grown Organically Software and Bonsai.pdf
byAgile Austria Conference
 
PDF
When to Use Google Ads Instead of Local Service Ads: 4 Clear Scenarios
byDigital Harvest
 
PPTX
alkali metal physical and chemical properties
byolurantifaith43
 
PDF
AAC2026_Durcevic_Seeing the System: Making Relationships Visible to Unlock th...
byAgile Austria Conference
 
PDF
AAC2026_Liebisch_It Starts With You - How Personal Growth Builds the Foundati...
byAgile Austria Conference
 
PPTX
Febelfin Academy Presentation - Breakfast
byFinTech Belgium
 
PDF
What’s New in the Latest Roblox Update (2026)
bytenaciousmunir77
 
PDF
🇮🇳🇮🇳🇮🇳 Welcome to India 🇮🇳🇮🇳🇮🇳.pdf
bymanjusuhag88
 
PDF
AAC2026_Lister_Understand your 'Scrum DNA' !pdf
byAgile Austria Conference
 
PDF
Top 10 Tips To Buy Verified Binance Accounts_ An Educational Guide topselleri...
bygsghhgf8
 
PPTX
Street and Hip-hop Dance - Power Point -
byrhanzcerhamirez16
 
PPTX
Bob Stewart Greater 02 08 2026.pptx
byFamilyWorshipCenterD
 
Presentation - French Revolution (Student Lecture).pdf
byacdebussy
 
Overview-Swine-Production-and-Breeds-of-Swine.pptx
bypanangcadjhonemart2
 
French Revolution { Class 9 } Presentation .pdf
bymanjusuhag88
 
Novel training approaches for the public administration in the context of Gre...
byProf. Dr. Fotios Fitsilis
 
Designing Spaces that Open Science and Foster Creative Activities
bynami575695
 
2026-02-08 Hebrews 06 (shared slides).pptx
byDale Wells
 
AAC2026_Lavrova_ The Second Spear of Agility Why We Need Meaning, Not Metrics...
byAgile Austria Conference
 
MENTAL HEALTH SESSION BY REHEMA MUTHONI MUCHAI.
by2402232
 
AAC2026_Tanzer_Grown Organically Software and Bonsai.pdf
byAgile Austria Conference
 
When to Use Google Ads Instead of Local Service Ads: 4 Clear Scenarios
byDigital Harvest
 
alkali metal physical and chemical properties
byolurantifaith43
 
AAC2026_Durcevic_Seeing the System: Making Relationships Visible to Unlock th...
byAgile Austria Conference
 
AAC2026_Liebisch_It Starts With You - How Personal Growth Builds the Foundati...
byAgile Austria Conference
 
Febelfin Academy Presentation - Breakfast
byFinTech Belgium
 
What’s New in the Latest Roblox Update (2026)
bytenaciousmunir77
 
🇮🇳🇮🇳🇮🇳 Welcome to India 🇮🇳🇮🇳🇮🇳.pdf
bymanjusuhag88
 
AAC2026_Lister_Understand your 'Scrum DNA' !pdf
byAgile Austria Conference
 
Top 10 Tips To Buy Verified Binance Accounts_ An Educational Guide topselleri...
bygsghhgf8
 
Street and Hip-hop Dance - Power Point -
byrhanzcerhamirez16
 
Bob Stewart Greater 02 08 2026.pptx
byFamilyWorshipCenterD
 

KGCTutorial_AIISC_2022.pptx

  • 1.
    Knowledge Infused Reinforcement Learning Use-case:Conversational Systems Spread the word! Tweet, post on LinkedIn your insights, screenshots, and questions #KGC2022 #ProcessKnowledge #KnowledgeInfusedLearning #safeAI #InterpretableML #ExplainbleAI http://aiisc.ai/kirl
  • 2.
    About the Tutors 2 QiZhang, Professor, CS@UofSC Planning and Reinforcement Learning in Multi-Agent Systems, Knowledge-infused Reinforcement Learning Amit Sheth, Professor, CS@UofSC Director, AI Institute @ UofSC Knowledge-infused Learning, Knowledge Graphs, Semantic Web, Natural Language Understanding, Conversational Systems Manas Gaur, Ph.D. in CS @ UofSC Researcher, AI Institute @ UofSC Knowledge-infused Learning, Natural Language Understanding, Knowledge Graphs, Kaushik Roy, Ph.D. in CS @ UofSC Researcher, AI Institute @ UofSC Knowledge-infused Reinforcement Learning, Conversational Systems
  • 3.
    D O M A I N S 3 Conversational AI in[Healthcare] Sheth et al. 2018, Roy et al. 2021, Gaur et al. 2022 [Pandemic/Crisis] Matching Support Seekers with Support Providers (Gaur et al. 2021) [Education] ALLURE Collaborative Rubik’s Cube Solver Agostinelli et al. 2021 (https://tinyurl.com/AIISC-Rubiks) [Food] Allergy-Aware Recipe Understanding (Khandelwal et al. 2022) (https://tinyurl.com/AIISC-Allergies)
  • 4.
    How to Builda Conversational Agent 4 Attention Neural Network (Vaswani et al. NIPS’17) Large Amount of Textual Conversational Data in General Domain Trained Generative Model Picture Credit: https://keyreply.com/conversational-ai- for-healthcare/
  • 5.
    Agent tasked toConverse with Patient with Mental healthcare Condition 5 Do you feel nervous? More than half the days Do you feel irritated or self destructive? Do you feel something extreme might happen to you? Are you able to relax? The model can generate and ask either of these questions They are either bad questions or irrelevant ( A clinician won’t ask either of these) A model trained to asked questions Risky
  • 6.
    6 Unsafe Question Generation Doyou feel nervous? More than half the days Do you feel irritated or self destructive? Do you feel something extreme might happen to you? Are you able to relax? The model can generate and ask either of these questions A model trained to asked questions
  • 7.
    Large Language ModelsHallucinate 7 ❏ Generate Factually Incorrect Response ❏ Blenderbot 1 [Roller et al. 2020] ❏ Blenderbot 2 [https://tinyurl.com/Meta-bbot2] ❏ Generate Questions that aren’t Safe ❏ OpenAI GPT-3 [Brown et al. 2020] ❏ DeepMind’s RETRO [Borgeaud et al. 2021] ❏ Google’s LaMDA [Thoppilan et al. 2022] ❏ Utilize Google’s long list of Safety Guidelines to place constraint on Safety.
  • 8.
    8 Do you feelnervous? More than half the days Do you feel irritated or self destructive? Do you feel something extreme might happen to you? Are you able to relax? Do you feel nervous? More than half the days Do you feel Irritated? Are you bothered by becoming easily annoyed or irritable? Are you bothered by any relaxation troubles? Knowledge Infusion using Medical Questionnaire (MedQ) These questions are medically valid and safe. Safety Checks Knowledge fix : Safety Checks Roy and Gaur et al. ACL’22
  • 9.
    9 Do you feelnervous? More than half the days T5 (Raffel et. al. ACL’20) KI Attention Model (Ours) 30.6% 17.1% 10.6% 13.3% KI: Knowledge Infusion Do you feel Irritated? Are you bothered by becoming easily annoyed or irritable? Are you bothered by any relaxation troubles? (Ours) Roy and Gaur et al. ACL (under review) Effect of Knowledge fix : Safety Checks
  • 10.
    Attention Model I feelbothered by little interest and have least pleasure in doing anything Did you check your dopamine levels? Do you feel your brain is affected? Did you intend to indulge in risky behaviors I feel bothered by little interest and have least pleasure in doing anything What does “lack of pleasure” mean to you? Do you feel little pleasure doing things you used to enjoy? How long have you struggled with lack of interest in things you used to enjoy? KI Attention Model DSM-5 Lexicons for Depression SCID for Depression PHQ-9 Questionnair e Gaur et al. WWW’19; Yazdavar et al. ASONAM’17 0.83 0.78 0.71 0.33 0.21 0.19
  • 11.
    Safety Checks 11 If iscause then symptom If is symptom then medication If is medication then treatment Probability next question generation is Process Do you feel nervous? More than half the days Do you feel Irritated? Are you bothered by becoming easily annoyed or irritable? Are you bothered by any relaxation troubles?
  • 12.
    Uncertainty and Risk 12 GenerationTask Handling Uncertainty or Risk Human Annotation Experience (e.g. History) Web Search Corpus Expert Guidelines Generative Output Classification Output Labeled Dataset
  • 13.
    Safety Checks: Amethod of infuse Knowledge 13 Generation Task Handling Uncertainty or Risk Generative Output Classification Output Labeled Dataset Information graph (KG, Lexicons, etc.) Kashyap and Sheth CIKM’94
  • 14.
    Viable forms ofknowledge to infuse 14 Generative Output Classification Output Labeled Dataset Interpretability User-level Explainability Information graph (KG, Lexicons, etc.) Kashyap and Sheth CIKM’94 Generative Output Classification Output Labeled Dataset Interpretability User-level Explainability ConceptNet Healthcare lexicons
  • 15.
    15 Example KG constructedeither from manual effort (A, B, C), automatically (D, E), or semi-automatically (F) (A) is empathi ontology designed to identify concepts in disaster scenarios (Gaur et al. 2019). (B) Chem2Bio2RDF (Chen et al. 2010). (C) ATOMIC (Sap et al. 2019). (D) Education Knowledge Graph by Embibe (Faldu et al. 2020). (E) Event Cascade Graph in WildFire (Jiang et al. 2019). (F) Opioid Drug Knowledge Graph (Kamdar et al. 2019)
  • 16.
    Process Knowledge :Suicide Risk 16 Columbia Suicide Severity Rating Scale (C-SSRS) Process Knowledge Structure of C-SSRS Posner et al. 2011 Am. Journal of Psychiatry
  • 17.
    Process Knowledge :Anxiety 17 Spitzer et al. 2006 Archives of Internal Medicine Bartolo et al. 2017 SciELO Brasil
  • 18.
    Knowledge Infused Learning(KiL) 18 Knowledge-infused Learning is a class of Neuro-Symbolic AI techniques that incorporate broader forms of knowledge (lexical, domain-specific, common-sense, and constraint-based) into addressing limitations of either symbolic or statistical AI approaches, such as model interpretations and user-level explanations. Compared to powerful statistical AI that exploit data, KiL benefit from data as well as knowledge.
  • 19.
    19 Shallow Infusion ofProcess Knowledge Deep Language Model for Classification Semantic Embedding Loss ❏ External Knowledge is utilized in Annotating the Dataset ❏ Patient Health Questionnaire-9 (PHQ-9): an instrument to measure severity of Depression. ❏ Input is matched to Question ❏ To perform user-level classification ❏ Encodes knowledge as either word embeddings (text) or graph embeddings (graphs,rules, trees) ❏ User-level Explainability and Safety: ❏ Model’s classification is checked by the questions answered correctly. ❏ Correctly answered questions are matched to concepts highlighted in the input. Agarwal and Gupta et al. ACL RR Under Review embedding
  • 20.
    Semi-Deep Infusion ofProcess Knowledge 20 Deep Language Model For Representation Bernoulli Loss embedding Decision Tree ❏ Utilizes knowledge in its original form to modify the parameters of the ML model ❏ We employ and adapt interpretable ML models E.g. Decision Trees, Naive Bayes, K-Nearest Neighbors ❏ Knowledge is incorporated in the objective function to achieve parameter tweaking. ❏ User-level Explainability, Safety, & Interpretability: ❏ Model’s outcome is a tree → Interpretability ❏ Predicted Tree is checked with PHQ-9 Questions → User Level Explainability ❏ Bernoulli Loss keep refining the tree and can be tweaked by experts. Roy and Gaur et al. Arxiv (https://tinyurl.com/AIISC-PKiL)
  • 21.
    Deep Infusion ofProcess Knowledge 21 ❏ Every layer within the deep neural network is checked for information divergence using external knowledge ❏ In case of Process Knowledge and in the context of Depression, we use Structured Clinical Interview information for Depression ❏ Knowledge Control: ❏ Checks if current learned representation can answer questions in SCID ❏ If none questions are answered, ❏ Information from SCID replace learned representation ❏ If some questions are answered incorrectly ( ) ❏ Learned representation is altered as follows: Structured Clinical Interview for Depression (SCID) backprop Kursuncu and Gaur et al. AAAI-MAKE’19
  • 22.
    22 Context Sensitive Capture: StatisticalAI is opinionated based on the text it sees and input is partial representation of the world. Uncertainty and Risk: Statistical AI, fail to establish the connection between input and output User-level Explainable: Statistical AI’s explanations are system-oriented and not rich enough for user-level understanding. Interpretable: A Statistical AI model that you can understand and control Task Transferable: Statistical AI learns the data and not the task Knowledge can highlight the context in input. Knowledge can assess risky prediction Knowledge can influence attention of statistical AI. Knowledge can enable User-level explanations Knowledge can help in generalize across tasks Benefits of Knowledge-infused Learning
  • 23.
    Tutorial Overview 23 ❏ Challengesin Conversational Agents in Healthcare ❏ Can a model capture context and handle uncertainty in input? ❏ Can user-level explanations be obtained from the success or failure of an AI model? ❏ Can we control an AI model by making it interpretable? ❏ How can we make an AI model self-explainable? ❏ Tutorial Highlights ❏ Reinforcement Learning for Co-operative Multi-Agent System Interactions ❏ Process Knowledge Integration in Reinforcement Learning ❏ Demonstration of Process Knowledge Infusion
  • 24.
    Tutorial Outline 24 ❏ Introduction ❏Reinforcement Learning ❏ Knowledge-infused Reinforcement Learning ❏ Demo ❏ Summary and Future Work
  • 25.
  • 26.
    What’s Reinforcement Learning? Learnto make good sequences of decisions 26
  • 27.
    What’s Reinforcement Learning? Learnto make good sequences of decisions ⬢ Repeated interactions with environment 27
  • 28.
    What’s Reinforcement Learning? Learnto make good sequences of decisions ⬢ Repeated interactions with environment ⬢ Reward to measure the quality of sequence of decisions 28
  • 29.
    What’s Reinforcement Learning? Learnto make good sequences of decisions ⬢ Repeated interactions with environment ⬢ Reward to measure the quality of sequence of decisions ⬢ Don’t know in advance how environment works 29
  • 30.
    Animal Reinforcements Negative reinforcements ⬢Pain and hunger Positive reinforcements ⬢ Pleasure and food 30 Reinforcements used to train animals Let’s do the same with machines! ⬢ Here reinforcements are called rewards. ⬢ Machines are called (reinforcement learning) agents.
  • 31.
    Rewards A reward r_tis a scalar feedback signal Indicates how well the agent is doing at step t The agent’s job is to maximize cumulative reward 31 Reinforcement learning is based on the reward hypothesis: All goals can be described by the maximization of expected cumulative reward
  • 32.
    Examples of Rewards Defeatthe world champion at Chess ⬢ +/- reward for winning/losing a game ⬢ No reward (i.e. 0 reward) in the middle of a game Manage an investment portfolio ⬢ + reward for each $ in bank Control a power station ⬢ + reward for producing power ⬢ − reward for exceeding safety thresholds Make a humanoid robot walk ⬢ + reward for forward motion ⬢ − reward for falling over 32
  • 33.
    The RL Loop ⬢Goal: select actions to maximize total future reward ⬢ May require sacrificing immediate reward to gain more long-term reward 33 Environment Agent Observation, Reward Action Examples: game playing (Go, Atari), operations research (pricing, vehicle routing), robotic control, conversational agents, autonomous vehicles, computational finance, etc.
  • 34.
    The RL Loop:Blood Pressure Control 34 Environment Agent Observation: Patient’s information Reward: +1 if healthy pressure; - 0.05 for side effects Action: Exercise or medication ⬢ Goal: select actions to maximize total future reward ⬢ May require sacrificing immediate reward to gain more long-term reward
  • 35.
    The RL Loop:Conversational Agent 35 Environment Agent Observation: User utterance Reward: Task completion, user satisfaction, etc. Action: Next utterance ⬢ Goal: select actions to maximize total future reward ⬢ May require sacrificing immediate reward to gain more long-term reward
  • 36.
    Multi-Agent RL: AOne-Slider 36 Environment Observation, Reward Action Agent N Agent 1 . . . Action 1 Action N Obs. & Reward 1 Obs. & Reward N
  • 37.
    Multi Agent example:Manufacturing 37
  • 38.
  • 39.
    Knowledge Infused ReinforcementLearning (KiRL) 39 Knowledge-infused Reinforcement Learning is a class of Neuro- Symbolic AI techniques that incorporates knowledge (lexical, domain-specific, common-sense, and constraint-based) into the policy function, resulting in better performance, interpretable models and user-level explanations.
  • 40.
    Data Knowledge Infused RL Staterepresentation Knowledge States, actions, rewards Knowledge infused Policy Explanation module Knowledge Infused RL architecture (single agent) 40
  • 41.
    Example: Adaptive ContagionControl 41 Roy et al. AAAI-MAKE’20
  • 42.
    Example: Adaptive ContagionControl 42 Roy et al. AAAI-MAKE’20
  • 43.
    43 End to endChatbot using KiRL
  • 44.
    Overall Execution Flow:Mental Health 44 ❏ User Profile Graphs: ❏ Historical Information about patients stored as Graph ❏ User Profile Graph + Clinical guidelines drives the high level decision making of the RL algorithm. ❏ The knowledge modifies the policy to tend towards the knowledge as a strong prior in a Bayesian formulation (We derive a MAP estimate by formulating it as an optimization problem)
  • 45.
    Example: Information Gathering 45 ❏Once the high level task is chosen, it is expanded to low level execution. ❏ For example what information to gather is obtained from a dialogue generation system that takes as input the patient profile ❏ The responses are parsed for new information to be added to the patient knowledge graph.
  • 46.
  • 47.
  • 48.
    Process Knowledge-based Userlevel explanation 54
  • 49.
    Process Knowledge-based Userlevel explanation 55 Process Knowledge Structure in C-SSRS I wish I could give a shit about what would make it to the front page. I have been there and got nothing. Same as my life. I do have a gun.’, ’I thought I was talking about it. I am not on a ledge or something, but I do have my gun in my lap.’, ’No. I made sure she got an education and she knows how to get a job. I also have recently bought her clothes to make her more attractive. She has told me she only loves me because I buy her things. 1. Wish to be dead - Yes 2. Non-specific Active Suicidal Thoughts - Yes 3. Active Suicidal Ideation with Some Intent to Act - Yes 4. Label: Suicide Behavior or Attempt Agreement with Experts 47% Process Knowledge (Ours) 70% XLNet Yang et al. NIPS’19 Gaur and Sheth et al. Internet Computing Under Review
  • 50.
  • 51.
  • 52.
    Example: Conversational InformationSeeking What is Conversational Information Seeking? Conversational Information Seeking (CIS) is an emerging research area within Conversation AI that attempts to seek information from end-users in order to understand and satisfy user’s needs. A CIS system can assist clinicians in pre-screening or triaging patients in healthcare. Here, the agent is the propeller of a conversation with a user What are Information Seeking Questions (ISQs) ? ISQs differ from other question type (e.g Clarifying questions, Follow-up questions) by having a ● Structure : semantic relations between questions and logical coherence ● Cover Objective Details ● Expand on the breath of topic Gaur et al. AAAI 2022 ® Samsung Research America
  • 53.
    Properties of Human'sInformation Seeking Questions 60 ❏ Task Oriented ❏Seeking information for preparing a delicious cuisine ❏Seeking information about a health condition ❏ Question Styles ❏Contextually, lexically, and syntactically diverse ❏Semantically related and have logical order ❏ Response Shaping ❏Require understanding procedural questions ❏Keep track of entities and actions Gaur et al. AAAI 2022 ® Samsung Research America
  • 54.
    ISEEQ: Information SEEkingQuestion generation using Dynamic Meta-Information Retrieval and Knowledge Graphs 61 Gaur et al. AAAI 2022 ® Samsung Research America
  • 55.
    Baseline T5 & ISEEQ 62 1.Deep Language Models generate irrelevant questions (blockchain and currencies) in absence of external knowledge 2. Without external knowledge, the model fails to (a) prevent redundancy (b) showcase diversity in question generation ISEEQ​ Baseline T5 Gaur et al. AAAI 2022 ® Samsung Research America
  • 56.
    Example 64 Sentence BERT Encoder SentenceBERT Encoder 1. What is gross_domestic_product? 2. What is the measure of gross_domestic product? 3. What is the reason nation income relations gross_domestic_product? 4. What is the influence of inflation to gross_domestic_product? 5. What is the meaning of unemployment in inflation? 6. What is the influence of inflation on cost_of_living? Title: Economy and Employment Statistics Description: Learn Information about key economic concepts including gdp, inflation, and the influence on employment Constituency Parsing Information + { economy, employment statistics, employment, influence employment, inflation influence employment, gdp, gdp influence employment, key economic concepts} economics economy inflation employment gdp gross domestic product unemployment gnp gross national product national income cost of living income personal income income tax ConceptNet Graph for Semantic Query Expansion ISQ by Generative Adversarial Reinforcement Learning Knowledge-guided Passage Retriever Gaur et al. AAAI 2022 ® Samsung Research America Query
  • 57.
    How ISEEQ showimprovement? (Context: Healthcare) 65 Bothered by feeling hopeless and depressed. Need Advice. Generator Network • Are you feeling bothered? • Are you depressed? • Do you feel depressed? Generated questions are do not (a) Capture User Context (b) Share Semantic Relations (c) Have Logical Order Qcurr Baseline Generator-only Approach (T5) Bothered by feeling hopeless and depressed. Need Advice. Generator Network Neural Passage Retriever Reward (Qcurr , Qtrue) • Are you feeling bothered? • Are you depressed? • Do you feel depressed? • Do you feel like you are depressed sometime? • Do you know depression can make you mentally slow? Generated questions (a) Captures the context (b) Share semantic relations (c) Follow Logical Order Generated questions are unsafe Baseline Generator-only Approach (T5) with Neural Passage Retriever Gaur et al. AAAI 2022 ® Samsung Research America
  • 58.
    66 How ISEEQ showimprovement? (Context: Healthcare) Bothered by feeling hopeless and depressed. Need Advice. Generator Network Neural Passage Retriever Context Reward (Qcurr , Qtrue) • Do you know what cause depression? • Can you describe hopelessness? • Do you think you feel hopeless and depressed? • What is the cause of sadness? Commonsense Knowledge Graph ISEEQ Variant 1 Generated questions are (a) Captures the context (b) Share semantic relations (c) In logical Order Gaur et al. AAAI 2022 ® Samsung Research America
  • 59.
    67 How ISEEQ showimprovement? (Context: Healthcare) Bothered by feeling hopeless and depressed. Need Advice. Generator Network Evaluator Network ISEEQ Variant 2 Context Reward (Qcurr , Qtrue) Order (Qcurr, Qprev) Neural Passage Retriever Qcurr: Current Generated Question Qprev: Previous Generated Question Qtrue: Ground truth Question 1. Do you think you feel down most of the time? 2. How often do you feel depressed or hopeless? 3. How long have you struggled with depression? 4. Do you know what cause depression? Generated questions are (a) Captures the context (b) Share semantic relations (c) In logical Order Gaur et al. AAAI 2022 ® Samsung Research America
  • 60.
  • 61.
    System 1 andSystem 2 Synchrony 69 Sheth and Thirunarayan, Duality of Data and Knowledge, IEEE Computer Society 2021 Low-level Data Sensors, Text, Image, and Collection Neural Network and Deep Learning Knowledge (rules, graphs, process knowledge, ..) Symbolic Reasoning Decisions/Actions System 1 System 2 Neural Network and Deep Learning Decisions/Actions System 1 Low-level Data Sensors, Text, Image, and Collection Daniel Kahneman - 2011 Thinking Fast and Slow
  • 62.
    Corpus of Linguistic Acceptability Summarizing Clinical Interviews DSM-5 andPHQ-9 Flesch Reading, Divergence, Theme Overlap Matthew Correlation Rich Evaluation metrics Stanford Sentiment Treebank Assessing Severity in User-generated Content Ordinal Error, Perceived Risk Measure, Ranked Precision/Recall DSM-5 and Drug Abuse Ontology Accuracy Question NLI ConceptNet and WordNet Concept Mover Distance, BLEURT F1-Score and Accuracy User-language Paraphrase Corpus Microsoft Paraphrase Corpus Recognizing Textual Entailment Conversational Information Seeking Process Knowledge NLG Gaur et al. JMIR’21 Gaur et al. Pone’21, WWW’19, ACL’ 22 Roy & Gaur et al. ACL Under Review ConceptNet, WikiNews, Wikipedia , MS-MARCO Logical Coherence, Semantic Relevance, BLEURT Accuracy PHQ-9, GAD-7, C-SSRS Accuracy Avg. # Unsafe Matches, Avg. #KG concept Matches, Avg. Sq. Rank Error Reagle & Gaur FirstMonday’22 General Language Understanding Evaluation (GLUE) (Wang et al. ICLR’19) Knowledge-intensive Language Understanding (KILU) (Sheth and Gaur IEEE IC’21) Gaur et al. AAAI’22 Publicly Available Datasets with Knowledge Sources
  • 63.
    71 Multimodal Conversational Systemfor Autism Assessment Using ADOS Autism Diagnostic Observation Schedule
  • 64.
    72 Personalized Knowledge Graphin Autism Assessment
  • 65.
    73 User-level Explanations forTrust in AI System for Autism Assessment
  • 66.
    References 74 ❏ Kaushik, R.,Gaur, M., Zhang, Q., & Sheth, A. (2022). Process Knowledge-infused Learning for Suicidality Assessment on Social Media. Scholar Commons. ❏ Sheth, Amit, Manas Gaur, Kaushik Roy, and Keyur Faldu. "Knowledge-intensive language understanding for explainable ai." IEEE Internet Computing 25, 2021. ❏ Manas Gaur, Kalpa Gunaratna, Vijay Srinivasan, and Hongxia Jin. "ISEEQ: Information Seeking Question Generation using Dynamic Meta-Information Retrieval and Knowledge Graphs." arXiv preprint arXiv:2112.07622 (2021). In AAAI 2022 ❏ Kaushik Roy, Qi Zhang, Manas Gaur, and Amit Sheth. "Knowledge Infused Policy Gradients for Adaptive Pandemic Control." (2021). In AAAI 2021 ❏ Kaushik Roy, Qi Zhang, Manas Gaur, and Amit Sheth. "Knowledge infused policy gradients with upper confidence bound for relational bandits." In ECML-PKDD 2021 ❏ Ugur Kursuncu, Manas Gaur, and Amit Sheth. "Knowledge infused learning (k-il): Towards deep incorporation of knowledge in deep learning." In AAAI 2019 ❏ Gaur, Manas, Keyur Faldu, and Amit Sheth. "Semantics of the black-box: Can knowledge graphs help make deep learning systems more interpretable and explainable?." IEEE Internet Computing, 2021 ❏ Sheth, Amit, Manas Gaur, Ugur Kursuncu, and Ruwan Wickramarachchi. "Shades of knowledge-infused learning for enhancing deep learning." IEEE Internet Computing, 2019
  • 67.
    Acknowledgement 75 Contribution to Demoon Virtual Assistant for Mental Health We acknowledge partial support from the National Science Foundation (NSF) award # 2133842 “EAGER: Advancing Neuro-symbolic AI with Deep Knowledge- infused Learning,” with PI Dr. Amit Sheth. We also acknowledge partial support from University of South Carolina ASPIRE Award Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation or University of South Carolina. Vedant Khandelwal Ph.D. Student, AIISC
  • 68.
    Questions? For further details,please send email to mgaur@email.sc.edu More Project on Knowledge Graphs and Knowledge-infused Learning: http://wiki.aiisc.ai/ 76

Editor's Notes

  • #6 Harmful ---- Hallucinations Google’s LaMDA We are focusing on domain-specific issues : https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=9195554
  • #9 One way we can constrain safety by using guidelines These put a bound on topics covered -- solving patient need and prevent system from going into unsafe turfs Medical Conditions are ok Medical conditions that we leave it for human --- constraint our agent to remain in the bounds
  • #10 Percentage of Questions generated are unsafe One Example -- Which is unsafe in Vaswani et al. Which is unsafe in Raffel et al. You blocked ---- Why KiLSTM model blocked it?
  • #12 With this process knowledge, the conversational agent can sense when the generated question is safe and when it is unsafe
  • #13 That information is collapsed into a label which results into information loss
  • #22 This is under construction Where I want to go --- come up with a way -- latent representation (low level) -- scale up to abstraction -- Perception -- representation by Neural Network Cognition -- representation by external knowledge
  • #23 Knowledge for proper interpretation of the data Interpretable : Knowledge give you additional guidance, what is the relevance to algorithmic choices How the data drove the particular pathway The information you are deriving is not meaningful to outside world
  • #43 Some relational features that are learned with their English descriptions. It can be seen that the features allow finer grained control at the level of individual shops, homes, residences, workplaces, and routes.
  • #70 System 1: Processes low level data and enables decisions/actions (This is insufficient as low level data is not at a level of abstraction that humans understand or that can generalize across tasks) System 1 and 2: System one can process low level data which can be lifted to a higher level of abstraction by mapping to external knowledge. Symbolic reasoning over the mapped knowledge is then performed to enable decisions/actions.
  • #71 Knowledge Intensive Language Understanding (X axis) vs GLUE (Y axis).
  • #73 Sensory data abstracted and compiled as a PKG and also processed through a multimodal modeling pipeline. The multimodal model (deep network) interacts with the PKG and provides the assessment of ASD.
  • #79 Add flowchart, put images on right and text on left. What is Knowledge Infused RL, limitations of current RL.
  • #81 Open questions in CLPsych