Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Learning to speak medicine


Published on

Keynote I gave at the Chatbot Workshop at ICWSM 2018

Published in: Technology
  • Be the first to comment

Learning to speak medicine

  1. 1. Learning To Speak Medicine Xavier Amatriain
  2. 2. Part I. The Medical Conversation
  3. 3. Medical Diagnosis ● Diagnosis (R.A. Miller 1990): ○ Mapping from patient’s data (history, examination, lab exams…) to a possible condition. ○ It depends on ability to: ■ Evoke history ■ Surface symptoms and findings ■ Generate hypotheses that suggest how to refine or pursue different hypothesis ○ In a compassionate, cost-effective manner ● Doctor: “How can I help?” ● Patient: Chief complaint ● Doctor: “Anything else?” ● Patient: …. ● Start of a doctor-led Q&A: ○ E.g. “Do you have X?” ● Doctor communicates actionable recommendation (diagnosis + treatment, triage, referral…)
  4. 4. Medicine is hard(er) ● Doctors have ~15 minutes to capture information* about a patient, diagnose, and recommend treatment ● *Information ○ Patient’s history ○ Patient’s symptoms ○ Medical knowledge ■ Learned years ago ■ Latest research findings ■ Different demographics ● Data is growing over time, so is complexity
  5. 5. The future of healthcare ● Patient-centered, AI-driven, doctor-in-the-loop ● Increased value of patient/doctor interaction ● Data-enabled applications ● Individual/personalized recommendations ● Multimodal (images, voice, audio…) ● Pro-active Prognosis/Diagnosis (eg. when sensor reading abnormal or new information available) ● Identifying and warning about side effect of medications ● Conversational systems are a core component Personalized AI agent Sensors Conversational System (eliciting and providing information, recommending outcomes, treatments, prevention)
  6. 6. Part II. Why now?
  7. 7. Is it a new idea? ● Internist -1 (1971) led by Jack Myers considered (one of) the best clinical diagnostic experts in the US ● Process for adding a disease requires 2-4 weeks of full-time effort and doctors reading 50 to 250 relevant publications ● Very structured heuristic-driven dialog system
  8. 8. Is it still hot?
  9. 9. Research efforts
  10. 10. Data Availability ● Large-scale patient-level clinical data ○ Electronic health records ○ Electronic imaging: x-rays, scans ○ Genes ● Smart devices as health sensors ○ Wearables ○ FDA-approved phone apps ○ High-quality images ● Electronic access to medical research ● Much of this only available in the last few years
  11. 11. The Language of Medicine ● ICD(10) ○ 10th revision of the International Statistical Classification of Diseases and Related Health Problems (ICD) ○ Codes for diseases, symptoms, findings, complaints... ○ Evolved from Bertillon Classification of Causes of Death (1893) ○ UN gave WHO responsibility for the ICD in 1946 ● Snomed Clinical Terms ○ Computer processable collection of medical terms used in clinical documentation and reporting. ○ Clinical findings, symptoms, diagnoses, procedures, body structures, organisms substances, pharmaceuticals, devices… ○ Started in 1965, has had multiple revisions and mutations ● UMLS ○ Compendium of many controlled vocabularies (1986) ○ Mapping structure among vocabularies ○ Allows to translate among the various terminology systems
  12. 12. Electronic health records ● Digital information about patients encounters with doctors or the health system ● An encounter may include ○ Doctor notes, medications, procedures, diagnosis ● Mix of structured data and unstructured text
  13. 13. Open Data ● Hnlp ● I2b2 ● Mimic-3 ● ….
  14. 14. Part III. Building Medical Dialogue Systems
  15. 15. Medical conversation as a task-oriented dialogue
  16. 16. Intent classification
  17. 17. Text representation ● Infer structured representation An 18-year-old male student presents with severe headache and fever that he has had for 3 days. Examination reveals fever, photophobia, and neck stiffness Demographics: ● Age: 18 years ● Gender: Male Symptoms: ● headache ○ Severity: severe ● Fever ○ Duration: 3 days ● Photophobia ● Neck stiffness
  18. 18. Slot filling in frames
  19. 19. Text representation ● Infer latent semantic space through embeddings A. L. Beam, B. Kompa, I. Fried, N. P. Palmer, X. Shi, T. Cai, and I. S. Kohane. 2018. Clinical Concept Embeddings Learned from Massive Sources of Medical Data. ArXiv e-prints (April 2018). 108,477 medical concepts using: ● insurance claims database of 60 million members ● 20 million clinical notes ● 1.7 million full text biomedical journal articles
  20. 20. Entity recognition
  21. 21. Entity recognition
  22. 22. The Deep Learning promise
  23. 23. Learning to infer structure in text A. Jagannatha and H. Yu, Bidirectional Recurrent Neural Networks for Medical Event Detection in Electronic Health Records, 2016 A. Vani Grounded Recurrent Neural Networks, arXiv, 2017 bleeding due to warfarin Adverse drug effect - - medication
  24. 24. End-to-end Deep Learning Dialog Systems SOTA Language model input: User: got home and when i 've been at work i 've been experiencing a pain like a 4 on my left side by my rib and some days i feel where i had my iv in my arm on march 2 when i went to the er for my constipation Dr: ah i see . have you tried anything for relief ? User: took an acetaminophen but that only last for about an hour my body burns off medicine really fast i learned that at a young age Language model output: Dr: i see . how bad is the pain ? on a scale of 1 - 10 10 being the worst how would you rate it ? User: 8 Dr: sounds rough ! applying a warm compress 2 - 3 times a day warm baths with epsom salt massaging...
  25. 25. End-to-end Deep Learning Dialog Systems
  26. 26. References
  27. 27. ● “An interlingua for electronic interchange of medical information: using frames to map between clinical vocabularies.” Masarie. 1991 ● “Health dialog systems for patients and consumers.” Bickmore. 2006 ● “Designing a Chatbot for diabetic patients”, Lokman. 2007 ● “Computer-assisted diagnostic decision support: history, challenges, and possible paths forward” Miller. 2009. ● “Serving the enterprise and beyond with informatics for integrating biology and the bedside (i2b2)” Murphy. 2010 ● “Mining Biomedical Ontologies and Data Using RDF Hypergraphs” Liu et al. 2013. ● “From health search to healthcare: explorations of intention and utilization via query logs and user surveys” White. 2014 ● “Pharmabot: A Pediatric Generic Medicine Consultant Chatbot”, Comendador. 2015 ● “Identifiable Phenotyping using Constrained Non-Negative Matrix Factorization”. Joshi et al. 2016. ● “Clinical Tagging with Joint Probabilistic Models” . Halpern et al. 2016. ● “MIMIC-III, a freely accessible critical care database” Johnson. 2016 ● “Disease named entity recognition by combining conditional random fields & bidirectional recurrent neural networks” Wei. 2016 ● “Named Entity Recognition Over Electronic Health Records Through a Combined Dictionary-based Approach”, Pomares. 2016 ● “Bidirectional Recurrent Neural Networks for Medical Event Detection in Electronic Health Records”, Jagannatha. 2016 ● “Bringing semantic structures to user intent detection in online medical queries”, Zhang. 2017 ● “A Conversational Chatbot Based on Kowledge-Graphs for Factoid Medical Questions”, A. Minutolo. 2017 ● “MANDY: Towards a Smart Primary Care Chatbot Application”. Ni. 2017 ● “Entity recognition from clinical texts via recurrent neural network”, Liu. 2017 ● “Knowledge-driven Entity Recognition and Disambiguation in Biomedical Text”. Siu. 2017 ● “Deep Learning for Dialogue Systems”. Vivian Chen. 2017 ● “Universal Language Model Fine-tuning for Text Classification”, Howard. 2018 ● “Clinical Concept Embeddings Learned from Massive Sources of Medical Data” A. L. Beam, et al. ArXiv e-prints (April 2018). References
  28. 28. Yes, we’re hiring!