Your SlideShare is downloading. ×
0
Dr. Eliot Siegel: Watson and Deep QA Software in Pursuit of Personalized Medicine
Dr. Eliot Siegel: Watson and Deep QA Software in Pursuit of Personalized Medicine
Dr. Eliot Siegel: Watson and Deep QA Software in Pursuit of Personalized Medicine
Dr. Eliot Siegel: Watson and Deep QA Software in Pursuit of Personalized Medicine
Dr. Eliot Siegel: Watson and Deep QA Software in Pursuit of Personalized Medicine
Dr. Eliot Siegel: Watson and Deep QA Software in Pursuit of Personalized Medicine
Dr. Eliot Siegel: Watson and Deep QA Software in Pursuit of Personalized Medicine
Dr. Eliot Siegel: Watson and Deep QA Software in Pursuit of Personalized Medicine
Dr. Eliot Siegel: Watson and Deep QA Software in Pursuit of Personalized Medicine
Dr. Eliot Siegel: Watson and Deep QA Software in Pursuit of Personalized Medicine
Dr. Eliot Siegel: Watson and Deep QA Software in Pursuit of Personalized Medicine
Dr. Eliot Siegel: Watson and Deep QA Software in Pursuit of Personalized Medicine
Dr. Eliot Siegel: Watson and Deep QA Software in Pursuit of Personalized Medicine
Dr. Eliot Siegel: Watson and Deep QA Software in Pursuit of Personalized Medicine
Dr. Eliot Siegel: Watson and Deep QA Software in Pursuit of Personalized Medicine
Dr. Eliot Siegel: Watson and Deep QA Software in Pursuit of Personalized Medicine
Dr. Eliot Siegel: Watson and Deep QA Software in Pursuit of Personalized Medicine
Dr. Eliot Siegel: Watson and Deep QA Software in Pursuit of Personalized Medicine
Dr. Eliot Siegel: Watson and Deep QA Software in Pursuit of Personalized Medicine
Dr. Eliot Siegel: Watson and Deep QA Software in Pursuit of Personalized Medicine
Dr. Eliot Siegel: Watson and Deep QA Software in Pursuit of Personalized Medicine
Dr. Eliot Siegel: Watson and Deep QA Software in Pursuit of Personalized Medicine
Dr. Eliot Siegel: Watson and Deep QA Software in Pursuit of Personalized Medicine
Dr. Eliot Siegel: Watson and Deep QA Software in Pursuit of Personalized Medicine
Dr. Eliot Siegel: Watson and Deep QA Software in Pursuit of Personalized Medicine
Dr. Eliot Siegel: Watson and Deep QA Software in Pursuit of Personalized Medicine
Dr. Eliot Siegel: Watson and Deep QA Software in Pursuit of Personalized Medicine
Dr. Eliot Siegel: Watson and Deep QA Software in Pursuit of Personalized Medicine
Dr. Eliot Siegel: Watson and Deep QA Software in Pursuit of Personalized Medicine
Dr. Eliot Siegel: Watson and Deep QA Software in Pursuit of Personalized Medicine
Dr. Eliot Siegel: Watson and Deep QA Software in Pursuit of Personalized Medicine
Dr. Eliot Siegel: Watson and Deep QA Software in Pursuit of Personalized Medicine
Dr. Eliot Siegel: Watson and Deep QA Software in Pursuit of Personalized Medicine
Dr. Eliot Siegel: Watson and Deep QA Software in Pursuit of Personalized Medicine
Dr. Eliot Siegel: Watson and Deep QA Software in Pursuit of Personalized Medicine
Dr. Eliot Siegel: Watson and Deep QA Software in Pursuit of Personalized Medicine
Dr. Eliot Siegel: Watson and Deep QA Software in Pursuit of Personalized Medicine
Dr. Eliot Siegel: Watson and Deep QA Software in Pursuit of Personalized Medicine
Dr. Eliot Siegel: Watson and Deep QA Software in Pursuit of Personalized Medicine
Dr. Eliot Siegel: Watson and Deep QA Software in Pursuit of Personalized Medicine
Dr. Eliot Siegel: Watson and Deep QA Software in Pursuit of Personalized Medicine
Dr. Eliot Siegel: Watson and Deep QA Software in Pursuit of Personalized Medicine
Dr. Eliot Siegel: Watson and Deep QA Software in Pursuit of Personalized Medicine
Dr. Eliot Siegel: Watson and Deep QA Software in Pursuit of Personalized Medicine
Dr. Eliot Siegel: Watson and Deep QA Software in Pursuit of Personalized Medicine
Dr. Eliot Siegel: Watson and Deep QA Software in Pursuit of Personalized Medicine
Dr. Eliot Siegel: Watson and Deep QA Software in Pursuit of Personalized Medicine
Dr. Eliot Siegel: Watson and Deep QA Software in Pursuit of Personalized Medicine
Dr. Eliot Siegel: Watson and Deep QA Software in Pursuit of Personalized Medicine
Dr. Eliot Siegel: Watson and Deep QA Software in Pursuit of Personalized Medicine
Dr. Eliot Siegel: Watson and Deep QA Software in Pursuit of Personalized Medicine
Dr. Eliot Siegel: Watson and Deep QA Software in Pursuit of Personalized Medicine
Dr. Eliot Siegel: Watson and Deep QA Software in Pursuit of Personalized Medicine
Dr. Eliot Siegel: Watson and Deep QA Software in Pursuit of Personalized Medicine
Dr. Eliot Siegel: Watson and Deep QA Software in Pursuit of Personalized Medicine
Dr. Eliot Siegel: Watson and Deep QA Software in Pursuit of Personalized Medicine
Dr. Eliot Siegel: Watson and Deep QA Software in Pursuit of Personalized Medicine
Dr. Eliot Siegel: Watson and Deep QA Software in Pursuit of Personalized Medicine
Dr. Eliot Siegel: Watson and Deep QA Software in Pursuit of Personalized Medicine
Dr. Eliot Siegel: Watson and Deep QA Software in Pursuit of Personalized Medicine
Dr. Eliot Siegel: Watson and Deep QA Software in Pursuit of Personalized Medicine
Dr. Eliot Siegel: Watson and Deep QA Software in Pursuit of Personalized Medicine
Dr. Eliot Siegel: Watson and Deep QA Software in Pursuit of Personalized Medicine
Dr. Eliot Siegel: Watson and Deep QA Software in Pursuit of Personalized Medicine
Dr. Eliot Siegel: Watson and Deep QA Software in Pursuit of Personalized Medicine
Dr. Eliot Siegel: Watson and Deep QA Software in Pursuit of Personalized Medicine
Dr. Eliot Siegel: Watson and Deep QA Software in Pursuit of Personalized Medicine
Dr. Eliot Siegel: Watson and Deep QA Software in Pursuit of Personalized Medicine
Dr. Eliot Siegel: Watson and Deep QA Software in Pursuit of Personalized Medicine
Dr. Eliot Siegel: Watson and Deep QA Software in Pursuit of Personalized Medicine
Dr. Eliot Siegel: Watson and Deep QA Software in Pursuit of Personalized Medicine
Dr. Eliot Siegel: Watson and Deep QA Software in Pursuit of Personalized Medicine
Dr. Eliot Siegel: Watson and Deep QA Software in Pursuit of Personalized Medicine
Dr. Eliot Siegel: Watson and Deep QA Software in Pursuit of Personalized Medicine
Dr. Eliot Siegel: Watson and Deep QA Software in Pursuit of Personalized Medicine
Dr. Eliot Siegel: Watson and Deep QA Software in Pursuit of Personalized Medicine
Dr. Eliot Siegel: Watson and Deep QA Software in Pursuit of Personalized Medicine
Dr. Eliot Siegel: Watson and Deep QA Software in Pursuit of Personalized Medicine
Dr. Eliot Siegel: Watson and Deep QA Software in Pursuit of Personalized Medicine
Dr. Eliot Siegel: Watson and Deep QA Software in Pursuit of Personalized Medicine
Dr. Eliot Siegel: Watson and Deep QA Software in Pursuit of Personalized Medicine
Dr. Eliot Siegel: Watson and Deep QA Software in Pursuit of Personalized Medicine
Dr. Eliot Siegel: Watson and Deep QA Software in Pursuit of Personalized Medicine
Dr. Eliot Siegel: Watson and Deep QA Software in Pursuit of Personalized Medicine
Dr. Eliot Siegel: Watson and Deep QA Software in Pursuit of Personalized Medicine
Dr. Eliot Siegel: Watson and Deep QA Software in Pursuit of Personalized Medicine
Dr. Eliot Siegel: Watson and Deep QA Software in Pursuit of Personalized Medicine
Dr. Eliot Siegel: Watson and Deep QA Software in Pursuit of Personalized Medicine
Dr. Eliot Siegel: Watson and Deep QA Software in Pursuit of Personalized Medicine
Dr. Eliot Siegel: Watson and Deep QA Software in Pursuit of Personalized Medicine
Dr. Eliot Siegel: Watson and Deep QA Software in Pursuit of Personalized Medicine
Dr. Eliot Siegel: Watson and Deep QA Software in Pursuit of Personalized Medicine
Dr. Eliot Siegel: Watson and Deep QA Software in Pursuit of Personalized Medicine
Dr. Eliot Siegel: Watson and Deep QA Software in Pursuit of Personalized Medicine
Dr. Eliot Siegel: Watson and Deep QA Software in Pursuit of Personalized Medicine
Dr. Eliot Siegel: Watson and Deep QA Software in Pursuit of Personalized Medicine
Dr. Eliot Siegel: Watson and Deep QA Software in Pursuit of Personalized Medicine
Dr. Eliot Siegel: Watson and Deep QA Software in Pursuit of Personalized Medicine
Dr. Eliot Siegel: Watson and Deep QA Software in Pursuit of Personalized Medicine
Dr. Eliot Siegel: Watson and Deep QA Software in Pursuit of Personalized Medicine
Dr. Eliot Siegel: Watson and Deep QA Software in Pursuit of Personalized Medicine
Dr. Eliot Siegel: Watson and Deep QA Software in Pursuit of Personalized Medicine
Dr. Eliot Siegel: Watson and Deep QA Software in Pursuit of Personalized Medicine
Dr. Eliot Siegel: Watson and Deep QA Software in Pursuit of Personalized Medicine
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Dr. Eliot Siegel: Watson and Deep QA Software in Pursuit of Personalized Medicine

1,266

Published on

The Jeopardy match between the two best human players of all time and the IBM Deep Q/A software, “Watson,” captured the spotlight and stimulated the imagination of the entire world. The subsequent …

The Jeopardy match between the two best human players of all time and the IBM Deep Q/A software, “Watson,” captured the spotlight and stimulated the imagination of the entire world. The subsequent announcement of IBM’s involvement in the creation of “Dr. Watson” has created a high level of interest in the healthcare community about the potential of this breakthrough technology as well as the potential pitfalls of the use of “artificial intelligence” in medicine. Dr. Siegel is currently working together with IBM engineers to explore how Dr. Watson can work together with physicians and medical specialists. His presentation, which was delivered on March 28th, provided a high level overview of the uniqueness of Deep Q/A Software and how it differs from other previous artificial intelligence applications.

0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
1,266
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
47
Comments
0
Likes
1
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  1. CBIIT Speaker Series Watson and Deep Q/A Software In Pursuit of Personalized Medicine Eliot Siegel, M.D., FACR, FSIIM Professor and Vice Chair University of Maryland Department of Diagnostic Radiology Chief Imaging VA Maryland Healthcare System1
  2. caBIG Mission • Widespread, sustainable availability of critical standards-based, interoperable academic/commercial biomedical capabilities • Large and diverse cancer research data sets sustainably available for analysis, integration, and mining • Rather than 2% or 3% of patients’ data captured in clinical trials, capture all patient data for decision and treatment support and data driven research2
  3. Clinical Scenario3
  4. Clinical Challenge VINCI – VA Informatics and Computing Infrastructure4
  5. Year of Artificial Intelligence in Medicine • 2011 will likely be remembered as the year of the re-emergence of artificial intelligence in medicine with Watson and of course, Siri, arguably the best feature of the new iPhone 4S • 2011 may well be the year that AI finally gets real traction in the medical informatics community and in medicine in general including the lay population • Biggest contribution of Dr. Watson software in addition to Deep Q/A may be excitement to overcome inertia of the past5
  6. IBM and Jeopardy: A New Era? • The Jeopardy match between the two best human players of all time and the IBM Deep Q/A software, “Watson” captured the spotlight and stimulated the imagination of the entire world • The subsequent announcement of IBM’s involvement in the creation of “Dr. Watson” has created an incredible interest in the healthcare community about the potential breakthrough technology as well as the potential pitfalls of the use of “artificial intelligence” in medicine.6
  7. Dr. Watson Overview and History • Initially had opportunity to visit IBM team about a year and a half ago • Engaged Jeopardy team and discussed the potential for medical applications as next steps after Jeopardy Challenge • Began initial research with IBM approximately one year ago • Current grant with IBM for initial exploratory work with physician helping team to understand the medical domain and challenges • Worked together on deeper understanding of the medical domain using multiple resources7
  8. Introduction • Deep Q/A is unique and exciting because it represents a fundamentally new approach that creates tools to rapidly mine a dynamic and non- predefined database • Represents a potential fundamental change in opportunities for Artificial Intelligence applications in medicine • But in some ways Watson is a “special needs” student • How does one train a system that is so remarkable at Jeopardy! questions and apply to medicine?8
  9. • Watson can process 500 gigabytes, the equivalent of a million books, per second • Hardware cost has been estimated at about $3 million • 80 TeraFLOPs , 49th in the Top 50 Supercomputers list • Content was stored in Watsons RAM for the game because data stored on hard drives too slow to process9
  10. Deep Q/A • Massively parallel, component based pipeline architecture • Uses extensible set of structured and unstructured content sources • Uses broad range of pluggable search and scoring components10
  11. Deep Q/A • These allow integration of many different analytic techniques • Input from scorers is weighed and combined using machine learning to generate a set of ranked candidate answers and associated confidence values • Each answer is linked to its supporting evidence11
  12. Deep Q/A • Does not map question to database of answers • Represents software architecture for analyzing natural language content in both questions and knowledge sources • Discovers and evaluates potential answers and gathers and scores evidence for those answers using unstructured sources such as natural language documents and structured sources such as relational and knowledge databases12
  13. Hardware • Cluster of ninety IBM Power 750 servers (plus additional I/O, network and cluster controller nodes in 10 racks) with a total of 2880 POWER7 processor cores and 16 Terabytes of RAM • Each Power 750 server uses a 3.5 GHz POWER7 eight core processor, with four threads per core • The POWER7 processors massively parallel processing capability is an ideal match for Watsons IBM DeepQA software which is embarrassingly parallel (that is a workload that is easily split up into multiple parallel tasks)13
  14. Software • Watsons software was written in both Java and C++ and uses Apache Had0op framework for distributed computing • Apache UIMA (Unstructured Information Management Architecture) framework • IBM’s DeepQA software and SUSE Linux Enterprise Server 11 operating system • “More than 100 different techniques are used to analyze natural language, identify sources, find and generate hypotheses, find and score evidence, and merge and rank hypotheses.”14
  15. High Level View of DeepQA Architecture15
  16. Deep QA Process • Analyzes input question and generates many possible candidate answers through broad search of volumes of content • Hypothesis is formed based on considerate of each candidate answer in context of original question and topic • For each of these, DeepQA spawns independent thread attempting to prove it • Searches content sources for evidence supporting or refuting each hypothesis • Applies hundreds of algorithms for each evidence hypothesis pair that dissects and analyzes along different dimensions of evidence16
  17. Types of Dimensions of Evidence • Type classification • Time • Geography • Popularity • Passage support • Source reliability • Semantic relatedness17
  18. Dimensions of Evidence for Jeopardy!18
  19. Scoring Features • These features/scores are then combined based on their learned potential for predicting the right answer resulting in a ranked list of candidate answers, each with a confidence score indicating degree to which the answer is believed to be correct, along with links back to the evidence19
  20. Deep QA for Differential Diagnosis20
  21. Advantages of Dr. Watson Approach • Represents new architecture for evaluating unstructured content • Different from traditional expert systems using forward reasoning (data to conclusions) or backward reasoning • Unlike systems such as Stanford’s Mycin that used If-Then statements: • If • The stain of the organism is grampos and the morphology of the organism is coccus and the growth conformation of the organism is chains • Then • There is suggestive evidence that the identity of the organism is streptococcus21
  22. Advantages of Watson Approach • If then approach is costly and difficult to develop and maintain • Traditional expert systems are brittle because underlying reasoning engine requires perfect match between input data and existing rule forms • Not all rule forms can be known in advance for all forms that the input data may take22
  23. Advantages of Watson Approach • Watson uses NLP and variety of search techniques to generate likely candidate answers in hypothesis generation (analogous to forward chaining”) • Uses evidence collection and scoring (analogous to “backward chaining”) • These make DeepQA more flexible, maintainable, and scalable as well as cost efficient in terms of staying current with vast amounts of new information23
  24. Clinical Setting • Deep QA can develop diagnostic support tool using the context of an input case (information about patient’s medical condition) • Generates ranked list of differential diagnoses with associated confidences • The dimensions of evidence include • Symptoms • Findings • Patient history • Family history • Demographics • Current medications • Many others24
  25. Is There A Need for Artificial Intelligence In Medicine? Do Physicians Need Assistance?25
  26. Motivation for Artificial Intelligence Software in Medicine • Schiff • Diagnostic errors far outnumber other medical errors by 2-4X • Elstein • Diagnostic error rate of about 15% in line with autopsy studies • Singh and Graber • Diagnostic errors are single largest contributor to ambulatory malpractice claims (40% in some studies) and cost about $300,000 per claim • Graber • Literature review of causes of diagnostic error suggest 65% system related (e.g. communication) and 75% had cognitive related factors26
  27. Cognitive Errors Graber et al Diagnostic Error in Internal Medicine, Arch Intern Med 2005; 165:1493-1499 • Cognitive errors primary due to “faulty synthesis or flawed processing of the available information” • Predominant cause of cognitive error was premature closure (satisfaction of search in diagnostic imaging) • Failure to continue considering reasonable alternatives after an initial diagnosis was reached27
  28. Cognitive Errors • Other contributors to cognitive errors • Faulty context generation – lack of awareness of aspects of patient info relevant to diagnosis • Misjudging salience of a finding • Faulty detection or perception • Failed use of heuristics – assuming single rather than multifactorial cause of patient symptoms28
  29. Cognitive Errors • Graber suggested augmenting “a clinician’s inherent metacognitive skills by using expert systems” • Suggested that clinicians continue to miss diagnostic information and “one likely contributing factor is the overwhelming volume of alerts, reminders, and other diagnostic information in the Electronic Health Record”29
  30. Previous Attempts at Artificial Intelligence in Medicine • Mycin- Stanford • Doctoral dissertation of Edward Shortliffe designed to identify bacterial etiology in patients with sepsis and meningitis and to recommend antibiotics • Had simple inference engine and knowledge base of 600 rules • Proposed acceptable therapy in 69% of cases which was better than most ID experts • Never actually used in practice largely due to lack of access and time for physician entry >30 minutes • Caduceus – similar inference engine to Mycin and based on Harry Pope from U of Pittsburgh’s interviews with Dr. Jack Myers with database of up to 1,000 diseases30
  31. Previous Attempts at Artificial Intelligence in Medicine • Internist I and II – Covered 70-80% of possible diagnoses in internal medicine, also based on Jack Myers’ expertise • Worked best on only single disease • Long training and unwieldy interface took 30 to 90 minutes to interact with system • Was succeeded by “Quick Medical Reference” which was discontinued ten years ago and evolved into more of a reference system than diagnostic system • Each differential diagnosis includes linkes to origin evidence to provide meaningful use of EMR’s and supports adoption of evidence based medicine/practice31
  32. Medical Diagnostic Systems • Dxplain used structured knowledge similar to Internetist I, but added hierarchical lexicon of findings • Iliad system developed in ‘90s added probabilistic reasoning • Each disease had associated a priori probability of disease in population for which it was assigned32
  33. Diagnostic Systems using Unstructured Knowledge • ISABEL uses information retrieval software developed by “Autonomy” • First CONSULT allows search of medical books, journals, and guidelines by chief complaints and age group • PEPID DDX is diagnosis generator33
  34. Diagnosis Systems Using Clinical Rules • Acute cardiac ischemia time insensitive predictive instrument uses ECG features and clinical information to predict probability of ischemia and is incorporated into heart monitor/defibrillator • CaseWalker system uses four item questionnaire to diagnose major depressive disorder • PKC advisor provides guidance on 98 patient problems such as abdominal pain and vomiting34
  35. Reasons Current Diagnostic Systems Aren’t Widely Used • They aren’t integrated into day to day operations and workflow of health organizations and patient information is scattered in outpatient clinic visits and hospital visits and their primary provider and specialists • Entry of patient data is difficult – requires too much manual entry of information • They aren’t focused enough on recommendations for next steps for follow up • Unable to interact with practitioner for missing information to increase confidence and more definitive diagnosis • Have difficulty staying up to date35
  36. Watson in the News This Week As Oncology Librarian • March 22, 2012 -- Memorial Sloan-Kettering Cancer Center (MSKCC) and IBM plan to collaborate on the development of a powerful tool built on IBMs Watson artificial intelligence platform that will provide medical professionals with improved access to current and comprehensive cancer data and practices, MSKCC said. • The initiative will combine the computational and language-processing ability of IBM Watson with MSKCCs clinical knowledge, existing molecular and genomic data, and repository of cancer case histories in order to create an outcome and evidence-based decision-support system, according to MSKCC36
  37. Watson in the News as Research Librarian • Development work has begun for the first applications, which include lung, breast, and prostate cancers. The goal is to begin testing the tool with a small group of oncologists in late 2012, with wider distribution planned for late 2013, MSKCC said. • The computer will assist doctors in making diagnoses and treatment decisions by mining current information and alerting doctors to new developments and research,37
  38. • "Sloan-Kettering and IBM are already developing the first applications using Watson related to lung, breast, and prostate cancers, and aim to begin piloting the solutions to some oncologists in late 2012, with wider distribution planned for late 2013.”38
  39. Watson as Diagnostician39
  40. 40
  41. Google Search41
  42. 42
  43. 43
  44. Google Search: Uveitis Cause44
  45. 45
  46. Google Search: Uveitis, Arthritis, Circular Rash, Headache46
  47. 47
  48. 48
  49. 49
  50. User Information Design for Decision Support50
  51. My Involvement in Helping to Train Dr. Watson • Initial research and grant to help educate Watson in medical domain • Could Watson software for Jeopardy! be successfully ported into the medical domain? • Began discussing challenge associated with NEJM Clinico-Pathological Conference • Talked about books and journals and other sources that could augment the general knowledge built into the Jeopardy! playing software51
  52. After Jeopardy! Match: Initial Reactions/Expectations • E-mails and interviews from all over the world: • Most were incredibly impressed with potential for medicine and opportunities for the future • Some however: • SKYNET and end of world as we know it • Pre-medical students speculating that it really doesn’t make sense to attend medical school any more • Physicians writing blogs predicting that they would be replaced by the computer within a short period of time52
  53. Taking Watson to Medical School • Want 3 components similar to medical students education • Book knowledge • Sim Human Model • Experiential learning from actual EMR53
  54. Book Learning • Textbook, journal, and Internet resource knowledge. Quiz materials • Like medical student this alone not enough don’t want to make hypochondriac54
  55. Advancing Deep Q/A’s Medical Knowledge • Continue to develop medical knowledge database • Harrison’s • Merck • Current Medical Diagnosis and Treatment • American College of Physicians Medicine • Stein’s Internal Medicine • medical Knowledge Self Assessment Program • NLM’s Clinical Question Repository55
  56. Advancing Deep Q/A’s Medical Knowledge • Use New England Journal of Medicine 130 CPC cases and quiz material • Additional CPC cases at U of Maryland • Begin developing interactive capability to develop hypotheses and refine them depending on the answer to those questions • Develop a tool that allows for physician feedback to the system for various hypotheses so community can interact and teach Watson56
  57. SIM Human • SIM Human model of physiology – work done at the University of Maryland School of Medicine and UMBC by Dr. Bruce Jarrell and colleagues • Want to have understanding from model of physiology • Work has been done to create simulations of disease processes and then observe how it affects other physiology in the body57
  58. Clinical/Hospital “Experience” • Consumption of electronic medical record which is largely just paper represented digitally, cannot search for “rash” for example • Access to records at U of Maryland and VA but also larger repositories from the VA in de-identified manner58
  59. Electronic Medical Record Challenges and Limitations • Epic system at the University of Maryland • VA’s VISTA System • University of Maryland EPIC system • EMR • Electronic version of paper records • Review large number of discharge summaries • Review progress notes and structured and unstructured additional information from EMR59
  60. IBM and VA Team Review of EMR • Patient EMR such as VA’s highly publicized and praised VISTA revealed numerous challenges60
  61. Despite the fact that virtually 100% of patient information is available in the electronic EMR with records going back more than 15 years • Not possible to search for a term within or among patient records such as “rash” • Majority of data is unstructured and in free text format • Much of the text in progress notes and other types of notes is highly redundant since interns and residents and attending physicians typically cut and paste information from lab and radiology and other studies and other notes • Information is entered with abbreviations that are not consistent and misspellings61
  62. Patient Problem List • Patient problem list has no “sheriff” and each physician is free to add “problems” but very few delete them for “problems” that are temporary • The problem lists themselves often have contradictory information62
  63. Medical Domain Adaptation • 5000 questions from American College of Physicians Doctor’s Dilemma competition • E.g. • The syndrome characterized by joint pain, abdominal pain, palpable purpura, and a nephritic sediment • Henoch-Schonlein Purpura • Familial adenomatous polyposis is caused by mutations of this gene: APC gene • Syndrome characterized by narrowing of the extrahepatic bile duct from mechanical compression by a gallstone impacted in the cystic duct: Mirizzi’s Syndrome63
  64. 3 Areas of Adaptation for Deep QA • Content • Organizing domain content for hypothesis and evidence generation such as textbooks, dictionaries, clinical guidelines, research articles • Tradeoff between reliability and recency • Training • Adding data in the form of sample training questions and correct answers from the target domain so system can learn appropriate weights for its components when estimating answer confidence • Functional • Adding new question analysis, candidate generation, and hypothesis evidencing analytics specialized for the domain64
  65. Content Adaptation • Text content is converted into XML format used as input for indexing • Text analyzed for medical concepts and semantic types using Unified Medical Language System terminology to provide for structured query based lookup • “Corpus expansion technique” used by DeepQA searches web for similar passages given description of symptoms for example and generates pseudo documents from web search results65
  66. Medical Content Sources for Watson Include: • ACP (American College of Physicians) Medicine • Merck Manual of Diagnosis and Therapy • PIER (collection of guidelines and evidence summaries) • MKSAP (Medical Knowledge Self Assessment Program study guide from ACP) • Journals and Textbooks66
  67. Discovering the Untapped, Disconnected Gold Mines of Clinical and Research Data • Despite all of the advances in computer technology we are arguably still at the paper stage of research as far as ability to discover and combine important data • Research data including those associated with major medical journals and clinical trials are typically created for a single purpose and beyond a one or two manuscripts, remain largely locked up or inaccessible • Even when the data are made accessible, they are typically associated with limited access through a proprietary Internet portal or even by requesting data on a hard drive • Often requires submission of a research plan and data and then a considerable wait for permission to use the data which is often not granted67
  68. ADNI • Alzheimer’s Disease Neuroimaging Initiative • Excellent example of patient data and associated images with great sharing model • However requires access through their own portal and requires permission from ADNI Data Sharing and Publications Committee68
  69. CTEP (NIH Cancer Therapy Evaluation Program) Pediatric Brain Tumor Consortium One of the Better Sources of Data • As an NCI funded Consortium, the Pediatric Brain Tumor Consortium (PBTC) is required to make research data available to other investigators for use in research projects • An investigator who wishes to use individual patient data from one or more of the Consortiums completed and published studies must submit in writing: • Description of the research project • Specific data requested • List of investigators involved with the project • Affiliated research institutions • Copy of the requesting investigators CV must also be provided. • The submitted research proposal and CV shall be distributed to the PBTC Steering Committee for review • Once approved, the responsible investigator will be required to complete a Material and Data Transfer Agreement as part of the conditions for data release • Requests for data will only be considered once the primary study analyses have been published69
  70. Institutional Database General Practice Research Database70
  71. Institutional Database: VA’s Corporate Data Warehouse Vinci71
  72. Disease Specific Databases Alzheimer’s, Parkinson’s, Schizophrenia72
  73. Cornucopia of Sources of Data for Dr. Watson • University hospital databases • Large medical system e.g. Kaiser Permanente data warehouse • Insurance databases such as WellPoint • State level databases73
  74. Discovering and Consuming Databases • At best, freely sharable databases are accessed using their own idiosyncratic web portal • Currently no index of databases or their content • No standards exist to describe how databases can “advertise” their content and availability (free or business model) and their data provenance and sources and peer review, etc. • Would be wonderful project for AMIA or NLM to investigate the creation of an XML standard for describing the content of databases • This will be critical to the continuing success of the Dr. Watson project in my opinion74
  75. Medical Guidelines • Medical guidelines are increasingly being put into machine intelligible form although this is not an easy process • Incorporating these into Watson software could serve multiple purposes including health surveillance, could factor into diagnostic decision making, and could be an early implementation of the Watson technology75
  76. Peleska et al: General Graphic Model Making Guidelines “Formal” and Machine Readable 2003 European Guidelines on Cardiovascular Disease Prevention and 2003 ESH/ESC Hypertension Guidelines76
  77. The Electronic Medical Record • The transition to the 3rd year of medical school begins a new phase in education from theoretical to empirical • Medical students are exposed for the first time to the wards and of course, importantly, to one of their major jobs for the next few years: • Maintenance and review of patient charts, nowadays the Electronic Medical Record77
  78. Introducing Dr. Watson to the Electronic Medical Record78
  79. Watson and the EMR • Despite the tremendous strides we have made toward an electronic medical record, we are really just at the 1.0 stage and arguably most current EMR systems really represent just a digital form of paper • The Watson development team was really surprised when we reviewed the EMR at how primitive it was, even in 2011 • Lack of ability to search for terms within a patient’s record • Lack of ability to search across patient records • Lack of ability to perform basic statistics or have access to basic decision support tools in EMR79
  80. EMR • The diagnosis of a specific type of pneumonia, for example, can be made according to patient signs and symptoms using journal articles and textbooks • But it can also be made more reliably by a system such as Watson by also mining the local EMR database as to what diagnoses have been made over the past few days, weeks, months, etc. locally80
  81. EMR • It can then be further refined by not necessarily being constrained to tentative diagnoses that have been made but the microbiology/pathology proven causes of pneumonia • The EMR provides empirical data about the association of these signs and symptoms with diagnoses and the means to verify what was found by lab tests etc.81
  82. EMR Challenges • Challenges mining EMR • Unstructured free text with abbreviations, variable terms (e.g. MRI terminology) • Difficulty in having Watson technology analyze large databases such as VA’s EMR due to PHI concerns and need to stay within the firewall • Watson needs to incorporate the concept of changing signs and symptoms in a patient over time which creates added dimension to diagnosis of a single patient presentation • Challenge is the fragmentation of electronic medical records by multiple hospitals, clinics, outpatient settings, etc.82
  83. EMR Opportunities • Watson can gain empirical knowledge of vast numbers of physicians and patients in a way that would not be possible for any single practitioner • Watson could use EMR to perform research and discovery in healthcare such as unanticipated drug responses and interactions and factors impacting patient response to therapy • Watson can be impetus to medical community for the development of more structured EMR in a more friendly machine readable format83
  84. Personal Health Records May Help Ameliorate Fragmentation of EMR’s Hospital and Clinics and Offices • PHR’s will enable Watson to get all information in one place when patients centralize and take control of their own electronic health records • Patients will be able to control level of access to their information84
  85. ADDITIONAL APPLICATIONS FOR DR. WATSON85
  86. Additional Applications for Dr. Watson • Surveillance – e.g. Los Alamos Labs • Bioterrorism • Drug • Infectious Disease86
  87. Chart Review and Patient Problem List Sheriff • Review for patient safety issues • Computerized patient problem list • IBM team and I found patient problem list typically poorly maintained and updated • Problems not deleted when they are no longer important • Contradictions in patient problem list • Patient on medications not corresponding to problems on the list87
  88. Personalized Medicine • Dr. Watson software can utilize genomic and proteomic information in addition to patient signs and symptoms to provide personalized diagnostic and treatment information • Will be able to utilize an increasing number of genomic and proteomic databases such as The Cancer Genome Atlas and The Million Veteran Program88
  89. Synthesis/Display of Complex Information in EMR89
  90. Utilizing the NCI caBIG Semantics and Technologies To Support Phenotype/Genotype Clinical Analysis for Personalized Medicine in the Diagnosis of Glioblastoma Multiforme90
  91. Current Dr. Watson Opportunities for Improvement • Need to understand to listen and human speech including accents • Needs to have improved ability to understand abbreviations and medical jargon • Needs mechanism to obtain feedback (learn) from physicians using it • Continue to refine and improve user interface to allow feedback and refinement of algorithms91
  92. Interactive • Emergency Department Scenario • Requires “real-time” decision making • Cannot use same model with all information entered into the chart before Watson makes its assessment and recommendations • Need better systems to capture information at point of care • Vital signs and lab and signal monitoring • Do we need additional methods of inputting data? • Do we need to capture live conversations with providers and patients?92
  93. Current Opportunities for Improvement • Could use more personality • Female voice chosen for Siri after much research and feedback • Needs to understand nuances of communication such as patients questions expressing emotions such as fear etc.93
  94. Siri: Artificial Intelligence Devices Say the Darnest Things94
  95. Watson Opportunity: As Unifier for Interoperability and Test Bed • Potential for Watson to be bridge to allow connectivity and interoperability since so many islands currently being set up with health information exchanges at city and state and other levels • Watson or Watson like technology may provide test bed for standards in medicine and may improve interoperability95
  96. Teaching Dr. Watson Bedside Manners • According to a study done by the Mayo Clinic in 2006, the most important characteristics patients feel a good doctor must possess are entirely human • According to the study, the ideal physician is confident, empathetic, humane, personal, forthright, respectful, and thorough • Watson may have proved his cognitive superiority, but can a computer ever be taught these human attributes needed to negotiate through patient fear, anxiety, and confusion? Could such a computer ever come across as sincere?96
  97. Turing Test • Introduced by Alan Turing in his 1950 paper “Computing Machinery and Intelligence” • Opens with the words “I propose to consider the question, ‘Can machines think?” • Asks whether a computer could fool a human being in another room into thinking it was a human being • Modified Dr. Watson Turing Test might ask: Can a computer fool a human being into thinking it was a doctor?97
  98. Ultimate Challenge: Medical Imaging Scientific American June 2011 Testing for Consciousness Alternative to Turning Test Christof Koch and Giulio Tononi98
  99. Imaging May Be Ultimate/Future Frontier For Dr. Watson99
  100. Does Watson Obviate Need for Standards and Structure? • No, in order to achieve their full potential we will need to make our medical records more structured and standardized, and rethink how we can make our clinical trial and other research databases more readily discoverable and reusable • These changes will also accelerate interoperability and information exchange which will improve healthcare100
  101. Conclusions • I am absolutely convinced that natural language processing and Artificial Intelligence applications such as IBM’s Dr. Watson will have a major impact on the practice of medicine in the very near future • It will result in more cost effective, higher quality care and will help to decrease the disparities of care that we currently see geographically, socioeconomically, and according to subspecialty • It will also allow us to finally achieve true personalized medicine, taking clinical signs and symptoms and history and laboratory information and diagnostic imaging and genomics and proteomics into account to personalize treatment recommendations101
  102. Conclusion • Dr. Watson will evolve as an amiable, knowledgeable, fast, and reliable assistant • If there are any pre-med students out there in the audience, please do plan to attend medical school and rest assured that Dr. Watson will require your wisdom, common sense, and humanity in order to be a continuing and evolving success102
  103. • The Watson Q/A technology and Jeopardy demonstration have captured the imagination of many people including those in healthcare and this may provide a critical springboard to revive many of the excellent initiatives on artificial intelligence applications in medicine • The potential of these to revolutionize medicine is tremendous and exciting103
  104. DR. WATSON – A PROMISING STUDENT IN PURSUIT OF SMARTER MEDICINE Eliot Siegel, M.D. Professor and Vice Chair University of Maryland Department of Diagnostic Radiology104

×