The document discusses how IBM's Watson technology can be applied to healthcare to improve clinical decision making and reduce diagnostic errors. It describes Watson's ability to analyze large amounts of structured and unstructured data, generate differential diagnoses, consider various hypotheses, and provide evidence and a confidence level for its responses without making a definitive diagnosis. The document also outlines how electronic health records could be enhanced with Watson to better record assessments, generate checklists to aid decision making, and provide relevant knowledge resources to clinicians.

1. Clinical Decision Support: DeepQA Martin S. Kohn, MD, MS, FACEP, FACPE Chief Medical Scientist, Care Delivery Systems IBM Research

3. Per Capita Health Spending and 15-Year Survival for 45-Year-Old Women Copyright ©2010 by Project HOPE, all rights reserved. Peter A. Muennig and Sherry A. Glied, What Changes In Survival Rates Tell Us About US Health Care, Health Affairs, Vol 0, Issue 2010, hlthaff.2010.0073v1-101377201

8. A value-based health system must appropriately balance resources expended in keeping people healthy Value-based Health Care System 20% of people generate 80% of costs Health care spending Early Symptoms Health Status Source: IBM Global Business Services and IBM Institute for Business Value Healthy / Low Risk High Risk At Risk Active Disease Early Clinical Symptoms

10. “ Smarter Healthcare” + + Improve operational effectiveness Achieve better quality and outcomes Collaborate for prevention and wellness A smarter health system collaborates across health and care settings, activating individuals in their own health and making best use of resources to treat conditions, keep people healthy and deliver greater individual value. Intelligent Instrumented Interconnected

11. Health systems will need to emphasize different competencies to thrive in a changing healthcare environment Empower Patients Empower members to assume accountability and make more informed health and financial choices Collaborate with Providers Help providers become successful in a value-base reimbursement environment Innovate Collaboratively innovate products and services, operational processes, and business models Optimize Operational Efficiencies Continue driving costs down in order to maximize margins in a highly regulated industry Enable through Information Technology Flexible applications, BI, on-demand information, effective operations/management & governance Competencies

12. Informed Decision Making: Search vs. Expert Q&A Decision Maker Search Engine Finds Documents containing Keywords Delivers Documents based on Popularity Has Question Distills to 2-3 Keywords Reads Documents, Finds Answers Finds & Analyzes Evidence Expert Understands Question Produces Possible Answers & Evidence Delivers Response, Evidence & Confidence Analyzes Evidence, Computes Confidence Asks NL Question Considers Answer & Evidence Decision Maker

14. The Jeopardy! Challenge: A compelling and notable way to drive and measure the technology of automatic Question Answering along 5 Key Dimensions $600 In cell division, mitosis splits the nucleus & cytokinesis splits this liquid cushioning the nucleus $200 If you're standing, it's the direction you should look to check out the wainscoting . $2000 Of the 4 countries in the world that the U.S. does not have diplomatic relations with, the one that’s farthest north $1000 The first person mentioned by name in ‘The Man in the Iron Mask’ is this hero of a previous book by the same author. Broad/Open Domain Complex Language High Precision Accurate Confidence High Speed

15. Keyword Evidence Keyword Matching Keyword Matching Keyword Matching Keyword Matching Keyword Matching celebrated India In May 1898 400th anniversary arrival in Portugal India In May Gary explorer celebrated anniversary in Portugal arrived in In May , Gary arrived in India after he celebrated his anniversary in Portugal . In May 1898 Portugal celebrated the 400th anniversary of this explorer’s arrival in India. This evidence suggests “Gary” is the answer BUT the system must learn that keyword matching may be weak relative to other types of evidence

17. The Missing Link On hearing of the discovery of George Mallory's body, he told reporters he still thinks he was first. TV remote controls, Buttons Shirts, Telephones Mt Everest He was first Edmund Hillary

18. DeepQA : The Technology Behind Watson Massively Parallel Probabilistic Evidence-Based Architecture Generates and scores many hypotheses using a combination of 1000’s Natural Language Processing , Information Retrieval , Machine Learning and Reasoning Algorithms. These gather, evaluate, weigh and balance different types of evidence to deliver the answer with the best support it can find. . . . 1000’s of Pieces of Evidence Multiple Interpretations 100,000’s Scores from many Deep Analysis Algorithms 100’s sources 100’s Possible Answers Balance & Combine Answer Scoring Models Answer & Confidence Question Evidence Sources Models Models Models Models Models Primary Search Candidate Answer Generation Hypothesis Generation Hypothesis and Evidence Scoring Final Confidence Merging & Ranking Synthesis Answer Sources Question & Topic Analysis Evidence Retrieval Deep Evidence Scoring Learned Models help combine and weigh the Evidence Hypothesis Generation Hypothesis and Evidence Scoring Question Decomposition

19. Baseline 12/06 v0.1 12/07 v0.3 08/08 v0.5 05/09 v0.6 10/09 v0.8 11/10 v0.4 12/08 DeepQA: Incremental Progress in Answering Precision on the Jeopardy Challenge: 6/2007-11/2010 v0.2 05/08 IBM Watson Playing in the Winners Cloud V0.7 04/10

23. Why is Watson Technology ideal for Healthcare? Source: IBM Research, MI, SCIP, BCG analysis Interprets and understands natural language questions Understands ambiguous and imprecise questions using sophisticated natural language algorithms Analyzes large volumes of unstructured data Synthesizes broad domain of unstructured data from a variety of selected public, licensed and private sources Quantifies degrees of confidence in potential answers Generates hypotheses and ranks degrees of confidence in a range of potential answers based on evidence Supports iterative dialogue to refine results Internal iterative and interactive question and answering to refine and improve results Adapts and learns to improve results over time Learns from additional evidence, additional questions and mistakes to improve accuracy over time What condition has red eye, pain, inflammation, blurred vision, floating spots and sensitivity to light? Physician Notes, Medical Journals, Pathology results, Clinical Trials, Wikipedia, etc/ Family History, Physical Exam, Current Medications, etc. New Clinical Recommendations. New Drugs. Approved use of Drugs, etc. Uveitis 91% Iritis 48% Keratitis 29%

24. A Range of Watson-enabled Healthcare Solutions Patient Caregiver…Nurse…Physician Assistant Clinician Specialty Diagnosis & Treatment Options Longitudinal Patient Electronic Health Information Specialty Research Genomic-based Analysis Coding Analysis & Automation Caregiver Education Consumer Portal Patient Workup Differential Diagnosis Treatment Options Patient Inquiry On-going Treatment Treatment Protocol Analysis Treatment Authorization Population Analysis & Care Mgmt Second Opinion Care Consideration Automation

25. Key Elements of the Clinical Diagnostic Reasoning Process Bowen J. N Engl J Med 2006;355:2217-2225

26. Graber, et al. Diagnostic Error in Internal Medicine, Arch Int Med 2005; 165:1493-1499

32. [email_address]

33. Convergence of Market and Technical Forces Create an Opportunity to Leverage Watson for Healthcare 8/2/2011 Advances in Analytics Advances in Mobile Computing Watson / DeepQA platform Partner Ecosystem Next Generation Solutions for Healthcare Pharma Medical Research Centers Information Providers Private, Public Insurers Medical Devices Providers Public Health Demand for Healthcare Reform Spiraling Costs Medical Information Overload Quality and Safety Concerns Aging Population Watson for Healthcare

34. Watson: How It Works Runtime Pipeline Input CASes Answers & Confidence Question and Answer Key Output CASes A variety of NLP algorithms analyze the question and the context to attempt to figure out what is being asked. (named entities, relations, LAT detection, question class) Retrieve content related to the question using index search on documents, passages and structured repositories From retrieved content, extract the words or phrases that could be possible answers Consider all the scored evidence to produce a final ranked list of answers with confidence For each candidate answer, retrieve more content that relates that answer to the question Many algorithms attempt to determine the degree to which the retrieved evidence supports the candidate answers. Training/Testing questions with medically vetted answers WaaS API Initial Scoring of candidate answers independent of supporting passages Build an abstract query from question analysis Removes candidates that should not proceed to remaining phases Question Analysis Query Builder Context-Independent Answer Scoring Candidate Answer Filtering Supporting Evidence Retrieval Context-Dependent Answer Scoring Final Merger Primary Search Search Result Processing Candidate Answer Generation Context Analysis