Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Bill Evans, Rock Heatlh on Machine Learning @ DIABETESMINE UNIVERSITY

194 views

Published on

Bill Evans of Rock Health presents on market & regulatory aspects of Machine Learning in Healthcare at DIABETESMINE UNIVERSITY – our new innovation program that encompasses our annual Innovation Summit and D-Data ExChange forums, held Nov.1-2 at UCSF Mission Bay.

Published in: Healthcare
  • Be the first to comment

  • Be the first to like this

Bill Evans, Rock Heatlh on Machine Learning @ DIABETESMINE UNIVERSITY

  1. 1. PRESENTATION © 2018 ROCK HEALTH AI & MACHINE LEARNING IN DIGITAL HEALTH Nov 2018
  2. 2. AN OVERVIEW OF ROCK HEALTH
  3. 3. Rock Health’s mission is to make healthcare massively better for every human PRESENTATION © 2018 ROCK HEALTH $1.4B+ raised by portfolio 50+ portfolio companies 18% of portfolio companies have exited FOUR KEY PILLARS OF ROCK HEALTH We publish data-driven research on innovation Our partners are our allies who support us We are the first venture fund dedicated to digital health We host three signature events annually THOUGHT LEADERSHIP PARTNERSHIPS INVESTINGCOMMUNITY
  4. 4. ROCK HEALTH RESEARCH: AI/ML IN HEALTH
  5. 5. PRESENTATION © 2018 ROCK HEALTH With machine and deep learning at the “peak of inflated expectations,” backlash to AI hype anticipates a potential move to the “trough of disillusionment.” WE’RE AT PEAK HYPE 5%
 Health systems self-reporting they already leverage AI1 Source: “Top Trends in the Gartner Hype Cycle for Emerging Technologies, 2017,” August 15, 2017; http://www.gartner.com/smarterwithgartner/ top-trends-in-the-gartner-hype-cycle-for-emerging-technologies-2017/, Sullivan, Tom, “Half of hospitals to adopt artificial intelligence within 5 years,” April 11, 2017, http://www.healthcareitnews.com/news/half-hospitals-adopt-artificial-intelligence-within-5-years; “Gartner Says AI Technologies Will Be in Almost Every New Software Product by 2020,” July 18, 2017, http://www.gartner.com/newsroom/id/3763265; Rock Health analysis. ADOPTION STILL LAGGING 1: Survey by Healthcare IT News and HIMSS Analytics survey includes responses from CEOs, CIOs, CFOs, COOS, chief medical information officers, chief clinical officers, and professionals working at the IT_director level and above. I think there is more hype and buzz than reality. I’ve seen these bubbles burst. I am concerned. “ “ “ Digital health entrepreneur, PhD The huge increase in startups […] all claiming to offer AI products without any real differentiation is confusing buyers. Gartner Innovation
 Trigger Peak of Inflated Expectations Trough of Disillusion Slope of Enlighten- ment Plateau of Productivity Deep learning Machine learning “
  6. 6. PRESENTATION © 2018 ROCK HEALTH With key developments advancing machine learning, there is reason to believe we won’t have another “AI Winter.” BETTER ALGORITHMS MORE INFRASTRUCTURE FASTER PROCESSING MORE HEALTH-RELATED DATA 1 2 3 FOUR MACHINE LEARNING ACCELERANTS 4
  7. 7. PRESENTATION © 2018 ROCK HEALTH The exponential growth in the availability of healthcare data will fuel returns to investment in AI/ML, even as algorithmic innovation slows. Source: Kelnar, David, “The fourth industrial revolution: a primer on Artificial Intelligence (AI),” December 2, 2016, https:// medium.com/mmc-writes/the-fourth-industrial-revolution-a-primer-on-artificial-intelligence-ai-ff5e7fffcae1?welcomeRedirect=true; “DNA Sequencing Costs: Data,” National Human Genome Research Institute, https://www.genome.gov/sequencingcostsdata/; “Office-based Physician Electronic Health Record Adoption,” The Office of the National Coordinator for Health Information Technology, https://dashboard.healthit.gov/quickstats/pages/physician-ehr-adoption-trends.php; Rock Health analysis. COST OF SEQUENCING A HUMAN GENOME (LOGARITHMIC SCALE) 2003–2015 2003 2005 2007 2009 2011 2013 2015 $40.2M $13.8M $7.1M $70.3K $7,743 $5,096 $1,245 $40.2M
  8. 8. PRESENTATION © 2018 ROCK HEALTH The increase in resolution & (slow but) steady improvements in linkages among data sets represents a relatively unexplored and growing “natural resource.” OFFICE-BASED PHYSICIAN EHR ADOPTION 2005–2015 0% 25% 50% 75% 100% 2005 2007 2009 2011 2013 2015 24% 35% 48% 57% 78% 87% Source: Kelnar, David, “The fourth industrial revolution: a primer on Artificial Intelligence (AI),” December 2, 2016, https:// medium.com/mmc-writes/the-fourth-industrial-revolution-a-primer-on-artificial-intelligence-ai-ff5e7fffcae1?welcomeRedirect=true; “DNA Sequencing Costs: Data,” National Human Genome Research Institute, https://www.genome.gov/sequencingcostsdata/; “Office-based Physician Electronic Health Record Adoption,” The Office of the National Coordinator for Health Information Technology, https://dashboard.healthit.gov/quickstats/pages/physician-ehr-adoption-trends.php; Rock Health analysis.
  9. 9. PRESENTATION © 2018 ROCK HEALTH $7.2M $10.8M $9.9M $15.6M $13.5M $14.6M $11.2M DEAL COUNT $780M $585M $390M TOTAL AI/ML FUNDING 15 30 45 20172013 2014 2015 201620122011 $195M 60 58 52 3434 16 8 4 $33.1M $86.4M $146.7M $525.3M $481.4M $776.4M $698.4M AVERAGE DEAL SIZE $8.3M $10.8M $9.2M $15.4M $14.2M $14.9M $12.0M TOTAL FUNDING FOR AI/ML-POWERED DIGITAL HEALTH COMPANIES 2011–2017 121 AI/ML companies in digital health have raised a total of $2.7B through 206 deals from 2011 through 2017. Source: Rock Health Digital Health Funding Database, interviews and analysis. Note: Only includes U.S. deals >$2M
  10. 10. PRESENTATION © 2018 ROCK HEALTH VALUE PROPOSITIONS IN AI/ML 2011–2017 AI/ML can be applied to nearly every use case in healthcare. But the least sophisticated use cases (collectively) attract the most investment. Source: Rock Health Digital Health Funding Database. RESEARCH & DEVELOPMENT CATALYST 13 7 12 2 7 13 16 6 12 19 5 8 16 22 1 2 4 8 4 POPULATION HEALTH MANAGEMENT CLINICAL WORKFLOW HEALTH BENEFITS ADMINISTRATION DIAGNOSIS OF DISEASE TREATMENT OF DISEASE NON-CLINICAL WORKFLOW DATA INFRASTRUCTURE & INTEROPERABILITY FITNESS & WELLNESS PREVENTION OF DISEASE CUSTOMER ACQUISITION & RELATIONSHIP MANAGEMENT PATIENT ADHERENCE ON-DEMAND HEALTHCARE SERVICES MARKETPLACE LEGEND Number of companies Cumulative funding MONITORING OF DISEASE CLINICAL DECISION SUPPORT & PRECISION MEDICINE $650.5M $523.8M $514.8M $469.5M $330.4M $281.2M $269.6M $214.9M $123.9M $103.1M $102.2M $85.8M MEDICAL REFERENCE CARE COORDINATION $77.2M $77.2M $73M $58.6M $33.2M $2.6M CONSUMER HEALTH INFORMATION $255.3M $219.9M $200.7M $155.2M $123.9M $109.3M $110.5M
  11. 11. PRESENTATION © 2018 ROCK HEALTH The top six value propositions accounted for 85% of all AI funding and range from research and development to diagnosis of disease. Source: Rock Health Digital Health Funding Database. TOP VALUE PROPOSITIONS SINCE 2011 2011–2017 MONITORING OF DISEASE POPULATION HEALTH MANAGEMENT DIAGNOSIS OF DISEASE Welltok ($215M) $ 524M $ 515M Omada Health ($126M) $ 281M RESEARCH & DEVELOPMENT CATALYST $ 650M Flatiron Health ($313M) Butterfly Network ($80M) CLINICAL WORKFLOW Number of companies: 22 Number of companies: 16 Number of companies: 8 Number of companies: 19 Number of companies: 12 $ 330M HEALTH BENEFITS ADMINISTRATION $ 470M Number of companies: 5 Welltok ($215M) Flatiron Health ($313M) LEGEND All other companiesMost funded company
  12. 12. PRESENTATION © 2018 ROCK HEALTH “AI/ML” STARTUPS AND PRACTITIONERS ENTERPRISE LEADERS INVESTORS REGULATORS Investment momentum in AI/ML continues to build, but significant threats stand in the way of AI/ML realizing its full potential. SEVEN THREATS TO AI 1 2 3 4 5 6 7 Over-focusing on “shiny objects” vs. the UX and business value. Smart algorithms are being trained on dumb and dirty data.
 Practitioners are building “black boxes” even they can’t understand. Though they’re the key customers, most enterprise organizations don’t know where to begin.
 Major incumbents possess—but fail to capitalize on—the most valuable commodity: Data.
 Hype allows some companies to masquerade as “AI” companies.
 Regulation of AI/ML still needs to come into focus.
  13. 13. ROCK HEALTH RESEARCH: AI/ML WORKING GROUP
  14. 14. PRESENTATION © 2018 ROCK HEALTH With key developments advancing machine learning, there is reason to believe we won’t have another “AI Winter.” WHAT IS GROUND TRUTH & HOW TO ESTABLISH IT? WHAT LEVEL OF TRANSPARENCY IS NEEDED? HOW TO REGULATE “CONTINUOUS” REFITTING? ATTRIBUTES OF A PRECERT WORTHY AI COMPANY? 1 2 3 FOUR QUESTIONS FOR PRACTITIONERS 4
  15. 15. PRESENTATION © 2018 ROCK HEALTH When seeking ground truth data (see definition in the graphic below), how should entrepreneurs navigate the following challenges? 1 WHAT IS GROUND TRUTH & HOW TO ESTABLISH IT? • Consensus ground truth data often not readily available • Uncertainty about what may be acceptable to regulators • Issues re: retrospective vs prospective data/studies • With some tech, prior benchmarks may not be possible
  16. 16. PRESENTATION © 2018 ROCK HEALTH How should industry regulators approach data—the “active ingredient” in AI/ML? Should transparency and/or reproducibility on similar data be considered? 2 WHAT LEVEL OF TRANSPARENCY IS NEEDED? • Intended use vs training data quality • Best practice in managing training vs test/validation data • Publicly available data (test? training?) vs proprietary • Managing statistical biases (& codification of human bias)
  17. 17. PRESENTATION © 2018 ROCK HEALTH What is the appropriate regulatory approach for refitting models to enable them to learn continuously (as opposed to remaining static)? 3 HOW TO REGULATE “CONTINUOUS” REFITTING? 10111101001 • Initial model development vs refitting (same approval?) • How to spot (and address) overfitting • Overfitting the hold-out data (over multiple generations)
  18. 18. PRESENTATION © 2018 ROCK HEALTH How can FDA’s PreCert pilot take AI/ML into account? How should we define “a quality of culture” so that it is relevant to companies that heavily employ AI/ML? 4 ATTRIBUTES OF A PRECERT WORTHY AI COMPANY? • Particular attributes of an AI/ML “culture of quality” • Considerations for QMS for data as well as code • Anomaly detection in post-market
  19. 19. PRESENTATION © 2018 ROCK HEALTH Special thanks to our AI/ML FDA regulatory working group for sharing their experiences, insights, and feedback! John Axerio-Cilies, PhD, Arterys
 Joshua Bloom, PhD, GE Digital
 Ian Blumenfeld, PhD, Clover Health
 Alison Darcy, PhD, Woebot
 Erik Douglas, PhD, CellScope
 Bill Evans, Rock Health
 Luca Foschini, PhD, Evidation
 Leo Grady, PhD, HeartFlow
 Liam Kaufman, WinterLight Labs
 Christine Lemke, Evidation
 Janine Morris, Lilly
 Katie Planey, PhD, Mantra Bio
 Ryan Quan, Omada Health
 Sarah Smith, Bodyport
 Megan Zweig, Rock Health
  20. 20. QUESTIONS?

×