SlideShare a Scribd company logo
Evidence	
  Farming1:	
  Implications	
  for
       Open	
  Architecture
                                Ida	
  Sim,	
  MD,	
  PhD
  Director,	
  Center	
  for	
  Clinical	
  and	
  Translational	
  Informatics
               University	
  of	
  California	
  San	
  Francisco
                                   May	
  5,	
  2011

 1With	
  thanks	
  to	
  Rich	
  Kravitz	
  MD,	
  UC	
  Davis	
  and	
  Naihua	
  Duan,	
  Columbia
Rephrasing	
  “Does	
  it	
  Work?”

(Complexes of)
                                             Outcome
  Exposures      strength of association?   Increased
  Text4Baby            individual         breastfeeding


                                        population
Current	
  Approaches:	
  RCT
                              Asthma App                    ER visits at 1 year
                               50 people
    100 people
                               Usual Care                   ER visits at 1 year
                               50 people                                population


•   Tests	
  prespecified	
  interventions	
  and	
  outcomes
•   To	
  confirm	
  a	
  hypothesis	
  at	
  the	
  population	
  level
•   Strong	
  internal	
  validity
•   Problems:	
  slow	
  to	
  set-­‐up,	
  expensive,	
  short-­‐term,	
  lack
    relevance	
  to	
  the	
  real	
  world
Current	
  Approaches:	
  Data	
  Mining

  EHR
                             Exposures               Outcomes
                                               ?
  Apps
                                          population


• Exposures	
  and	
  outcomes	
  from	
  care	
  process	
  systems
• To	
  generate	
  hypotheses	
  at	
  the	
  population	
  level
• Problems:	
  limited	
  to	
  data	
  collected,	
  weak	
  internal
  validity	
  (data	
  not	
  complete	
  or	
  systematic)
Current	
  Approaches:
               N-­‐of-­‐1	
  Studies
            Asthma app                Usual Care           Asthma app
                          peak flow                peak flow
             Usual Care               Asthma app               Usual Care
                                                                  individual

• Within-­‐subject	
  multiple	
  crossover
• Only	
  formal	
  method	
  for	
  determining	
  individual
  treatment	
  effectiveness
• Problems:	
  complicated	
  to	
  set	
  up,	
  analysis	
  is
  difficult,	
  little	
  known,	
  not	
  widely	
  used
Evidence	
  Extraction

• Evidence	
  is	
  something	
  to	
  be	
  extracted
  from	
  the	
  care	
  process
   – mining	
  it	
  from	
  the	
  data
   – directly	
  manipulating	
  the	
  care	
  process	
  with
     rigid	
  and	
  pre-­‐defined	
  protocols
Evidence	
  Strip	
  Mining
Evidence	
  Farming




          Hay, et al. J Eval Clin Prac 14(2008):707-713.
Rooting	
  for	
  Evidence
Industrial	
  Evidence	
  Farming
             Asthma App   ER visits at 1 year
             50 people
100 people
             Usual Care   ER visits at 1 year
             50 people            population
Personal	
  Evidence	
  Gardens
   Asthma app               Usual Care           Asthma app
                peak flow                peak flow
   Usual Care               Asthma app               Usual Care
                                                        individual
Personal	
  Evidence	
  Gardens
     Flovent               Flovent PRN               Flovent
                 dancing                 dancing
   Flovent PRN               Flovent               Flovent PRN
                                                      individual
Crowdsourcing	
  What	
  Matters
• (Complexes	
  of)	
  Exposures
   – does	
  chocolate	
  trigger	
  (my)	
  asthma?
   – testing	
  common	
  regimens	
  (ACEI,	
  statin,	
  b-­‐blocker),
     complementary	
  medicines
• (Complexes	
  of)	
  Outcomes
   – what	
  outcomes	
  do	
  patients	
  care	
  about?
Evidence	
  Macrosystem
Rooting for   Industrial Evidence   Personal Evidence
 Evidence          Farming              Gardens
How	
  can	
  we	
  scale	
  evaluation?
Stovepiped
     mHealth
• Health	
  apps	
  built
  independently
    – little	
  data	
  sharing	
  and
      interoperability
• Limits	
  efficiency	
  and
  impact	
  of	
  quality
  mHealth
Internet	
  Hourglass	
  Model
• Standardize	
  and
  make	
  open	
  the
  “narrow	
  waist”
• Reduces	
  duplication,
  spurs	
  community
  innovation,	
  supports
  commercial	
  and	
  non-­‐
  profit	
  uses
OpenmHealth.org




        Estrin DE, Sim I. Science; 330: 759-60. 2010.
OpenmHealth.org



• The	
  waist	
  should	
  support
  the	
  evidence	
  macrosystem
Open	
  Architecture	
  for	
  an
           Evidence	
  Macrosystem
• Modules	
  for	
  usage	
  analytics
    – #	
  of	
  text	
  messages,	
  #	
  of	
  sessions,	
  etc.
• Rooting	
  for	
  (glocal)	
  evidence
    – data	
  sharing	
  with	
  shared	
  syntax	
  and	
  semantics
• Industrial	
  farming,	
  e.g.,	
  with	
  RCTs
    – modules	
  for	
  informed	
  consent,	
  randomization,	
  adaptive
      treatment	
  strategy,	
  mixed	
  methods,	
  etc.
• Personal	
  evidence	
  gardening,	
  e.g.,	
  N-­‐of-­‐1
    – modules	
  for	
  scripting	
  and	
  analyzing	
  individualized	
  N-­‐of-­‐
      1	
  protocols,	
  etc.
Open	
  Architecture	
  for	
  an
        Evidence	
  Macrosystem
• Social	
  media	
  for	
  discovery	
  of	
  exposures	
  and
  outcomes	
  that	
  matter
• Shared	
  libraries	
  of	
  validated	
  measures	
  and
  instruments	
  (e.g.,	
  PROMIS)
    – measures	
  that	
  get	
  at	
  finer-­‐grained	
  mechanisms	
  based
      on	
  theoretical	
  models	
  of	
  change,	
  etc.
Goal	
  for	
  mHealth	
  Evidence
• A	
  learning	
  community	
  coupled	
  with	
  an
  open	
  architecture	
  for	
  broad,	
  rapid,	
  and
  iterative	
  dissemination	
  of	
  evaluation
  methods	
  and	
  findings	
  that	
  matter
• Ida	
  Sim	
  ida.sim@ucsf.edu
• Deborah	
  Estrin	
  destrin@cs.ucla.edu
• http://openmhealth.org/

More Related Content

What's hot

Atul Butte's presentation at the From Data to Discovery symposium at Westat
Atul Butte's presentation at the From Data to Discovery symposium at WestatAtul Butte's presentation at the From Data to Discovery symposium at Westat
Atul Butte's presentation at the From Data to Discovery symposium at Westat
University of California, San Francisco
 
297 vickers
297 vickers297 vickers
297 vickers
297 vickers297 vickers
2015-04-28 Atul Butte's presentation to the NIH Precision Medicine Initiative...
2015-04-28 Atul Butte's presentation to the NIH Precision Medicine Initiative...2015-04-28 Atul Butte's presentation to the NIH Precision Medicine Initiative...
2015-04-28 Atul Butte's presentation to the NIH Precision Medicine Initiative...
University of California, San Francisco
 
Forecasting patient outflow from wards having no real-time clinical data
Forecasting patient outflow from wards having no real-time clinical dataForecasting patient outflow from wards having no real-time clinical data
Forecasting patient outflow from wards having no real-time clinical data
Shivapratap Gopakumar
 
Public Data Archiving in Ecology and Evolution: How well are we doing?
Public Data Archiving in Ecology and Evolution: How well are we doing?Public Data Archiving in Ecology and Evolution: How well are we doing?
Public Data Archiving in Ecology and Evolution: How well are we doing?
Sandra Binning
 
iHT2 Health IT Summit San Francisco 2013 - Christopher Longhurst, MD, CMIO, L...
iHT2 Health IT Summit San Francisco 2013 - Christopher Longhurst, MD, CMIO, L...iHT2 Health IT Summit San Francisco 2013 - Christopher Longhurst, MD, CMIO, L...
iHT2 Health IT Summit San Francisco 2013 - Christopher Longhurst, MD, CMIO, L...
Health IT Conference – iHT2
 
Atul Butte NIPS 2017 ML4H
Atul Butte NIPS 2017 ML4HAtul Butte NIPS 2017 ML4H
Atul Butte NIPS 2017 ML4H
University of California, San Francisco
 
C3 - How Do We Use HTA and Real-World Evidence to Implement “Reassessments” E...
C3 - How Do We Use HTA and Real-World Evidence to Implement “Reassessments” E...C3 - How Do We Use HTA and Real-World Evidence to Implement “Reassessments” E...
C3 - How Do We Use HTA and Real-World Evidence to Implement “Reassessments” E...
Emilie Adams
 
Future of Healthcare: 3 Disruptive Trends
Future of Healthcare: 3 Disruptive TrendsFuture of Healthcare: 3 Disruptive Trends
Future of Healthcare: 3 Disruptive Trends
Sean Koon, MD, MS
 
Secondary Data Analysis
Secondary Data AnalysisSecondary Data Analysis
Secondary Data Analysis
REY DECASTRO
 
Prediction research in a pandemic: 3 lessons from a living systematic review ...
Prediction research in a pandemic: 3 lessons from a living systematic review ...Prediction research in a pandemic: 3 lessons from a living systematic review ...
Prediction research in a pandemic: 3 lessons from a living systematic review ...
Laure Wynants
 
Atul Butte presentation on 2019-02-05 for Accelerating biology 2019: Towards ...
Atul Butte presentation on 2019-02-05 for Accelerating biology 2019: Towards ...Atul Butte presentation on 2019-02-05 for Accelerating biology 2019: Towards ...
Atul Butte presentation on 2019-02-05 for Accelerating biology 2019: Towards ...
University of California, San Francisco
 
Precision Medicine World Conference 2017
Precision Medicine World Conference 2017Precision Medicine World Conference 2017
Precision Medicine World Conference 2017
University of California, San Francisco
 
Atul Butte's presentation to the Association of Medical School Pediatric Depa...
Atul Butte's presentation to the Association of Medical School Pediatric Depa...Atul Butte's presentation to the Association of Medical School Pediatric Depa...
Atul Butte's presentation to the Association of Medical School Pediatric Depa...
University of California, San Francisco
 
Atul Butte's presentation for the FDA 5th Annual Scientific Computing Days
Atul Butte's presentation for the FDA 5th Annual Scientific Computing DaysAtul Butte's presentation for the FDA 5th Annual Scientific Computing Days
Atul Butte's presentation for the FDA 5th Annual Scientific Computing Days
University of California, San Francisco
 
Unifying Genomics, Phenomics, and Environments
Unifying Genomics, Phenomics, and EnvironmentsUnifying Genomics, Phenomics, and Environments
Unifying Genomics, Phenomics, and Environments
Anne Thessen
 
Issues Related to Gene Therapy Assignment - Virtual High School (VHS) - SBI4U
Issues Related to Gene Therapy Assignment - Virtual High School (VHS) - SBI4UIssues Related to Gene Therapy Assignment - Virtual High School (VHS) - SBI4U
Issues Related to Gene Therapy Assignment - Virtual High School (VHS) - SBI4U
Michael Taylor
 
Diabetes, PHRs,at teams - Hopkins Capstone
Diabetes, PHRs,at teams - Hopkins CapstoneDiabetes, PHRs,at teams - Hopkins Capstone
Diabetes, PHRs,at teams - Hopkins Capstone
Wade Schuette
 
Presentation on Research Reproducibility at Friends of the National Library o...
Presentation on Research Reproducibility at Friends of the National Library o...Presentation on Research Reproducibility at Friends of the National Library o...
Presentation on Research Reproducibility at Friends of the National Library o...
University of California, San Francisco
 

What's hot (20)

Atul Butte's presentation at the From Data to Discovery symposium at Westat
Atul Butte's presentation at the From Data to Discovery symposium at WestatAtul Butte's presentation at the From Data to Discovery symposium at Westat
Atul Butte's presentation at the From Data to Discovery symposium at Westat
 
297 vickers
297 vickers297 vickers
297 vickers
 
297 vickers
297 vickers297 vickers
297 vickers
 
2015-04-28 Atul Butte's presentation to the NIH Precision Medicine Initiative...
2015-04-28 Atul Butte's presentation to the NIH Precision Medicine Initiative...2015-04-28 Atul Butte's presentation to the NIH Precision Medicine Initiative...
2015-04-28 Atul Butte's presentation to the NIH Precision Medicine Initiative...
 
Forecasting patient outflow from wards having no real-time clinical data
Forecasting patient outflow from wards having no real-time clinical dataForecasting patient outflow from wards having no real-time clinical data
Forecasting patient outflow from wards having no real-time clinical data
 
Public Data Archiving in Ecology and Evolution: How well are we doing?
Public Data Archiving in Ecology and Evolution: How well are we doing?Public Data Archiving in Ecology and Evolution: How well are we doing?
Public Data Archiving in Ecology and Evolution: How well are we doing?
 
iHT2 Health IT Summit San Francisco 2013 - Christopher Longhurst, MD, CMIO, L...
iHT2 Health IT Summit San Francisco 2013 - Christopher Longhurst, MD, CMIO, L...iHT2 Health IT Summit San Francisco 2013 - Christopher Longhurst, MD, CMIO, L...
iHT2 Health IT Summit San Francisco 2013 - Christopher Longhurst, MD, CMIO, L...
 
Atul Butte NIPS 2017 ML4H
Atul Butte NIPS 2017 ML4HAtul Butte NIPS 2017 ML4H
Atul Butte NIPS 2017 ML4H
 
C3 - How Do We Use HTA and Real-World Evidence to Implement “Reassessments” E...
C3 - How Do We Use HTA and Real-World Evidence to Implement “Reassessments” E...C3 - How Do We Use HTA and Real-World Evidence to Implement “Reassessments” E...
C3 - How Do We Use HTA and Real-World Evidence to Implement “Reassessments” E...
 
Future of Healthcare: 3 Disruptive Trends
Future of Healthcare: 3 Disruptive TrendsFuture of Healthcare: 3 Disruptive Trends
Future of Healthcare: 3 Disruptive Trends
 
Secondary Data Analysis
Secondary Data AnalysisSecondary Data Analysis
Secondary Data Analysis
 
Prediction research in a pandemic: 3 lessons from a living systematic review ...
Prediction research in a pandemic: 3 lessons from a living systematic review ...Prediction research in a pandemic: 3 lessons from a living systematic review ...
Prediction research in a pandemic: 3 lessons from a living systematic review ...
 
Atul Butte presentation on 2019-02-05 for Accelerating biology 2019: Towards ...
Atul Butte presentation on 2019-02-05 for Accelerating biology 2019: Towards ...Atul Butte presentation on 2019-02-05 for Accelerating biology 2019: Towards ...
Atul Butte presentation on 2019-02-05 for Accelerating biology 2019: Towards ...
 
Precision Medicine World Conference 2017
Precision Medicine World Conference 2017Precision Medicine World Conference 2017
Precision Medicine World Conference 2017
 
Atul Butte's presentation to the Association of Medical School Pediatric Depa...
Atul Butte's presentation to the Association of Medical School Pediatric Depa...Atul Butte's presentation to the Association of Medical School Pediatric Depa...
Atul Butte's presentation to the Association of Medical School Pediatric Depa...
 
Atul Butte's presentation for the FDA 5th Annual Scientific Computing Days
Atul Butte's presentation for the FDA 5th Annual Scientific Computing DaysAtul Butte's presentation for the FDA 5th Annual Scientific Computing Days
Atul Butte's presentation for the FDA 5th Annual Scientific Computing Days
 
Unifying Genomics, Phenomics, and Environments
Unifying Genomics, Phenomics, and EnvironmentsUnifying Genomics, Phenomics, and Environments
Unifying Genomics, Phenomics, and Environments
 
Issues Related to Gene Therapy Assignment - Virtual High School (VHS) - SBI4U
Issues Related to Gene Therapy Assignment - Virtual High School (VHS) - SBI4UIssues Related to Gene Therapy Assignment - Virtual High School (VHS) - SBI4U
Issues Related to Gene Therapy Assignment - Virtual High School (VHS) - SBI4U
 
Diabetes, PHRs,at teams - Hopkins Capstone
Diabetes, PHRs,at teams - Hopkins CapstoneDiabetes, PHRs,at teams - Hopkins Capstone
Diabetes, PHRs,at teams - Hopkins Capstone
 
Presentation on Research Reproducibility at Friends of the National Library o...
Presentation on Research Reproducibility at Friends of the National Library o...Presentation on Research Reproducibility at Friends of the National Library o...
Presentation on Research Reproducibility at Friends of the National Library o...
 

Similar to Evidence Farming and Open Architecture

Impact Of a Clinical Decision Support Tool on Asthma Patients with Current As...
Impact Of a Clinical Decision Support Tool on Asthma Patients with Current As...Impact Of a Clinical Decision Support Tool on Asthma Patients with Current As...
Impact Of a Clinical Decision Support Tool on Asthma Patients with Current As...
Yiscah Bracha
 
Standards of Prevention: Ethics in the Era of ART Prophylaxis
Standards of Prevention: Ethics in the Era of ART ProphylaxisStandards of Prevention: Ethics in the Era of ART Prophylaxis
Standards of Prevention: Ethics in the Era of ART Prophylaxis
CDC NPIN
 
What if we never agree on a common health information model?
What if we never agree on a common health information model?What if we never agree on a common health information model?
What if we never agree on a common health information model?
Koray Atalag
 
UCSF Informatics Day 2014 - Keith R. Yamamoto, "Precision Medicine"
UCSF Informatics Day 2014 - Keith R. Yamamoto, "Precision Medicine"UCSF Informatics Day 2014 - Keith R. Yamamoto, "Precision Medicine"
UCSF Informatics Day 2014 - Keith R. Yamamoto, "Precision Medicine"
CTSI at UCSF
 
Health Systems Research - 2011
Health Systems Research - 2011Health Systems Research - 2011
Health Systems Research - 2011
Health OER Network
 
Health Systems Research I
Health Systems Research IHealth Systems Research I
Health Systems Research I
Saide OER Africa
 
Watson – from Jeopardy to healthcare
Watson – from Jeopardy to healthcareWatson – from Jeopardy to healthcare
Watson – from Jeopardy to healthcare
Health Informatics New Zealand
 
Module 2: Evidence-Based Dental Public Health
Module 2: Evidence-Based Dental Public HealthModule 2: Evidence-Based Dental Public Health
Module 2: Evidence-Based Dental Public Health
University of Texas Health Science Center San Antonio Libraries
 
Worldwide Lessons in Healthcare IT
Worldwide Lessons in Healthcare ITWorldwide Lessons in Healthcare IT
Worldwide Lessons in Healthcare IT
Social Media Today
 
ISB Prosperity Partnership Presentation by John Aitchison
ISB Prosperity Partnership Presentation by John AitchisonISB Prosperity Partnership Presentation by John Aitchison
ISB Prosperity Partnership Presentation by John Aitchison
Institute for Systems Biology
 
Qi tools in wound infection care finale
Qi tools in wound infection care finaleQi tools in wound infection care finale
Qi tools in wound infection care finale
Evanvs
 
Introduction to the science of improving patient safety
Introduction to the science of improving patient safetyIntroduction to the science of improving patient safety
Introduction to the science of improving patient safety
ht3
 
Clinical Genomics and Medicine
Clinical Genomics and MedicineClinical Genomics and Medicine
Clinical Genomics and Medicine
Warren Kibbe
 
107 satellife
107 satellife107 satellife
Mark Lipsitch: "Simulation and Deliberation to Prepare for Clinical Trials in...
Mark Lipsitch: "Simulation and Deliberation to Prepare for Clinical Trials in...Mark Lipsitch: "Simulation and Deliberation to Prepare for Clinical Trials in...
Mark Lipsitch: "Simulation and Deliberation to Prepare for Clinical Trials in...
The Petrie-Flom Center for Health Law Policy, Biotechnology, and Bioethics
 
MedicalResearch.com - Medical Research Week in Review
MedicalResearch.com - Medical Research  Week in ReviewMedicalResearch.com - Medical Research  Week in Review
MedicalResearch.com - Medical Research Week in Review
Marie Benz MD FAAD
 
The mHealth Triad + Fund Failure
The mHealth Triad + Fund FailureThe mHealth Triad + Fund Failure
The mHealth Triad + Fund Failure
Ernesto Ramirez
 
Mexico final congresso
Mexico final congressoMexico final congresso
Mexico final congresso
celinadeleon
 
The Translational Medicine
The Translational MedicineThe Translational Medicine
The Translational Medicine
Joanne Luciano
 
evidence based practice, EBP
evidence based practice, EBPevidence based practice, EBP
evidence based practice, EBP
pankaj rana
 

Similar to Evidence Farming and Open Architecture (20)

Impact Of a Clinical Decision Support Tool on Asthma Patients with Current As...
Impact Of a Clinical Decision Support Tool on Asthma Patients with Current As...Impact Of a Clinical Decision Support Tool on Asthma Patients with Current As...
Impact Of a Clinical Decision Support Tool on Asthma Patients with Current As...
 
Standards of Prevention: Ethics in the Era of ART Prophylaxis
Standards of Prevention: Ethics in the Era of ART ProphylaxisStandards of Prevention: Ethics in the Era of ART Prophylaxis
Standards of Prevention: Ethics in the Era of ART Prophylaxis
 
What if we never agree on a common health information model?
What if we never agree on a common health information model?What if we never agree on a common health information model?
What if we never agree on a common health information model?
 
UCSF Informatics Day 2014 - Keith R. Yamamoto, "Precision Medicine"
UCSF Informatics Day 2014 - Keith R. Yamamoto, "Precision Medicine"UCSF Informatics Day 2014 - Keith R. Yamamoto, "Precision Medicine"
UCSF Informatics Day 2014 - Keith R. Yamamoto, "Precision Medicine"
 
Health Systems Research - 2011
Health Systems Research - 2011Health Systems Research - 2011
Health Systems Research - 2011
 
Health Systems Research I
Health Systems Research IHealth Systems Research I
Health Systems Research I
 
Watson – from Jeopardy to healthcare
Watson – from Jeopardy to healthcareWatson – from Jeopardy to healthcare
Watson – from Jeopardy to healthcare
 
Module 2: Evidence-Based Dental Public Health
Module 2: Evidence-Based Dental Public HealthModule 2: Evidence-Based Dental Public Health
Module 2: Evidence-Based Dental Public Health
 
Worldwide Lessons in Healthcare IT
Worldwide Lessons in Healthcare ITWorldwide Lessons in Healthcare IT
Worldwide Lessons in Healthcare IT
 
ISB Prosperity Partnership Presentation by John Aitchison
ISB Prosperity Partnership Presentation by John AitchisonISB Prosperity Partnership Presentation by John Aitchison
ISB Prosperity Partnership Presentation by John Aitchison
 
Qi tools in wound infection care finale
Qi tools in wound infection care finaleQi tools in wound infection care finale
Qi tools in wound infection care finale
 
Introduction to the science of improving patient safety
Introduction to the science of improving patient safetyIntroduction to the science of improving patient safety
Introduction to the science of improving patient safety
 
Clinical Genomics and Medicine
Clinical Genomics and MedicineClinical Genomics and Medicine
Clinical Genomics and Medicine
 
107 satellife
107 satellife107 satellife
107 satellife
 
Mark Lipsitch: "Simulation and Deliberation to Prepare for Clinical Trials in...
Mark Lipsitch: "Simulation and Deliberation to Prepare for Clinical Trials in...Mark Lipsitch: "Simulation and Deliberation to Prepare for Clinical Trials in...
Mark Lipsitch: "Simulation and Deliberation to Prepare for Clinical Trials in...
 
MedicalResearch.com - Medical Research Week in Review
MedicalResearch.com - Medical Research  Week in ReviewMedicalResearch.com - Medical Research  Week in Review
MedicalResearch.com - Medical Research Week in Review
 
The mHealth Triad + Fund Failure
The mHealth Triad + Fund FailureThe mHealth Triad + Fund Failure
The mHealth Triad + Fund Failure
 
Mexico final congresso
Mexico final congressoMexico final congresso
Mexico final congresso
 
The Translational Medicine
The Translational MedicineThe Translational Medicine
The Translational Medicine
 
evidence based practice, EBP
evidence based practice, EBPevidence based practice, EBP
evidence based practice, EBP
 

Recently uploaded

June Patch Tuesday
June Patch TuesdayJune Patch Tuesday
June Patch Tuesday
Ivanti
 
5th LF Energy Power Grid Model Meet-up Slides
5th LF Energy Power Grid Model Meet-up Slides5th LF Energy Power Grid Model Meet-up Slides
5th LF Energy Power Grid Model Meet-up Slides
DanBrown980551
 
Dandelion Hashtable: beyond billion requests per second on a commodity server
Dandelion Hashtable: beyond billion requests per second on a commodity serverDandelion Hashtable: beyond billion requests per second on a commodity server
Dandelion Hashtable: beyond billion requests per second on a commodity server
Antonios Katsarakis
 
JavaLand 2024: Application Development Green Masterplan
JavaLand 2024: Application Development Green MasterplanJavaLand 2024: Application Development Green Masterplan
JavaLand 2024: Application Development Green Masterplan
Miro Wengner
 
Leveraging the Graph for Clinical Trials and Standards
Leveraging the Graph for Clinical Trials and StandardsLeveraging the Graph for Clinical Trials and Standards
Leveraging the Graph for Clinical Trials and Standards
Neo4j
 
Astute Business Solutions | Oracle Cloud Partner |
Astute Business Solutions | Oracle Cloud Partner |Astute Business Solutions | Oracle Cloud Partner |
Astute Business Solutions | Oracle Cloud Partner |
AstuteBusiness
 
Energy Efficient Video Encoding for Cloud and Edge Computing Instances
Energy Efficient Video Encoding for Cloud and Edge Computing InstancesEnergy Efficient Video Encoding for Cloud and Edge Computing Instances
Energy Efficient Video Encoding for Cloud and Edge Computing Instances
Alpen-Adria-Universität
 
Generating privacy-protected synthetic data using Secludy and Milvus
Generating privacy-protected synthetic data using Secludy and MilvusGenerating privacy-protected synthetic data using Secludy and Milvus
Generating privacy-protected synthetic data using Secludy and Milvus
Zilliz
 
Harnessing the Power of NLP and Knowledge Graphs for Opioid Research
Harnessing the Power of NLP and Knowledge Graphs for Opioid ResearchHarnessing the Power of NLP and Knowledge Graphs for Opioid Research
Harnessing the Power of NLP and Knowledge Graphs for Opioid Research
Neo4j
 
Northern Engraving | Nameplate Manufacturing Process - 2024
Northern Engraving | Nameplate Manufacturing Process - 2024Northern Engraving | Nameplate Manufacturing Process - 2024
Northern Engraving | Nameplate Manufacturing Process - 2024
Northern Engraving
 
Main news related to the CCS TSI 2023 (2023/1695)
Main news related to the CCS TSI 2023 (2023/1695)Main news related to the CCS TSI 2023 (2023/1695)
Main news related to the CCS TSI 2023 (2023/1695)
Jakub Marek
 
Programming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup SlidesProgramming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup Slides
Zilliz
 
Essentials of Automations: Exploring Attributes & Automation Parameters
Essentials of Automations: Exploring Attributes & Automation ParametersEssentials of Automations: Exploring Attributes & Automation Parameters
Essentials of Automations: Exploring Attributes & Automation Parameters
Safe Software
 
Y-Combinator seed pitch deck template PP
Y-Combinator seed pitch deck template PPY-Combinator seed pitch deck template PP
Y-Combinator seed pitch deck template PP
c5vrf27qcz
 
Monitoring and Managing Anomaly Detection on OpenShift.pdf
Monitoring and Managing Anomaly Detection on OpenShift.pdfMonitoring and Managing Anomaly Detection on OpenShift.pdf
Monitoring and Managing Anomaly Detection on OpenShift.pdf
Tosin Akinosho
 
"Choosing proper type of scaling", Olena Syrota
"Choosing proper type of scaling", Olena Syrota"Choosing proper type of scaling", Olena Syrota
"Choosing proper type of scaling", Olena Syrota
Fwdays
 
“How Axelera AI Uses Digital Compute-in-memory to Deliver Fast and Energy-eff...
“How Axelera AI Uses Digital Compute-in-memory to Deliver Fast and Energy-eff...“How Axelera AI Uses Digital Compute-in-memory to Deliver Fast and Energy-eff...
“How Axelera AI Uses Digital Compute-in-memory to Deliver Fast and Energy-eff...
Edge AI and Vision Alliance
 
Introduction of Cybersecurity with OSS at Code Europe 2024
Introduction of Cybersecurity with OSS  at Code Europe 2024Introduction of Cybersecurity with OSS  at Code Europe 2024
Introduction of Cybersecurity with OSS at Code Europe 2024
Hiroshi SHIBATA
 
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...
Jason Yip
 
The Microsoft 365 Migration Tutorial For Beginner.pptx
The Microsoft 365 Migration Tutorial For Beginner.pptxThe Microsoft 365 Migration Tutorial For Beginner.pptx
The Microsoft 365 Migration Tutorial For Beginner.pptx
operationspcvita
 

Recently uploaded (20)

June Patch Tuesday
June Patch TuesdayJune Patch Tuesday
June Patch Tuesday
 
5th LF Energy Power Grid Model Meet-up Slides
5th LF Energy Power Grid Model Meet-up Slides5th LF Energy Power Grid Model Meet-up Slides
5th LF Energy Power Grid Model Meet-up Slides
 
Dandelion Hashtable: beyond billion requests per second on a commodity server
Dandelion Hashtable: beyond billion requests per second on a commodity serverDandelion Hashtable: beyond billion requests per second on a commodity server
Dandelion Hashtable: beyond billion requests per second on a commodity server
 
JavaLand 2024: Application Development Green Masterplan
JavaLand 2024: Application Development Green MasterplanJavaLand 2024: Application Development Green Masterplan
JavaLand 2024: Application Development Green Masterplan
 
Leveraging the Graph for Clinical Trials and Standards
Leveraging the Graph for Clinical Trials and StandardsLeveraging the Graph for Clinical Trials and Standards
Leveraging the Graph for Clinical Trials and Standards
 
Astute Business Solutions | Oracle Cloud Partner |
Astute Business Solutions | Oracle Cloud Partner |Astute Business Solutions | Oracle Cloud Partner |
Astute Business Solutions | Oracle Cloud Partner |
 
Energy Efficient Video Encoding for Cloud and Edge Computing Instances
Energy Efficient Video Encoding for Cloud and Edge Computing InstancesEnergy Efficient Video Encoding for Cloud and Edge Computing Instances
Energy Efficient Video Encoding for Cloud and Edge Computing Instances
 
Generating privacy-protected synthetic data using Secludy and Milvus
Generating privacy-protected synthetic data using Secludy and MilvusGenerating privacy-protected synthetic data using Secludy and Milvus
Generating privacy-protected synthetic data using Secludy and Milvus
 
Harnessing the Power of NLP and Knowledge Graphs for Opioid Research
Harnessing the Power of NLP and Knowledge Graphs for Opioid ResearchHarnessing the Power of NLP and Knowledge Graphs for Opioid Research
Harnessing the Power of NLP and Knowledge Graphs for Opioid Research
 
Northern Engraving | Nameplate Manufacturing Process - 2024
Northern Engraving | Nameplate Manufacturing Process - 2024Northern Engraving | Nameplate Manufacturing Process - 2024
Northern Engraving | Nameplate Manufacturing Process - 2024
 
Main news related to the CCS TSI 2023 (2023/1695)
Main news related to the CCS TSI 2023 (2023/1695)Main news related to the CCS TSI 2023 (2023/1695)
Main news related to the CCS TSI 2023 (2023/1695)
 
Programming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup SlidesProgramming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup Slides
 
Essentials of Automations: Exploring Attributes & Automation Parameters
Essentials of Automations: Exploring Attributes & Automation ParametersEssentials of Automations: Exploring Attributes & Automation Parameters
Essentials of Automations: Exploring Attributes & Automation Parameters
 
Y-Combinator seed pitch deck template PP
Y-Combinator seed pitch deck template PPY-Combinator seed pitch deck template PP
Y-Combinator seed pitch deck template PP
 
Monitoring and Managing Anomaly Detection on OpenShift.pdf
Monitoring and Managing Anomaly Detection on OpenShift.pdfMonitoring and Managing Anomaly Detection on OpenShift.pdf
Monitoring and Managing Anomaly Detection on OpenShift.pdf
 
"Choosing proper type of scaling", Olena Syrota
"Choosing proper type of scaling", Olena Syrota"Choosing proper type of scaling", Olena Syrota
"Choosing proper type of scaling", Olena Syrota
 
“How Axelera AI Uses Digital Compute-in-memory to Deliver Fast and Energy-eff...
“How Axelera AI Uses Digital Compute-in-memory to Deliver Fast and Energy-eff...“How Axelera AI Uses Digital Compute-in-memory to Deliver Fast and Energy-eff...
“How Axelera AI Uses Digital Compute-in-memory to Deliver Fast and Energy-eff...
 
Introduction of Cybersecurity with OSS at Code Europe 2024
Introduction of Cybersecurity with OSS  at Code Europe 2024Introduction of Cybersecurity with OSS  at Code Europe 2024
Introduction of Cybersecurity with OSS at Code Europe 2024
 
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...
 
The Microsoft 365 Migration Tutorial For Beginner.pptx
The Microsoft 365 Migration Tutorial For Beginner.pptxThe Microsoft 365 Migration Tutorial For Beginner.pptx
The Microsoft 365 Migration Tutorial For Beginner.pptx
 

Evidence Farming and Open Architecture

  • 1. Evidence  Farming1:  Implications  for Open  Architecture Ida  Sim,  MD,  PhD Director,  Center  for  Clinical  and  Translational  Informatics University  of  California  San  Francisco May  5,  2011 1With  thanks  to  Rich  Kravitz  MD,  UC  Davis  and  Naihua  Duan,  Columbia
  • 2. Rephrasing  “Does  it  Work?” (Complexes of) Outcome Exposures strength of association? Increased Text4Baby individual breastfeeding population
  • 3. Current  Approaches:  RCT Asthma App ER visits at 1 year 50 people 100 people Usual Care ER visits at 1 year 50 people population • Tests  prespecified  interventions  and  outcomes • To  confirm  a  hypothesis  at  the  population  level • Strong  internal  validity • Problems:  slow  to  set-­‐up,  expensive,  short-­‐term,  lack relevance  to  the  real  world
  • 4. Current  Approaches:  Data  Mining EHR Exposures Outcomes ? Apps population • Exposures  and  outcomes  from  care  process  systems • To  generate  hypotheses  at  the  population  level • Problems:  limited  to  data  collected,  weak  internal validity  (data  not  complete  or  systematic)
  • 5. Current  Approaches: N-­‐of-­‐1  Studies Asthma app Usual Care Asthma app peak flow peak flow Usual Care Asthma app Usual Care individual • Within-­‐subject  multiple  crossover • Only  formal  method  for  determining  individual treatment  effectiveness • Problems:  complicated  to  set  up,  analysis  is difficult,  little  known,  not  widely  used
  • 6. Evidence  Extraction • Evidence  is  something  to  be  extracted from  the  care  process – mining  it  from  the  data – directly  manipulating  the  care  process  with rigid  and  pre-­‐defined  protocols
  • 8. Evidence  Farming Hay, et al. J Eval Clin Prac 14(2008):707-713.
  • 10. Industrial  Evidence  Farming Asthma App ER visits at 1 year 50 people 100 people Usual Care ER visits at 1 year 50 people population
  • 11. Personal  Evidence  Gardens Asthma app Usual Care Asthma app peak flow peak flow Usual Care Asthma app Usual Care individual
  • 12. Personal  Evidence  Gardens Flovent Flovent PRN Flovent dancing dancing Flovent PRN Flovent Flovent PRN individual
  • 13. Crowdsourcing  What  Matters • (Complexes  of)  Exposures – does  chocolate  trigger  (my)  asthma? – testing  common  regimens  (ACEI,  statin,  b-­‐blocker), complementary  medicines • (Complexes  of)  Outcomes – what  outcomes  do  patients  care  about?
  • 14. Evidence  Macrosystem Rooting for Industrial Evidence Personal Evidence Evidence Farming Gardens
  • 15. How  can  we  scale  evaluation?
  • 16. Stovepiped mHealth • Health  apps  built independently – little  data  sharing  and interoperability • Limits  efficiency  and impact  of  quality mHealth
  • 17. Internet  Hourglass  Model • Standardize  and make  open  the “narrow  waist” • Reduces  duplication, spurs  community innovation,  supports commercial  and  non-­‐ profit  uses
  • 18. OpenmHealth.org Estrin DE, Sim I. Science; 330: 759-60. 2010.
  • 19. OpenmHealth.org • The  waist  should  support the  evidence  macrosystem
  • 20. Open  Architecture  for  an Evidence  Macrosystem • Modules  for  usage  analytics – #  of  text  messages,  #  of  sessions,  etc. • Rooting  for  (glocal)  evidence – data  sharing  with  shared  syntax  and  semantics • Industrial  farming,  e.g.,  with  RCTs – modules  for  informed  consent,  randomization,  adaptive treatment  strategy,  mixed  methods,  etc. • Personal  evidence  gardening,  e.g.,  N-­‐of-­‐1 – modules  for  scripting  and  analyzing  individualized  N-­‐of-­‐ 1  protocols,  etc.
  • 21. Open  Architecture  for  an Evidence  Macrosystem • Social  media  for  discovery  of  exposures  and outcomes  that  matter • Shared  libraries  of  validated  measures  and instruments  (e.g.,  PROMIS) – measures  that  get  at  finer-­‐grained  mechanisms  based on  theoretical  models  of  change,  etc.
  • 22.
  • 23. Goal  for  mHealth  Evidence • A  learning  community  coupled  with  an open  architecture  for  broad,  rapid,  and iterative  dissemination  of  evaluation methods  and  findings  that  matter
  • 24. • Ida  Sim  ida.sim@ucsf.edu • Deborah  Estrin  destrin@cs.ucla.edu • http://openmhealth.org/