Evidence Farming1: Implications for Open Architecture Ida Sim, MD, PhD Director, Center for Clinical and Translational Informatics University of California San Francisco May 5, 2011 1With thanks to Rich Kravitz MD, UC Davis and Naihua Duan, Columbia
Rephrasing “Does it Work?”(Complexes of) Outcome Exposures strength of association? Increased Text4Baby individual breastfeeding population
Current Approaches: RCT Asthma App ER visits at 1 year 50 people 100 people Usual Care ER visits at 1 year 50 people population• Tests prespeciﬁed interventions and outcomes• To conﬁrm a hypothesis at the population level• Strong internal validity• Problems: slow to set-‐up, expensive, short-‐term, lack relevance to the real world
Current Approaches: Data Mining EHR Exposures Outcomes ? Apps population• Exposures and outcomes from care process systems• To generate hypotheses at the population level• Problems: limited to data collected, weak internal validity (data not complete or systematic)
Current Approaches: N-‐of-‐1 Studies Asthma app Usual Care Asthma app peak flow peak flow Usual Care Asthma app Usual Care individual• Within-‐subject multiple crossover• Only formal method for determining individual treatment eﬀectiveness• Problems: complicated to set up, analysis is diﬃcult, little known, not widely used
Evidence Extraction• Evidence is something to be extracted from the care process – mining it from the data – directly manipulating the care process with rigid and pre-‐deﬁned protocols
Stovepiped mHealth• Health apps built independently – little data sharing and interoperability• Limits eﬃciency and impact of quality mHealth
Internet Hourglass Model• Standardize and make open the “narrow waist”• Reduces duplication, spurs community innovation, supports commercial and non-‐ proﬁt uses
OpenmHealth.org Estrin DE, Sim I. Science; 330: 759-60. 2010.
OpenmHealth.org• The waist should support the evidence macrosystem
Open Architecture for an Evidence Macrosystem• Modules for usage analytics – # of text messages, # of sessions, etc.• Rooting for (glocal) evidence – data sharing with shared syntax and semantics• Industrial farming, e.g., with RCTs – modules for informed consent, randomization, adaptive treatment strategy, mixed methods, etc.• Personal evidence gardening, e.g., N-‐of-‐1 – modules for scripting and analyzing individualized N-‐of-‐ 1 protocols, etc.
Open Architecture for an Evidence Macrosystem• Social media for discovery of exposures and outcomes that matter• Shared libraries of validated measures and instruments (e.g., PROMIS) – measures that get at ﬁner-‐grained mechanisms based on theoretical models of change, etc.
Goal for mHealth Evidence• A learning community coupled with an open architecture for broad, rapid, and iterative dissemination of evaluation methods and ﬁndings that matter
• Ida Sim firstname.lastname@example.org• Deborah Estrin email@example.com• http://openmhealth.org/