Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Heterogeneity in Trial Data: Learning from Difference<br />Rick Rheingans, PhD<br />University of Florida<br />SHARE Resea...
What Works Best? It Depends<br />Sector debate at the interface of science and policy<br />Based on reasonable questions: ...
Meta-analyses and Heterogeneity<br />
Variability across and within studies<br />Depends on behaviors<br />Depends on who – higher protection among the most vul...
Sources of Variability within Trials<br />Different effect levels in different sub-populations due to behavior or vulnerab...
Analytical Tools for Teasing Out Difference<br />Random effects models<br />Did the intervention work differently in diffe...
Grappling with Differences: School Water, Sanitation and Hygiene Impacts<br />SWASH+ <br />Collaborative applied research ...
What’s the Question?<br />National and global policy and advocacy interest in estimating the main effects<br />Days of abs...
Differential Impacts of School Water, Sanitation and Hygiene<br />Absenteeism (Freeman et al, 2011)<br />Strong impact for...
Conduct across 3 Districts in western Kenya<br />Differing socio-economic and exposure conditions<br />
Trying to Explain Differences in School-Cluster Performance<br />Reveals challenges in sustaining hand washing facilities ...
Trying to Explain Differences: New Pathways<br />In schools receiving new latrines, children had increases in fecal hand c...
Implications?<br />What to invest in: <br />De-worming? <br />School uniforms? <br />More teachers? <br />School WASH?<br />
Different Conditions and Impact Variability: A Hypothetical Exercise<br />Overall impact estimates provide us with the ‘av...
Different Conditions and Impact Variability: A Hypothetical Exercise<br />Overall impact estimates provide us with the ‘av...
What Happens with Greater Community Exposures<br />65% Preventable?<br />Comm<br />If there is heterogeneity in level of c...
Variability in Community Level Exposure<br />One measure may be population density of people without sanitation<br />Based...
Variability in Community Level Exposure<br />One measure may be population density of people without sanitation<br />Based...
Other Sources of Differences<br />Could also consider heterogeneity in vulnerability (e.g., nutritional status)<br />Incre...
Differences in Impact?<br />Median<br />Median 16.7%<br />
Differences in Impact?<br />How does sanitation impact vary across the space?<br />Could heterogeneity in impact trial dat...
Variability within Provinces<br />
Salvador, Brazil Sanitation Trial<br />Not a randomized trial – repeated cross-sectional study before and after city-wide ...
Trial Heterogeneity and Generalizability<br />Trials often focus on settings with high levels of burden and homogeneity<br...
Example Deworming and Soil Transmitted Helminths<br />Miguel and Kremer tested the impact of deworming for STH on educatio...
Translating to Heterogeneous Settings<br />Pullan and colleagues developed spatial estimates of national burden to identif...
Schistosomiasis Control in China<br />Liang et al examined the impact environmental and chemotherapy interventions for Sch...
Upcoming SlideShare
Loading in …5
×

UNC Water and Health Conference 2011: Heterogeneity in trial data: learning from difference, Professor Rick Rheingans, University of Florida and SHARE

573 views

Published on

  • Be the first to comment

  • Be the first to like this

UNC Water and Health Conference 2011: Heterogeneity in trial data: learning from difference, Professor Rick Rheingans, University of Florida and SHARE

  1. 1. Heterogeneity in Trial Data: Learning from Difference<br />Rick Rheingans, PhD<br />University of Florida<br />SHARE Research Consortium<br />
  2. 2. What Works Best? It Depends<br />Sector debate at the interface of science and policy<br />Based on reasonable questions: what will work best here?<br />Differences between studies<br />Meta-analyses<br />Differences within studies<br />Analytical focus on main effects<br />Differences outside of studies<br />
  3. 3. Meta-analyses and Heterogeneity<br />
  4. 4. Variability across and within studies<br />Depends on behaviors<br />Depends on who – higher protection among the most vulnerable<br />Depends on initial water quality and other exposures<br />
  5. 5. Sources of Variability within Trials<br />Different effect levels in different sub-populations due to behavior or vulnerability<br />Opportunity for targeting<br />Spatial differences environmental conditions affecting exposure<br />Opportunity for geographic targeting<br />Differences between settings based on implementation<br />Opportunity to adjust adjust the intervention<br />
  6. 6. Analytical Tools for Teasing Out Difference<br />Random effects models<br />Did the intervention work differently in different communities – especially for cluster randomized trials<br />Effect modification<br />Are there characteristics of individuals or communities that change the impact of the intervention<br />Stratification to look at discrete populations<br />Focus is usually on the main effect<br />
  7. 7. Grappling with Differences: School Water, Sanitation and Hygiene Impacts<br />SWASH+ <br />Collaborative applied research and advocacy project led by CARE in western Kenya<br />Cluster-randomized trial in 185 schools<br />Included hygiene promotion, water treatment, sanitation infrastructure, and water supply<br />Objective:<br />Estimate the impact of school WASH interventions on health (helminthes and diarrhea), educational outcomes (absenteeism and performance), and behaviors (e.g., diffusion to homes)<br />
  8. 8. What’s the Question?<br />National and global policy and advocacy interest in estimating the main effects<br />Days of absence avoided<br />Percent reduction in diarrhea<br />Compare it to other school investments<br />Compare it to other WASH investments<br />What if the most important answer is - it depends?<br />
  9. 9. Differential Impacts of School Water, Sanitation and Hygiene<br />Absenteeism (Freeman et al, 2011)<br />Strong impact for girls (Odds Ratio 0.4), no measureable impact for boys<br />Helminthsreinfections - <br />Differences by gender<br />Ascaris for girls; especially poorest<br />Hookworm for boys; especially poorest<br />Differences by behavior<br />Reduced hookworm reinfection among boys without shoes<br />Diffusion of behavior change (water treatment) to homes<br />Strongest effect among the poorest households <br />Differences between schools and regions<br />
  10. 10. Conduct across 3 Districts in western Kenya<br />Differing socio-economic and exposure conditions<br />
  11. 11. Trying to Explain Differences in School-Cluster Performance<br />Reveals challenges in sustaining hand washing facilities and water treatment<br />In compliance adjusted analysis, both having HW facilities and treated water are associated with reduced absence<br />
  12. 12. Trying to Explain Differences: New Pathways<br />In schools receiving new latrines, children had increases in fecal hand contamination<br />Suggests <br />Importance of latrine cleanliness<br />Interdependence of hand-washing and sanitation<br /> Need for anal cleansing materials<br />
  13. 13. Implications?<br />What to invest in: <br />De-worming? <br />School uniforms? <br />More teachers? <br />School WASH?<br />
  14. 14. Different Conditions and Impact Variability: A Hypothetical Exercise<br />Overall impact estimates provide us with the ‘average’ setting, but what will it be in a particular setting?<br />Assume a setting where on average 35% of under-5 diarrhea preventable through improved sanitation<br />Part of diarrhea burden is due to non-sanitation related exposures<br />Some due to whether the household has sanitation<br />Some due to whether they have to share that facility<br />Some due depending how community’s coverage<br />
  15. 15. Different Conditions and Impact Variability: A Hypothetical Exercise<br />Overall impact estimates provide us with the ‘average’ setting, but what will it be in a particular setting?<br />Assume a setting where on average 35% of under-5 diarrhea preventable through improved sanitation<br />Part of diarrhea burden is due to non-sanitation related exposures<br />Some due to whether the household has sanitation<br />Some due to whether they have to share that facility<br />Some due depending how community’s coverage<br />Comm<br />35% Preventable<br />Household<br />Other<br />
  16. 16. What Happens with Greater Community Exposures<br />65% Preventable?<br />Comm<br />If there is heterogeneity in level of community exposure –<br />Would severe diarrhea rates go up?<br />Would the preventable fraction with sanitation go up?<br />50% Preventable?<br />Comm<br />35% Preventable<br />25% Preventable?<br />Comm<br />Household<br />Household<br />Household<br />Household<br />Other<br />Other<br />Other<br />Other<br />
  17. 17. Variability in Community Level Exposure<br />One measure may be population density of people without sanitation<br />Based on cluster-level coverage and population density <br />Varies widely within countries and provinces <br />
  18. 18. Variability in Community Level Exposure<br />One measure may be population density of people without sanitation<br />Based on cluster-level coverage and population density <br />Varies widely within countries and provinces <br />Does sanitations impact change?<br />
  19. 19. Other Sources of Differences<br />Could also consider heterogeneity in vulnerability (e.g., nutritional status)<br />Increased odds diarrhea mortality with low weight for age (Caulfield et al, 2004)<br />Increased risk of illness, for a given exposure<br />Increased risk of mortality, given illness<br />May not affect the fraction preventable through sanitation, but would increase the number of severe cases preventable <br />
  20. 20. Differences in Impact?<br />Median<br />Median 16.7%<br />
  21. 21. Differences in Impact?<br />How does sanitation impact vary across the space?<br />Could heterogeneity in impact trial data help us understand how much?<br />Same is likely true for other WASH interventions<br />
  22. 22. Variability within Provinces<br />
  23. 23. Salvador, Brazil Sanitation Trial<br />Not a randomized trial – repeated cross-sectional study before and after city-wide sanitation project (Barreto et al, 2010)<br />Took advantage of different levels of household and neighborhood change to estimate impact on childhood diarrhea and helminth infections<br />Lessons from heterogeneity within the trial<br />Changes in community sanitation coverage were more important than whether households received a connection<br />Impacts on helminth reduction were strongest among the poorest<br />Showed that intervention reduced the impact of SES on diarrhea disparities <br />
  24. 24. Trial Heterogeneity and Generalizability<br />Trials often focus on settings with high levels of burden and homogeneity<br />Increase the measureable impact<br />Reduce the size of the intervention needed<br />However lack of heterogeneity within the trial can make it hard to generalize to a broader setting<br />External validity<br />
  25. 25. Example Deworming and Soil Transmitted Helminths<br />Miguel and Kremer tested the impact of deworming for STH on educational outcomes in western Kenya<br />Found that deworming can significantly reduce absenteeism (Miguel and Kremer, 2004); spillover effects; and increased long-term earnings (Baird et al, 2011)<br />However the prevalence of STH was uniformly high within the study, compared to the rest of the country<br />
  26. 26. Translating to Heterogeneous Settings<br />Pullan and colleagues developed spatial estimates of national burden to identify where mass treatment would be most appropriate<br />
  27. 27. Schistosomiasis Control in China<br />Liang et al examined the impact environmental and chemotherapy interventions for Schisto control<br />Developed mathematical models of transmission<br />Used data from intervention trials to calibrate the models in different settings<br />Identified patterns for generalizing<br />
  28. 28. Using Variability to Make a Difference<br />Getting more out of trials<br />Analyzing factors modifying intervention effect<br />Better characterizing mechanisms – connecting interventions to outcomes<br />Exposure and environmental studies<br />Modeling and new analytical techniques<br />Deliberate attention to external validity and generalizability in trial design<br />Better translation<br />Using non-trial outcome data to better understand what happens in more diverse settings<br />Better characterizing contexts for which we would like to know the effect of interventions<br />Better policy signals - to encourage more effective intervention selection<br />

×