Heterogeneity in Trial Data: Learning from Difference<br />Rick Rheingans, PhD<br />University of Florida<br />SHARE Resea...
What Works Best? It Depends<br />Sector debate at the interface of science and policy<br />Based on reasonable questions: ...
Meta-analyses and Heterogeneity<br />
Variability across and within studies<br />Depends on behaviors<br />Depends on who – higher protection among the most vul...
Sources of Variability within Trials<br />Different effect levels in different sub-populations due to behavior or vulnerab...
Analytical Tools for Teasing Out Difference<br />Random effects models<br />Did the intervention work differently in diffe...
Grappling with Differences: School Water, Sanitation and Hygiene Impacts<br />SWASH+ <br />Collaborative applied research ...
What’s the Question?<br />National and global policy and advocacy interest in estimating the main effects<br />Days of abs...
Differential Impacts of School Water, Sanitation and Hygiene<br />Absenteeism (Freeman et al, 2011)<br />Strong impact for...
Conduct across 3 Districts in western Kenya<br />Differing socio-economic and exposure conditions<br />
Trying to Explain Differences in School-Cluster Performance<br />Reveals challenges in sustaining hand washing facilities ...
Trying to Explain Differences: New Pathways<br />In schools receiving new latrines, children had increases in fecal hand c...
Implications?<br />What to invest in: <br />De-worming? <br />School uniforms? <br />More teachers? <br />School WASH?<br />
Different Conditions and Impact Variability: A Hypothetical Exercise<br />Overall impact estimates provide us with the ‘av...
Different Conditions and Impact Variability: A Hypothetical Exercise<br />Overall impact estimates provide us with the ‘av...
What Happens with Greater Community Exposures<br />65% Preventable?<br />Comm<br />If there is heterogeneity in level of c...
Variability in Community Level Exposure<br />One measure may be population density of people without sanitation<br />Based...
Variability in Community Level Exposure<br />One measure may be population density of people without sanitation<br />Based...
Other Sources of Differences<br />Could also consider heterogeneity in vulnerability (e.g., nutritional status)<br />Incre...
Differences in Impact?<br />Median<br />Median 16.7%<br />
Differences in Impact?<br />How does sanitation impact vary across the space?<br />Could heterogeneity in impact trial dat...
Variability within Provinces<br />
Salvador, Brazil Sanitation Trial<br />Not a randomized trial – repeated cross-sectional study before and after city-wide ...
Trial Heterogeneity and Generalizability<br />Trials often focus on settings with high levels of burden and homogeneity<br...
Example Deworming and Soil Transmitted Helminths<br />Miguel and Kremer tested the impact of deworming for STH on educatio...
Translating to Heterogeneous Settings<br />Pullan and colleagues developed spatial estimates of national burden to identif...
Schistosomiasis Control in China<br />Liang et al examined the impact environmental and chemotherapy interventions for Sch...
Upcoming SlideShare
Loading in …5
×

UNC Water and Health Conference 2011: Heterogeneity in trial data: learning from difference, Professor Rick Rheingans, University of Florida and SHARE

482 views
428 views

Published on

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
482
On SlideShare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
1
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

UNC Water and Health Conference 2011: Heterogeneity in trial data: learning from difference, Professor Rick Rheingans, University of Florida and SHARE

  1. 1. Heterogeneity in Trial Data: Learning from Difference<br />Rick Rheingans, PhD<br />University of Florida<br />SHARE Research Consortium<br />
  2. 2. What Works Best? It Depends<br />Sector debate at the interface of science and policy<br />Based on reasonable questions: what will work best here?<br />Differences between studies<br />Meta-analyses<br />Differences within studies<br />Analytical focus on main effects<br />Differences outside of studies<br />
  3. 3. Meta-analyses and Heterogeneity<br />
  4. 4. Variability across and within studies<br />Depends on behaviors<br />Depends on who – higher protection among the most vulnerable<br />Depends on initial water quality and other exposures<br />
  5. 5. Sources of Variability within Trials<br />Different effect levels in different sub-populations due to behavior or vulnerability<br />Opportunity for targeting<br />Spatial differences environmental conditions affecting exposure<br />Opportunity for geographic targeting<br />Differences between settings based on implementation<br />Opportunity to adjust adjust the intervention<br />
  6. 6. Analytical Tools for Teasing Out Difference<br />Random effects models<br />Did the intervention work differently in different communities – especially for cluster randomized trials<br />Effect modification<br />Are there characteristics of individuals or communities that change the impact of the intervention<br />Stratification to look at discrete populations<br />Focus is usually on the main effect<br />
  7. 7. Grappling with Differences: School Water, Sanitation and Hygiene Impacts<br />SWASH+ <br />Collaborative applied research and advocacy project led by CARE in western Kenya<br />Cluster-randomized trial in 185 schools<br />Included hygiene promotion, water treatment, sanitation infrastructure, and water supply<br />Objective:<br />Estimate the impact of school WASH interventions on health (helminthes and diarrhea), educational outcomes (absenteeism and performance), and behaviors (e.g., diffusion to homes)<br />
  8. 8. What’s the Question?<br />National and global policy and advocacy interest in estimating the main effects<br />Days of absence avoided<br />Percent reduction in diarrhea<br />Compare it to other school investments<br />Compare it to other WASH investments<br />What if the most important answer is - it depends?<br />
  9. 9. Differential Impacts of School Water, Sanitation and Hygiene<br />Absenteeism (Freeman et al, 2011)<br />Strong impact for girls (Odds Ratio 0.4), no measureable impact for boys<br />Helminthsreinfections - <br />Differences by gender<br />Ascaris for girls; especially poorest<br />Hookworm for boys; especially poorest<br />Differences by behavior<br />Reduced hookworm reinfection among boys without shoes<br />Diffusion of behavior change (water treatment) to homes<br />Strongest effect among the poorest households <br />Differences between schools and regions<br />
  10. 10. Conduct across 3 Districts in western Kenya<br />Differing socio-economic and exposure conditions<br />
  11. 11. Trying to Explain Differences in School-Cluster Performance<br />Reveals challenges in sustaining hand washing facilities and water treatment<br />In compliance adjusted analysis, both having HW facilities and treated water are associated with reduced absence<br />
  12. 12. Trying to Explain Differences: New Pathways<br />In schools receiving new latrines, children had increases in fecal hand contamination<br />Suggests <br />Importance of latrine cleanliness<br />Interdependence of hand-washing and sanitation<br /> Need for anal cleansing materials<br />
  13. 13. Implications?<br />What to invest in: <br />De-worming? <br />School uniforms? <br />More teachers? <br />School WASH?<br />
  14. 14. Different Conditions and Impact Variability: A Hypothetical Exercise<br />Overall impact estimates provide us with the ‘average’ setting, but what will it be in a particular setting?<br />Assume a setting where on average 35% of under-5 diarrhea preventable through improved sanitation<br />Part of diarrhea burden is due to non-sanitation related exposures<br />Some due to whether the household has sanitation<br />Some due to whether they have to share that facility<br />Some due depending how community’s coverage<br />
  15. 15. Different Conditions and Impact Variability: A Hypothetical Exercise<br />Overall impact estimates provide us with the ‘average’ setting, but what will it be in a particular setting?<br />Assume a setting where on average 35% of under-5 diarrhea preventable through improved sanitation<br />Part of diarrhea burden is due to non-sanitation related exposures<br />Some due to whether the household has sanitation<br />Some due to whether they have to share that facility<br />Some due depending how community’s coverage<br />Comm<br />35% Preventable<br />Household<br />Other<br />
  16. 16. What Happens with Greater Community Exposures<br />65% Preventable?<br />Comm<br />If there is heterogeneity in level of community exposure –<br />Would severe diarrhea rates go up?<br />Would the preventable fraction with sanitation go up?<br />50% Preventable?<br />Comm<br />35% Preventable<br />25% Preventable?<br />Comm<br />Household<br />Household<br />Household<br />Household<br />Other<br />Other<br />Other<br />Other<br />
  17. 17. Variability in Community Level Exposure<br />One measure may be population density of people without sanitation<br />Based on cluster-level coverage and population density <br />Varies widely within countries and provinces <br />
  18. 18. Variability in Community Level Exposure<br />One measure may be population density of people without sanitation<br />Based on cluster-level coverage and population density <br />Varies widely within countries and provinces <br />Does sanitations impact change?<br />
  19. 19. Other Sources of Differences<br />Could also consider heterogeneity in vulnerability (e.g., nutritional status)<br />Increased odds diarrhea mortality with low weight for age (Caulfield et al, 2004)<br />Increased risk of illness, for a given exposure<br />Increased risk of mortality, given illness<br />May not affect the fraction preventable through sanitation, but would increase the number of severe cases preventable <br />
  20. 20. Differences in Impact?<br />Median<br />Median 16.7%<br />
  21. 21. Differences in Impact?<br />How does sanitation impact vary across the space?<br />Could heterogeneity in impact trial data help us understand how much?<br />Same is likely true for other WASH interventions<br />
  22. 22. Variability within Provinces<br />
  23. 23. Salvador, Brazil Sanitation Trial<br />Not a randomized trial – repeated cross-sectional study before and after city-wide sanitation project (Barreto et al, 2010)<br />Took advantage of different levels of household and neighborhood change to estimate impact on childhood diarrhea and helminth infections<br />Lessons from heterogeneity within the trial<br />Changes in community sanitation coverage were more important than whether households received a connection<br />Impacts on helminth reduction were strongest among the poorest<br />Showed that intervention reduced the impact of SES on diarrhea disparities <br />
  24. 24. Trial Heterogeneity and Generalizability<br />Trials often focus on settings with high levels of burden and homogeneity<br />Increase the measureable impact<br />Reduce the size of the intervention needed<br />However lack of heterogeneity within the trial can make it hard to generalize to a broader setting<br />External validity<br />
  25. 25. Example Deworming and Soil Transmitted Helminths<br />Miguel and Kremer tested the impact of deworming for STH on educational outcomes in western Kenya<br />Found that deworming can significantly reduce absenteeism (Miguel and Kremer, 2004); spillover effects; and increased long-term earnings (Baird et al, 2011)<br />However the prevalence of STH was uniformly high within the study, compared to the rest of the country<br />
  26. 26. Translating to Heterogeneous Settings<br />Pullan and colleagues developed spatial estimates of national burden to identify where mass treatment would be most appropriate<br />
  27. 27. Schistosomiasis Control in China<br />Liang et al examined the impact environmental and chemotherapy interventions for Schisto control<br />Developed mathematical models of transmission<br />Used data from intervention trials to calibrate the models in different settings<br />Identified patterns for generalizing<br />
  28. 28. Using Variability to Make a Difference<br />Getting more out of trials<br />Analyzing factors modifying intervention effect<br />Better characterizing mechanisms – connecting interventions to outcomes<br />Exposure and environmental studies<br />Modeling and new analytical techniques<br />Deliberate attention to external validity and generalizability in trial design<br />Better translation<br />Using non-trial outcome data to better understand what happens in more diverse settings<br />Better characterizing contexts for which we would like to know the effect of interventions<br />Better policy signals - to encourage more effective intervention selection<br />

×