Your SlideShare is downloading.
×

- 1. META ANALYSIS - AN OVERVIEW Tulasi Raman P
- 2. DEFINITION Meta-analysis is a quantitative approach for systematically combining results of previous research to arrive at conclusions about the body of research. Quantitative: numbers Systematic: methodical Combining: putting together Previous research: what's already done Conclusions: new knowledge A study collects data from individual subjects (such as 100 subjects = 100 “data points”) A meta-analysis collects data from individual studies (such as 100 studies = 100 “data points”)
- 3. HISTORY Karl Pearson is probably the first medical researcher to use formal techniques to combine data from different studies (1904): He synthesized data from several studies on efficacy of typhoid vaccination His rationale for pooling data: “Many of the groups… are far too small to allow of any definite opinion being formed at all, having regard to the size of the probable error involved.”
- 4. THE TERM META-ANALYSIS 1952: Hans J. Eysenck concluded that there were no favorable effects of psychotherapy, starting a raging debate which 25 years of evaluation research and hundreds of studies failed to resolve 1978: To proved Eysenck wrong, Gene V. Glass statistically aggregated the findings of 375 psychotherapy outcome studies Glass called the method “meta-analysis”
- 5. META ANALYSIS IN CONTEXT Traditional Systematic Meta- review Review analysis Author An individual A team A team Search Individual Based on a Based on a strategy protocol protocol Summary Author‟s Can be Summary Conclusion judgement qualitative statistical techniques
- 6. META-ANALYSES INDEXED IN PUBMED 2500 2000 1500 1000 500 0 1992 1994 1996 1998 2000
- 7. FUNCTION OF META-ANALYSIS 1. Identify heterogeneity in effects among multiple studies and, where appropriate, provide summary measure 2. Increase statistical power and precision to detect an effect 3. Develop, refine and test hypothesis 4. Reduce the subjectivity of study comparisons by using systematic and explicit comparison procedure 5. Identify data gap in the knowledge base and suggest direction for future research 6. Calculate sample size for future studies
- 8. COULD WE JUST ADD THE DATA FROM ALL THE TRIALS TOGETHER? One approach to combining trials would be to add all the treatment groups together, add all the control groups together, and compare the totals This is wrong for several reasons, and it can give the wrong answer
- 9. If we just add up the columns we get From a meta-analysis, we get 34.3% vs 32.5% , a RR of 1.06, RR=0.96 , a lower death rate a higher death rate in the steroids group in the steroids group
- 10. PROBLEMS WITH SIMPLE ADDITION OF STUDIES Breaks the power of randomization Imbalances within trials introduce bias
- 11. * # In effect we are comparing this experimental group directly with this control group – this is not a randomized comparison
- 12. STEPS IN META-ANALYSIS 1. Define the research question and specific hypotheses 2. Define the criteria for including and excluding studies 3. Locate research studies 4. Determine which studies are eligible for inclusion 5. Classify and code important study characteristics (e.g., sample size; length of follow-up; definition of outcome; drug brand and dose) 6. Select or translate results from each study using a common metric 7. Aggregate findings across studies, generating weighted pooled estimates of effect size. 8. Evaluate the statistical homogeneity of pooled studies 9. Perform sensitivity analyses to assess the impact of excluding or down-weighting unpublished studies, studies of lower quality, out-of-date studies, etc.
- 13. PUBLICATION BIAS Studies with significant results are more likely to be published to be published in English to be cited by others to produce multiple publications Including only published studies can introduce publication bias Most reviews do not look for publication bias
- 14. APPROACHES TO PUBLICATION BIAS Fail-safe N Publications are filled with the 5% of studies with Type I error but file drawers are filled with the 95% with non- significant effects Funnel plots Trim and fill method Small studies are removed from funnel plot until it is symmetric Then replace the small studies and balance them with studies on the opposite side of the funnel Statistical analogues of funnel plot Egger test
- 15. FUNNEL PLOT
- 16. ASSESSING STUDY QUALITY
- 17. QUALITY SCORE Some meta-analysts score the quality of a study. 1. Examples (scored yes=1, no=0): 2. Published in a peer-reviewed journal? 3. Experienced researchers? 4. Research funded by impartial agency? 5. Study performed by impartial researchers? 6. Subjects selected randomly from a population? 7. Subjects assigned randomly to treatments? 8. High proportion of subjects entered and/or finished the study? 9. Subjects blind to treatment? 10. Data gatherers blind to treatment? 11. Analysis performed blind?
- 18. AVERAGING STUDIES Starting with the summary statistic for each study, how should we combine these? A simple average gives each study equal weight This seems intuitively wrong Some studies are more likely to give an answer closer to the „true‟ effect than others
- 19. WEIGHTING STUDIES More weight to the studies which give us more information More participants More events Lower variance Calculated using the effect estimate and its variance Inverse-variance method: 1 1 weight variance of estimate SE2 sum of (estimate weight) pooled estimate sum of weights
- 20. FOR EXAMPLE Headache Caffeine Decaf Weight Amore-Coffea 2000 2/31 10/34 6.6% Deliciozza 2004 10/40 9/40 21.9% Mama-Kaffa 1999 12/53 9/61 22.2% Morrocona 1998 3/15 1/17 2.9% Norscafe 1998 19/68 9/64 26.4% Oohlahlazza 1998 4/35 2/37 5.1% Piazza-Allerta 2003 8/35 6/37 14.9%
- 21. WHAT IS HETEROGENEITY Heterogeneity is variation between the studies‟ results
- 22. CAUSES OF HETEROGENEITY 1. Differences between studies with respect to: 2. Patients: diagnosis, in- and exclusion criteria, etc. 3. Interventions: type, dose, duration, etc. 4. Outcomes: type, scale, cut-off points, duration of follow-up, etc. 5. Quality and methodology: randomised or not, allocation concealment, blinding, etc.
- 23. HOW TO LOOK FOR HETEROGENEITY? Visually Forest plot: do confidence intervals of studies overlap with each other and the summary effect? Statistically Chi-square test for heterogeneity (Mantel-Haenszel test or Cochran Q test) Tests whether the individual effects are farther away from the common effect, beyond what is expected by chance Has poor power P-value < 0.10 indicates significant heterogeneity
- 24. HOW TO DEAL WITH HETEROGENEITY 1. Do not pool at all 2. Ignore heterogeneity: use fixed effect model 3. Allow for heterogeneity: use random effects model 4. Explore heterogeneity
- 25. FIXED EFFECT MODEL Fixed effects model assumes that the true effect of treatment is the same value in each study (fixed); the differences between studies is solely due to random error Specific methods for combining odds ratio 1. Mantel- Haenszel method 2. Peto‟s method 3. Maximum-Likelihood techniques 4. Exact methods of interval estimation
- 26. RANDOM EFFECT MODEL In random effects models, the treatment effects for the individual studies are assumed to vary around some overall average treatment effect Allows for random error plus inter-study variability Results in wider confidence intervals (conservative) Studies tend to be weighted more equally (relatively more weight is given to smaller studies) There are five approach for this model: 1. Weighted least squares 2. Un-weighted least squares 3. Maximum likelihood 4. Restricted Maximum likelihood 5. Exact approach to random effects of binary data.
- 27. SENSITIVITY ANALYSIS Several features of the meta-analysis can be altered to gauge the robustness of the results: Modifying the inclusion criteria Including and excluding questionable studies Including and excluding unpublished studies Weighting the analysis by study quality Trying different ways to impute missing data Removing each study, one by one, to see the change
- 28. FOREST PLOT The graphical display of results from individual studies on a common scale is a “Forest plot”. In the forest plot each study is represented by a black square and a horizontal line (CI:95%).The area of the black square reflects the weight of the study in the meta-analysis. A logarithmic scale should be used for plotting the Relative Risk.
- 29. FOREST PLOTS Headache at 24 hours HEADINGS EXPLAIN THE COMPARISON
- 30. FOREST PLOTS Headache at 24 hours LIST OF INCLUDED STUDIES
- 31. FOREST PLOTS Headache at 24 hours RAW DATA FOR EACH STUDY
- 32. FOREST PLOTS Headache at 24 hours TOTAL DATA FOR ALL STUDIES
- 33. FOREST PLOTS Headache at 24 hours WEIGHT GIVEN TO EACH STUDY
- 34. FOREST PLOTS Headache at 24 hours EFFECT ESTIMATE FOR EACH STUDY, WITH CI
- 35. FOREST PLOTS Headache at 24 hours EFFECT ESTIMATE FOR EACH STUDY, WITH CI
- 36. FOREST PLOTS Headache at 24 hours SCALE AND DIRECTION OF BENEFIT
- 37. FOREST PLOTS Headache at 24 hours POOLED EFFECT ESTIMATE FOR ALL STUDIES, WITH CI
- 38. INTERPRETING CONFIDENCE INTERVALS always present estimate with a confidence interval precision • point estimate is the best guess of the effect • CI expresses uncertainty – range of values we can be reasonably sure includes the true effect significance • if the CI includes the null value rarely means evidence of no effect effect cannot be confirmed or refuted by the available evidence • consider what level of change is clinically important
- 39. LIMITATIONS OF META-ANALYSIS A meta-analysis reflects only what's published or searchable. It's focused on mean effects and differences between studies. But what really matters is effects on individuals. (Aggression bias) Relation between group rates or and means may not resemble the relation between individual values of exposure and outcome. This phenomenon is known as aggregation bias or ecologic bias.
- 40. WHAT IS AN IPD META-ANALYSIS? Involves the central collection, checking and analysis of updated individual patient data Include all properly randomised trials, published and unpublished Include all patients in an intention-to-treat analysis Individual patient data used Analysis stratified by trial IPD does not mean that all patients are combined into a single mega trial The major advantage of a IPD over an MA is the use of individual-based rather than group-based data.
- 41. GUIDELINES FOR REPORTING META-ANALYSIS The QUORUM Statement: Quality of Reporting of Meta-analyses – For clinical Randomized Controlled Trials (RCT‟s) MOOSE guidelines: Meta-analysis Of Observational Studies in Epidemiology
- 42. META-ANALYSIS SOFTWARE Free Commercial RevMan [Review Comprehensive Meta- Manager] analysis Meta-Analyst Meta-Win Epi Meta WEasy MA Easy MA Meta-Test General stats packages Meta-Stat Stata SAS S-Plus
- 43. THANK YOU