Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Introduction & rationale for meta-analysis

Meta-analysis

Source & acknowledgement

https://www.terripigott.com/meta-analysis
https://mason.gmu.edu/~dwilsonb/ma.html
https://www.campbellcollaboration.org/research-resources/training-courses.html
https://handbook-5-1.cochrane.org/
https://www.youtube.com/user/Biostat100
https://www.youtube.com/user/collaborationtube
https://www.meta-analysis.com/pages/videotutorials.php?cart=BWJJ3522355

  • Login to see the comments

  • Be the first to like this

Introduction & rationale for meta-analysis

  1. 1. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 02/10 Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course Introduction to meta-analysis Dr. S. A. Rizwan M.D., Public Health Specialist & Lecturer, Saudi Board of Community Medicine – Riyadh, Ministry of Health, KSA With sincere thanks to David B. Wilson 24.11.2019
  2. 2. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 02/10 Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course Outline • History of meta-analysis • Logic of meta-analysis • When to do meta-analysis • Basics of meta-analysis • Strengths and weaknesses of meta-analysis 24.11.2019
  3. 3. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 02/10 Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course The great debate • 1952: Hans J. Eysenck concluded that there were no favorable effects of psychotherapy, starting a raging debate • 20 years of evaluation research and hundreds of studies failed to resolve the debate • 1978: To prove Eysenck wrong, Gene V. Glass statistically aggregated the findings of 375 psychotherapy outcome studies • Glass (and Smith) concluded that psychotherapy did indeed work • Glass called his method “meta-analysis” 24.11.2019
  4. 4. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 02/10 Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course The emergence of meta-analysis • Ideas behind meta-analysis predate Glass’ work by several decades • Karl Pearson (1904) • Averaged correlations for studies of the effectiveness of inoculation for typhoid fever • R. A. Fisher (1944) • “When a number of quite independent tests of significance have been made, it sometimes happens that although few or none can be claimed individually as significant, yet the aggregate gives an impression that the probabilities are on the whole lower than would often have been obtained by chance” (p. 99). • Source of the idea of cumulating probability values 24.11.2019
  5. 5. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 02/10 Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course The emergence of meta-analysis • Ideas behind meta-analysis predate Glass’ work by several decades • W. G. Cochran (1953) • Discusses a method of averaging means across independent studies • Laid-out much of the statistical foundation that modern meta-analysis is built upon (e.g., Inverse variance weighting and homogeneity testing) 24.11.2019
  6. 6. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 02/10 Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course The logic of meta-analysis • Traditional methods of review focus on statistical significance testing • Significance testing is not well suited to this task • Highly dependent on sample size • Null finding does not carry the same “weight” as a significant finding • Significant effect is a strong conclusion • Non-significant effect is a weak conclusion • Meta-analysis focuses on the direction and magnitude of the effects across studies, not statistical significance • Isn’t this what we are interested in anyway? • Direction and magnitude are represented by the effect size 24.11.2019
  7. 7. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 02/10 Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course When can you do meta-analysis? • Meta-analysis is applicable to collections of research that • Are empirical, rather than theoretical • Produce quantitative results, rather than qualitative findings • Examine the same constructs and relationships • Have findings that can be configured in a comparable statistical form (e.g., as effect sizes, correlation coefficients, odds-ratios, proportions) • Are “comparable” given the question at hand 24.11.2019
  8. 8. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 02/10 Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course The replication continuum Pure Replications Conceptual Replications • You must be able to argue that the collection of studies you are meta-analyzing examine the same relationship. • This may be at a broad level of abstraction, such as the relationship between criminal justice interventions and recidivism or between school-based prevention programs and problem behavior. • Alternatively it may be at a narrow level of abstraction and represent pure replications. • The closer to pure replications your collection of studies, the easier it is to argue comparability. 24.11.2019
  9. 9. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 02/10 Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course Forms of research findings suitable to meta-analysis • Central tendency research • Prevalence rates • Pre-post contrasts • Group contrasts • Experimentally created groups • Comparison of outcomes between treatment and comparison groups • Naturally occurring groups • Comparison of spatial abilities between boys and girls • Rates of morbidity among high and low risk groups 24.11.2019
  10. 10. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 02/10 Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course Forms of research findings suitable to meta-analysis • Association between variables • Measurement research • Validity generalization • Individual differences research • Correlation between personality constructs 24.11.2019
  11. 11. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 02/10 Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course Basics of meta-analysis • Describe the distribution, including its mean • Establish a confidence interval around the mean • Test that the mean differs from zero • Test whether studies tell a consistent story (are homogeneous) • Explore the relationship between study features and effect size 24.11.2019
  12. 12. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 02/10 Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course Determining the mean effect size • Problem: some effect sizes are more accurate than others • What we need is an index of precision • Standard error is a direct measure of precision • Hedges and Olkin solution: • Weight by the inverse variance • Provides a statistical basis for: • Standard error of the mean effect size • Confidence intervals • Homogeneity testing 24.11.2019
  13. 13. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 02/10 Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course 24.11.2019
  14. 14. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 02/10 Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course 24.11.2019
  15. 15. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 02/10 Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course 24.11.2019
  16. 16. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 02/10 Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course 24.11.2019
  17. 17. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 02/10 Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course A meta-analysis is a synthesis • Howweapproachthesynthesis depends on our researchquestion • Whatkindsofstudiesweincludedependson our researchquestion • Thekindofanalysisdependsonourresearch question • Theconclusions dependon ourresearch question and the informationavailable 24.11.2019
  18. 18. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 02/10 Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course Compared to primary study • Providesacontext – Thetreatmenteffectissimilaracrossthesetof studies – Thetreatmenteffectvariesin unknownways – Thetreatmenteffectvariesinwayswecanmodel 24.11.2019
  19. 19. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 02/10 Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course When the treatment effect is consistent across studies… Meta-analysis • Provides a more precise estimate of effect size • Provides increased statistical power 24.11.2019
  20. 20. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 02/10 Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course When studies vary in unknown ways… Meta-analysis • Provides more precise estimate of effect size • Provides increased statistical power • Allows us to measure the heterogeneity of the effect 24.11.2019
  21. 21. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 02/10 Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course Studies vary in ways we can model • Meta-analysis allows us to assess treatment effect (and variation) within subgroups • Also, to measure the impact of moderator(s) on the treatment effect 24.11.2019
  22. 22. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 02/10 Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course Bottom line • Can limit the analysis to studies that are essentially identical, and get more precise estimate of the common effect • Can include studies that vary in random ways, assess variation in effect • Can include studies that vary on potentially important factors, assess the impact of these factors 24.11.2019
  23. 23. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 02/10 Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course Strengths of meta-analysis • Imposes a discipline on the process of summing up research findings • Represents findings in a more differentiated and sophisticated manner than conventional reviews • Capable of finding relationships across studies that are obscured in other approaches • Protects against over-interpreting differences across studies • Can handle a large numbers of studies (this would overwhelm traditional approaches to review) 24.11.2019
  24. 24. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 02/10 Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course Weaknesses of meta-analysis • Requires a good deal of effort • Mechanical aspects don’t lend themselves to capturing more qualitative distinctions between studies • “Apples and oranges” criticism • Most meta-analyses include “blemished” studies to one degree or another (e.g., a randomized design with attrition) • Selection bias posses a continual threat • Negative and null finding studies that you were unable to find • Outcomes for which there were negative or null findings that were not reported • Analysis of between study differences is fundamentally correlational 24.11.2019
  25. 25. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 02/10 Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course Thank you Kindly email your queries to sarizwan1986@outlook.com 24.11.2019

×