Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Moderator analysis in meta-analysis

Systematic review and meta-analysis

Source & acknowledgement

https://www.terripigott.com/meta-analysis
https://mason.gmu.edu/~dwilsonb/ma.html
https://www.campbellcollaboration.org/research-resources/training-courses.html
https://handbook-5-1.cochrane.org/
https://www.youtube.com/user/Biostat100
https://www.youtube.com/user/collaborationtube
https://www.meta-analysis.com/pages/videotutorials.php?cart=BWJJ3522355

  • Login to see the comments

  • Be the first to like this

Moderator analysis in meta-analysis

  1. 1. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 08/10 Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course Moderator analysis in meta-analysis Dr. S. A. Rizwan M.D., Public Health Specialist & Lecturer, Saudi Board of Preventive Medicine – Riyadh, Ministry of Health, KSA With thanks to Ryan Williams 26.11.2019 1
  2. 2. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 08/10 Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course Moderator analyses in meta-analysis • To test our hypotheses about whether variation among studies in effect size is associated with differences in study methods or participants • A priori ideas, incorporating these characteristics of studies into our coding forms • Two major forms: – categorical models analogous to ANOVA – meta-regression 26.11.2019 2
  3. 3. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 08/10 Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course Assumptions • Focus on random effects models • Based on the random effects variance component 26.11.2019 3
  4. 4. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 08/10 Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course Categorical moderators • When the moderator variable is categorical, estimate ANOVA analogue • Typically, comparing the mean effect sizes for 2 or more groups • For example, we will look at a meta-analysis where we compare the mean effect size for studies published in three different sources: journals, dissertations, and unpublished studies 26.11.2019 4
  5. 5. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 08/10 Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course Categorical moderators • With a one-way random effects ANOVA model, recall that we will compute – A mean effect size and standard error for each group, and then test whether these means are significantly different from one another • The mean effect size and standard error require an estimate of the variance component • Options: Will we assume that each group has the same variance component? Or, will we assume that each group has its own variance component? 26.11.2019 5
  6. 6. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 08/10 Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course Assumptions for separate subgroups • We believe that the variation among studies is different between groups. • For example, – if we are testing out an intervention and we have studies that use either a low-income and a high-income group of students, we might believe that there will be more variation in effectiveness among studies that have mostly low- income participants – the effectiveness of an intervention for juvenile delinquents will vary more for the group that had a prior arrest than for those that do not have a prior arrest 26.11.2019 6
  7. 7. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 08/10 Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course Assumptions for pooled estimate • We believe that the variation among effect sizes are the same no matter the group. • For an intervention review, we may assume that the variation among studies does not differ within the groups of interest • Caveat – We might have to use a pooled estimate if we have small sample sizes within subgroups – We need at least 5 cases (in general) to be able to estimate a separate variance component for each subgroup 26.11.2019 7
  8. 8. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 08/10 Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course Flowchart for choosing variance 26.11.2019 8
  9. 9. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 08/10 Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course Steps for a random effects ANOVA • Make a decision about the use of a pooled or a separate estimate of the variance component • Compute the group mean effect sizes, and their standard errors • Compare the group mean effect sizes to see if they are statistically different from one another 26.11.2019 9
  10. 10. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 08/10 Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course Eagly, Johannesen-Schmidt & van Engen (2003) • This synthesis examines the standardized mean difference estimated in primary studies for the difference between men and women in their use of transformational leadership. • Transformational leadership involves “establishing onself as a role model by gaining the trust and confidence of followers” (Eagly et al. 2003, p. 570). • The sample data is a subset of the studies in the full meta- analysis, a set of 24 studies that compare men and women in their use of transformational leadership • Positive effect sizes indicate males score higher than females 26.11.2019 10
  11. 11. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 08/10 Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course Sub-groups • Example 26.11.2019 11
  12. 12. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 08/10 Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course Sub-groups 26.11.2019 12
  13. 13. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 08/10 Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course Sub-groups 26.11.2019 13
  14. 14. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 08/10 Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course Summary of results – separate variance estimates for each group • Journals have a significant variance component, and the mean is not different from zero by magnitude • Dissertations and unpublished studies both have a non-significant variance component, but both have medium to large effect size Group k Mean 95% CI τ2 p Journals 13 -0.05 [-0.24, 0.12] 0.09 <0.001 Dissertations 7 -0.47 [-0.69,-0.26] 0.02 0.22 Unpublished 4 -0.16 [-0.30,-0.03] 0.00 0.87 TOTAL 24 -0.16 [-0.29, -0.03] 0.08 <0.001 26.11.2019 14
  15. 15. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 08/10 Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course Summary of results – separate variance estimates for each group • The test of the variance component as different from zero is exactly the fixed effects test of homogeneity. • To get this test, we compute the test of homogeneity within each group of studies. Group k Mean 95% CI τ2 p Journals 13 -0.05 [-0.24, 0.12] 0.09 <0.001 Dissertations 7 -0.47 [-0.69,-0.26] 0.02 0.22 Unpublished 4 -0.16 [-0.30,-0.03] 0.00 0.87 TOTAL 24 -0.16 [-0.29, -0.03] 0.08 <0.001 26.11.2019 15
  16. 16. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 08/10 Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course Test of between group differences • To test between group differences in a random effects model, we test whether the variance component for the variation among the random effects means is equal to zero • There are several ways to obtain this value • We will use a test of homogeneity of the three means – we will treat the three group means as a meta-analysis 26.11.2019 16
  17. 17. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 08/10 Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course Test of between group differences • We will compute a test of homogeneity using our three means as if this is a meta-analysis • We will use the means and their estimated variances to compute the sums we need to compute the homogeneity test 26.11.2019 17
  18. 18. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 08/10 Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course Computation of Q between groups Source Mean Var Wt Wt*Mean Wt*Mean2 Journals -0.05 0.008 122.53 -6.13 0.031 Dissertations -0.47 0.012 86.46 -40.64 19.10 Unpublished -0.16 0.005 211.32 -33.82 5.41 SUM 420.31 -80.59 24.54 Q = 24.54 - (-80.59 * -80.59) 420.31 = 9.09 Compare 9.09 to a chi-square with df=3-1=2. p-value is 0.011 26.11.2019 18
  19. 19. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 08/10 Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course Using the same variance for subgroups 26.11.2019 19
  20. 20. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 08/10 Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course Notes about the CMA forest plot • Like RevMan, the confidence intervals around each study are the fixed effects confidence intervals (they use the within-study fixed effects variance) • The group means are the random effects means computed using random effects weights. Their confidence intervals are also use random effects. • In this example, we are using the same variance component for all groups 26.11.2019 20
  21. 21. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 08/10 Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course Summary from CMA 26.11.2019 21
  22. 22. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 08/10 Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course Notes about CMA results • We assumed that the variance component was 0.08 for all studies • Compared to our separate variance estimates, this value is smaller than the separate variance estimate for journals, but larger than the separate estimate for dissertations and unpublished articles 26.11.2019 22
  23. 23. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 08/10 Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course Another subgroup example 26.11.2019 23
  24. 24. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 08/10 Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course Reporting random effects categorical analysis • The assumption made about the random effects variance: separate estimate for each group, or the same estimate for all groups. • Rationale for the choice of variance component • The random effects mean and CI • The value of the variance components (or variance component) • The test of the between-group differences, and its significance 26.11.2019 24
  25. 25. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 08/10 Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course What is meta-regression? • Meta-regression is a statistical technique used in a meta- analysis to examine how characteristics of studies are related to variation in effect sizes across studies • Meta-regression is analogous to regression analysis but using effect sizes as our outcomes, and information extracted from studies as moderators/predictors 26.11.2019 25
  26. 26. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 08/10 Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course What is meta-regression? • When we have a heterogeneous set of effect sizes, we can use statistical techniques to examine the association among characteristics of the study and variation among effect sizes • We have a plan for these analyses a priori – based on our understanding of the literature, and a logic model or framework • Meta-regression used when we have more than one predictor or moderator (either continuous or categorical) 26.11.2019 26
  27. 27. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 08/10 Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course Form of the meta-regression model • ! "↓# = $↓0 + $↓1  %↓#1 +…+ $↓& %↓#& • where ! "↓#  is our generic effect size estimate from study i, %↓#1 , %↓#2 , … %↓#&  are the moderator variable values for study i, and $↓0 , $↓1 , … $↓&  are the unknown regression coefficients. 26.11.2019 27
  28. 28. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 08/10 Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course Recall that the variance of the effect size • Depends on the sample size for all types of effects we have talked about • Thus, the precision of each study’s effect size depends on sample size • This is different from our typical application of regression where we assume every person has the same “weight” • Thus, we need to use weighted least squares regression to account for the fact that the precision of each effect size depends on sample size 26.11.2019 28
  29. 29. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 08/10 Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course Random effects meta-regression • As in the categorical analysis discussion, we will need an estimate of the random effects variance for our studies that will be used as our weights in the regression • There are many ways to compute the variance component in a random effects meta-regression • For now, let’s assume a single variance component for all studies. 26.11.2019 29
  30. 30. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 08/10 Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course Test for model fit in meta-regression • As in a standard regression model, we can use the regression ANOVA table for diagnostics about the fit of a meta- regression • Recall that in a standard regression analysis, we would get the following regression ANOVA table: 26.11.2019 30
  31. 31. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 08/10 Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course ANOVA table for normal regression 26.11.2019 31
  32. 32. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 08/10 Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course Test of model fit in meta-regression • In meta-regression, we use the ANOVA table to get two different Q statistics: • QM – model sum of squares, compare to chi-square distribution with p – 1 df (p is number of predictors in the model) • QR – residual sum of squares, compare to chi-square distribution with k - p – 1 df (k is the number of studies) 26.11.2019 32
  33. 33. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 08/10 Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course QM - the model sum of squares • Qmodel is the test of whether at least one of the regression coefficients (not including the intercept) is different from zero • We compare QM to a chi-square distribution with p–1 degrees of freedom with p = # of predictors in model • If QM is significant, then at least one of the regression coefficients is different from zero 26.11.2019 33
  34. 34. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 08/10 Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course QR - the error or residual sum of squares • QR is the test of whether there is more residual variation than we would expect IF the model “fits” the data • We compare QR to a chi-square distribution with k - p – 1 degrees of freedom with k = # of studies/effect sizes, and p = # of predictors in model • If QR is significant, then we have more error or residual variation to explain, or that is not accounted for by the variables we have in the model 26.11.2019 34
  35. 35. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 08/10 Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course Significance of regression coefficients • In a standard regression analysist-tests on the printout tell which regression coefficients are significantly different from zero • Those significant regression coefficients indicate that these predictors are associated with the outcome 26.11.2019 35
  36. 36. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 08/10 Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course Results of meta-regression 26.11.2019 36
  37. 37. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 08/10 Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course Plot of points & regression line – bubble plot 26.11.2019 37
  38. 38. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 08/10 Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course Example 26.11.2019 38
  39. 39. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 08/10 Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course What to report in a random effects meta- regression? • The software and/or method used to compute the results • The method used to compute the random effects variance component • The goodness of fit tests: QModel , and QResidual • The regression coefficients and their test of significance 26.11.2019 39
  40. 40. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 08/10 Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course Software • RevMan does not have the capacity for meta-regression • CMA only allows one predictor in the meta-regression • STATA, R programs more flexible 26.11.2019 40
  41. 41. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 08/10 Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course Take home messages • Moderator analysis is a way of understanding heterogeneity in our meta-analysis • Categorical moderators are analyzed individually by subgroups like ANOVA model • Multiple and/or continuous variables are analyzed by meta- regression 26.11.2019 41
  42. 42. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 08/10 Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course THANK YOU Kindly email your queries to sarizwan1986@outlook.com 26.11.2019 42

×