Call girls Service Phullen / 9332606886 Genuine Call girls with real Photos a...
Moderator analysis in meta-analysis
1. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 08/10
Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course
Moderator analysis in meta-analysis
Dr. S. A. Rizwan M.D.,
Public Health Specialist & Lecturer,
Saudi Board of Preventive Medicine – Riyadh,
Ministry of Health, KSA
With thanks to Ryan Williams
26.11.2019 1
2. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 08/10
Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course
Moderator analyses in meta-analysis
• To test our hypotheses about whether variation among studies in effect
size is associated with differences in study methods or participants
• A priori ideas, incorporating these characteristics of studies into our
coding forms
• Two major forms:
– categorical models analogous to ANOVA
– meta-regression
26.11.2019 2
3. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 08/10
Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course
Assumptions
• Focus on random effects models
• Based on the random effects variance component
26.11.2019 3
4. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 08/10
Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course
Categorical moderators
• When the moderator variable is categorical, estimate ANOVA analogue
• Typically, comparing the mean effect sizes for 2 or more groups
• For example, we will look at a meta-analysis where we compare the mean
effect size for studies published in three different sources: journals,
dissertations, and unpublished studies
26.11.2019 4
5. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 08/10
Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course
Categorical moderators
• With a one-way random effects ANOVA model, recall that we will compute
– A mean effect size and standard error for each group, and then test whether these
means are significantly different from one another
• The mean effect size and standard error require an estimate of the
variance component
• Options: Will we assume that each group has the same variance
component? Or, will we assume that each group has its own variance
component?
26.11.2019 5
6. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 08/10
Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course
Assumptions for separate subgroups
• We believe that the variation among studies is different
between groups.
• For example,
– if we are testing out an intervention and we have studies that use either a
low-income and a high-income group of students, we might believe that there
will be more variation in effectiveness among studies that have mostly low-
income participants
– the effectiveness of an intervention for juvenile delinquents will vary more for
the group that had a prior arrest than for those that do not have a prior arrest
26.11.2019 6
7. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 08/10
Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course
Assumptions for pooled estimate
• We believe that the variation among effect sizes are the same no matter
the group.
• For an intervention review, we may assume that the variation among
studies does not differ within the groups of interest
• Caveat
– We might have to use a pooled estimate if we have small sample sizes within subgroups
– We need at least 5 cases (in general) to be able to estimate a separate variance
component for each subgroup
26.11.2019 7
8. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 08/10
Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course
Flowchart for choosing variance
26.11.2019 8
9. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 08/10
Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course
Steps for a random effects ANOVA
• Make a decision about the use of a pooled or a separate
estimate of the variance component
• Compute the group mean effect sizes, and their standard
errors
• Compare the group mean effect sizes to see if they are
statistically different from one another
26.11.2019 9
10. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 08/10
Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course
Eagly, Johannesen-Schmidt & van Engen (2003)
• This synthesis examines the standardized mean difference estimated in
primary studies for the difference between men and women in their use
of transformational leadership.
• Transformational leadership involves “establishing onself as a role model
by gaining the trust and confidence of followers” (Eagly et al. 2003, p.
570).
• The sample data is a subset of the studies in the full meta- analysis, a set
of 24 studies that compare men and women in their use of
transformational leadership
• Positive effect sizes indicate males score higher than females
26.11.2019 10
11. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 08/10
Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course
Sub-groups
• Example
26.11.2019 11
12. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 08/10
Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course
Sub-groups
26.11.2019 12
13. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 08/10
Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course
Sub-groups
26.11.2019 13
14. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 08/10
Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course
Summary of results – separate variance
estimates for each group
• Journals have a significant
variance component, and
the mean is not different
from zero by magnitude
• Dissertations and
unpublished studies both
have a non-significant
variance component, but
both have medium to large
effect size
Group k Mean 95% CI τ2 p
Journals 13 -0.05 [-0.24, 0.12] 0.09 <0.001
Dissertations 7 -0.47 [-0.69,-0.26] 0.02 0.22
Unpublished 4 -0.16 [-0.30,-0.03] 0.00 0.87
TOTAL 24 -0.16 [-0.29, -0.03] 0.08 <0.001
26.11.2019 14
15. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 08/10
Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course
Summary of results – separate variance
estimates for each group
• The test of the variance
component as different
from zero is exactly the
fixed effects test of
homogeneity.
• To get this test, we
compute the test of
homogeneity within each
group of studies.
Group k Mean 95% CI τ2 p
Journals 13 -0.05 [-0.24, 0.12] 0.09 <0.001
Dissertations 7 -0.47 [-0.69,-0.26] 0.02 0.22
Unpublished 4 -0.16 [-0.30,-0.03] 0.00 0.87
TOTAL 24 -0.16 [-0.29, -0.03] 0.08 <0.001
26.11.2019 15
16. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 08/10
Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course
Test of between group differences
• To test between group differences in a random effects model, we test
whether the variance component for the variation among the random
effects means is equal to zero
• There are several ways to obtain this value
• We will use a test of homogeneity of the three means – we will treat the
three group means as a meta-analysis
26.11.2019 16
17. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 08/10
Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course
Test of between group differences
• We will compute a test of homogeneity using our three
means as if this is a meta-analysis
• We will use the means and their estimated variances to
compute the sums we need to compute the homogeneity test
26.11.2019 17
18. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 08/10
Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course
Computation of Q between groups
Source Mean Var Wt Wt*Mean Wt*Mean2
Journals -0.05 0.008 122.53 -6.13 0.031
Dissertations -0.47 0.012 86.46 -40.64 19.10
Unpublished -0.16 0.005 211.32 -33.82 5.41
SUM 420.31 -80.59 24.54
Q = 24.54 - (-80.59 * -80.59)
420.31
= 9.09 Compare 9.09 to a chi-square
with df=3-1=2. p-value is 0.011
26.11.2019 18
19. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 08/10
Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course
Using the same variance for subgroups
26.11.2019 19
20. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 08/10
Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course
Notes about the CMA forest plot
• Like RevMan, the confidence intervals around each study are the fixed
effects confidence intervals (they use the within-study fixed effects
variance)
• The group means are the random effects means computed using random
effects weights. Their confidence intervals are also use random effects.
• In this example, we are using the same variance component for all groups
26.11.2019 20
21. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 08/10
Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course
Summary from CMA
26.11.2019 21
22. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 08/10
Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course
Notes about CMA results
• We assumed that the variance component was 0.08 for all
studies
• Compared to our separate variance estimates, this value is
smaller than the separate variance estimate for journals, but
larger than the separate estimate for dissertations and
unpublished articles
26.11.2019 22
23. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 08/10
Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course
Another subgroup
example
26.11.2019 23
24. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 08/10
Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course
Reporting random effects categorical
analysis
• The assumption made about the random effects variance: separate
estimate for each group, or the same estimate for all groups.
• Rationale for the choice of variance component
• The random effects mean and CI
• The value of the variance components (or variance component)
• The test of the between-group differences, and its significance
26.11.2019 24
25. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 08/10
Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course
What is meta-regression?
• Meta-regression is a statistical technique used in a meta-
analysis to examine how characteristics of studies are related
to variation in effect sizes across studies
• Meta-regression is analogous to regression analysis but using
effect sizes as our outcomes, and information extracted from
studies as moderators/predictors
26.11.2019 25
26. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 08/10
Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course
What is meta-regression?
• When we have a heterogeneous set of effect sizes, we can use statistical
techniques to examine the association among characteristics of the study
and variation among effect sizes
• We have a plan for these analyses a priori – based on our understanding
of the literature, and a logic model or framework
• Meta-regression used when we have more than one predictor or
moderator (either continuous or categorical)
26.11.2019 26
27. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 08/10
Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course
Form of the meta-regression model
• ! "↓# = $↓0 + $↓1 %↓#1 +…+ $↓& %↓#&
• where ! "↓# is our generic effect size estimate from study i,
%↓#1 , %↓#2 , … %↓#& are the moderator variable values for
study i, and $↓0 , $↓1 , … $↓& are the unknown regression
coefficients.
26.11.2019 27
28. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 08/10
Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course
Recall that the variance of the effect size
• Depends on the sample size for all types of effects we have talked about
• Thus, the precision of each study’s effect size depends on sample size
• This is different from our typical application of regression where we
assume every person has the same “weight”
• Thus, we need to use weighted least squares regression to account for the
fact that the precision of each effect size depends on sample size
26.11.2019 28
29. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 08/10
Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course
Random effects meta-regression
• As in the categorical analysis discussion, we will need an estimate of the
random effects variance for our studies that will be used as our weights in
the regression
• There are many ways to compute the variance component in a random
effects meta-regression
• For now, let’s assume a single variance component for all studies.
26.11.2019 29
30. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 08/10
Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course
Test for model fit in meta-regression
• As in a standard regression model, we can use the regression
ANOVA table for diagnostics about the fit of a meta-
regression
• Recall that in a standard regression analysis, we would get
the following regression ANOVA table:
26.11.2019 30
31. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 08/10
Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course
ANOVA table for normal regression
26.11.2019 31
32. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 08/10
Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course
Test of model fit in meta-regression
• In meta-regression, we use the ANOVA table to get two different Q
statistics:
• QM – model sum of squares, compare to chi-square distribution with p – 1
df (p is number of predictors in the model)
• QR – residual sum of squares, compare to chi-square distribution with k -
p – 1 df (k is the number of studies)
26.11.2019 32
33. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 08/10
Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course
QM - the model sum of squares
• Qmodel is the test of whether at least one of the regression coefficients (not
including the intercept) is different from zero
• We compare QM to a chi-square distribution with p–1 degrees of freedom
with p = # of predictors in model
• If QM is significant, then at least one of the regression coefficients is
different from zero
26.11.2019 33
34. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 08/10
Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course
QR - the error or residual sum of squares
• QR is the test of whether there is more residual variation than we would
expect IF the model “fits” the data
• We compare QR to a chi-square distribution with k - p – 1 degrees of
freedom with k = # of studies/effect sizes, and p = # of predictors in model
• If QR is significant, then we have more error or residual variation to
explain, or that is not accounted for by the variables we have in the model
26.11.2019 34
35. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 08/10
Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course
Significance of regression coefficients
• In a standard regression analysist-tests on the printout tell
which regression coefficients are significantly different from
zero
• Those significant regression coefficients indicate that these
predictors are associated with the outcome
26.11.2019 35
36. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 08/10
Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course
Results of meta-regression
26.11.2019 36
37. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 08/10
Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course
Plot of points & regression line – bubble plot
26.11.2019 37
38. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 08/10
Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course
Example
26.11.2019 38
39. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 08/10
Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course
What to report in a random effects meta-
regression?
• The software and/or method used to compute the results
• The method used to compute the random effects variance
component
• The goodness of fit tests: QModel , and QResidual
• The regression coefficients and their test of significance
26.11.2019 39
40. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 08/10
Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course
Software
• RevMan does not have the capacity for meta-regression
• CMA only allows one predictor in the meta-regression
• STATA, R programs more flexible
26.11.2019 40
41. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 08/10
Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course
Take home messages
• Moderator analysis is a way of understanding heterogeneity in
our meta-analysis
• Categorical moderators are analyzed individually by subgroups
like ANOVA model
• Multiple and/or continuous variables are analyzed by meta-
regression
26.11.2019 41
42. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 08/10
Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course
THANK YOU
Kindly email your queries to sarizwan1986@outlook.com
26.11.2019 42