Fixed-effect and random-effects models in meta-analysis

Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 06/10
Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course
Fixed-effect vs. Random-effects
models in meta-analysis
Dr. S. A. Rizwan M.D.,
Public Health Specialist & Lecturer,
Saudi Board of Preventive Medicine – Riyadh,
Ministry of Health, KSA
25.11.2019 1
With thanks to Michael Borenstein

Outline
• Provide a description of fixed and of random effects models
• Outline the underlying assumptions of these two models in order to
clarify the choices a reviewer has in a meta-analysis
• Discuss how to estimate key parameters in the model
25.11.2019 2

Our choice between the two depends on
• Our assumption about how the effect sizes vary in our meta-analysis
• The two models are based on different assumptions about the nature
of the variation among effect sizes in our research synthesis
25.11.2019 3

Sampling assumptions
• Fixed-effect
• Sampling takes place at one level only
• Any between-study variance will be ignored when assigning weights
• Random-effects
• Sampling takes place at two levels
• Any between-study variance will be used when assigning weights
25.11.2019 4

Fixed-effect
• When there is reason to believe that all the studies are functionally identical
• When our goal is to compute the common effect size, for the studies in the
analysis
• Example of drug company has run five studies to assess the effect of a drug
• Note that the effect size from each study estimate a single common mean –
the fixed-effect
• We know that each study will give us a different effect size, but each effect
size is an estimate of a common mean, designated in the prior picture as θ
25.11.2019 5

Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course 25.11.2019 6

Random-effects
• We assume two components of variation:
• – Sampling variation as in our fixed-effect model assumption
• – Random variation because the effect sizes themselves are sampled from
population of effect sizes
• In a random effects model, we know that our effect sizes will differ
because they are sampled from an unknown distribution
• Our goal in the analysis will be to estimate the mean and variance of
the underlying population of effect sizes
• All the studies are functionally equivalent
• Example of studies culled from publications
25.11.2019 7

Random-effects
• We see in the picture that each distribution has its own mean that is
sampled from the underlying population distribution of effect sizes
• That underlying population distribution also has its own variance, τ2,
commonly called the variance component
• Thus, each effect size has two components of variation, one due to
sampling error, and one from the underlying distribution
25.11.2019 8

Definition of combined effect
• Fixed effect model
• There is one true effect
• Summary effect is estimate of this value
• Random effects model
• There is a distribution of effects
• Summary effect is mean of distribution
25.11.2019 10

How weights are assigned?
25.11.2019 11

W = 1/ (V1)
Fixed-effects model
25.11.2019 12

Random-effects model
25.11.2019 13
1W = 1/ (V +T 2
)

190
How weights shift?
• If within-study variance only, W=1/V
• If between-study variance only, W=1/T2
• If both, W=1/(V+T2)
25.11.2019 14

Random vs. Fixed
25.11.2019 17

Random vs. Fixed
25.11.2019 18
RE weights are more balanced

Random vs. Fixed
25.11.2019 19
RE confidence interval is wider

Large study has less impact under RE
25.11.2019 20

Small study has more impact under RE
25.11.2019 21

FE
25.11.2019 22

Summary effect in Fixed model
25.11.2019 23
244.215
101.171
= 0.414FE

Variance of summary effect in fixed model
25.11.2019 24
1
244.215
= 0.004FE

RE
25.11.2019 25

Summary effect in random model
25.11.2019 26
90.284
32.342
= 0.358RE

Variance of summary effect in random model
25.11.2019 27
1
90.284
= 0.011RE

Why does it matter?
• One matches the sampling
• One does not match the sampling
• Wrong model yields incorrect weights
• Estimate of mean is wrong
• Estimate of CI is wrong
• MUST choose based on sampling model
• The meaning of the ES is different
• Relative weights are closer under RE (effect size will shift)
• Absolute weights are smaller under RE (CI will become wider)
• p-value will change (less significant in long run but can go either way)
25.11.2019 28

Test of null
25.11.2019 29
M
M
Z =
SE
M*
M*
Z* =
SE
Fixed
Random
Two
sources
of error
One
source
of error

Implications of the choice
• If you expect that the effect size from each study arises from a different
population then use random effects
• Random effects model is more likely to give non-significant estimates
• Recall that all our analyses in a meta-analysis are weighted by the inverse of
the variance of the effect size, i.e., by the precision of the effect size estimate
• Because we have more variation assumed in a random effects model, our
weights for each study will be more equal to one another
• In other words, in a fixed effect model, we will more heavily weight larger
studies. In a random effects model, the larger studies will not be weighted as
heavily
25.11.2019 30

Question!
• Suppose we had four studies, each with
• N =1,000,000, and a true (mean) effect size of 50.
• Under the two models,
• What would the forest plot look like?
• What would the diamond look like?
25.11.2019 31

Which model should we use?
• Base decision on the model that matched the way the data were
collected
• Not on test of homogeneity
25.11.2019 34

What you may hear
• Fixed-effect is simple model
• Random-effects is more complicated
25.11.2019 35

Actually
• Fixed-effect is more restricted model
• Random-effects makes less assumptions
25.11.2019 36

An alternate view
• Random-effects model only makes sense if we have a clear picture of
the sampling frame
• Otherwise, we should report the mean and CI for the studies in our
sample without attempt to generalize to a larger universe
25.11.2019 37

Goals in a fixed effect model
• In a fixed effect model, we will be most interested in estimating our
common effect, θ, and its standard error
• We will also want to know if there is heterogeneity present by
computing Q
• HOWEVER, WE WILL NOT decide on the basis of a significant Q that
we should really do a random effects model
• WE WILL make a decision about fixed effect versus random effects
models because of our substantive knowledge of the area of the
systematic review
25.11.2019 38

Goals in a random effects model
• We also want to estimate a mean effect size, but now this is the mean
effect size from the underlying population, μ
• We also want to estimate the variance of the underlying effect size, τ2
• We will test heterogeneity by testing whether τ2 is different from 0.
• Biggest difficulty: how to estimate τ2
25.11.2019 39

Estimating the variance component, τ2
• There are two main methods for computing τ2
• Variously called the method of moments, the DerSimonian/Laird estimate
• Restricted maximum likelihood
• The method of moments estimator is easy to compute and is based
on the value of Q, the homogeneity statistic
• Restricted maximum likelihood requires an iterative solution
25.11.2019 40

Method of moments estimator
25.11.2019 41

Restricted maximum likelihood estimator
• Many statisticians do not like the method of moments estimator
• Can estimate τ2 using HLM, SAS Proc Mixed, or R
25.11.2019 42

Differences between the two
models
25.11.2019 43

ES assumptions
Fixed effect model
• We assume that the treatment
effect is the same in all trials.
• We use only the sampling
variation within the trials.
Random effects model
• We assume that the treatment
effect is the not same in all trials.
• The trials are a sample from a
population of possible of trials
where the treatment effect varies.
• We use the sampling variation
within the trials and the sampling
variation between trials.
25.11.2019 44

Sampling assumptions
Fixed effect model
• If the treatment effect is the
same in all trials, it is more
powerful and easier.
• No assumption about
representativeness.
• Less powerful because P values are
larger and confidence intervals are
wider.
• The trials are a sample from a
population of possible of trials
where the treatment effect varies.
• They must be a representative or
random sample.
• Very strong assumption.
25.11.2019 45

Variance
Fixed effect model
• Variance of treatment effect in
trial = standard error squared.
• Weight = 1/variance = 1/SE2
• Variance of treatment effect in trial
= standard error squared plus inter-
trial variance
• Weight = 1/variance.
• 1
= --------------------------------
SE2 + inter-trial variance
• Inter-trial variance has degrees of
freedom given by number of trials
minus one.
• Typically small.
25.11.2019 46

When heterogeneity exists
Fixed effect model
• When heterogeneity exists we
get:
• a pooled estimate which may give
too much weight to large studies,
• a confidence interval which is too
narrow,
• a P-value which is too small.
• When heterogeneity exists we
get:
• possibly a different pooled
estimate with a different
interpretation,
• a wider confidence interval,
• a larger P-value.
25.11.2019 47

When heterogeneity doesn’t exist
Fixed effect model
• When heterogeneity does not
exists:
• a pooled estimate which is
correct,
• a confidence interval which is
correct,
• a P-value which is correct.
• When heterogeneity does not
exist:
• a pooled estimate which is
correct,
• a confidence interval which is too
wide,
• a P-value which is too large.
25.11.2019 48

Thank you
Kindly email your queries to sarizwan1986@outlook.com
25.11.2019 50

Fixed-effect and random-effects models in meta-analysis

More Related Content

What's hot

Similar to Fixed-effect and random-effects models in meta-analysis

More from Rizwan S A

Recently uploaded

Fixed-effect and random-effects models in meta-analysis