1.
META ANALYSIS - AN OVERVIEW
Tulasi Raman P

2.
DEFINITION
 Meta-analysis is a quantitative approach for
systematically combining results of previous
research to arrive at conclusions about the body
of research.
 Quantitative: numbers
 Systematic: methodical
 Combining: putting together
 Previous research: what's already done
 Conclusions: new knowledge
 A study collects data from individual subjects
 (such as 100 subjects = 100 “data points”)
 A meta-analysis collects data from individual studies
 (such as 100 studies = 100 “data points”)

3.
HISTORY
 Karl Pearson is probably the first
medical researcher to use formal
techniques to combine data from
different studies (1904):
 He synthesized data from several studies
on efficacy of typhoid vaccination
 His rationale for pooling data:
 “Many of the groups… are far too small to
allow of any definite opinion being formed
at all, having regard to the size of the
probable error involved.”

4.
THE TERM META-ANALYSIS
 1952: Hans J. Eysenck concluded that there were
no favorable effects of psychotherapy, starting a
raging debate which 25 years of evaluation
research and hundreds of studies failed to resolve
 1978: To proved Eysenck wrong, Gene V. Glass
statistically aggregated the findings of 375
psychotherapy outcome studies
 Glass called the method “meta-analysis”

5.
META ANALYSIS IN CONTEXT
Traditional Systematic Meta-
review Review analysis
Author An individual A team A team
Search Individual Based on a Based on a
strategy protocol protocol
Summary Author‟s Can be Summary
Conclusion judgement qualitative statistical
techniques

6.
META-ANALYSES INDEXED IN PUBMED
2500
2000
1500
1000
500
0
1992 1994 1996 1998 2000

7.
FUNCTION OF META-ANALYSIS
1. Identify heterogeneity in effects among multiple
studies and, where appropriate, provide summary
measure
2. Increase statistical power and precision to detect
an effect
3. Develop, refine and test hypothesis
4. Reduce the subjectivity of study comparisons by
using systematic and explicit comparison
procedure
5. Identify data gap in the knowledge base and
suggest direction for future research
6. Calculate sample size for future studies

8.
COULD WE JUST ADD THE DATA FROM ALL
THE TRIALS TOGETHER?
 One approach to combining trials would be to add
all the treatment groups together, add all the control
groups together, and compare the totals
 This is wrong for several reasons, and it can give
the wrong answer

9.
If we just add up the columns we get From a meta-analysis, we get
34.3% vs 32.5% , a RR of 1.06, RR=0.96 , a lower death rate
a higher death rate in the steroids group in the steroids group

10.
PROBLEMS WITH SIMPLE ADDITION OF
STUDIES
 Breaks the power of randomization
 Imbalances within trials introduce bias

11.
*
#
In effect we are comparing
this experimental group directly
with this control group – this is
not a randomized comparison

12.
STEPS IN META-ANALYSIS
1. Define the research question and specific hypotheses
2. Define the criteria for including and excluding studies
3. Locate research studies
4. Determine which studies are eligible for inclusion
5. Classify and code important study characteristics
(e.g., sample size; length of follow-up; definition of outcome;
drug brand and dose)
6. Select or translate results from each study using a common
metric
7. Aggregate findings across studies, generating weighted
pooled estimates of effect size.
8. Evaluate the statistical homogeneity of pooled studies
9. Perform sensitivity analyses to assess the impact of
excluding or down-weighting unpublished studies, studies of
lower quality, out-of-date studies, etc.

13.
PUBLICATION BIAS
 Studies with significant results are more likely
 to be published
 to be published in English
 to be cited by others
 to produce multiple publications
 Including only published studies can introduce
publication bias
 Most reviews do not look for publication bias

14.
APPROACHES TO PUBLICATION BIAS
 Fail-safe N
 Publications are filled with the 5% of studies with Type I
error but file drawers are filled with the 95% with non-
significant effects
 Funnel plots
 Trim and fill method
 Small studies are removed from funnel plot until it is symmetric
 Then replace the small studies and balance them with studies
on the opposite side of the funnel
 Statistical analogues of funnel plot
 Egger test

15.
FUNNEL PLOT

16.
ASSESSING STUDY QUALITY

17.
QUALITY SCORE
 Some meta-analysts score the quality of a study.
1. Examples (scored yes=1, no=0):
2. Published in a peer-reviewed journal?
3. Experienced researchers?
4. Research funded by impartial agency?
5. Study performed by impartial researchers?
6. Subjects selected randomly from a population?
7. Subjects assigned randomly to treatments?
8. High proportion of subjects entered and/or finished the
study?
9. Subjects blind to treatment?
10. Data gatherers blind to treatment?
11. Analysis performed blind?

18.
AVERAGING STUDIES
 Starting with the summary statistic for each
study, how should we combine these?
 A simple average gives each study equal weight
 This seems intuitively wrong
 Some studies are more likely to give an answer
closer to the „true‟ effect than others

19.
WEIGHTING STUDIES
 More weight to the studies which give us more
information
 More participants
 More events
 Lower variance
 Calculated using the effect estimate and its
variance
 Inverse-variance method:
1 1
weight
variance of estimate SE2
sum of (estimate weight)
pooled estimate
sum of weights

20.
FOR EXAMPLE
Headache Caffeine Decaf Weight
Amore-Coffea 2000 2/31 10/34 6.6%
Deliciozza 2004 10/40 9/40 21.9%
Mama-Kaffa 1999 12/53 9/61 22.2%
Morrocona 1998 3/15 1/17 2.9%
Norscafe 1998 19/68 9/64 26.4%
Oohlahlazza 1998 4/35 2/37 5.1%
Piazza-Allerta 2003 8/35 6/37 14.9%

21.
WHAT IS HETEROGENEITY
 Heterogeneity is variation between the studies‟
results

22.
CAUSES OF HETEROGENEITY
1. Differences between studies with respect to:
2. Patients: diagnosis, in- and exclusion criteria, etc.
3. Interventions: type, dose, duration, etc.
4. Outcomes: type, scale, cut-off points, duration of
follow-up, etc.
5. Quality and methodology: randomised or
not, allocation concealment, blinding, etc.

23.
HOW TO LOOK FOR HETEROGENEITY?
 Visually
 Forest plot: do confidence intervals of studies overlap
with each other and the summary effect?
 Statistically
 Chi-square test for heterogeneity (Mantel-Haenszel test
or Cochran Q test)
 Tests whether the individual effects are farther away from the
common effect, beyond what is expected by chance
 Has poor power
 P-value < 0.10 indicates significant heterogeneity

24.
HOW TO DEAL WITH HETEROGENEITY
1. Do not pool at all
2. Ignore heterogeneity: use fixed effect model
3. Allow for heterogeneity: use random effects model
4. Explore heterogeneity

25.
FIXED EFFECT MODEL
 Fixed effects model assumes that the true effect of
treatment is the same value in each study (fixed);
the differences between studies is solely due to
random error
 Specific methods for combining odds ratio
1. Mantel- Haenszel method
2. Peto‟s method
3. Maximum-Likelihood techniques
4. Exact methods of interval estimation

26.
RANDOM EFFECT MODEL
 In random effects models, the treatment effects for the
individual studies are assumed to vary around some
overall average treatment effect
 Allows for random error plus inter-study variability
 Results in wider confidence intervals (conservative)
 Studies tend to be weighted more equally (relatively more
weight is given to smaller studies)
 There are five approach for this model:
1. Weighted least squares
2. Un-weighted least squares
3. Maximum likelihood
4. Restricted Maximum likelihood
5. Exact approach to random effects of binary data.

27.
SENSITIVITY ANALYSIS
 Several features of the meta-analysis can be
altered to gauge the robustness of the results:
 Modifying the inclusion criteria
 Including and excluding questionable studies
 Including and excluding unpublished studies
 Weighting the analysis by study quality
 Trying different ways to impute missing data
 Removing each study, one by one, to see the change

28.
FOREST PLOT
 The graphical display of results from individual
studies on a common scale is a “Forest plot”.
 In the forest plot each study is represented by a
black square and a horizontal line (CI:95%).The
area of the black square reflects the weight of the
study in the meta-analysis.
 A logarithmic scale should be used for plotting the
Relative Risk.

29.
FOREST PLOTS
Headache at 24 hours
HEADINGS EXPLAIN THE COMPARISON

30.
FOREST PLOTS
Headache at 24 hours
LIST OF INCLUDED STUDIES

31.
FOREST PLOTS
Headache at 24 hours
RAW DATA FOR EACH STUDY

32.
FOREST PLOTS
Headache at 24 hours
TOTAL DATA FOR ALL STUDIES

33.
FOREST PLOTS
Headache at 24 hours
WEIGHT GIVEN TO EACH STUDY

34.
FOREST PLOTS
Headache at 24 hours
EFFECT ESTIMATE FOR EACH STUDY, WITH CI

35.
FOREST PLOTS
Headache at 24 hours
EFFECT ESTIMATE FOR EACH STUDY, WITH CI

36.
FOREST PLOTS
Headache at 24 hours
SCALE AND DIRECTION OF BENEFIT

37.
FOREST PLOTS
Headache at 24 hours
POOLED EFFECT ESTIMATE FOR ALL
STUDIES, WITH CI

38.
INTERPRETING CONFIDENCE INTERVALS
 always present estimate with a confidence interval
 precision
• point estimate is the best guess of the effect
• CI expresses uncertainty – range of values we can be reasonably
sure includes the true effect
 significance
• if the CI includes the null value
 rarely means evidence of no effect
 effect cannot be confirmed or refuted by the available evidence
• consider what level of change is clinically important

39.
LIMITATIONS OF META-ANALYSIS
 A meta-analysis reflects only what's published or
searchable.
 It's focused on mean effects and differences
between studies. But what really matters is effects
on individuals. (Aggression bias)
 Relation between group rates or and means may
not resemble the relation between individual values
of exposure and outcome.
 This phenomenon is known as aggregation bias or
ecologic bias.

40.
WHAT IS AN IPD META-ANALYSIS?
 Involves the central collection, checking and
analysis of updated individual patient data
 Include all properly randomised trials, published
and unpublished
 Include all patients in an intention-to-treat analysis
 Individual patient data used
 Analysis stratified by trial
 IPD does not mean that all patients are combined
into a single mega trial
 The major advantage of a IPD over an MA is the
use of individual-based rather than group-based
data.

41.
GUIDELINES FOR REPORTING META-ANALYSIS
 The QUORUM Statement:
 Quality of Reporting of Meta-analyses – For clinical
Randomized Controlled Trials (RCT‟s)
 MOOSE guidelines:
 Meta-analysis Of Observational Studies in Epidemiology

42.
META-ANALYSIS SOFTWARE
 Free  Commercial
 RevMan [Review  Comprehensive Meta-
Manager] analysis
 Meta-Analyst  Meta-Win
 Epi Meta  WEasy MA
 Easy MA
 Meta-Test  General stats packages
 Meta-Stat  Stata
 SAS
 S-Plus

43.
THANK YOU