This document outlines key concepts and methodologies related to performing meta-analyses, including types of models (fixed and random effects), methods for combining study effects, and approaches to addressing heterogeneity and publication bias. It discusses the importance of study quality assessment and details reporting guidelines such as PRISMA and MOOSE. The document emphasizes the use of various statistical tests and methods for exploring and evaluating the validity of meta-analytical results.

1. Meta-analysis II
Adrian V. Hernandez, M.D., Ph.D.
Assistant Professor of Medicine
Quantitative Health Sciences
October 21, 2010

2. OUTLINE SECOND PART
SECOND PART: 50 MINUTES
• Analysis (models, methods, heterogeneity, publication
bias, quality, subgroup analysis)
• Reporting of meta-analysis (PRISMA, MOOSE guidelines)
Meta-analysis II l
October 10, 2012 l

3. MODELS
• Fixed effects
-Assumption: Effect is the same across
studies and differences due to chance
-Common effect unknown
-Objective: Estimation of common effect with
more precision
-Pool studies using weights ↔ sample size
Meta-analysis II l
October 10, 2012 l

4. MODELS (2)
• Random effects
-Assumption: Effect is different across
studies and there is an average effect
-Average effect unknown
-Objective: Estimation of average effect of
studies
-Pool studies using similar weights
Meta-analysis II l
October 10, 2012 l

5. When are effects similar between models?
 Large effect
 Balanced arms
 Study sizes similar
 Low heterogeneity of effects
HF Fixed MH 1.59 (1.34-1.89)
Random 1.56 (1.32-1.86)
Edema Fixed MH 2.04 (1.85-2.26)
Random 2.41 (1.91-3.04)
Hernandez AV et al. 2010 (submitted)
Meta-analysis II l
October 10, 2012 l

6. METHODS TO COMBINE STUDY EFFECTS
• Inverse Variance (IV)
Common, flexible
Binary/continuous data
Log OR, Log RR, log HR, standardized ratios
Meta-analysis II l
October 10, 2012 l

7. METHODS TO COMBINE STUDY EFFECTS (2)
• Mantel-Haenzel (MH)
Binary outcomes only
Special cases: sparse outcomes, unbalanced
arms
Correction for zeros in arms
Other situations: Effects similar to IV
Meta-analysis II l
October 10, 2012 l

8. METHODS TO COMBINE STUDY EFFECTS (3)
• Peto
Binary outcomes only, a few outcomes
Small effect
Balanced arms
No correction for zeros in arms
HF Fixed MH 1.59 (1.34-1.89)
Peto 1.59 (1.34-1.88)
Meta-analysis II l
October 10, 2012 l

9. Meta-analysis II l
October 10, 2012 l

10. Meta-analysis II l
October 10, 2012 l

11. Meta-analysis II l
October 10, 2012 l

12. METHODS TO COMBINE STUDY EFFECTS (4)
• DerSimonian and Laird
Any type of effect measures
Random model
Larger CI of the pooled effect
More weight to smaller studies
Meta-analysis II l
October 10, 2012 l

13. HETEROGENEITY
• Degree of dissimilarity in effects of individual studies
• Why?
Participants
Interventions
Co-interventions
Outcomes
Biases of studies (according to hierarchy), etc.
Meta-analysis II l
October 10, 2012 l

14. Meta-analysis II l
October 10, 2012 l

15. HETEROGENEITY: Pseudo-tests
• Eyeballing forest plots
• Point estimates
• Significance level
• Confidence intervals
Meta-analysis II l
October 10, 2012 l

16. HETEROGENEITY: Test it!
• Cochrane Q test (Х2, p)
AND
• I2: Amount of heterogeneity (0-100%)
It needs 95% CI (<25% Low; 25-50% Mod; >50 High)
(Ioannidis JP. J Eval Clin Pract 2008:14:951-7)
 Bad news for both: Low power
 Bad news for I2: No software for CIs
Meta-analysis II l
October 10, 2012 l

17. Meta-analysis II l
October 10, 2012 l

18. HETEROGENEITY: Address it!
• Check data again
• Do not perform a meta-analysis
• Explore vs. ignore
• Use random-effects models
• Change effect measure (e.g. MD →SMD)
• Exclude studies
Meta-analysis II l
October 10, 2012 l

19. EXPLORING HETEROGENEITY
• Subgroup analysis
 Exploratory only
 Low power to detect significant effects
 Better pre-specify in protocol
 Generates hypotheses
→ Editors and reviewers like subgroup analysis
Meta-analysis II l
October 10, 2012 l

20. EXPLORING HETEROGENEITY (2)
• How to perform subgroup analysis?
→ By baseline characteristics (e.g. age, gender)
→ By quality
→ By sample size
→ By follow-up time
Meta-analysis II l
October 10, 2012 l

21. Meta-analysis II l
October 10, 2012 l

22. Meta-analysis II l
October 10, 2012 l

23. Use of thiazolidinediones and risk of heart failure and
peripheral edema in patients at high risk of diabetes and
type 2 diabetes:
A systematic review and meta-analysis of placebo-
controlled randomized trials
Follow-up OR (95% CI) RR (95% CI)
≥12 months
MH 1.57 [1.31-1.87] 1.50 [1.28-1.76]
Random 1.67 [1.16-2.40] 1.66 [1.10-2.50]
<12 months
MH 2.71 [0.94-7.79] 2.68 [0.93-7.67]
Random 2.56 [0.88-7.44] 2.52 [0.88-7.25]
Hernandez AV et al. 2010 (Submitted)
Meta-analysis II l
October 10, 2012 l

24. EXPLORING HETEROGENEITY (3)
• Meta-regression
→Evaluates factors that explain heterogeneity of
effects
→Bad news: Low power
Lack of data
Meta-analysis II l
October 10, 2012 l

25. Meta-analysis II l
October 10, 2012 l

26. EVALUATION OF PUBLICATION BIAS
Funnel Plot: Size effect vs. SE/SS ; Asymmetry?
De Luca G et al. Am Heart J 2007; 153:343-53
Meta-analysis II l
October 10, 2012 l

27. EVALUATION OF PUBLICATION BIAS (2)
Asymmetrical: Only due to publication bias?
Dentali F et al. Ann Intern Med 2007; 146: 278-88
Meta-analysis II l
October 10, 2012 l

28. EVALUATION OF PUBLICATION BIAS (3)
Pseudo-test: Visual inspection of funnel plot
Test!: Begg-Mazumdar test
Asymmetry regression test
Kendall test, etc
→ Bad news: All have low power
Meta-analysis II l
October 10, 2012 l

29. EVALUATION OF PUBLICATION BIAS (4)
In 95% of MAs, the use of asymmetry regression
tests is inappropriate:
→ Highly heterogeneous (I2 > 50%)
→ <10 studies
Meta-analysis II l
October 10, 2012 l

30. QUALITY OF STUDIES: Observational
• Design: Prospective cohort
Retrospective cohort
Case-control
• Quality of measurement of factors
• Patient enrollment (consecutive vs no)
Meta-analysis II l
October 10, 2012 l

31. QUALITY OF STUDIES: RCTs
• Difficult to define
Design/conduct/analysis?
Clinical relevance?
Reporting?
• Several scales: 39
Egger M et al. Systematic reviews in health care. Meta-analysis in
context. 2nd Edition, BMJ London 2001. pp87-108.
Meta-analysis II l
October 10, 2012 l

32. QUALITY OF STUDIES: RCTs (2)
TC Chalmers et al. Control Clin Trials 1981; 2: 31-49
30 items, complex
- Internal validity (R, Blinding, Attrition,
stat analysis)
- External validity
- Data presentation/Organizational aspects
→ Low weight to internal validity
Meta-analysis II l
October 10, 2012 l

33. QUALITY OF STUDIES: RCTs (3)
AR Jadad et al. Control Clin Trials 1996; 17: 1-12
5 items, 5 points, ≥3 high quality
- Randomization: Description of method? 1
Appropriate? 1
- Double blinding: Description of method? 1
Appropriate? 1
- Description of withdrawal/dropouts? 1
→ More weight to reporting than methodology
Meta-analysis II l
October 10, 2012 l

34. REPORTING
• RCTs: PRISMA (Preferred Reporting Items for
Systematic reviews and Meta-Analyses)
• Observational: MOOSE (Meta-analysis Of
Observational Studies in Epidemiology)
Meta-analysis II l
October 10, 2012 l

35. PRISMA
Replace and improve the old QUOROM (1999) guidelines
27 items
Title, Abstract, Introduction, Methods, Results, Discussion
and Funding.
Meta-analysis II l
October 10, 2012 l

36. PRISMA flow chart
Meta-analysis II l
October 10, 2012 l

37. PRISMA guidelines: Improvements
• Clear description of objective (PICOS)
• Improve description of selection of studies
(search strategy). Publish at least one.
• Improve evaluation of risk of bias within studies
(quality)
• Improve description and evaluation of
publication bias.
• Suggest publishing the protocol of the MA
Meta-analysis II l
October 10, 2012 l

38. MOOSE
35 items, 1 point to each
Background, Search Strategy, Methods, Results,
Discussion & Conclusion
Meta-analysis II l
October 10, 2012 l

39. Meta-analysis II l
October 10, 2012 l