Second part of meta-analysis lecture, Cleveland Clinic Lerner College of Medicine of Case Western Reserve University, Cleveland, OH

1. Meta-analysis II
Adrian V. Hernandez, M.D., Ph.D.
Assistant Professor of Medicine
Quantitative Health Sciences
October 21, 2010

2. OUTLINE SECOND PART
SECOND PART: 50 MINUTES
• Analysis (models, methods, heterogeneity, publication
bias, quality, subgroup analysis)
• Reporting of meta-analysis (PRISMA, MOOSE guidelines)
Meta-analysis II l
October 10, 2012 l

3. MODELS
• Fixed effects
-Assumption: Effect is the same across
studies and differences due to chance
-Common effect unknown
-Objective: Estimation of common effect with
more precision
-Pool studies using weights ↔ sample size
Meta-analysis II l
October 10, 2012 l

4. MODELS (2)
• Random effects
-Assumption: Effect is different across
studies and there is an average effect
-Average effect unknown
-Objective: Estimation of average effect of
studies
-Pool studies using similar weights
Meta-analysis II l
October 10, 2012 l

5. When are effects similar between models?
 Large effect
 Balanced arms
 Study sizes similar
 Low heterogeneity of effects
HF Fixed MH 1.59 (1.34-1.89)
Random 1.56 (1.32-1.86)
Edema Fixed MH 2.04 (1.85-2.26)
Random 2.41 (1.91-3.04)
Hernandez AV et al. 2010 (submitted)
Meta-analysis II l
October 10, 2012 l

6. METHODS TO COMBINE STUDY EFFECTS
• Inverse Variance (IV)
Common, flexible
Binary/continuous data
Log OR, Log RR, log HR, standardized ratios
Meta-analysis II l
October 10, 2012 l

7. METHODS TO COMBINE STUDY EFFECTS (2)
• Mantel-Haenzel (MH)
Binary outcomes only
Special cases: sparse outcomes, unbalanced
arms
Correction for zeros in arms
Other situations: Effects similar to IV
Meta-analysis II l
October 10, 2012 l

8. METHODS TO COMBINE STUDY EFFECTS (3)
• Peto
Binary outcomes only, a few outcomes
Small effect
Balanced arms
No correction for zeros in arms
HF Fixed MH 1.59 (1.34-1.89)
Peto 1.59 (1.34-1.88)
Meta-analysis II l
October 10, 2012 l

9. Meta-analysis II l
October 10, 2012 l

10. Meta-analysis II l
October 10, 2012 l

11. Meta-analysis II l
October 10, 2012 l

12. METHODS TO COMBINE STUDY EFFECTS (4)
• DerSimonian and Laird
Any type of effect measures
Random model
Larger CI of the pooled effect
More weight to smaller studies
Meta-analysis II l
October 10, 2012 l

13. HETEROGENEITY
• Degree of dissimilarity in effects of individual studies
• Why?
Participants
Interventions
Co-interventions
Outcomes
Biases of studies (according to hierarchy), etc.
Meta-analysis II l
October 10, 2012 l

14. Meta-analysis II l
October 10, 2012 l

15. HETEROGENEITY: Pseudo-tests
• Eyeballing forest plots
• Point estimates
• Significance level
• Confidence intervals
Meta-analysis II l
October 10, 2012 l

16. HETEROGENEITY: Test it!
• Cochrane Q test (Х2, p)
AND
• I2: Amount of heterogeneity (0-100%)
It needs 95% CI (<25% Low; 25-50% Mod; >50 High)
(Ioannidis JP. J Eval Clin Pract 2008:14:951-7)
 Bad news for both: Low power
 Bad news for I2: No software for CIs
Meta-analysis II l
October 10, 2012 l

17. Meta-analysis II l
October 10, 2012 l

18. HETEROGENEITY: Address it!
• Check data again
• Do not perform a meta-analysis
• Explore vs. ignore
• Use random-effects models
• Change effect measure (e.g. MD →SMD)
• Exclude studies
Meta-analysis II l
October 10, 2012 l

19. EXPLORING HETEROGENEITY
• Subgroup analysis
 Exploratory only
 Low power to detect significant effects
 Better pre-specify in protocol
 Generates hypotheses
→ Editors and reviewers like subgroup analysis
Meta-analysis II l
October 10, 2012 l

20. EXPLORING HETEROGENEITY (2)
• How to perform subgroup analysis?
→ By baseline characteristics (e.g. age, gender)
→ By quality
→ By sample size
→ By follow-up time
Meta-analysis II l
October 10, 2012 l

21. Meta-analysis II l
October 10, 2012 l

22. Meta-analysis II l
October 10, 2012 l

23. Use of thiazolidinediones and risk of heart failure and
peripheral edema in patients at high risk of diabetes and
type 2 diabetes:
A systematic review and meta-analysis of placebo-
controlled randomized trials
Follow-up OR (95% CI) RR (95% CI)
≥12 months
MH 1.57 [1.31-1.87] 1.50 [1.28-1.76]
Random 1.67 [1.16-2.40] 1.66 [1.10-2.50]
<12 months
MH 2.71 [0.94-7.79] 2.68 [0.93-7.67]
Random 2.56 [0.88-7.44] 2.52 [0.88-7.25]
Hernandez AV et al. 2010 (Submitted)
Meta-analysis II l
October 10, 2012 l

24. EXPLORING HETEROGENEITY (3)
• Meta-regression
→Evaluates factors that explain heterogeneity of
effects
→Bad news: Low power
Lack of data
Meta-analysis II l
October 10, 2012 l

25. Meta-analysis II l
October 10, 2012 l

26. EVALUATION OF PUBLICATION BIAS
Funnel Plot: Size effect vs. SE/SS ; Asymmetry?
De Luca G et al. Am Heart J 2007; 153:343-53
Meta-analysis II l
October 10, 2012 l

27. EVALUATION OF PUBLICATION BIAS (2)
Asymmetrical: Only due to publication bias?
Dentali F et al. Ann Intern Med 2007; 146: 278-88
Meta-analysis II l
October 10, 2012 l

28. EVALUATION OF PUBLICATION BIAS (3)
Pseudo-test: Visual inspection of funnel plot
Test!: Begg-Mazumdar test
Asymmetry regression test
Kendall test, etc
→ Bad news: All have low power
Meta-analysis II l
October 10, 2012 l

29. EVALUATION OF PUBLICATION BIAS (4)
In 95% of MAs, the use of asymmetry regression
tests is inappropriate:
→ Highly heterogeneous (I2 > 50%)
→ <10 studies
Meta-analysis II l
October 10, 2012 l

30. QUALITY OF STUDIES: Observational
• Design: Prospective cohort
Retrospective cohort
Case-control
• Quality of measurement of factors
• Patient enrollment (consecutive vs no)
Meta-analysis II l
October 10, 2012 l

31. QUALITY OF STUDIES: RCTs
• Difficult to define
Design/conduct/analysis?
Clinical relevance?
Reporting?
• Several scales: 39
Egger M et al. Systematic reviews in health care. Meta-analysis in
context. 2nd Edition, BMJ London 2001. pp87-108.
Meta-analysis II l
October 10, 2012 l

32. QUALITY OF STUDIES: RCTs (2)
TC Chalmers et al. Control Clin Trials 1981; 2: 31-49
30 items, complex
- Internal validity (R, Blinding, Attrition,
stat analysis)
- External validity
- Data presentation/Organizational aspects
→ Low weight to internal validity
Meta-analysis II l
October 10, 2012 l

33. QUALITY OF STUDIES: RCTs (3)
AR Jadad et al. Control Clin Trials 1996; 17: 1-12
5 items, 5 points, ≥3 high quality
- Randomization: Description of method? 1
Appropriate? 1
- Double blinding: Description of method? 1
Appropriate? 1
- Description of withdrawal/dropouts? 1
→ More weight to reporting than methodology
Meta-analysis II l
October 10, 2012 l

34. REPORTING
• RCTs: PRISMA (Preferred Reporting Items for
Systematic reviews and Meta-Analyses)
• Observational: MOOSE (Meta-analysis Of
Observational Studies in Epidemiology)
Meta-analysis II l
October 10, 2012 l

35. PRISMA
Replace and improve the old QUOROM (1999) guidelines
27 items
Title, Abstract, Introduction, Methods, Results, Discussion
and Funding.
Meta-analysis II l
October 10, 2012 l

36. PRISMA flow chart
Meta-analysis II l
October 10, 2012 l

37. PRISMA guidelines: Improvements
• Clear description of objective (PICOS)
• Improve description of selection of studies
(search strategy). Publish at least one.
• Improve evaluation of risk of bias within studies
(quality)
• Improve description and evaluation of
publication bias.
• Suggest publishing the protocol of the MA
Meta-analysis II l
October 10, 2012 l

38. MOOSE
35 items, 1 point to each
Background, Search Strategy, Methods, Results,
Discussion & Conclusion
Meta-analysis II l
October 10, 2012 l

39. Meta-analysis II l
October 10, 2012 l