Definition Steps Appreciate the benefits and limitations Some basic concepts and plots ( forest, funnel, bubble plots) Useful resources.

1. Systematic Reviews
and Meta-Analyses
Nouran Hamza, B Pharm, MSc, DPH
Biostatistician

2. AgendaAgenda
• Definition
• Steps
• Appreciate the benefits and limitations
• Some basic concepts and plots ( forest, funnel,
bubble plots)
• Useful resources.

3. DefinitionsDefinitions
• Systematic Reviews: systematic search
for all available studies to answer a
specific research question (PICO??)
• Meta-Analyses: is the statistical
pooling of the included studies’ results

4. Steps:Steps:
1. Clear answerable research question
2. Reproducible search strategy
3. Assessment of literature quality
4. Summary of the evidence
5. Statistical analyses and Interpretation
6. Conclusions, recommendations

5. Hierarchy Of ScientificHierarchy Of Scientific
EvidenceEvidence
Product A
• Feature 1
• Feature 2
• Feature 3
Product B
• Feature 1
• Feature 2
• Feature 3

6. BENEFITS:BENEFITS:
• Identifies all available evidence rather than “picking”.
• Increase the number of participants increases the power.
• Increase precision.
• Ability to determine if effects are consistent across
different clinical population.
• Generate new hypothesis.
• Explore difference between studies

7. LimitationLimitation
• Heterogeneity
• Risk of bias
• Publication bias
• GIGO????

8. HeterogeneityHeterogeneity
• . There are two basic types of heterogeneity.
• The heterogeneity could be about patients (e.g., are
the populations similar across studies?, [sometimes
called clinical heterogeneity])
• the heterogeneity could be focused on the results
(e.g., are the results consistent across studies?,
[sometimes called statistical heterogeneity]).

9. Risk of bias
• GIGO ( garbage in
garbage out)
• The meta-analysis is
only good (or bad) as
the studies it includes.
• Cochrane risk of bias
tool: low (green),
unclear (yellow) and
high risk of bias (red)

10. PUBLICATION BIASPUBLICATION BIAS
• Studies with negative findings may be difficult to get
published (Although negative finding is a finding)
• If only studies with positive results are published
then a meta-analysis of the published papers will give
a positive result

11. Forest plotForest plot
Weight given to
each study

12. Forest plotForest plot
Effect estimate for
each study with CI

13. Forest plotForest plot
Scale and direction
of benefit

14. Forest plotForest plot
Pooled effect of
estimate with CIStatistical
heterogeneity

15. Funnel plotFunnel plot
Standard
error
Log risk
ratio

16. Funnel plotFunnel plot
• One way of looking for selection bias is to construct a
funnel plot where the largest studies (with the most
precision) are close to the top and smaller studies
are nearer the X axis. If there is no bias then the
studies selected should be symmetrically distributed
around the results from the larger studies which
should be closer to the true result and less
influenced by random error

17. Bubble plot
( meta regression)
When there is increased
variability between studies,
the question arise here is,
why???
One way to investigate is
using (meta regression) that
R2
= proportion of
heterogeneity
 A bubble chart plots x
values, y values, and z (size)
values.
Difference in
means in each
individual trial
The size of bubbles refer
to the % weight of study
in the meta analysis

18. Useful ResourcesUseful Resources
• The Cochrane Collaboration www.thecochranelibrary.com/
o Cochrane Handbook for Systematic Reviews of Interventions
• CRD www.crd.york.ac.uk/
o The Centre for Reviews and Dissemination is a department of the
University of York and is part of the National Institute for Health Research
• www.gradeworkinggroup.org
o The Grading of Recommendations Assessment, Development and
Evaluation (short GRADE) working group
• EPPI-Centre www.eppi.ioe.ac.uk/
o The Evidence for Policy and Practice Information and Co-ordinating
Centre, Social Science Research Unit, Institute of Education, University of
London.

19. Thank you !