This document provides an overview of systematic reviews and meta-analysis. It defines a systematic review as a review that uses explicit and reproducible methods to minimize bias in identifying, appraising, and synthesizing results from relevant studies. It notes key characteristics of systematic reviews include having a focused question, comprehensive literature search strategies, explicit inclusion/exclusion criteria, quality assessment of studies, and qualitative or quantitative data synthesis. Meta-analysis is defined as a statistical approach to pooling quantitative results from multiple studies. The document outlines the steps involved in conducting systematic reviews and meta-analyses, including developing a protocol, searching for evidence, screening studies, extracting and analyzing data, and reporting results. It highlights the importance of systematic reviews in providing reliable evidence to inform

1. Systematic
Reviews
and
Meta-Analysis

2. Today’s
Agenda
1 Systemic review vs. Narrative review
2
3
4 How to conduct a systemic review?
Meta-Analysis
5
Importance of systemic reviews
What is systemic review?

4. Systemic
Reviewvs.
Narrative
Review
Feature Systematic Review Narrative Review
Question Focused clinical question Often broad in scope
Purpose
Evidence-base practice and
decision making
Inform current literature
and trends
Data sources
Comprehensive sources &
explicit search strategy
Not usually specified
Study selection Criterion-based selection Not usually specified
Quality assessment Critical appraisal Variable
Evidence synthesis
Qualitative/ Quantitative
summary
Often a qualitative
summary
Inferences
Based on all available
evidence
Based on a sample of the
evidence

5. SYSTEMIC REVIEW
A systematic review is an attempt to collate all empirical
evidence that fits pre-specified eligibility criteria in order
to answer a specific research question.
It uses explicit, systematic methods documented ahead of
time (protocol) to minimize bias
It provides reliable findings which could be used to draw
conclusions and make important decisions.
Source: Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, Welch VA (editors). Cochrane Handbook for Systematic Reviews of
Interventions version 6.2 (updated February 2021).Cochrane, 2021. Available from www.training.cochrane.org/handbook.

6. WHY ARE SYSTEMATIC REVIEWS
NECESSARY?
• “The volume of published material makes it impractical for an individual clinician to
remain up to date on a variety of common conditions. This is further complicated
when individual studies report conflicting conclusions, a problem that is prevalent
when small patient samples and retrospective designs are used.”
Margaliot, Zvi, Kevin C. Chung. “Systematic Reviews: A Primer for Plastic Surgery Research”. PRS Journal. 120/7 (2007) p.1834

7. WHAT IS THE SIGNIFICANCE OF
SYSTEMATIC REVIEWS?
• The large amount of medical literature requires clinicians and researchers alike to
rely on systematic reviews in order to make an informed decision.
• Systematic Reviews minimize bias. “A systematic review is a more scientific method
of summarizing literature because specific protocols are used to determine which
studies will be included in the review.”
Kevin C. Chung, MD, Patricia B. Burns, MPH, H. Myra Kim, ScD, “Clinical Perspective: A Practical Guide to Meta-Analysis.” The Journal of
Hand Surgery. Vol. 31A No.10 December 2006. p.1671

8. KEY CHARACTERISTICS OF SYSTEMATIC
REVIEWS
• Clearly stated title and objectives
• Comprehensive strategy to search for relevant studies (unpublished and published)
• Explicit and justified criteria for the inclusion or exclusion of any study
• Clear presentation of characteristics of each study included and an analysis of
methodological quality
• Comprehensive list of all studies excluded and justification for exclusion
Linda N. Meurer, MD, MPH Department of Family and Community Medicine. “Systematic Synthesis of the Literature: Introduction to
Meta-analysis”. Power Point Presentation.

9. CHARACTERISTICS OF SYSTEMATIC
REVIEWS (CONT.)
• Clear analysis of the results of the eligible studies
• statistical synthesis of data (meta-analysis) if appropriate and possible;
• or qualitative synthesis
• Structured report of the review clearly stating the aims, describing
the methods and materials and reporting the results
Linda N. Meurer, MD, MPH Department of Family and Community Medicine. “Systematic Synthesis of the Literature: Introduction to
Meta-analysis”. Power Point Presentation.

10. HOW TO
CONDUCT
SYSTEMATIC
REVIEW
It is recommended that two reviewers should independently screen the abstracts and full text articles with any discrepancies resolved by a third reviewer

11. IS IT THE RIGHT STUDY DESIGN?

13. GATHERING YOUR TEAM
• A systematic review must have a team of two or greater.
• A systematic review cannot be completed by one person.
• Choose team members wisely and based on areas of
expertise.
• Minimum of two reviewers with a third to serve as a
tiebreaker
• They are to resolve disagreements for reviewers 1 and 2
for stages of the review that are blinded (screening,
data extraction, critical appraisal) and are completed by
two independent reviewers.

14. QUESTIONING
• The PICOT/PICOS format is commonly used to
define the research question into one that is a
searchable question.
• A systematic review question should be narrow in
scope. The purpose of a systematic review is to
draw conclusions based on the evidence to answer
that one well-defined and narrow question.

15. PLANNING
• Create a protocol, plan methods & strategies, register
protocol
• Having a plan in place is essential to a good quality review,
and by spending more time planning before the review takes
place, you could avoid issues or errors that may slow down
the process or be detrimental to the review.
• Planning includes seeing if the review is feasible?
• Setting goals and timelines for the review is important and
mapping out how the review project will be managed.
• Create a document called a protocol. Registering the protocol
is optional but highly recommended. - PROSPERO
• The protocol also includes defining a priori what the
selection criteria will be for the review in terms of inclusion
and exclusion criteria for what studies should be screened
for inclusion in the review.

16. SEARCHING/SCREENING
• Exhaustive, transparent & repeatable searching for
evidence/selecting studies
• Includes searching multiple databases, grey
literature/clinical trial registries and hand-searching of the
literature (performed by the subject matter expert).
• It is best practice to involve a librarian or an information
specialist in creating the comprehensive search,
translating the search for databases or grey literature,
documenting the search and deduplicating the repeating
references in a citation manager and writing the search
methods for the review.

17. DATA
EXTRACTION/SYNTHESIZING
THE EVIDENCE
• This stage includes appraising the evidence, interpreting
results, performing a qualitative (narrative analysis)
and/or a quantitative/meta-analysis.
• A meta-analysis is optional and is only done if it is feasible.
A biostatistician or advanced training in statistics is
recommended if doing a meta-analysis. There are many
specialized tools designed to assist with this process.
• Evidence from studies is assessed using critical appraisal
or Risk of Bias tools and/or checklists by study design.

18. DRAWING CONCLUSIONS,
WRITING & PUBLISHING
• After completing these steps, the results of the
review will be shared.
• What is the level of evidence? Is there evidence in
support of the question or are more studies
needed to draw conclusions? What are your
recommendations for future studies? What are the
limitations to your systematic review?
• How do these findings from your review change
what is known on the topic or question?

19. ACTUAL SUGGESTED TIMELINE FOR COCHRANE
AND BQ
Activity Cochrane Timeline in
Months
BQ Timeline in Months
Preparation of protocol 1-2 1
Searches for published and
unpublished studies
3-8 1-2
Screening of Articles 2-3 2-3
Screening of Articles - round 2 3-8 3-4
Pilot test of data collection 3 4-5
Data entry/extraction 3-10 5
Analysis 8-10 5-6
Preparation of review report 1-11 7

20. SOFTWARE FOR SLR
JBI SUMARI facilitates the entire systematic review process from protocol to report,
and includes team and contributor management for effective and efficient
collaboration

21. SUMARI

22. SUMARI

23. DATA SYNTHESIS
• “Once the data have been extracted and their quality and validity assessed, the
outcomes of individual studies within a systematic review may be pooled and
presented as summary outcome or effect”
Margaliot, Zvi, Kevin C. Chung. “Systematic Reviews: A Primer for Plastic Surgery Research.” PRS Journal. 120/7 (2007) p.1840

24. META-ANALYSIS

26. META-ANALYSIS
• “While all meta-analyses are based on systematic review of literature, not all
systematic reviews necessarily include meta-analysis.”
Margaliot, Zvi, Kevin C. Chung. “Systematic Reviews: A Primer for Plastic Surgery Research.” PRS Journal. 120/7 (2007) p.1840

27. META-ANALYSIS
Meta-analysis is a quantitative, formal, epidemiological study
design used to systematically assess the results of previous
research to derive conclusions about that body of research.
Typically, but not necessarily, the study is based on randomized,
controlled clinical trials.

28. STEPS OF META-ANALYSIS
• Define the Research Question
• Perform the literature search
• Select the studies
• Extract the data
• Analyze the data
• Report the results

29. THE VALIDITY OF A META-ANALYSIS
• “The validity of a meta-analysis depends on the quality of the studies included, and
an assessment of quality is a necessary part of the process.”
Kevin C. Chung, MD, Patricia B. Burns, MPH, H. Myra Kim, ScD. “Clinical Perspective: A Practical Guide to Meta-Analysis.” The Journal
of Hand Surgery. Vol.31A No.10 December 2006. p. 1674

30. META-ANALYSIS: EXTRACTING THE DATA
• “The type of data to be extracted from each study should be determined in the
design phase and a standardized form is constructed to record the data.”
Kevin C. Chung, MD, Patricia B. Burns, MPH, H. Myra Kim, ScD. “Clinical Perspective: A Practical Guide to Meta-Analysis.” The Journal
of Hand Surgery. Vol.31A No.10 December 2006. p. 1674

31. • SAS, STATA, SPSS, R, spreadsheets, RevMan
• Graphics quality will differ
• Easiest to start with data in a spreadsheet
• Will need, at minimum, study identifier, effect size, and measure of error
39
STATISTICAL PROGRAMS

32. META-ANALYSIS: DATA
• What are the examples of data commonly extracted?
• Study design, descriptions of study groups, diagnostic information, treatments, length of
follow-up evaluation, and outcome measures.

33. META-ANALYSIS: DATA
• “The difficulty with data extraction is that studies often use different outcome
metrics, which make combining the data awkward. The data should be converted to
a uniform metric for pooling.”
Kevin C. Chung, MD, Patricia B. Burns, MPH, H. Myra Kim, ScD. “Clinical Perspective: A Practical Guide to Meta-Analysis.” The Journal
of Hand Surgery. Vol.31A No.10 December 2006. p. 1674

34. • Fixed effect
• Assumes there is one true effect size to be estimated
• Pooled estimate is the common effect size
• Weighting is based entirely on the size of the study
• Only source of error is within studies
• Random effects
• Allows the true effect to vary from study to study
• Trying to estimate the mean of a distribution of true effects
• Weights assigned are more balanced
• Can be error within and between studies 42
ANALYTICAL METHODS

35. META-ANALYSIS: REPORTING THE RESULTS
• A meta-analysis should include:
• A title, abstract, an introduction
• Methods, results, and discussion sections

36. A FOREST PLOT

37. A FUNNEL PLOT
• “A funnel plot is used as a way to assess publication
bias in meta-analysis.”
Kevin C. Chung, MD, Patricia B. Burns, MPH, H. Myra Kim, ScD. “Clinical Perspective: A Practical Guide to
Meta-Analysis.” The Journal of Hand Surgery. Vol.31A No.10 December 2006. p. 1676

38. Report
Structure
REPORT CONTENT
• Protocol
• Eligibilitycriteria
• Data sources
• Study strategy
• Study selection
• Data extraction
• Quality assessment
Results
• PRISMA flow diagram
• Study characteristics
• Patient characteristics
• Efficacy outcomes
• Safety outcomes
Quality assessment
Conclusion
Appendices
References
Title page
Executive summary
Background
Research objectives
Methods

39. References
1. Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, Welch VA (editors).
Cochrane Handbook for Systematic Reviews of Interventions version 6.3
(updated February 2022). Cochrane, 2022. Available from:
www.training.cochrane.org/handbook.
2. Systematic reviews: CRD's guidance for undertaking reviews in health care. Available
from: https://www.york.ac.uk/media/crd/Systematic_Reviews.pdf
3. Page MJ, et al. The PRISMA 2020 statement: an updated guideline for reporting
systematic reviews. BMJ 2021;372:n71.
4. International prospective register of systemic reviews - PROSPERO. Available
from: https://www.crd.york.ac.uk/prospero/
5. National Library of Medicine. PubMed Online Training. Available from:
https://www.nlm.nih.gov/bsd/disted/pubmedtutorial/020_010.html
6. Scottish Intercollegiate Guidelines Network. Search filters – SIGN. Available from:
http://www.sign.ac.uk/search-filters.html

40. THANK YOU