Results of isolated randomized controlled trials are frequently contradicted by subsequent studies (Ioannidis, 2005). Under the most rigorous study design conditions, a well-planned single study, even if prospective and randomized, rarely provides definitive results, and primary studies tend to overestimate treatment effects. A recent study reported that 32% of a set of studies with at least 1,000 citations were either contradicted by or reported stronger effects than subsequent studies (Ioannidis, 2005). Relying on single high-profile clinical trials can therefore be harmful to patients' health. Well-designed randomized controlled trials are excellent when looking at effectiveness, though many fall short in quality reporting of safety and adverse events associated with an intervention. Quality systematic reviews often have increased power and decreased bias as compared with the individual studies they include, and the careful pooling of treatment effects can provide the most accurate overall assessment of an intervention.
The life-saving potential of a systematic review conducted to examine infant sleeping position and sudden infant death syndrome. Although advice to place infants on their backs to sleep was widely available in the early 1990s, the authors of the sudden infant death syndrome study showed that the mortality benefit of this sleep position would have been apparent if a systematic review had been performed any time after 1970 (Gilbert et al., 2005). Such a review potentially could have saved 60,000 infant lives in the United Kingdom, Europe, the United States, and Australia.
We use meta-analysis to produce a more precise estimate of treatment effect so that small but clinically important effects become apparent among a group of apparently conflicting single trials.
Systematic review and meta analaysis course - part 1
Systematic Review and Meta-analysis
AHMED SAID NEGIDA
1. Why systematic reviews?
2. Systematic reviews vs. Literature review
3. Types of Systematic reviews
4. Steps of systematic reviews
5. Writing a systematic review protocol
6. Searching the literature, exporting results and checking duplications
7. Strategies for abstract screening
Steps of systematic review
Summarize the Literature
Less Bias: Transparent and reproducible
Source for Evidence-Based practice
No, because about 32% of a set of
studies with at least 1,000 citations
were either contradicted by or
reported stronger effects than
The NEED for systematic reviews
Can we reply on high
single studies rather than
Yes. For example: A systematic review about
sudden infant death syndrome and sleeping
position could have saved 60,000 infant lives in
the United Kingdom, Europe, the United States,
and Australia if it was performed any time after
Systematic reviews may have
a life-saving potential !
Systematic reviews are rapidly replacing
traditional narrative reviews as a means to
summarize and appraise literature.
Systematic review vs. Literature review
Systematic review Literature review
EVIDENCE High evidence Low evidence
METHOD Systematic steps according to
a detailed protocol
Expert opinion “no protocol
and no systematic steps”
BIAS Low bias High bias
REPRODUCIBILITY Yes Not reproducible
SCOPE Narrow scope Wide scope
synthesis; a process of
studies together to
give one effect size
with a large power
(large sample size).
Combined Effect Estimate
Advantages of meta-analysis?
• When there are conflicting studies.
• To produce a precise effect estimate.
• Large power so a small but clinically significant
effect size will be apparent.
Types of systematic reviews
(Cross sectional, Cohort,
and case control)
Diagnostic test accuracy
Steps of systematic review
#1: Formulating the Research Question
1. Specific “narrow scope”.
3. Supported by clear eligibility criteria.
#2: Define eligibility criteria
Criteria based on which you will include
some studies and exclude others
Major domains of eligibility criteria
• Age range? Minimum sample size? Gender?
• Definition of condition? Disease stage?Population
• Form of drug? Mode of administration? Dose?
• First line of second line? Accepted adjuvants?Intervention
• Placebo? or active comparator?
• Form of drug? Mode of administration? Dose?Comparator
• Efficacy measured on (….. / ….. / ….).
• Safety “reporting adverse events”…..Outcome
• Interventional? Randomized or Quasi experimental?
• Observational? Prospective or retrospective?
• Minimum follow up duration? Cross over or parallel?
How to write a Systematic Review Protocol?
Open the lecture notes (PRISMA-P checklist)
1- Formulate a research question
2- Choose relevant keywords
3- Define eligibility criteria
#3: Searching the literature
• Try to involve all studies relevant to your research question.
• Search multiple databases.
• Seek help from experts in your specialty.
#NOW: open lecture notes (Searching Medical Electronic Databases)
Do NOT forget:
1- To remove duplications if you retrieve studies from
different sources. [Using EndNote or Mendeley]
2- Search results and screening are reported as PRIMA
flow diagram. [Using Review Manager]
#NOW: open lecture notes (Checking duplicates + PRIMA flow diagram)
How to perform screening?
Hand screening (Printed titles/abstracts)
Online sheet (e.g. Google spreadsheet)
Using End Note software
Semi-automated (using Abstrackr @CEBM)
Hand screening of printed titles/abstracts
• REASON “code”
Using a printed screening form
for each paper
Reason code is used to save time from
repeating reasons. See our example in
lecture notes (Page no )
It is important that each step (Literature search,
screening, selection of full text article, data extraction,
and quality assessment) is done by multiple reviewers
to avoid possible individual errors.
Disagreements should be resolved by discussion
or by opinion of another reviewer.