Introduction to systematic reviews

1. Introduction to
Systematic Reviews
By Omar Midani

2. Outline
• Introduction to SR and MA
• Formulating the question?
• The protocol
• Finding appropriate studies
• Data Extraction

3. Systematic Review
• As the name implies, typically involve a detailed and comprehensive plan and
search strategy derived a priori, with the goal of reducing bias by identifying,
appraising, and synthesizing all relevant studies on a particular topic.
• Often, systematic reviews include a meta-analysis component which involves
using statistical techniques to synthesize the data from several studies into a
single quantitative estimate or summary effect size.
Smith, C. J. "Systematic Reviews And Meta-Analyses". Phlebology 26.6
(2011): 271-273.

4. Significance of SR
• “The volume of published material makes it impractical for an individual clinician to remain
up to date on a variety of common conditions. This is further complicated when individual
studies report conflicting conclusions, a problem that is prevalent when small patient samples
and retrospective designs are used.”
Margaliot, Zvi, Kevin C. Chung. “Systematic Reviews: A Primer for
Plastic Surgery Research”. PRS Journal. 120/7 (2007) p.1834

5. Characteristics of SR
• Clearly stated title and objectives
• Comprehensive strategy to search for relevant studies (unpublished and published)
• Explicit and justified criteria for the inclusion or exclusion of any study
• Clear presentation of characteristics of each study included and an analysis of methodological quality
• Comprehensive list of all studies excluded and justification for exclusion
• Clear analysis of the results of the eligible studies
– Statistical synthesis of data (meta-analysis) if appropriate and possible;
– or qualitative synthesis
• Structured report of the review clearly stating the aims, describing the methods and materials and
reporting the results

6. ‘Traditional’ reviews Systematic reviews
Unclear how included studies were
selected
Protocol specifies exact research
question/inclusion criteria
Based on incomplete set of studies Comprehensive search to find all
relevant studies
May not consider quality of included
studies
Standardised critical appraisal across
included studies
Unclear how conclusions follow from
included studies
Methods transparent and reproducible
Why not traditional reviews?

7. Hierarchy of evidence

8. How to start?
• Define review question
• Develop protocol – provides transparency; defines exact inclusion and
exclusion criteria and methods
• Identify relevant studies – usually comprehensive search across multiple
bibliographic databases plus reference checking
• Assess eligibility – careful matching of studies against inclusion criteria
• Extract relevant data
• Critically Appraise
• Synthesize, Interpret and Report Data

9. The Question

10. The Question
• Questions may be broad or narrow
• Well-formulated questions may guide many aspects of the review
process
Advantages Disadvantages
Broad question Comprehensive summary of all
evidence
May ‘lump’ interventions that are
heterogeneous – apples/ pears
Findings are likely to be
generalisable
May be a lot of studies to manage (difficult for
narrative synthesis)
Narrow question More discrete and manageable
inclusion criteria
May be too few studies to provide meaningful
result (although this can be useful evidence of
‘gap’)
Less heterogeneity May not be generalizable at all

11. The Question
PICO is for effectiveness questions which usually use controlled trials

12. The Question
SPICE is designed for questions around public health interventions
1. Setting what is the context of the question?
e.g. European countries
2. Perspective who are the users/potential users of the outcomes?
e.g. chronically ill
3. Intervention what is being done to them?
e.g. increased cost-sharing
4. Comparison what are the alternatives?
e.g. no increase
5. Evaluation how will you measure if the intervention is successful?
e.g. access to health services

13. The Protocol

14. The Protocol
• A protocol is an essential component of the systematic review process
• Helps ensure:
– Consistency
– Transparency
– Integrity
• The authors must compose a comprehensive list of inclusion and
exclusion criteria. To avoid selection bias, inclusion and exclusion
criteria should be agreed upon and formalized before data extraction
and analysis.”

15. Literature Search

16. Literature Search
• A comprehensive and reproducible literature search is the foundation of a
systematic review
• Be aware of Database bias (thus we use more than one database), Publication
bias (thus we need to have access to unpublished studies), Language bias
(different language articles should not be excluded).

17. Literature Search
• Electronic databases according to the topic area: MEDLINE,
EMBASE, Cochrane databases, Google scholars, etc.
• Clinical trial database
• Check existing systematic reviews in a similar area for ideas
• Download results to reference management software preferably
ENDNOTE.

18. Literature Search

19. Selecting Included
Studies according to inclusion criteria

20. Data Extraction form will be
provided

21. Critical Appraisal

22. Critical Appraisal
• The CASP Appraisal checklist
1. Clear aims of research (goals, why it is important, relevance)
2. Appropriate methodology
3. Sampling strategy
4. Data collection
5. Relationship between researcher and participants
6. Ethical issues
7. Data analysis
8. Findings
9. Value of research (context dependent)

23. Quality Assessment

24. Quality Assessment (Jadad Score)
• Randomization (2 points possible)
– 1 point if study described as randomized
– Add 1 point if randomization method described and appropriate (e.g. random numbers generated)
– Deduct 1 point randomization described and inappropriate
• Double-blinding (2 points possible)
– 1 point if study described as double-blinded
– Add 1 point if method of double-blinding described and appropriate
– Deduct 1 point if double-blinding described and inappropriate
• Withdrawals (1 point possible)
– Give 1 point for a description of withdrawals and drop-outs

25. Data Synthesis