Systematic Reviews
PeplerHead
FacultyLibrarian:EconomicandManagementSciences
ElizabethMoll-Willard
FacultyLibrarian:Agrisciences
2
What we’ll discuss
• What is a Systematic Review (SR)
• Protocol overview
• Why and when to conduct a SR
• Types of reviews
• Steps in the SR process
• Searching and documenting
• Reporting/Write up
3
Definition
“Review of the literature that is conducted systematically,
using explicit, accountable methods” (Gough, Oliver &
Thomas, 2012)
"Systematic review describes a specific methodology for
conducting reviews of literature. This methodology
prescribes explicit, reproducible, and transparent
processes for collating the best available evidence in
answer to specific questions” (Lewis-Beck, Bryman & Liao,
2004)
4
Protocol
You’ll need a protocol to follow to complete your
systematic review
• Roadmap of the steps you need
• It sets out your methodology for your study
• The recipe of how the review will be
conducted
• Ensures transparency & reproducibility
• A good basic example is the PRISMA protocol
which is a generic one for all systematic
reviews and meta-analyses
5
Protocol
6
Reviews can inform us about…
• What is known
• How it is known
• How this knowledge varies across studies
• What is known from previous research
Why & When
7
We need Reviews because…
• Any research study may be fallible
• Any research may have limited scope
• Provides a wider picture based on all the
research
• Keeping abreast of all research is cumbersome
• Undertaking new primary studies without
being informed may result in unnecessary,
inappropriate, irrelevant, or unethical research.
Why & When
8
When Systematic Reviews are needed?
• Uncertainty about the topic or intervention
• Evidence of topic or intervention is needed
• A lot of research, but still unanswered
questions
• Overview of a topic
• Accurate picture of past research
Why & When
9
Types of Reviews
Traditional literature reviews Systematic reviews Meta-analyses
Present research findings
related to a specific topic
Methodical literature search,
with clear purpose
Methodical literature search,
with clear purpose
Summarize what is known, to
contextualise a study
Provides evidence of search
strategy and studies selected
Provides evidence of search
strategy and studies selected
Provide details of studies that
are considered important,
without explaining why
The search becomes the
method with the ultimate aim
of synthesizing what is found,
discovered
The search becomes the
method with the ultimate aim
of synthesizing what is found,
discovered
Qualitative reporting of results
– written summary / narrative
Quantitative reporting of results
– combining data to answer
questions previous studies
didn’t ask
10
Important to remember
Remember
• Has its origin in Health Sciences
• Relatively new in other disciplines
• Not equally developed in all areas of research –
E.G. MeSH terms (may cause confusion)
• MeSH terms only in PubMed & Medical
databases
Scenario 1
11
While testing your search phrase, you find very
little or no information on your subject. What do
you do?
Scenario 2
12
When conducting your searching for a meta-
analyses, you do find sufficient information, but
not a lot of data out of the resources
(documents/papers). What now?
Steps in the SR
13
14
Getting started:
Whatever the question, always start with the
considerations of the phenomenon, topic, issue or
question that you want to answer, and
responsibilities of the team/reviewer
Step 1: Research questions
15
Your Turn:
Write down your research question and any sub-
objectives that impact your search strategy
Remember: you can have separate search
strategies for different sub-objectives if needed,
BUT it must relate to your main research question.
Step 1: Research questions
16
Searching for studies
• Each electronic source (database) is different and
may require its own search string
• Search strategy must be tested beforehand for
efficacy
• A log of searches must be kept, as well as the
number of results
Step 2: Identification
17
Searching for studies
• Log must include:
ü Where search is conducted
ü Search parameters (strategy)
ü When the search was conducted
ü Number of results
Keep in mind – identification from other
methods, reference list & citation scanning
Step 2: Identification
18
Effective searching
Step 2: Identification
19
Searching:
• The goal here is an exhaustive search by means
of an inclusive search strategy to remove
unwanted bias from your results
• Searching exhaustively attempts to minimize
publication bias
Step 2: Identification
20
Search limitations:
• Even exhaustive searches are less likely to find
negative and non-statistically significant results
• Lack of access to information (the entire
information universe)
• Important to search grey literature (reports,
theses, etc.) – discipline dependent
Step 2: Identification
21
Types of sources:
• Academic databases
• Specialist databases
• Specialist registries
• Organizations
• Internet
• Hand searching books & journals
• Reference list & Citation searching/scanning
Step 2: Identification
22
Live Demonstration:
• Example of what a search strategy might look
like and how to capture information
• Scopus searching
• Grey literature
Step 2: Identification
23
Your turn:
Conduct a search in a database using your search
strategy and log it accordingly in the worksheet
provided.
Remember: This is just to practice, if you struggle
with your search strategy, see your librarian after
this session!
Step 2: Identification
24
Screening - Inclusion/exclusion criteria
• Decide what is relevant for your review
• Applied to search results
• Applied to full text resources
• Criteria could include:
• Specific dates
• Unobtainable full text documents
• Language (English only)
Step 3: Screening
25
Step 3: Screening
Find full text of all relevant resources
• Keep track of items removed due to irrelevance
(screen again)
• Also keep track of which sources are ‘in file’, ‘on
order’ & ‘not available’
• Interlibrary loan service available for full text
delivery
26
Live Demonstration.
Scopus
(effect OR impact) AND (“canopy position” OR maturity OR storage) AND
Mealiness AND (pear OR “forelle pear” OR “Pyrus Communis L.”)
Grey
(intitle:effect OR intitle:impact) AND (intitle:“canopy
position” OR intitle:maturity OR intitle:storage) AND
mealiness AND (pear OR “forelle pear” OR “Pyrus Communis
L.”)
Step 3: Screening
27
Your turn:
Identify criteria for inclusion or exclusion for your
study and add it to your worksheet.
Step 3: Screening
28
Step 4: Included
Inclusion and synthesis
• Final number of resources determined
• Text analysis for patterns, themes, etc. OR
statistical analysis
• Answering your research question
• MS Excel, R, RevMan, Rayyan, Mendeley,
Atlas.ti (Atlas.ti available at SU)
29
The write up
Most reviews follow a basic IMRD structure:
Introduction
Methods
Results
Discussion
30
Examples of write up
31
Examples of write up
32
Examples of write up
Reason for study/review
Research Questions stand central!
33
Examples of write up
Justification – Database selection, search string development,
inclusion/exclusion criteria, additional searching (bibliography
scanning)
34
Examples of write up
Unpacks each step of the review in the results, reporting screening
process, data analysis etc.
35
Examples of write up
Discussion of themes as part of results
36
Examples of write up
Usually aspects of your inclusion/exclusion or replicability issues, for
instance, when using Google or Google Scholar
37
Examples of write up
38
Examples of write up
Immediately jumps into what the study’s aims are
39
Examples of write up
The literature review unpacks the various research questions
40
Examples of write up
In the case of this article, each step of the SR process is unpacked in
the methods, where the SR is also justified
41
Examples of write up
Followed by the findings, and unpacking of the trends
42
Examples of write up
Finally, the limitations
43
Help
Speak to your Faculty Librarian – effective
searching, source selection, where to publish
Writing Lab for structure and flow
Centre for Statistical Consultation – statistical
analysis

Systematic Review

  • 1.
  • 2.
    2 What we’ll discuss •What is a Systematic Review (SR) • Protocol overview • Why and when to conduct a SR • Types of reviews • Steps in the SR process • Searching and documenting • Reporting/Write up
  • 3.
    3 Definition “Review of theliterature that is conducted systematically, using explicit, accountable methods” (Gough, Oliver & Thomas, 2012) "Systematic review describes a specific methodology for conducting reviews of literature. This methodology prescribes explicit, reproducible, and transparent processes for collating the best available evidence in answer to specific questions” (Lewis-Beck, Bryman & Liao, 2004)
  • 4.
    4 Protocol You’ll need aprotocol to follow to complete your systematic review • Roadmap of the steps you need • It sets out your methodology for your study • The recipe of how the review will be conducted • Ensures transparency & reproducibility • A good basic example is the PRISMA protocol which is a generic one for all systematic reviews and meta-analyses
  • 5.
  • 6.
    6 Reviews can informus about… • What is known • How it is known • How this knowledge varies across studies • What is known from previous research Why & When
  • 7.
    7 We need Reviewsbecause… • Any research study may be fallible • Any research may have limited scope • Provides a wider picture based on all the research • Keeping abreast of all research is cumbersome • Undertaking new primary studies without being informed may result in unnecessary, inappropriate, irrelevant, or unethical research. Why & When
  • 8.
    8 When Systematic Reviewsare needed? • Uncertainty about the topic or intervention • Evidence of topic or intervention is needed • A lot of research, but still unanswered questions • Overview of a topic • Accurate picture of past research Why & When
  • 9.
    9 Types of Reviews Traditionalliterature reviews Systematic reviews Meta-analyses Present research findings related to a specific topic Methodical literature search, with clear purpose Methodical literature search, with clear purpose Summarize what is known, to contextualise a study Provides evidence of search strategy and studies selected Provides evidence of search strategy and studies selected Provide details of studies that are considered important, without explaining why The search becomes the method with the ultimate aim of synthesizing what is found, discovered The search becomes the method with the ultimate aim of synthesizing what is found, discovered Qualitative reporting of results – written summary / narrative Quantitative reporting of results – combining data to answer questions previous studies didn’t ask
  • 10.
    10 Important to remember Remember •Has its origin in Health Sciences • Relatively new in other disciplines • Not equally developed in all areas of research – E.G. MeSH terms (may cause confusion) • MeSH terms only in PubMed & Medical databases
  • 11.
    Scenario 1 11 While testingyour search phrase, you find very little or no information on your subject. What do you do?
  • 12.
    Scenario 2 12 When conductingyour searching for a meta- analyses, you do find sufficient information, but not a lot of data out of the resources (documents/papers). What now?
  • 13.
  • 14.
    14 Getting started: Whatever thequestion, always start with the considerations of the phenomenon, topic, issue or question that you want to answer, and responsibilities of the team/reviewer Step 1: Research questions
  • 15.
    15 Your Turn: Write downyour research question and any sub- objectives that impact your search strategy Remember: you can have separate search strategies for different sub-objectives if needed, BUT it must relate to your main research question. Step 1: Research questions
  • 16.
    16 Searching for studies •Each electronic source (database) is different and may require its own search string • Search strategy must be tested beforehand for efficacy • A log of searches must be kept, as well as the number of results Step 2: Identification
  • 17.
    17 Searching for studies •Log must include: ü Where search is conducted ü Search parameters (strategy) ü When the search was conducted ü Number of results Keep in mind – identification from other methods, reference list & citation scanning Step 2: Identification
  • 18.
  • 19.
    19 Searching: • The goalhere is an exhaustive search by means of an inclusive search strategy to remove unwanted bias from your results • Searching exhaustively attempts to minimize publication bias Step 2: Identification
  • 20.
    20 Search limitations: • Evenexhaustive searches are less likely to find negative and non-statistically significant results • Lack of access to information (the entire information universe) • Important to search grey literature (reports, theses, etc.) – discipline dependent Step 2: Identification
  • 21.
    21 Types of sources: •Academic databases • Specialist databases • Specialist registries • Organizations • Internet • Hand searching books & journals • Reference list & Citation searching/scanning Step 2: Identification
  • 22.
    22 Live Demonstration: • Exampleof what a search strategy might look like and how to capture information • Scopus searching • Grey literature Step 2: Identification
  • 23.
    23 Your turn: Conduct asearch in a database using your search strategy and log it accordingly in the worksheet provided. Remember: This is just to practice, if you struggle with your search strategy, see your librarian after this session! Step 2: Identification
  • 24.
    24 Screening - Inclusion/exclusioncriteria • Decide what is relevant for your review • Applied to search results • Applied to full text resources • Criteria could include: • Specific dates • Unobtainable full text documents • Language (English only) Step 3: Screening
  • 25.
    25 Step 3: Screening Findfull text of all relevant resources • Keep track of items removed due to irrelevance (screen again) • Also keep track of which sources are ‘in file’, ‘on order’ & ‘not available’ • Interlibrary loan service available for full text delivery
  • 26.
    26 Live Demonstration. Scopus (effect ORimpact) AND (“canopy position” OR maturity OR storage) AND Mealiness AND (pear OR “forelle pear” OR “Pyrus Communis L.”) Grey (intitle:effect OR intitle:impact) AND (intitle:“canopy position” OR intitle:maturity OR intitle:storage) AND mealiness AND (pear OR “forelle pear” OR “Pyrus Communis L.”) Step 3: Screening
  • 27.
    27 Your turn: Identify criteriafor inclusion or exclusion for your study and add it to your worksheet. Step 3: Screening
  • 28.
    28 Step 4: Included Inclusionand synthesis • Final number of resources determined • Text analysis for patterns, themes, etc. OR statistical analysis • Answering your research question • MS Excel, R, RevMan, Rayyan, Mendeley, Atlas.ti (Atlas.ti available at SU)
  • 29.
    29 The write up Mostreviews follow a basic IMRD structure: Introduction Methods Results Discussion
  • 30.
  • 31.
  • 32.
    32 Examples of writeup Reason for study/review Research Questions stand central!
  • 33.
    33 Examples of writeup Justification – Database selection, search string development, inclusion/exclusion criteria, additional searching (bibliography scanning)
  • 34.
    34 Examples of writeup Unpacks each step of the review in the results, reporting screening process, data analysis etc.
  • 35.
    35 Examples of writeup Discussion of themes as part of results
  • 36.
    36 Examples of writeup Usually aspects of your inclusion/exclusion or replicability issues, for instance, when using Google or Google Scholar
  • 37.
  • 38.
    38 Examples of writeup Immediately jumps into what the study’s aims are
  • 39.
    39 Examples of writeup The literature review unpacks the various research questions
  • 40.
    40 Examples of writeup In the case of this article, each step of the SR process is unpacked in the methods, where the SR is also justified
  • 41.
    41 Examples of writeup Followed by the findings, and unpacking of the trends
  • 42.
    42 Examples of writeup Finally, the limitations
  • 43.
    43 Help Speak to yourFaculty Librarian – effective searching, source selection, where to publish Writing Lab for structure and flow Centre for Statistical Consultation – statistical analysis

Editor's Notes