Dr. Buna Bhandari, Ph.D (UNSW,
Australia)
Assistant Professor, Central
Department of Public Health,
Tribhuvan University Institute of
Medicine, Kathmandu, Nepal
Lown Scholar, Department of Global
Health and Population, Harvard T.H.
Chan School of Public Health, USA
Author AID Steward and INASP
Associate
Systematic review
 A systematic review (SR) is a rigorous and organized method to
synthesize the evidence from multiple studies on a particular
research question or topic.
 The purpose of a systematic review is to identify, appraise, and
summarize all available evidence relevant to a specific research
question in a transparent and replicable manner.
 It aims to provide a comprehensive overview of academic literature
concerning a particular research question of topic.
A statistical technique employed in systematic reviews to integrate
and analyze the results from various included studies on a
particular research question or topic in SR.
 It involves pooling numerical data from similar individual studies
in a systematic review to reach a new statistical conclusion
Not all systematic reviews include meta-analysis, but all meta-
analyses are mostly conducted in systematic reviews.
1.1. Planning a systematic review
- Formation of the team for systematic review
- Developing a research question and objectives
- Developing a systematic review protocol
- Registering a systematic review protocol
2. Conducting a systematic review
3. Publishing a review
4. Updating a review
Planning a systematic review
Understand why to do a systematic review
To address the gap in the literature
Provide evidence on particular topic
Please go through literature - before deciding topic
To ensure no repetition of the topic
Determine a need for review
Chances of getting article to be included in your review.
If there is Systematic review (or protocol) already exists on your topic
 Is the date of last update longer than three years ago(depends)?
Is it a high quality, well conducted systematic review?
Do the methods reflect the specific criteria of interest for your topic?
Is there a specific gap in terms of population or intervention outcome that has not
been addressed in the identified review ?
If yes go ahead
Formation of the team
 Systematic reviews should be undertaken by a team of researchers with adequate expertise in
given topics and process of systematic review
 Team of expertise requires for the different task of SRM such as the selection of studies for
eligibility, data extraction and rating the certainty of the evidence.
 It requires at least two people independently reviewing and extracting data that minimize the
likelihood of errors.
Have knowledge of general systematic review methodology
An information scientist or research librarian
Specific methodological expertise such as knowledge of the
statistical methods to be used, experience in qualitative synthesis,
or experience with economic analyses for economic evaluations.
Knowledge of the topic area.
Other related experts
DEVELOPING A SYSTEMATIC REVIEW PROTOCOL
There should be clear protocol of the systematic review that outline the
systematic process/methods of conducting the systematic review (
with or without meta analysis)
It helps to complete a systematic review efficiently and accurately,
ensures greater understanding among team members, and makes
writing the manuscript far easier.
The PRISMA reporting standard lists information about the systematic
review protocol as an "essential element" (PRISMA 2020 item 24)
SR
Protocol
Consideration of
a priori
methods/analysi
s
Guide
reviewers/
prevent
arbitrary
decisions
Reduce
unintended
duplication
Facilitate
detection of
selective
reporting
Foster
collaboration
Plan task and
allocate
resources
Background and Rational
Key research questions
• Description of PICO
• Inclusion and exclusion criteria
• Search strategies
• Search methods/data base
• Data extraction /data management – analysis
strategies
• Methods of quality/risk of bias assessment of
individual studies
Detail plan of methods of review
 Write the background of review in context with the existing body of knowledge
 Should have description of the condition and its significance
Description of the intervention ( if related )
How the intervention might work
Gap in the literature
Why it is important to do systematic review now in the particular topic ??
Background and rational of the review
Research question and objectives
 Key research questions based on PICO (or other structured research
question) components
 The structured question will determine the inclusion and exclusion
criteria:
What is the population of interest?
What are the interventions?
What are the outcomes of interest?
What study designs are appropriate?
Effectiveness:
 Does the intervention work/not work?
 Who does it work/not work for?
Other important questions:
 How does the intervention work?
 Is the intervention appropriate?
 Is the intervention feasible?
 Is the intervention and comparison relevant?
For example, if the population is cancer patients, then researchers should decide , is it all cancer patients or just a
segment of them like breast cancer or cervical cancer?
Element Definition Example
Population Who is my question
focused on?
Women with cancers
Intervention What is the intervention I
am interested in?
Education
Comparison What is the issue I am
interested in?
Women without cancers
Outcome What, in relation to the
issue, do I want to
examine?
Quality of life
 There are around at least 25 other question frameworks apart from the PICO.
 These frameworks are PEO, SPIDER, SPICE, and ECLIPS that can help you
formulate a focused research question for your systematic review.
Plan of methods of review
Description of PICO ( or any other )
 Inclusion/exclusion criteria: Clearly specify what rules to decide which studies are
included in the review .
 It should be based on some or all of your PICO components included in objectives
 Also define the eligible study designs, time period of study, study area, population age,
types of intervention etc.
 Decide the outcome measures and eligibility accordingly.
Search strategies
 Based on eligibility criteria develop and detailed search terms that might search
all the related articles related to the topic.
 Plan the search strategy in collaboration with a librarian trained in conducting
systematic reviews and ensure it address each key research questions.
 Conduct independent peer review of the search methodology
 Search bibliographic databases + indexes
 Search literature cited by eligible studies
Plan of searching the database
 Plan and write all the main database where search of the literature would be done
such as PUBMED/MEDLINE, EMBASE, CINAHL etc.
 Decide data base based on scope of your review
 Mention all the important database- shows comprehensiveness
 Endnote
 Zotero, etc
 How will you organize your search results, and identify and remove
any duplicate
 Reasons for exclusion
 Importing references to analytic software
Plan of screening of articles
 First title/abstract screening: Write how and who is responsible for screening and
conflict resolution
 During this phase, two individuals typically review articles independently
 If there is conflict then how will be resolved
 Studies that are included by both the reviewers move on to the full text review
section.
 Then include how the full text review will be done and how the conflicts will be
resolved
Data extraction/data management strategies
 Clearly describe how the data will be extracted and what information will be
extracted.
 Be explicit about your data analysis approach, descriptive only or also meta-
analysis will be done .
 If yes also mention the detailed method to be followed.
 A plan for how the different populations, interventions, outcomes and study
designs within the scope of the review will be grouped for analysis.
 A list of planned subgroup analyses
Methods of quality/risk of bias of individual studies
 Grading the evidence for each key question: There are various risk of bias
assessment tools.
 So please clearly indicate/plan which tool are you going to use and why.
 Commonly used risk of bias tools:

Rob 2.0 for randomized controlled trials

ROBINS-I for non-randomized (observational) studies or cohorts of interventions

ROBINS-E for non-randomized (observational) studies or cohorts of exposures other
than interventions, including environmental and occupational exposures
Risk of Bias assessment of the study

Newcastle-ottawa scale (NOS) for observational studies; case control or cohort
studies

QUADAS-2 for diagnostic accuracy

AXIS for cross-sectional/prevalence studies

Mixed methods appraisal tool (MMAT) for mixed-methods srs with a variety of
study types including both qualitative & quantitative,

IHE case series studies critical appraisal checklist for case series

JBI critical appraisal checklist for case reports for case reports
Different Protocol templates for systematic
reviews

PRISMA for systematic review protocols (prisma-p):checklist and explanation of what should
be included in a systematic review protocol.

The PROSPERO systematic review protocol template

An OSF inclusive systematic review registration form template: this template can used across
different review types (i.E., Scoping review, review of qualitative studies, meta-analysis, or
any other type of review)

JBI scoping review protocol template
Registering a systematic review protocol
 After developing a review protocol, it should be registered in the review registry to
avoid the duplication of same work by others.
 There are different review registries available such as Cochrane
library, PROSPERO, JBI site or OSF. This registration is free and open to anyone
conducting a systematic reviews.
 Some of the journals also publish systematic review protocols as a manuscript. But
please remember registering a protocol and publication of protocol paper is two
different aspects.
 The most common and fast ways of registering SR protocol is in
PROSPERO.
PROSPERO: A registry for systematic review protocols

Registering a protocol in cochrane site

Registering in the JBI systematic review registry
 The effects of lifestyle interventions in mid-life in reducing cardiovascular events at later life : A Cochrane
Review- with team of UNSW Australia.
 Effects of Beverage consumption and cardiovascular mortality: A Systematic review and Meta
analysis - with team of UNSW Australia.
 “Mixed method systematic review of Interventions addressing antimicrobial resistance (AMR)
across one health spectrum in lower- and middle-income countries employing an inter-
disciplinary social-ecological framework”- With Mentee/students
 Lasserson TJ, Thomas J, Higgins JPT. Chapter 1: Starting a review. In: Higgins JPT, Thomas J, Chandler J, Cumpston M,
Li T, Page MJ, Welch VA (editors). Cochrane Handbook for Systematic Reviews of Interventions version 6.3 (updated
February 2022). Cochrane, 2022. Available from www.training.cochrane.org/handbook
 Aromataris E, Munn Z. Chapter 1: JBI Systematic Reviews. In: Aromataris E, Munn Z (Editors). JBI Manual for Evidence
Synthesis. JBI, 2020. Available from https://synthesismanual.jbi.global.. https://doi.org/10.46658/JBIMES-20-02
 Introduction to Systematic Review and Meta-Analysis. Course era online course https://www.coursera.org/learn/systematic-
review
 https://guides.mclibrary.duke.edu/sysreview/write
 https://training.cochrane.org/online-learning/core-software/revman
 https://ktdrr.org/resources/sr-resources/tools.htm
email: bhbuna@iom.edu.np
Twitter: bunabh
Linekdin – Buna Bhandari

Developing a Systematic Review Protocol

  • 1.
    Dr. Buna Bhandari,Ph.D (UNSW, Australia) Assistant Professor, Central Department of Public Health, Tribhuvan University Institute of Medicine, Kathmandu, Nepal Lown Scholar, Department of Global Health and Population, Harvard T.H. Chan School of Public Health, USA Author AID Steward and INASP Associate
  • 2.
    Systematic review  Asystematic review (SR) is a rigorous and organized method to synthesize the evidence from multiple studies on a particular research question or topic.  The purpose of a systematic review is to identify, appraise, and summarize all available evidence relevant to a specific research question in a transparent and replicable manner.  It aims to provide a comprehensive overview of academic literature concerning a particular research question of topic.
  • 3.
    A statistical techniqueemployed in systematic reviews to integrate and analyze the results from various included studies on a particular research question or topic in SR.  It involves pooling numerical data from similar individual studies in a systematic review to reach a new statistical conclusion Not all systematic reviews include meta-analysis, but all meta- analyses are mostly conducted in systematic reviews.
  • 6.
    1.1. Planning asystematic review - Formation of the team for systematic review - Developing a research question and objectives - Developing a systematic review protocol - Registering a systematic review protocol 2. Conducting a systematic review 3. Publishing a review 4. Updating a review
  • 7.
    Planning a systematicreview Understand why to do a systematic review To address the gap in the literature Provide evidence on particular topic Please go through literature - before deciding topic To ensure no repetition of the topic Determine a need for review Chances of getting article to be included in your review.
  • 8.
    If there isSystematic review (or protocol) already exists on your topic  Is the date of last update longer than three years ago(depends)? Is it a high quality, well conducted systematic review? Do the methods reflect the specific criteria of interest for your topic? Is there a specific gap in terms of population or intervention outcome that has not been addressed in the identified review ? If yes go ahead
  • 9.
    Formation of theteam  Systematic reviews should be undertaken by a team of researchers with adequate expertise in given topics and process of systematic review  Team of expertise requires for the different task of SRM such as the selection of studies for eligibility, data extraction and rating the certainty of the evidence.  It requires at least two people independently reviewing and extracting data that minimize the likelihood of errors.
  • 10.
    Have knowledge ofgeneral systematic review methodology An information scientist or research librarian Specific methodological expertise such as knowledge of the statistical methods to be used, experience in qualitative synthesis, or experience with economic analyses for economic evaluations. Knowledge of the topic area. Other related experts
  • 11.
    DEVELOPING A SYSTEMATICREVIEW PROTOCOL There should be clear protocol of the systematic review that outline the systematic process/methods of conducting the systematic review ( with or without meta analysis) It helps to complete a systematic review efficiently and accurately, ensures greater understanding among team members, and makes writing the manuscript far easier. The PRISMA reporting standard lists information about the systematic review protocol as an "essential element" (PRISMA 2020 item 24)
  • 12.
  • 13.
    Background and Rational Keyresearch questions • Description of PICO • Inclusion and exclusion criteria • Search strategies • Search methods/data base • Data extraction /data management – analysis strategies • Methods of quality/risk of bias assessment of individual studies Detail plan of methods of review
  • 14.
     Write thebackground of review in context with the existing body of knowledge  Should have description of the condition and its significance Description of the intervention ( if related ) How the intervention might work Gap in the literature Why it is important to do systematic review now in the particular topic ?? Background and rational of the review
  • 15.
    Research question andobjectives  Key research questions based on PICO (or other structured research question) components  The structured question will determine the inclusion and exclusion criteria: What is the population of interest? What are the interventions? What are the outcomes of interest? What study designs are appropriate?
  • 16.
    Effectiveness:  Does theintervention work/not work?  Who does it work/not work for? Other important questions:  How does the intervention work?  Is the intervention appropriate?  Is the intervention feasible?  Is the intervention and comparison relevant?
  • 17.
    For example, ifthe population is cancer patients, then researchers should decide , is it all cancer patients or just a segment of them like breast cancer or cervical cancer? Element Definition Example Population Who is my question focused on? Women with cancers Intervention What is the intervention I am interested in? Education Comparison What is the issue I am interested in? Women without cancers Outcome What, in relation to the issue, do I want to examine? Quality of life
  • 18.
     There arearound at least 25 other question frameworks apart from the PICO.  These frameworks are PEO, SPIDER, SPICE, and ECLIPS that can help you formulate a focused research question for your systematic review.
  • 19.
    Plan of methodsof review Description of PICO ( or any other )  Inclusion/exclusion criteria: Clearly specify what rules to decide which studies are included in the review .  It should be based on some or all of your PICO components included in objectives  Also define the eligible study designs, time period of study, study area, population age, types of intervention etc.  Decide the outcome measures and eligibility accordingly.
  • 20.
    Search strategies  Basedon eligibility criteria develop and detailed search terms that might search all the related articles related to the topic.  Plan the search strategy in collaboration with a librarian trained in conducting systematic reviews and ensure it address each key research questions.  Conduct independent peer review of the search methodology  Search bibliographic databases + indexes  Search literature cited by eligible studies
  • 24.
    Plan of searchingthe database  Plan and write all the main database where search of the literature would be done such as PUBMED/MEDLINE, EMBASE, CINAHL etc.  Decide data base based on scope of your review  Mention all the important database- shows comprehensiveness
  • 25.
     Endnote  Zotero,etc  How will you organize your search results, and identify and remove any duplicate  Reasons for exclusion  Importing references to analytic software
  • 26.
    Plan of screeningof articles  First title/abstract screening: Write how and who is responsible for screening and conflict resolution  During this phase, two individuals typically review articles independently  If there is conflict then how will be resolved  Studies that are included by both the reviewers move on to the full text review section.  Then include how the full text review will be done and how the conflicts will be resolved
  • 27.
    Data extraction/data managementstrategies  Clearly describe how the data will be extracted and what information will be extracted.  Be explicit about your data analysis approach, descriptive only or also meta- analysis will be done .  If yes also mention the detailed method to be followed.  A plan for how the different populations, interventions, outcomes and study designs within the scope of the review will be grouped for analysis.  A list of planned subgroup analyses
  • 28.
    Methods of quality/riskof bias of individual studies  Grading the evidence for each key question: There are various risk of bias assessment tools.  So please clearly indicate/plan which tool are you going to use and why.  Commonly used risk of bias tools:  Rob 2.0 for randomized controlled trials  ROBINS-I for non-randomized (observational) studies or cohorts of interventions  ROBINS-E for non-randomized (observational) studies or cohorts of exposures other than interventions, including environmental and occupational exposures
  • 29.
    Risk of Biasassessment of the study  Newcastle-ottawa scale (NOS) for observational studies; case control or cohort studies  QUADAS-2 for diagnostic accuracy  AXIS for cross-sectional/prevalence studies  Mixed methods appraisal tool (MMAT) for mixed-methods srs with a variety of study types including both qualitative & quantitative,  IHE case series studies critical appraisal checklist for case series  JBI critical appraisal checklist for case reports for case reports
  • 30.
    Different Protocol templatesfor systematic reviews  PRISMA for systematic review protocols (prisma-p):checklist and explanation of what should be included in a systematic review protocol.  The PROSPERO systematic review protocol template  An OSF inclusive systematic review registration form template: this template can used across different review types (i.E., Scoping review, review of qualitative studies, meta-analysis, or any other type of review)  JBI scoping review protocol template
  • 34.
    Registering a systematicreview protocol  After developing a review protocol, it should be registered in the review registry to avoid the duplication of same work by others.  There are different review registries available such as Cochrane library, PROSPERO, JBI site or OSF. This registration is free and open to anyone conducting a systematic reviews.  Some of the journals also publish systematic review protocols as a manuscript. But please remember registering a protocol and publication of protocol paper is two different aspects.
  • 35.
     The mostcommon and fast ways of registering SR protocol is in PROSPERO. PROSPERO: A registry for systematic review protocols  Registering a protocol in cochrane site  Registering in the JBI systematic review registry
  • 41.
     The effectsof lifestyle interventions in mid-life in reducing cardiovascular events at later life : A Cochrane Review- with team of UNSW Australia.  Effects of Beverage consumption and cardiovascular mortality: A Systematic review and Meta analysis - with team of UNSW Australia.  “Mixed method systematic review of Interventions addressing antimicrobial resistance (AMR) across one health spectrum in lower- and middle-income countries employing an inter- disciplinary social-ecological framework”- With Mentee/students
  • 42.
     Lasserson TJ,Thomas J, Higgins JPT. Chapter 1: Starting a review. In: Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, Welch VA (editors). Cochrane Handbook for Systematic Reviews of Interventions version 6.3 (updated February 2022). Cochrane, 2022. Available from www.training.cochrane.org/handbook  Aromataris E, Munn Z. Chapter 1: JBI Systematic Reviews. In: Aromataris E, Munn Z (Editors). JBI Manual for Evidence Synthesis. JBI, 2020. Available from https://synthesismanual.jbi.global.. https://doi.org/10.46658/JBIMES-20-02  Introduction to Systematic Review and Meta-Analysis. Course era online course https://www.coursera.org/learn/systematic- review  https://guides.mclibrary.duke.edu/sysreview/write  https://training.cochrane.org/online-learning/core-software/revman  https://ktdrr.org/resources/sr-resources/tools.htm
  • 43.