The document discusses risk of bias assessment, which is used to evaluate systematic errors in studies. It describes different tools used to assess risk of bias based on study design, such as AMSTAR-2 for systematic reviews and Cochrane RoB for randomized trials. The document provides a table matching study types with the appropriate assessment tools. It explains that risk of bias assessment helps ensure findings are accurate and appropriate by identifying factors like selection bias or reporting bias. Selecting the correct risk of bias tool is important to properly evaluate studies and avoid systematic errors.

1. Copyright © 2020 pubrica. All rights reserved 1
What is the Risk of Bias Assessment and Different Tools used to Assess
Systematic Review
Dr. Nancy Agens, Head,
Technical Operations, Pubrica
sales@pubrica.com
In Brief
The risk of bias is defined as the risk of
systematic error or a deviation from
reporting the truth or an appropriate
evidence finding. It occurs when there is
a systematic flaw or limitations in the
study design or the conduct of the study.
This article describes different
assessment tools and the types of study
designs for which the tool is applicable
along with the flow of how to avoid the
risk of bias.
I. INTRODUCTION
A systematic review guideline will often
determine the study design to answer the
formulated question, and it is not enough
in trusting the evidence of systematic
review over observational studies. There
are several terms like quality assessment,
critical appraisal, or internal validity,
which are used for the evaluation of
studies supported by the guideline.
Whereas in the risk of bias, the bias
determines the factor that can affect the
observations and findings of the study
systematically and report it to be different
from the actual conclusion. In other words,
a study affected by bias can be inaccurate
and thus leads to an inappropriate
guideline recommendation. Therefore,
inadequate study design or the conduct of
the study will give false findings which
result in wasting time and resources, and
missing opportunity for effective
intervention.
Quality assessment or risk of bias
assessment helps in regulating and
establishing transparency of evidence
synthesis (data collection methods, search
strategies, etc.) and findings and it is often
performed for each study in a review; thus
the collected evidence eliminate bias in
the outcomes. Generally, a quality
assessment outside of a systematic review
does not require evidence synthesis
method; however, it completely depends
upon the utilization of evidence synthesis
method. The risk of bias assessment can be
done by professionals who may be
conducted by the guideline development
group or requires a methodological expert
or by experiences systematic review
researchers. Once the evaluation is done,
then it can be used for the synthesis of
study results and findings and combined
into the complete assessment of the
evidence. There are certain common
factors which can introduce bias in many
research areas, that include
 Selection bias – Participant or
Population comparability in a study
 Performance bias – The influence of
estimate effect (aspects other than the
intervention or exposure of interest)
 Reporting bias – Problems with
measurements or classification of
outcome (missing information)
The risk of bias is assessed frequently by
evaluating the study design and the
conduct of the study because it is
impossible to determine a specific study
has been affected by bias or not.
II. DIFFERENT TOOLS USED TO
ASSESS THE RISK OF BIAS IN THE
SYSTEMATIC REVIEW
AMSTAR 2 - A MeaSurement Tool to
Assess systematic Reviews
AHRQ RRB - AHRQ tool for
evaluating the Risk of Reporting Bias

2. Copyright © 2020 pubrica. All rights reserved 2
AHRQ ORB – AHRQ Outcome and
analysis Reporting Bias framework
GRADE – Grading of
Recommendations Assessment,
Development and Evaluation
GRADE-CERQual – GRADE
Confidence in Evidence from Reviews
of Qualitative research
SAQAT – Semi-Automated Quality
Assessment Tool
NMA Quality – Framework for
evaluating the quality of evidence from
a Network Meta-Analysis
NOS - Newcastle-Ottawa Scale
AXIS – Appraisal tool for Cross-
Sectional Studies
QUIPS - QUality In Prognosis Studies
tool
RoB 1.0 – Cochrane Risk of Bias tool
for randomized trials
RoB 2.0 - Revised tool for Assessing
Risk of Bias in randomized trials
SYRCLE RoB – SYstematic Review
Centre for Laboratory animal
Experimentation Risk of Bias tool
SciRAP - Science in Risk Assessment
and Policy
SIGN - The Scottish Intercollegiate
Guidelines Network
ROBIS - Risk Of Bias in Systematic
Reviews
RoBANS - Risk of Bias Assessment
Tool for Nonrandomized Studies
RoBINS I - Risk of Bias In Non-
randomized Studies of Interventions
tool
ORBIT 1 – Outcome Reporting Bias
In Trials classification system for
benefit outcomes
QUADAS 2 - Quality Assessment of
Diagnostic Accuracy Studies
PROBAST - Prediction model Risk
Of Bias ASsessment Tool
ORBIT 2 - Outcome Reporting Bias
In Trials classification system for harm
outcomes
Reid – Selective reporting bias
algorithm
RTI 2012 – RTI Item Bank for
Assessment of Risk of Bias and
Precision for Observational Studies of
Interventions or Exposures
RTI 2013 – RTI Item Bank for
Assessing Risk of Bias and
Confounding for Observational Studies
of Interventions or Exposures
JBI - the Joanna Briggs Institute
OHAT - Office of Health Assessment
and Translation
Downs Black - Downs Black tool
III. HOW TO ASSESS THE RISK OF
BIAS
1. Plan your approach
2. Identifying an appropriate risk of bias
assessment tool
3. Be aware of related issues
4. Appraise each study
5. Report the assessment process
6. Use your appraisals to inform the
guideline
IV. DETERMINING AN APPROPRIATE
RISK OF BIAS ASSESSMENT TOOL

3. Copyright © 2020 pubrica. All rights reserved 2
Study type Assessment tools Year Source
Systematic reviews
AMSTAR - 2 2017 https://amstar.ca/Amstar-2.php
ROBIS 2016
www.bristol.ac.uk/population-health-
sciences/projects/robis/
SIGN checklist 2014
https://www.sign.ac.uk/checklists-and-
notes.html
Randomized trials
Cochrane RoB 2.0
Tool
2016 www.riskofbias.info
SIGN checklist 2014
https://www.sign.ac.uk/checklists-and-
notes.html
Non-randomised studies of
interventions
(case-control, cohort, etc.)
ROBINS-I 2016 www.riskofbias.info
SIGN checklist 2014
https://www.sign.ac.uk/checklists-and-
notes.html
Prognostic:
 Prognostic factors
 Risk prediction
modelling
 Overall
prognosis/baseline
risk/prevalence/inciden
ce
JBI checklist for
prevalence studies
2017
http://joannabriggs.org/research/critica
l-appraisal-tools.html
PROBAST 2014 www.systematic-reviews.com/probast
QUIPS 2013
https://methods.cochrane.org/prognosi
s/welcome
Diagnostic
SIGN checklist 2014
https://www.sign.ac.uk/checklists-and-
notes.html
QUADAS-2 2011
www.bristol.ac.uk/population-health-
sciences/projects/quadas/quadas-2
Qualitative
CASP Qualitative
Checklist
2018
https://casp-uk.net/wp-
content/uploads/2018/01/CASP-
Qualitative-Checklist-2018.pdf
GRADE-CERQual 2017 https://www.cerqual.org/
JBI Checklist for
Qualitative Research
2017
http://joannabriggs.org/research/critica
l-appraisal-tools.html
Observational studies of
exposures (human
epidemiology, wildlife)
Navigation Guide
risk of bias checklist
2018 https://www.cosmin.nl/
OHAT tool 2015
https://ntp.niehs.nih.gov/ntp/ohat/pubs
/riskofbiastool_508.pdf
In vivo animal studies
Navigation Guide
risk of bias checklist
2018 https://www.cosmin.nl/
SciRAP tool 2018 http://www.scirap.org/
OHAT tool 2015
https://ntp.niehs.nih.gov/ntp/ohat/pubs
/riskofbiastool_508.pdf
CRED 2016
www.ecotoxcentre.ch/projects/risk-
assessment/cred
SYRCLE tool 2014
Systematic Review Centre for
Laboratory Animal Experimentation
In vitro studies OHAT tool 2015
https://ntp.niehs.nih.gov/ntp/ohat/pubs
/riskofbiastool_508.pdf
Obtained from: https://www.nhmrc.gov.au/guidelinesforguidelines/develop/assessing-risk-bias#

4. Copyright © 2020 pubrica. All rights reserved 2
In summary, risk of bias assessment is a
tool used to regulate findings which are
accurate and appropriate, and it is essential
to select the risk of bias tool rightly. There
are many researches available to help, and
it makes things easier to find appropriate
tools for assessing the risk of bias. This
article describes different assessment tools
for a systematic review and the types of
study designs for which the tool is
applicable along with the flow of how to
avoid the risk of bias.
REFERENCE
1. Bero, L. A. Why the Cochrane risk of bias tool
should include funding source as a standard item.
Cochrane Database of Systematic Reviews, 2013.
2. Higgins, J. P., J. Savović, et al., Eds. Revised
Cochrane the risk of bias tool for randomized trials,
20 October 2016.
3. Marshall, I. J., A. Noel-Storr, et al. Machine
learning for identifying Randomized Controlled
Trials: An evaluation and practitioner's guide.
Research Synthesis Methods, 2016.
4. Reeves, et al. AMSTAR 2: a critical appraisal tool
for systematic reviews that include randomized or
non-randomised studies of healthcare interventions,
or both. British Medical Journal 358, 2017.
5. Whiting, P., J. Savović, et al. ROBIS: A new tool to
assess the risk of bias in systematic reviews was
developed. Journal of Clinical Epidemiology 69:
225-234, 2016.