Critical Appraisal Methods
for Clinical Studies
SYSTEMATIC REVIEWS
RANDOMIZED
CONTROLLED TRIALS
COHORT STUDIES
CASE-CONTROLLED STUDIES
CASE SERIES / CASE REPORTS
EDITORIALS / EXPERT OPINIONS
LEVELS OF EVIDENCE
prognosis &
etiology/harm
questions
Holy grail of articles
Therapy/prevention &
diagnosis questions
What is Critical Appraisal?
A process of carefully and
systematically examining research to
judge its trustworthiness, value and
relevance in a particular context
(Burls 2009)
Why is it important?
- To ensure that the data presented is valid
- To determine whether the study results
are applicable to your population
- To ensure you are practicing an evidence-
based practice in your clinical practice
Controversies in Medical Research
- Research findings need to be evaluated
carefully to avoid making inaccurate
conclusions!
Autism & MMR
Wakefield, 1998, The Lancet
HRT & Breast
Cancer
“Million Women Study” 2003
Common Misconceptions
about Research Articles
It was published in a scientific journal so it
must be correct…right?
Wrong!
- It should have gone through a peer review
process but needs to be appraised to avoid
misinterpreting the results
- Some journals get paid by authors to
publish, regardless of quality of research 
Critical Appraisal: How to do it
Research Question
Literature Search (PICO)
Identify titles relevant to your study
Read abstract to get rough idea about
study setting, methodology & findings
Doesn’t stop here!
Critical Appraisal: How to do it
Obtain full text of article
Read whole article!
Intro – Background & objectives of study
Methodology—study of population, setting,
study design, data collection
Results—analysis, any confounding factors
Discussion—compare & contrast findings,
supported by literature
References—reliable references? Any
potential articles to be snowballed?
Main Considerations (CEBM)
1. Does this study address a clearly focused
question?
2. Did the study use valid methods to address
this question?
3. Are the valid results of this study
important?
4. Are these valid, important results
applicable to my patient or population?
Critical Appraisal Checklist
- Critical Appraisal Checklists (from Center
for Evidence-Based Medicine):
https://www.cebm.net/2014/06/critical-
appraisal/
Questions?
1. Title of Paper
- Does it relate to the topic you are
searching for? Does it contain important
keywords?
- Does it mention where the study was
done?
2. Author of Paper
- Who are the authors?
- What are their affiliations? (relevant
expertise, location of institution)
3. Abstract
- Can be structured or unstructured,
depending on requirements of journal
- Includes:
- Brief intro of research problem /
question & study objectives
- Summary of methodology
- Summary of results
- Discussion or conclusion
4. Introduction
- Should provide a background of why the
study was conducted, relevant literature
regarding the topic
- Final paragraph usually states the objective
/ aim of study
- May contain several key references that
you should search for and read as well
5. Methodology
- Study population: where and what type of
population? i.e. community setting,
primary care setting, specialist setting?
Specific disease conditions? Asian or
Western?
- The above will affect whether the findings
can be comparable to your own results
- If no local or regional studies, then
international studies can be useful
6. Results
- Presentation of data in tables, charts
that are easy to understand
- Statistical significance?
- Strength of correlation?
7. Discussion
- Comparing and contrasting results with
other studies
- Reasonable and logical explanation for the
differences if any
- What are the strengths and limitations of
the study? Are there other strengths and
limitations not stated by the author(s)?
8. Conclusion
- Is the conclusion from the study valid
(appropriate methodology, valid tools,
correct analysis)?
- Is the conclusion based on the findings of
the study?
- Are the findings of the study relevant to
your research topic? Can it be applied to
your research proposal?

Critical Appraisal Methods

  • 1.
  • 2.
    SYSTEMATIC REVIEWS RANDOMIZED CONTROLLED TRIALS COHORTSTUDIES CASE-CONTROLLED STUDIES CASE SERIES / CASE REPORTS EDITORIALS / EXPERT OPINIONS LEVELS OF EVIDENCE prognosis & etiology/harm questions Holy grail of articles Therapy/prevention & diagnosis questions
  • 3.
    What is CriticalAppraisal? A process of carefully and systematically examining research to judge its trustworthiness, value and relevance in a particular context (Burls 2009)
  • 5.
    Why is itimportant? - To ensure that the data presented is valid - To determine whether the study results are applicable to your population - To ensure you are practicing an evidence- based practice in your clinical practice
  • 6.
    Controversies in MedicalResearch - Research findings need to be evaluated carefully to avoid making inaccurate conclusions! Autism & MMR Wakefield, 1998, The Lancet HRT & Breast Cancer “Million Women Study” 2003
  • 7.
    Common Misconceptions about ResearchArticles It was published in a scientific journal so it must be correct…right? Wrong! - It should have gone through a peer review process but needs to be appraised to avoid misinterpreting the results - Some journals get paid by authors to publish, regardless of quality of research 
  • 9.
    Critical Appraisal: Howto do it Research Question Literature Search (PICO) Identify titles relevant to your study Read abstract to get rough idea about study setting, methodology & findings Doesn’t stop here!
  • 10.
    Critical Appraisal: Howto do it Obtain full text of article Read whole article! Intro – Background & objectives of study Methodology—study of population, setting, study design, data collection Results—analysis, any confounding factors Discussion—compare & contrast findings, supported by literature References—reliable references? Any potential articles to be snowballed?
  • 11.
    Main Considerations (CEBM) 1.Does this study address a clearly focused question? 2. Did the study use valid methods to address this question? 3. Are the valid results of this study important? 4. Are these valid, important results applicable to my patient or population?
  • 12.
    Critical Appraisal Checklist -Critical Appraisal Checklists (from Center for Evidence-Based Medicine): https://www.cebm.net/2014/06/critical- appraisal/
  • 13.
  • 14.
    1. Title ofPaper - Does it relate to the topic you are searching for? Does it contain important keywords? - Does it mention where the study was done?
  • 15.
    2. Author ofPaper - Who are the authors? - What are their affiliations? (relevant expertise, location of institution)
  • 16.
    3. Abstract - Canbe structured or unstructured, depending on requirements of journal - Includes: - Brief intro of research problem / question & study objectives - Summary of methodology - Summary of results - Discussion or conclusion
  • 17.
    4. Introduction - Shouldprovide a background of why the study was conducted, relevant literature regarding the topic - Final paragraph usually states the objective / aim of study - May contain several key references that you should search for and read as well
  • 18.
    5. Methodology - Studypopulation: where and what type of population? i.e. community setting, primary care setting, specialist setting? Specific disease conditions? Asian or Western? - The above will affect whether the findings can be comparable to your own results - If no local or regional studies, then international studies can be useful
  • 19.
    6. Results - Presentationof data in tables, charts that are easy to understand - Statistical significance? - Strength of correlation?
  • 20.
    7. Discussion - Comparingand contrasting results with other studies - Reasonable and logical explanation for the differences if any - What are the strengths and limitations of the study? Are there other strengths and limitations not stated by the author(s)?
  • 21.
    8. Conclusion - Isthe conclusion from the study valid (appropriate methodology, valid tools, correct analysis)? - Is the conclusion based on the findings of the study? - Are the findings of the study relevant to your research topic? Can it be applied to your research proposal?

Editor's Notes

  • #3 Also mention cross sectional, and review terms: prospective, retrospective, blind