An easy-to-use template and guide for Critical Appraisal

1. Journal Club: Critical Appraisal of an Article on Therapy
Sudan Evidence-Based Medicine Association Template
Developed by:
Dr. Imad Salah Ahmed Hassan
MD (UK) FACP FRCPI MemAcadMEd (UK) MSc (UK) MBBS (U of K)

3. Clinical Scenario

4. Formulate a Well-Built Clinical
Question
Patient
Intervention
Comparison
Outcome

5. Outline your Search Strategy:
Search Terms/Keywords/Mesh terms
Search site/Search Engine
Pubmed via PICO
Pubmed Search Box & Limits
Pubmed Clinical Queries
Pubmed Advanced Search
Cochrane Library
Other

6. Outline Current Practice
 Tip: From the Introduction Section or from a
Guideline

7. Select a Reasonable Piece of
Research
Tip:
Paste the title, authors, journal details
here
Type of Study:  RCT  Retrospective  Cohort  Case-Control
 Other:
Tip: Study Strength: RCT>cohort>case control>case series
Date published:
Number of Pages:
Quality of journal i.e. impact factor (get from Google!)
Linked-In: Author Details/Picture

8. Reason(s) for Selecting this Piece of
Research
 Tip: Common Reasons
Answers my QUESTION
Free article
Recent Article
Reputable Journal
Other

9. Summarize the Results of your
Article
 Tip: Paste the Abstract/Conclusion from the
Abstract here
 Paste Abstract here: use several slides if it is
long-remember the 6X6 rule in power-point
presentations!

10. My Appraisal Map
 Appropriateness of chosen Title?
 Trustworthy Authors?
 Truthful Abstract?
 Comprehensiveness of Introduction section?
 Relevance: Importance for Practice
 Validity: Appropriate Methodology (Validity:
Internal)?
 Results: Trustworthy Results?
 Applicability?
 Impact of Findings on Current Practice and
Future Recommendations?

11. Apply Your Appraisal Scheme
 Tip: Use the RVRA Scheme: Based on the National & Gulf
Center for Evidence-based Health Practice
Relevance
Validity
Results
Applicability

12. Apply Your Appraisal Scheme
 Tip: Use the RVRA Scheme
Relevance Personal Knowledge
Validity Methods Section
Results
(Impact)
Results Section
Applicability Personal
Interpretation/
Experience

13. Apply Your Appraisal Scheme
Relevance
Does the study
address a
common
problem in your
practice?
Does the study
address an
important
outcome to you
or to your
patient? (DOE
vs. POEM)
Assuming that
the study
conclusion is
true; would it

14. Apply Your Appraisal Scheme
Validity
Describe the Research
Design
Tip:
 Name the Study Design

15. Apply Your Appraisal Scheme
Validity
Describe the Selection
Criteria, Interventions,
Outcome Measures &
Statistical Tests used
Tip: From Method Section or Copy the Method
Summary from the Abstract.
 Copy the Inclusion & Exclusion Criteria
 Copy the Study Flow Chart to your
Presentation

16. Apply Your Appraisal Scheme
Validity
Copy the Inclusion &
Exclusion Criteria
Inclusion Criteria: Exclusion Criteria:

17. Apply Your Appraisal Scheme
Validity
Interventions

18. Apply Your Appraisal Scheme
Validity
Outcome Measures:
Primary & Secondary
Primary: Secondary:

19. Apply Your Appraisal Scheme
Validity
Statistical Tests
(Appropriateness)

20. Apply Your Appraisal Scheme
Validity
Answer the following
Questions:
• Was the
assignment of
patients to
treatment
randomized?
• Was the
assignment
concealed?
 Method of Randomization: Describe
 Selection bias: before intervention starts

21. Apply Your Appraisal Scheme
Validity
Were the groups
similar at the start of
the trial?
Baseline prognostic factors (demographics, co-
morbidity, disease severity, other known
confounders) balanced?
If different, were these adjusted for?
Tip:
Copy Table 1 (usually describes the different groups
in the study).

22. Apply Your Appraisal Scheme
Validity
Were patients,
their clinicians,
and study
personnel 'blind'
to treatment?
If NO
 Was it possible
 Can this induce bias
 Levels of blinding: Open-label, Single, Double-blind,
Triple-blind etc.
Aside from the
experimental
intervention,
were the groups
treated equally?
Performance bias:
 Co-intervention
 Contamination
 Compliance

23. Validity
Were all patients
who entered the
trial properly
accounted for
and attributed at
its conclusion?
(ITT & Follow-up)
Intention-to-Treat ITT: Were patients analyzed in the
groups to which they were randomized (intention to treat
analysis)?
Attrition Bias: Was follow-up complete & long enough?
Sensitivity Analysis (WCS)

24. Validity
* Were all
clinically
important
outcomes
considered?
* Baseline
characteristics
(Prognostic
Factors)
influence on
Outcomes?
 Are substitute endpoints/surrogate tests valid?
 Was the outcome assessor blind
(Ascertainment/Detection bias or Measurement
bias)
 Clinical Outcomes
 Quality of Life
 Harm/Side-effects etc.

25. Validity
Overall, is the
method of the
study valid?
[ ] Yes (100%) [ ] No (0%) [ ] In-between ( ___%) [ ] Can't
Tell

26. Apply Your Appraisal Scheme
Results clinical significance (in MP : Magnitude &
Precision)
Magnitude of
the effect: How
large was the
treatment
effect &
Specified Time
Factor
(Duration of
Study)?
 Risk difference (RD or ARR)
Risk Ratio (RR)
Relative risk reduction (RRR)
NNT
NNT for different outcomes
NNH

27. Apply Your Appraisal Scheme
Results clinical significance (in MP : Magnitude &
Precision)
Precision of
the effect: How
precise was
the estimate of
the treatment
effect?
Confidence intervals? Narrow or Wide
Statistical vs. Clinical Significance

28. Summarize the Study Biases
Selection, Attrition, Performance,
Detection Biases, Conflict of
Interest etc.

29. Summarize the Study Strength
Tip: From the Discussion or
Letter to the Editor/Editorial
Comment etc.
E.G.: Number of Patients, Randomization Method,
Least Biases, Statistical Tests used, Outcome
Measures, Ascertainment, Drop-outs, Length of
Follow-up etc.

30. Summarize the Study
Weaknesses
Tip: From the Discussion or
Letter to the Editor/Editorial
Comment etc.
E.G.: Number of Patients, Randomization Method,
Least Biases, Statistical Tests used, Outcome
Measures, Ascertainment, Drop-outs, Length of
Follow-up etc.

31. Summarize the Study
Tip: From the Discussion or
Letter to the Editor/Editorial
Comment etc.

32. Unanswered Questions from the
Study
Tip: From the Discussion or
Letter to the Editor/Editorial
Comment etc.

33. Apply Your Appraisal Scheme
Applicability I (in IPPP: Intervention – Population -
Preferences-Potential Benefits - Harm
Can the results be
applied to my patient
care?
1. Can you do the
Intervention exactly
as it is described in
the paper
2. Is your Patient
similar to the
population of the
study?
3. Did you examine
your patient’s
Preferences?
4. Are the potential
benefits outweigh the
Availability-feasibility-cost-side
effects
Patients similar for demographics,
severity, co-morbidity and other
prognostic factors? (as per
Inclusion & Exclusion Criteria)
Compelling reason why the results
should not be applied?

34. Applicability
System
Preparedness:
Availability, Cost
etc
Staff Preparedness:
Knowledge, Skill
etc
Patient Factors &
Acceptability:
Similar to study
population,
Preferences, Values
etc

35. Resolution of your PICO
Question
Narrate/Write your PICO Question & Why is
it important.
Summarize the Study Findings
Summarize the weaknesses, biases, NNT
and NNH etc.
Indicate the Level of Evidence & whether
you are going to apply the findings to your
patient:
Which Implementation Tools you are going to Use:
Education
Pathways, Protocols, Order Set etc.
System Redesign/Quality Improvement Team

36. Resolution of your PICO Question:
Recommendations
New Research: Is there a need to do more
research of the subject? Y□ N □
Suggested research if the answer is yes:
Audit: Is there a need to audit your practice?
Y□ N □
Suggest an audit cycle.
Publication: Is it worth publishing your Journal
Club presentation in an EBM CAT journal or
website? May be start your own JC website?

38. Useful Links
 EBM Glossary of Terms: http://www.cebm.net/glossary/
 EBM Calculators:
http://ktclearinghouse.ca/cebm/practise/ca/calculators
 EBM NNT, LR etc. Websites:
 http://ktclearinghouse.ca/cebm/toolbox
 http://www.thennt.com/
 EBM Critical Appraisal Worksheets:
http://ktclearinghouse.ca/cebm/practise/ca/worksheets
 Diagnosis worksheet
 Harm worksheet
 Prognosis worksheet
 Systematic review (of therapy) worksheet
 Therapy worksheet