Prognosis Critical Appraisal Guide on a PowerPoint Template: for Journal Club Presentations

1. Journal Club: Critical Appraisal of an Article on
Prognosis
Sudan Evidence-Based Medicine Association Template
Developed by:
Dr. Imad Salah Ahmed Hassan
MD (UK) FACP FRCPI MemAcadMEd (UK) MSc (UK) MBBS (U of K)

3. Clinical Scenario

4. Formulate a Well-Built Clinical
Question
Patient
Intervention
Comparison
Outcome

5. Outline your Search Strategy:
Search Terms/Keywords/Mesh terms
Search site/Search Engine
Pubmed via PICO
Pubmed Search Box & Limits
Pubmed Clinical Queries
Pubmed Advanced Search
Cochrane Library
Other

6. Outline Current Knowledge
 Tip: From the Introduction Section or from a
Guideline

7. Select a Reasonable Piece of
Research
Tip:
Paste the title, authors, journal details
here
Type of Study:  RCT  Retrospective  Cohort  Case-Control
 Other:
Tip: Study Strength: RCT>cohort>case control>case series
Date published:
Number of Pages:
Quality of journal i.e. impact factor (get from Google!)
Linked-In: Author Details/Picture

8. Reason(s) for Selecting this Piece of
Research
 Tip: Common Reasons
Answers my QUESTION
Free article
Recent Article
Reputable Journal
Other

9. Summarize the Results of your
Article
 Tip: Paste the Abstract/Conclusion from the
Abstract here
 Paste Abstract here: use several slides if it is
long-remember the 6X6 rule in power-point
presentations!

10. My Appraisal Map
 Appropriateness of chosen Title?
 Trustworthy Authors?
 Truthful Abstract?
 Comprehensiveness of Introduction section?
 Relevance: Importance for Practice
 Validity: Appropriate Methodology (Validity:
Internal)?
 Results: Trustworthy Results?
 Applicability?
 Impact of Findings on Current Practice and
Future Recommendations?

11. Apply Your Appraisal Scheme
 Tip: Use the RVRA Scheme: Based on the National & Gulf
Center for Evidence-based Health Practice
Relevance
Validity
Results
Applicability

12. Apply Your Appraisal Scheme
 Tip: Use the RVRA Scheme
Relevance Personal Knowledge
Validity Methods Section
Results
(Impact)
Results Section
Applicability Personal
Interpretation/
Experience

13. Apply Your Appraisal Scheme
Relevance
Does the study
address a
common
problem in your
practice?
Does the study
address an
important
outcome to you
or to your
patient? (DOE
vs. POEM)
Assuming that
the study
conclusion is
true; would it

14. Apply Your Appraisal Scheme
Validity
Describe the Research
Design
Tip:
 Name the Study Design

15. Apply Your Appraisal Scheme
Validity
Recruitment: Was the
defined representative
sample of patients
assembled at a common
(usually early) point in
the course of their
disease i.e.
homogeneous sample?
Tip:
Selection and Sampling: )-----
‘‘Referral Bias’’: Primary vs Tertiary
Centres (Severity),
“Inception Bias”/ “Lead-time Bias”:
Timing of Inclusion/Homogeneous
Sample
Find me in the Methods Section:
Selection of Patients
 Copy the Inclusion &
Exclusion Criteria
 Copy the Study Flow Chart to
your Presentation

16. Apply Your Appraisal Scheme
Validity
Copy the Inclusion &
Exclusion Criteria
Inclusion Criteria: Exclusion Criteria:

17. Apply Your Appraisal Scheme
Validity
Maintenance: Was the
follow-up of these
patients sufficiently
long and complete?
Follow-up –Attrition
Bias: >20% loss of
follow-up: Non-
dependable Results
will vary depending on
the outcome (e.g., for
pregnancy outcomes,
nine months; for cancer,
many years)
Find me in the Results
Section.

18. Apply Your Appraisal Scheme
Validity
Outcome Measures:
Primary & Secondary
Detection Bias (Failure of Blinding): Objective
(Death, amputation, infection rates etc.) or strictly
defined Subjective outcomes (quality of life, pain,
dependence) PLUS Blinding (to clinical and
prognostic characteristics of the patients)
Find me in the Methods or Results Section.
Primary: Secondary:

19. Apply Your Appraisal Scheme
Validity
Measurement
(Objective & Blinded):
Were objective and
unbiased outcome
criteria clearly defined?
Consider: Did the
individual assessing the
outcome criteria know
whether or not the
patient had a potential
prognostic factor, i.e.
were they blinded?

20. Apply Your Appraisal Scheme
Validity
Adjustment: Was there
adjustment for
important prognostic
factors?
Consider:
Was there
standardization for
potentially important
prognostic factors e.g.
age?
Were different sub-
groups compared?

21. Apply Your Appraisal Scheme
Validity
Validation: Was there
validation in an
independent group
("test-set") of
patients?
Was there validation in an independent
group of patients?
Adjustment Bias: If the patient
groups are clinically dissimilar.
?Regression Analysis
Find me in the Methods or Results
Section.
Test vs Development Group in
Adjustment Bias
The most effective way to do this is to
create a second group, called the test
group, to analyze the results, that is
different from the first group (the
development group) and that is used
for determining the factors in question.
This eliminates the possibility that
chance was responsible for the initial
identification of prognostic factors.
If this second study effectively validates

22. Apply Your Appraisal Scheme
Validity
Statistical Tests
(Appropriateness)

23. Validity
Overall, is the
method of the
study valid?
[ ] Yes (100%) [ ] No (0%) [ ] In-between ( ___%) [ ] Can't
Tell

24. Results
Results clinical significance (in MP : Magnitude &
Precision)
Magnitude of the
effect: How likely
are the
outcomes over
time?

25. Results
Results clinical significance (in MP : Magnitude &
Precision)
Precision of the
effect: How
precise was the
estimate of the
treatment effect?
Confidence intervals? Narrow or Wide
Statistical vs. Clinical Significance

26. Summarize the Study Biases
Selection, Attrition, Performance,
Detection Biases, Conflict of
Interest etc.

27. Summarize the Study Strength
Tip: From the Discussion or
Letter to the Editor/Editorial
Comment etc.
E.G.: Number of Patients, Randomization Method,
Least Biases, Statistical Tests used, Outcome
Measures, Ascertainment, Drop-outs, Length of
Follow-up etc.

28. Summarize the Study
Weaknesses
Tip: From the Discussion or
Letter to the Editor/Editorial
Comment etc.
E.G.: Number of Patients, Randomization Method,
Least Biases, Statistical Tests used, Outcome
Measures, Ascertainment, Drop-outs, Length of
Follow-up etc.

29. Summarize the Study
Tip: From the Discussion or
Letter to the Editor/Editorial
Comment etc.

30. Unanswered Questions from the
Study
Tip: From the Discussion or
Letter to the Editor/Editorial
Comment etc.

31. Applicability
Applicability I (in IPPP: Intervention – Population -
Preferences-Potential Benefits - Harm
Applicability (External Validity): Can the results be applied to my patients?
Guide Tip Answer
Were the study patients
similar to this patient?
Article should list the patients’ important
clinical characteristics, along with the
definitions used for these characteristics
Will the results lead
directly to selecting or
avoiding a treatment?
Prognostic data often provide the basis for
sensible decisions about therapy
Are the results useful for
reassuring or counseling
my patient?
Will the evidence make a clinically
important impact on your conclusions
about what to offer or tell this patient?
Uniformly good prognosis –
Reassuring
Uniformly poor prognosis -
Counseling
Can the results be used
in my clinical practice?
Availability of diagnostic tests,
treatments, Cost etc.
Additional Points

32. Applicability
System
Preparedness:
Availability, Cost
etc
Staff Preparedness:
Knowledge, Skill
etc
Patient Factors &
Acceptability:
Similar to study
population,
Preferences, Values
etc

33. Resolution of your PICO
Question
Narrate/Write your PICO Question & Why is
it important.
Summarize the Study Findings
Summarize the weaknesses, biases,
Survival, Prediction Rule etc.
Indicate the Level of Evidence & whether
you are going to apply the findings to your
patient:

34. Resolution of your PICO Question:
Recommendations
New Research: Is there a need to do more
research of the subject? Y□ N □
Suggested research if the answer is yes:
Audit: Is there a need to audit your practice?
Y□ N □
Suggest an audit cycle.
Publication: Is it worth publishing your Journal
Club presentation in an EBM CAT journal or
website? May be start your own JC website?

36. Useful Links
 EBM Glossary of Terms: http://www.cebm.net/glossary/
 EBM Calculators:
http://ktclearinghouse.ca/cebm/practise/ca/calculators
 EBM NNT, LR etc. Websites:
 http://ktclearinghouse.ca/cebm/toolbox
 http://www.thennt.com/
 EBM Critical Appraisal Worksheets:
http://ktclearinghouse.ca/cebm/practise/ca/worksheets
 Diagnosis worksheet
 Harm worksheet
 Prognosis worksheet
 Systematic review (of therapy) worksheet
 Therapy worksheet