CAT, critical appraisal of scientific paper, journal club for evaluation of published papers

1. Amany Rashad Abo-El-Seoud
Prof.Community Medicine, Zagazig
University, EGYPT

2.  Proper selection of references used in review
and in discussion.
 Evaluate any scientific paper in journals or
on web sites. (for reviewers)

3.  Guidelines of CAT are arranged in the same
sequence of any paper (thesis) to facilitate
criticism for each item i.e. authors,
institution, title, abstract, introduction,
objectives, methods, results, discussion,
references and recommendation.

4.  Is it interesting?
 Not too long, not too short title (average 10-
12 words). Centered , all nouns are capital
 No abbreviations or formula in the title
 Should inform about what will be done in
that paper, on whom (subjects), where (the
place), when (the time).
 After finishing the paper I have to check the
title again to verify its consistency with the
included information in the paper.

5. Assessment
Evaluation
Output
Outcome
Impact

6.  A bad title :"Exam grades"
 A weak title :“ Sleep vs. Exam Grades“
 A good title : "Amount of Sleep Affects
Student Performance on Biology Exams“
 Excellent title: Effect of sleep pattern on
biological examination achievement in
Cairo Faculty of Medicine . A cross sectional
study

7.  Epidemiological study of pulmonary
Tuberculosis in Abbaseia chest hospital.
 Determinant of low birth weight in Cairo
governorate.
 Attitudes of nurses towards HIV patients in
Cairo governorate.

8.  Who share in the paper, their qualification
 Has the institute or university they belong to
a proven academic records?
 Authors from different specialties are
recommended

10. The Abstract should be written in
past tense and Not Contain :
• Lengthy background information.
• References ( literature ).
• Incomplete sentences.
• Abbreviations or terms that may
be confusing to readers.
• Any sort of illustration, figures
or tables.

11.  Does the introduction address the relevant
points? The importance of the subject and
the rationale of selection:
-To improve current situation
-To fill gaps /solution
-To give new products /solutions
-To improve old ones
 Is the study original?

12.  Are objectives clear and SMART? Specific,
measurable, attainable, reliable and time
limited.
 After finishing the paper I have to check if
authors reach all the stated objectives or
not.

13. Should cover:
 Technical design (study design)
 Operative design
 Administrative design
 Statistical design (Data management)

14. What is the type of study? studies are
arranged from up downwards according to
type of methodology into (meta-analysis,
systematic review, randomized controlled
trials, cohort study, case-control and the last
is the cross-sectional or case series studies).
Keep in mind that the results and conclusions
are limited in case series, cross sectional.

15.  Is the selection of subjects in the study sample
based on the rules of probability sampling? (proper
selection by random, systematic, stratified etc.)
 What are the inclusion and exclusion criteria of
cases selection?
 Is the sample size properly calculated and justified?
 Is the power and significance levels stated?
 Are the study groups comparable?(there must be
no significant difference between groups in
demographic data before intervention)

16.  Are the outcome measures stated and relevant?
 Are the measurements valid and reliable?
(questionnaire, laboratory tools, instruments..)
 Is there many missing data? (refusals to co-operate
must not exceed 10-15% and they must be
compensated in sample size from the start. If
refusals exceeds that there must be an explanation
and description of their characteristics, what was
done in statistics to overcome this problem.)
 Is there side effects or adverse outcome
documented?
 Is the time suitable for completeness of the study
and follow up?
 Are the ethical rules followed?

17.  Are the methods of statistical analysis
suitable for the type of data and correct in
interpretation?
 Are the methods described for each step?
 Are the tests used parametric or non
parametric?
 How missed data are handled?
 Is the level of significance stated? (p < 0.05,
0.01)

18.  What was the treatment effect? Or the
intervention action or the difference
between cases and controls? Is this
information clearly stated?
 Are the tables and graphs suitable and clear
to illustrate the results?
 Does each table or graph add benefit to the
result?
 Is the comments on results scientific?

19.  Were the objectives of study fulfilled?
 Were the sources of errors discussed?
 Are the relevant findings justified and
compared to other studies?
 Can the results be generalized to other
populations?
 What is the impact of that paper?
 Is the benefit of treatment used worth the
harm or cost?
 Is the discussion handled all the results
obtained?

20.  The conclusion must be deduced from results
of the paper
 The conclusion must be brief but summarize
all the findings

21.  Clear points and deduced from the results
and discussion
 Must be practical and can be done
 Should include recommendation for the
patients (subjects), for solving the problem
(or disease) and recommendation for further
studies in the same subjects.
 Recommendation for how to
overcome the faced difficulties

22.  Must be recent
 Must be written in the same format
 Either alphabetic or as their site in the paper
(both are right)
 Each reference is written in full details (book
name, publisher, author, year, pages). Web
cites must be clear and full provided that
anyone can reach that cite easily.