Critical appraisal of published article

Dr. Yogesh K Singhal
2nd Year PG student
Department of Community medicine
RNT Medical College
Udaipur (Raj.)
Dr YKS 2015 1

Purpose of this seminar
 The purpose of this seminar is to provide a brief
overview of the critical appraisal process.
Dr YKS 2015 2

Critical Appraisal
Critical appraisal is the process of carefully and
systematically analyze the research paper to judge its
trustworthiness, its value and relevance in a particular
context. (Amanda Burls 2009)
Dr YKS 2015 3

Critical Appraisal
 A critical review must identify the strengths
and limitations in a research paper and this
should be carried out in a systematic manner.
Dr YKS 2015 4

Critical Appraisal
 To practice medicine we need evidence.
 Of course Critical appraisal of an article will
give evidence.
• The Critical Appraisal helps in developing the
necessary skills to make sense of scientific
evidence, based on validity, results and
relevance.
Dr YKS 2015 5

Critical Appraisal – “Putting research
into practice”
Critical appraisal is an essential step in the
process of putting research into practice.
Asking questions about research methodology,
Scrutinizing data collection & statistical methods,
Evaluating the findings and results
will help to determine whether that article’s
conclusions should influence practical
decision-making.
Dr YKS 2015 6

Journal Clubs
 Are organized for critical evaluation of
research articles.
Dr YKS 2015 7

Critical Appraisal for a clinician
 Is essential to advance the knowledge
constantly for clinical practice.
Dr YKS 2015 8

Critical Appraisal for a public health
person
 to keep updated regarding the
contemporary practices in health care
administration, therapies, equipment,
diagnostics and the financial implications.
Dr YKS 2015 9

When should Critically Appraise?
• Conducting literature reviews for grant proposals.
• Evaluating the effectiveness, costs, and benefits of
health programs.
• Establishing new health programs.
• Implementing policies; and Public health decision
making, especially at the senior management level.
Dr YKS 2015 10

Why Critical Appraisal is essential?
 Many of the research are not in good quality
and many studies are biased and their results
are untrue.
 This can lead us to draw false conclusions.
 If, these false conclusions are applied to
population, may leads to a public health
problem.
Dr YKS 2015 11

Why do we need to critically appraise ?
 Article was published in a scientific journal so
it must be correct (it’s a myth)
 Many journals get payment from author for
publishing their articles.
Dr YKS 2015
Common misconception about research articles
12

Why do we need to critically appraise ?
 “It usually comes as a surprise to students to
learn that some published articles belongs in
the bin and should not be used to inform
practice”
(Trisha Greenhalgh, 2006)
Dr YKS 2015 13

How Critically Appraise a
published article
Dr YKS 2015 14

Step 1 : To Decide the Article
to appraise
 Conduct a literature search and obtain
full text articles.
 Look at the title : Is it interesting ?; Likely
to be useful in your practice? Yes/No.
Dr YKS 2015 15

Examples of few topics
 A trust-based consumer decision making model in
electronic commerce : role of trust, perceived risk, and
their antecedents
 Particle motion and the theory of charcoal analysis:
source area, transport, deposition, and sampling
 Long Term Effects of a Lifestyle Intervention on
Weight and Cardiovascular Risk Factors in Individuals
with Type 2 Diabetes: Four Year Results of the Look
AHEAD Trial
Dr YKS 2015 16

Examples of few topics
 A trust-based consumer decision making model in
electronic commerce : role of trust, perceived risk, and
their antecedents
 Particle motion and the theory of charcoal analysis:
source area, transport, deposition, and sampling
AHEAD Trial
Dr YKS 2015 17

Select the Topic
AHEAD Trial
Dr YKS 2015
START READING
First know about “Contents of a research article”
18

Anatomy of a Scientific Article
 Abstract
 Introduction
 Materials and Methods
 Results
 Discussion
 Conclusion/ Recommendation
Dr YKS 2015 19

Critically Appraise What You Read.
Dr YKS 2015
And Keep giving your
comments on a separate
paper
20

Step 2: Abstract
Use the information found in the abstract to answer the
questions below:
• Likely to be useful in your practice?
• Are your issues discussed there?
• What are the main findings of the research?
• Do you want to know more after reading the
abstract?
• Was the research done in a similar setting to
yours?
Dr YKS 2015 21

Abstract - after reading
 You may conclude that the study is not reliable merely
by reading the abstract.
 in such case, move on to another article!
Dr YKS 2015 22

Abstract – after reading
 You may conclude that the study is reliable merely by
reading the abstract.
 go ahead and start reading the article
Dr YKS 2015 23

OUR ARTICLE FOR APPRAISAL
 Long Term Effects of a Lifestyle Intervention
on Weight and Cardiovascular Risk Factors in
Individuals with Type 2 Diabetes: Four Year
Results of the Look AHEAD Trial
Dr YKS 2015
Abstract
24

Step 3 : Introduction
The introduction will help to
identify the key concepts,
goals, subjects, and themes
of the research.
Dr YKS 2015 26

Introduction : Questions to Ask
 Why was it done?
“clear statement of the purpose of the study”.
 Does the research question has some element
of novelty (is likely to add to existing
knowledge rather than reconfirming the
already well established facts).
Dr YKS 2015 27

Introduction : Questions to Ask
 The absence of such statements can imply that the
authors had no clear idea of what they were trying to
find out.
Or
they didn’t find anything but wanted
to publish!
Dr YKS 2015 28

Dr YKS 2015
Long Term Effects of a Lifestyle Intervention on Weight and
Cardiovascular Risk Factors in Individuals with Type 2 Diabetes: Four
Year Results of the Look AHEAD Trial - INTRODUCTION
29

Step 4 : Material & Methodology
 The Methodology will give a step-by-step description of
exactly how the study was carried out.
 Brief but should include enough detail to enable one to judge
quality.
 Type of study
 Mention of study population
 Calculation of the sample size
 method of sampling
 identified all the potential confounding factors (PCFs)
 Description of Questionnaire
 Physical instruments and techniques used
 Biases, if any
Dr YKS 2015 30

Methods : TYPES OF STUDIES
Dr YKS 2015
Different research questions
require different study
designs.
the best design for studies
evaluating the effectiveness
of an intervention or
treatment is a Randomized
controlled trial.
31

Error in study analysis ????
Dr YKS 2015
In a cohort study odds ratio was calculated ???
32

Confounders
 Confounders are minimizing by taking action
 during designing (Randomisation/ Restriction/
Matching)
OR
 during analysis (Standardisation / Stratified
analysis/ Mathematical modelling)
Dr YKS 2015 33

 Is the sample large enough to produce significant results?
 Has the data been presented in a simple, intelligible
form?
 Are the statistical methods appropriate?
Dr YKS 2015
Methodology : Statistical Analysis
35

Methodology: Questions to Ask
 Where the study was done?
 who was studied and how they were recruited.
 Is it primary or secondary data
 How the data was collected (interviews, focus groups,
questionnaires, surveys, observations, etc.).
 Basic demographics must be there.
Dr YKS 2015 36

Dr YKS 2015
Year Results of the Look AHEAD Trial – METHODOLOGY – 1 page
37

Dr YKS 2015
38

Limitation of study
 The study excluded patients younger than 45 years and
older than 76 years old. In fact, 45% of pre-screened
participants (n=26522) were excluded and the major
reason for exclusion was age (13.5%)
Dr YKS 2015 39

Dr YKS 2015
40

Limitation of study
 Only participants who could successfully complete a
maximal graded exercise test at screening were
included.
 maximal graded exercise test is not defined
Dr YKS 2015 41

Dr YKS 2015
Year Results of the Look AHEAD Trial - METHODOLOGY – 1 page
42

Limitation of study
 Participants were provided with meal replacements
free of charge which may be too costly to replicate in a
primary care setting.
 Patients in the ILI group received free partial meal
replacement which, if missed, may take away from the
motivation of patients to rightly follow dietary
recommendations.
 It is uncertain how the absence of free meal
replacements in primary care settings may lead to
different results from the study findings.
Dr YKS 2015 43

Dr YKS 2015
Year Results of the Look AHEAD Trial – METHODOLOGY - 1 page
44

Limitation of study
 Physical activity (brisk walking) was not scientifically
defined so that its interpretation can be subjective and
difficult to determine.
 Activities similar to intensity of brisk walking – who
will monitor in a self-monitored subject
Dr YKS 2015 45

Dr YKS 2015
Year Results of the Look AHEAD Trial - METHODOLOGY - 1
46

Limitation of study
 Subjects are self monitored and use of self-reported
data can’t be considered authentic.
Dr YKS 2015 47

Dr YKS 2015
48

Dr YKS 2015
49

Limitation of study
 For all the “DSE” participants in the Look AHEAD
trial, were seen yearly while “ILI” participants were
seen more frequently.
Dr YKS 2015 50

Dr YKS 2015
51

Dr YKS 2015
52

Limitation of study
 For all the participants in the Look AHEAD trial,
assessment of weight and many other outcomes were
conducted yearly after baseline measurements. The
interval between measurements may be long and
potentially important variability in outcome trends
could be missed.
Dr YKS 2015 53

Dr YKS 2015
54

Step 5 : Results
 The aims in the introduction should be addressed
in the results.
 Maintain the logical sequences (no bland
statements)
 All research has some flaws , the impact of the
flaws need to assessed.
Dr YKS 2015
 If the findings are ‘statistically significant’, are they also of
clinical/public health relevance?
 If the findings are ‘statistically non significant’ is it
possible that a real effect may have been missed due to
“low study power” as consequence of low sample size.
55

Results : Questions to Ask
 A- Are the results of the study valid?
 B-What are the results?
 C-Will the results help locally?
 Are the conclusions drawn by the authors
based on the actual findings of the study?
 Do you think the study results can be
gainfully utilized in your own clinical /
preventive / public health practice?
Dr YKS 2015 56

Dr YKS 2015
Year Results of the Look AHEAD Trial - RESULTS – 1 page
57

Dr YKS 2015
Year Results of the Look AHEAD Trial - RESULTS – 1 page
58

Strengths & Limitation of study
 Large sample size (n=5145) provided sufficient power
to detect clinically relevant differences between effects
of study arms.
 Excellent retention rate - Look AHEAD trial reported
that 94.1% of participants in the intervention arm (ILI)
and 93.1% in the comparator arm (DSE) completed the
4-year assessment.
 The Look AHEAD study does not provide information
about how missing data (dropouts/leftouts) were
handled.
Dr YKS 2015 59

Step 6 : Discussion & Conclusion
 How well are the results related to other
research on the same topic?
 In the discussion or conclusions section, is there a
review of how these results compare or contrast with
prior research?
 If this report found something different from previous
research, then it’s important to question on appraising
the reliability of the findings.
 The limitations of the research and possible
implications which are not mentioned in the
abstract.
Dr YKS 2015 60

Discussion : Questions to ask
 Compare the raw data
given in the tables with the
results analyzed in the
discussion and conclusions
 Are the results reported in the
conclusions consistent with
what is reported in the tables?
 Is the interpretation consistent
with what the actual findings
were?
Dr YKS 2015 61

Dr YKS 2015
Year Results of the Look AHEAD Trial – Discussion & Conclusion
62

Dr YKS 2015
Year Results of the Look AHEAD Trial – Discussion & Conclusion
63

Strengths of study
 The study has a longer duration than many others and
provides a more comprehensive long-term assessment
of behavioural intervention on weight loss and
associated health benefits.
Dr YKS 2015 64

Overall Strengths & Limitation of study
Dr YKS 2015
Strengths Limitation
1. Large sample size (n=5145) provided
sufficient power to detect clinically
relevant differences between effects of
study arms.
2. The study has a longer duration than
many others and provides a more
comprehensive long-term assessment
of behavioural intervention on weight
loss and associated health benefits.
3. Over 93% of participants were
assessed at each year for the four years
making it possible to evaluate more
comprehensively the durability of
measured outcomes and the long-term
effects of the intervention.
1. The study excluded patients younger
than 45 years and older than 76 years
old. In fact, 45% of pre-screened
participants (n=26522) were excluded
because of age factor.
2. Only participants who could
successfully complete a maximal
graded exercise test at screening were
included
3. Patients in the ILI group received
free partial meal replacement which
may not be offered in a primary care
setting and may take away from the
motivation of patients to rightly follow
dietary recommendations.
65

Thanks
Dr YKS 2015
Critical
Appraisal
of
my presentation
66

Critical appraisal of published article

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Critical appraisal of published article

Similar to Critical appraisal of published article (20)

Recently uploaded

Recently uploaded (20)

Critical appraisal of published article