1.
Guide to
Evaluating the Evidence
Paula Funnell
p.a.funnell@qmul.ac.uk
Senior Academic Liaison Librarian
(Medicine and Dentistry)
2.
Why?
To weigh up how valid and useful
the research will be
3.
Why? – to save time
• In order to keep up to date,
clinicians would have to read
17 articles a day, 365 days a
year
• Research is of variable
quality
• Only an estimated 1% is
judged clinically relevant
• Need to find the 1%
4.
Publication bias
Papers with “interesting” results are more
likely to be:
• Submitted and accepted for publication
• Published in a major journal
• Published in English
• Quoted by authors
• Quoted in newspapers
5.
Brainstorm
What factors should you be bearing in
mind when reading an article?
Think about
• the research described
• how it is reported
10.
Forest plots
Line of no effect
less than 1
1
more than 1
11.
Forest plots
Line of no effect
less than 1
1
more than 1
12.
Forest plots
Line of no effect
Best estimate
less than 1
1
more than 1
13.
Forest plots
Line of no effect
Confidence interval
Best estimate
less than 1
1
more than 1
14.
Forest plots
Line of no effect
Confidence interval
Best estimate
less than 1
1
more than 1
15.
Forest plots
Line of no effect
Confidence interval
Best estimate
Pooled result
less than 1
1
more than 1
16.
P-value
Could the result have
occurred by chance?
p = 0.001
(1 in 1000)
p = 0.2
(1 in 5)
A p-value of less than 0.05 (1 in 20) is
considered to be statistically significant
17.
How it works
• Involves answering a short questionnaire
• We use the CASP questionnaires at
http://www.sph.nhs.uk/what-we-do/publichealth-workforce/resources/critical-appraisalsskills-programme
• The questionnaires were devised by
doctors for doctors
18.
Summary
Validity
Results
Is it
trustworthy?
What does
it say?
Relevance
Will it help?
19.
Group critical appraisal
1) Did the review address a clearly-focused
question?
20.
Group critical appraisal
2) Did the authors look for the appropriate sort
of papers?
21.
Group critical appraisal
Is it worth continuing?
22.
Group critical appraisal
3) Do you think the important, relevant
studies were included?
23.
Group critical appraisal
4) Did the reviewers do enough to assess the
quality of the included studies?
24.
Group critical appraisal
5) If the results of the studies have been
combined, was it reasonable to do so?
25.
Group critical appraisal
6) What are the overall result of the reviews?
26.
Group critical appraisal
7) How precise are these results?
27.
Group critical appraisal
8) Can the results be applied to the local
population?
28.
Group critical appraisal
9) Were all important outcomes considered?
29.
Group critical appraisal
10) Are the benefits worth the harms and
costs?
Be the first to comment