0
Guide to

Evaluating the Evidence

Paula Funnell
p.a.funnell@qmul.ac.uk
Senior Academic Liaison Librarian
(Medicine and De...
Why?
To weigh up how valid and useful
the research will be
Why? – to save time
• In order to keep up to date,
clinicians would have to read
17 articles a day, 365 days a
year
• Rese...
Publication bias
Papers with “interesting” results are more
likely to be:
• Submitted and accepted for publication
• Publi...
Brainstorm
What factors should you be bearing in
mind when reading an article?
Think about
• the research described
• how ...
RCT checklist
How are the results presented?
•
•
•
•

Number needed to treat (NNT)
Odds Ratio
Relative risk
Mean difference
Odds and risk
10 horses running,
you bet on 1 horse
Odds of winning
1:9
You versus the rest

Risk of winning
1:10
You vers...
Forest plots

less than 1

1

more than 1
Forest plots
Line of no effect

less than 1

1

more than 1
Forest plots
Line of no effect

less than 1

1

more than 1
Forest plots
Line of no effect

Best estimate

less than 1

1

more than 1
Forest plots
Line of no effect
Confidence interval

Best estimate

less than 1

1

more than 1
Forest plots
Line of no effect
Confidence interval

Best estimate

less than 1

1

more than 1
Forest plots
Line of no effect
Confidence interval

Best estimate
Pooled result
less than 1

1

more than 1
P-value
Could the result have
occurred by chance?
p = 0.001

(1 in 1000)

p = 0.2

(1 in 5)

A p-value of less than 0.05 (...
How it works
• Involves answering a short questionnaire
• We use the CASP questionnaires at
http://www.sph.nhs.uk/what-we-...
Summary
Validity

Results

Is it
trustworthy?

What does
it say?
Relevance
Will it help?
Group critical appraisal
1) Did the review address a clearly-focused
question?
Group critical appraisal
2) Did the authors look for the appropriate sort
of papers?
Group critical appraisal
Is it worth continuing?
Group critical appraisal
3) Do you think the important, relevant
studies were included?
Group critical appraisal
4) Did the reviewers do enough to assess the
quality of the included studies?
Group critical appraisal
5) If the results of the studies have been
combined, was it reasonable to do so?
Group critical appraisal
6) What are the overall result of the reviews?
Group critical appraisal
7) How precise are these results?
Group critical appraisal
8) Can the results be applied to the local
population?
Group critical appraisal
9) Were all important outcomes considered?
Group critical appraisal
10) Are the benefits worth the harms and
costs?
Upcoming SlideShare
Loading in...5
×

SSC in Evidence Based Medicine - Evaluating the evidence

160

Published on

Session 4, workshop 3

Published in: Health & Medicine, Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
160
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
11
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Transcript of "SSC in Evidence Based Medicine - Evaluating the evidence"

  1. 1. Guide to Evaluating the Evidence Paula Funnell p.a.funnell@qmul.ac.uk Senior Academic Liaison Librarian (Medicine and Dentistry)
  2. 2. Why? To weigh up how valid and useful the research will be
  3. 3. Why? – to save time • In order to keep up to date, clinicians would have to read 17 articles a day, 365 days a year • Research is of variable quality • Only an estimated 1% is judged clinically relevant • Need to find the 1%
  4. 4. Publication bias Papers with “interesting” results are more likely to be: • Submitted and accepted for publication • Published in a major journal • Published in English • Quoted by authors • Quoted in newspapers
  5. 5. Brainstorm What factors should you be bearing in mind when reading an article? Think about • the research described • how it is reported
  6. 6. RCT checklist
  7. 7. How are the results presented? • • • • Number needed to treat (NNT) Odds Ratio Relative risk Mean difference
  8. 8. Odds and risk 10 horses running, you bet on 1 horse Odds of winning 1:9 You versus the rest Risk of winning 1:10 You versus all the runners
  9. 9. Forest plots less than 1 1 more than 1
  10. 10. Forest plots Line of no effect less than 1 1 more than 1
  11. 11. Forest plots Line of no effect less than 1 1 more than 1
  12. 12. Forest plots Line of no effect Best estimate less than 1 1 more than 1
  13. 13. Forest plots Line of no effect Confidence interval Best estimate less than 1 1 more than 1
  14. 14. Forest plots Line of no effect Confidence interval Best estimate less than 1 1 more than 1
  15. 15. Forest plots Line of no effect Confidence interval Best estimate Pooled result less than 1 1 more than 1
  16. 16. P-value Could the result have occurred by chance? p = 0.001 (1 in 1000) p = 0.2 (1 in 5) A p-value of less than 0.05 (1 in 20) is considered to be statistically significant
  17. 17. How it works • Involves answering a short questionnaire • We use the CASP questionnaires at http://www.sph.nhs.uk/what-we-do/publichealth-workforce/resources/critical-appraisalsskills-programme • The questionnaires were devised by doctors for doctors
  18. 18. Summary Validity Results Is it trustworthy? What does it say? Relevance Will it help?
  19. 19. Group critical appraisal 1) Did the review address a clearly-focused question?
  20. 20. Group critical appraisal 2) Did the authors look for the appropriate sort of papers?
  21. 21. Group critical appraisal Is it worth continuing?
  22. 22. Group critical appraisal 3) Do you think the important, relevant studies were included?
  23. 23. Group critical appraisal 4) Did the reviewers do enough to assess the quality of the included studies?
  24. 24. Group critical appraisal 5) If the results of the studies have been combined, was it reasonable to do so?
  25. 25. Group critical appraisal 6) What are the overall result of the reviews?
  26. 26. Group critical appraisal 7) How precise are these results?
  27. 27. Group critical appraisal 8) Can the results be applied to the local population?
  28. 28. Group critical appraisal 9) Were all important outcomes considered?
  29. 29. Group critical appraisal 10) Are the benefits worth the harms and costs?
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×