Critical Appraisal Overview


Published on

An overview of critical appraisal in medicine. Talk by Elspeth Hill and James Durrand as part of the Fastbleep Academic Masterclasses 2011.

Published in: Education
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Critical Appraisal Overview

  1. 1. Critical Appraisal Master class<br />Tues 10th May<br />
  2. 2. Critical Appraisal: WTF?<br />(well that’s fab)<br />
  3. 3. Introductions<br />Elspeth Hill<br /><ul><li>MRes Medical Sciences
  4. 4. PhD Education Student
  5. 5. Journal Reviewer
  6. 6. Cochrane Author
  7. 7. Nervous wreck</li></ul>James Durrand<br /><ul><li>BSc Pathology
  8. 8. MB ChB (nearly!)
  9. 9. Academic Foundation Post
  10. 10. RSM Rep
  11. 11. Massive geek</li></ul>You?<br />
  12. 12. Outline<br />Evidence-based medicine<br />Principles of critical appraisal<br />Jargon busting<br />Clinical studies<br />Lab stuff<br />Qualitative work<br />
  13. 13. Evidence-Based Medicine<br />What is it?<br />
  14. 14. Structure of EBM<br />
  15. 15. Critical Appraisal<br />Should I believe this paper?<br />Clinical<br />Should I treat my patients differently based on this paper?<br />Education<br />Should we include this in the medical school curriculum?<br />Lab Science<br />Should we divert resources to investigate this further?<br />
  16. 16. 3 Keys of Appraisal<br />Validity<br />Applicability<br />Impact<br />
  17. 17. Validity<br />Validity:<br />How far has this paper actually answered the question they set out to answer?<br />Question: <br />Does endovascular emergency AAA repair reduce mortality?<br />Answer:<br />Endovascular emergency AAA repair is associated with reduced blood loss.<br />
  18. 18. Applicability<br />Applicability<br />Does the evidence from this paper apply to my patients?<br />Trial Title<br />Cleverstatin reduces cardiac events in male patients with type II diabetes. <br />A randomized controlled trial<br />Population<br />1000 patients in Helsinki, Finland<br />Should I change my Wigan<br />patients to the more expensive <br />cleverstatin?<br />
  19. 19. Impact<br />Impact<br />How much of a difference would this make?<br />Trial Title<br />High flow oxygen increases mortality in acute myocardial infarction. A randomised controlled trial.<br />Recommendation<br />Patients should no longer be given maximal oxygen therapy in acute MI.<br />I am the SHO in A&E: a man presents with crushing chest pain <br />– am I really going to withhold oxygen?<br />
  20. 20. How to approach a paper!<br />
  21. 21. Case Study<br />
  22. 22. First impressions<br />What journal?<br /><ul><li>BMJ vs The Antarctic Journal of Homeopathic Resuscitation
  23. 23. Impact factors
  24. 24. Conflict of Interests
  25. 25. Track Record
  26. 26. Abstract?</li></li></ul><li>Research Question<br /><ul><li>What is the research question at the centre ofthis paper?
  27. 27. Is it an important question?
  28. 28. Did I learn anything from reading this paper?</li></li></ul><li>Introduction<br />. Outlines and references preceding work and ‘the story so far’.<br />. Points out weaknesses and gaps in preceding work.<br />. Justifies need and usefulness of the study.<br />. Puts the paper in context.<br />
  29. 29. Methods<br />
  30. 30. What kind of a study is it?<br />
  31. 31. Expert opinion<br />Thoughts of one person who has seen many cases<br />
  32. 32. Case Reports/Series<br />What happened with one patient?<br />What happened with a few patients?<br />
  33. 33. Case-control studies<br />Risk Factors<br />Time<br />
  34. 34. Cohort studies<br />Time<br />
  35. 35. Confounding factors<br />C<br />Smoking<br />Chewing Gum<br />Lung Cancer<br />A<br />B<br />
  36. 36. Randomized controlled trials (RCTs)<br />...Randomised…<br />– minimisebias by randomly assigning subjects to groups<br />...Controlled…<br />– compare your intervention to a control, where the only difference is the intervention<br />…Trial…<br /> – you don’t know what’s going to happen<br />
  37. 37. The Risk of Bias<br />‘Systematic differences between the two groups’<br />Selection bias<br />Treatment bias<br />Observer bias <br />
  38. 38. Minimising bias<br /><ul><li>Blinding
  39. 39. Double blinding
  40. 40. Triple blinding
  41. 41. Losing will to live?</li></li></ul><li>Results<br /><ul><li>Groups at baseline
  42. 42. Intention to treat
  43. 43. Selective outcome reporting
  44. 44. Subgroup analyses
  45. 45. Side effects</li></li></ul><li>Discussion<br />What does this mean?<br />How was the data interpreted?<br />What does this mean for practice?<br />Does it fit with other studies?<br />Have they acknowledged/justified the flaws?<br />Conclusion<br />Is this a reasonable conclusion?<br />Does it answer the research question?<br />
  46. 46. Lab Research<br />What is the research question?<br />Description – observation<br />Clarification – mechanism<br />Very similar to clinical studies, but watch out for relevance<br />cell biology vs animal studies vs human tissuevshuman trials<br /><ul><li>Does it follow from previous research – theoretical justification</li></li></ul><li><ul><li>n numbers, selective reporting
  47. 47. are the results meaningful – does it matter if the CRP is high if all the animals died?
  48. 48. good modeling – young animals do not represent aged obese smoking Salford residents
  49. 49. adequacy of controls – is that really the only difference
  50. 50. outcome assessment
  51. 51. are the controls normal
  52. 52. are the interventions different, how was this assessed
  53. 53. is there another explanation
  54. 54. how does it fit with existing research
  55. 55. do their conclusions support extrapolations</li></li></ul><li>Qualitative/Education<br />Scientific methods - leaves on trees<br />Did they need to use words?<br /><ul><li>Well-problematized topic
  56. 56. Clear research questions – relevance to the real world?
  57. 57. Appropriate methodology - RCT in education
  58. 58. Clear recipe methods
  59. 59. Limitations and conclusions</li></ul>Big problem<br />So many qualitative methods, <br />but 90% is description,<br /> very little is justification, <br />even less clarification<br />
  60. 60. Take home messages<br />take nothing at face value<br />healthy skepticism<br />anyone can do it!<br />
  61. 61. Further Reading<br />Books:<br />How to read a paper. Greenhalgh T. BMJ books. Blackwell publishing<br />The doctors guide to critical appraisal. Gosall N, Gosall G. Pastest books<br />Bad Science. Ben Goldacre<br />Papers:<br />Why Most Published Research Findings Are False. John Ioannidis<br />Courses:<br />Research methods and critical appraisal-<br /> (events)<br />