Diagnostic testing 2009

1,507 views

Published on

Published in: Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,507
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
48
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Diagnostic testing 2009

  1. 1. Medical Epidemiology <ul><li>Interpreting Medical Tests and Other Evidence </li></ul>
  2. 2. Developmental characteristics: test parameters <ul><li>2 types of Error: F alse P ositive and F alse N egative </li></ul>
  3. 3. Developmental characteristics: test parameters <ul><li>Complements of error rates as desirable test properties </li></ul><ul><li>Sensitivity = Pr(T+|D+) = a/(a+c) </li></ul><ul><li>Sensitivity is PID (Positive In Disease) [pelvic inflammatory disease] </li></ul><ul><li>Specificity = Pr(T-|D-) = d/(b+d) </li></ul><ul><li>Specificity is NIH (Negative In Health) [national institutes of health] </li></ul>
  4. 4. Typical setting for finding Sensitivity and Specificity <ul><li>Best if everyone who gets the new test also gets “ gold standard ” </li></ul><ul><li>What is a “gold standard”? </li></ul><ul><li>The perfect test, the truth </li></ul><ul><li>Doesn’t happen </li></ul><ul><li>Even reverse doesn’t happen </li></ul><ul><li>Not even a sample of each (case-control type) </li></ul><ul><li>Case series of patients who had both tests </li></ul>
  5. 5. Setting for finding Sensitivity and Specificity <ul><li>Sensitivity should not be tested in “sickest of sick” </li></ul><ul><li>Should include spectrum of disease </li></ul><ul><li>Specificity should not be tested in “healthiest of healthy” </li></ul><ul><li>Should include similar conditions. </li></ul>
  6. 6. Developmental characteristics: Cut-points and Receiver Operating Characteristic (ROC) Healthy
  7. 7. Developmental characteristics: Cut-points and Receiver Operating Characteristic (ROC) Healthy Sick
  8. 8. Developmental characteristics: Cut-points and Receiver Operating Characteristic (ROC) Fals pos= 20% True pos=82%
  9. 9. Developmental characteristics: Cut-points and Receiver Operating Characteristic (ROC) Fals pos= 9% True pos=70%
  10. 10. Developmental characteristics: Cut-points and Receiver Operating Characteristic (ROC) F pos= 100% T pos=100%
  11. 11. Developmental characteristics: Cut-points and Receiver Operating Characteristic (ROC) F pos= 50% T pos=90%
  12. 13. Developmental characteristics: Cut-points and Receiver Operating Characteristic (ROC) Receiver Operating Characteristic (ROC)
  13. 14. Developmental characteristics: Cut-points and Receiver Operating Characteristic (ROC) Receiver Operating Characteristic (ROC)
  14. 15. Receiver Operating Characteristic (ROC) <ul><li>ROC Curve allows comparison of different tests for the same condition without (before) specifying a cut-off point. </li></ul><ul><li>The test with the largest AUC (Area under the curve) is the best. </li></ul>
  15. 17. Clinical Interpretation: Predictive Values Most test positives below are sick. But this is because there are as many sick as healthy people overall. What if fewer people were sick, relative to the healthy?
  16. 18. Clinical Interpretation: Predictive Values Now most test positives below are healthy. This is because the number of false positives from the larger healthy group outweighs the true positives from the sick group. Thus, the chance that a test positive is sick depends on the prevalence of the disease in the group tested!
  17. 19. Clinical Interpretation: Predictive Values <ul><li>the chance that a test positive is sick, as well as the chance that a test negative is healthy, are what a physician needs to know. </li></ul><ul><li>These are not sensitivity and specificity! </li></ul><ul><li>The numbers a physician needs to know are the predictive values of the test. </li></ul>
  18. 20. Clinical Interpretation: Predictive Values <ul><li>Sensitivity (Se) </li></ul><ul><li>Pr{T+|D+} </li></ul><ul><li>true positives </li></ul><ul><li>total with the disease </li></ul><ul><li>Positive Predictive Value (PV+, PPV) </li></ul><ul><li>Pr{D+|T+} </li></ul><ul><li>true positives </li></ul><ul><li>total positive on the test </li></ul>
  19. 21. Positive Predictive Value <ul><li>Predictive value positive </li></ul><ul><li>The predictive value of a positive test. </li></ul><ul><li>If I have a positive test, does that mean I have the disease? </li></ul><ul><li>Then, what does it mean? </li></ul><ul><li>If I have a positive test what is the chance (probability) that I have the disease? </li></ul><ul><li>Probability of having the disease “after” you have a positive test ( posttest probability ) </li></ul><ul><li>(Watch for “ OF ”. It usually precedes the denominator </li></ul><ul><li>Numerator is always PART of the denominator) </li></ul>
  20. 22. Clinical Interpretation: Predictive Values T+ D+ T+ and D+
  21. 23. Clinical Interpretation: Predictive Value <ul><li>Specificity (Sp) </li></ul><ul><li>Pr{T-|D-} </li></ul><ul><li>true negatives </li></ul><ul><li>total without the disease </li></ul><ul><li>Negative Predictive Value (PV-, NPV) </li></ul><ul><li>Pr{D-|T-} </li></ul><ul><li>true negatives </li></ul><ul><li>total negative on the test </li></ul>
  22. 24. Negative Predictive Value <ul><li>Predictive value negative </li></ul><ul><li>If I have a negative test, does that mean I don’t have the disease? </li></ul><ul><li>What does it mean? </li></ul><ul><li>If I have a negative test what is the chance I don’t have the disease? </li></ul><ul><li>The predictive value of a negative test. </li></ul>
  23. 25. Mathematicians don’t Like PV- <ul><li>PV- “probability of no disease given a negative test result” </li></ul><ul><li>They prefer (1-PV-) “probability of disease given a negative test result” </li></ul><ul><li>Also referred to as “post-test probability” (of a negative test) </li></ul><ul><li>Ex: PV- = 0.95 “post-test probability for a negative test result = 0.05” </li></ul><ul><li>Ex: PV+ = 0.90 “post-test probability for a positive test result = 0.90” </li></ul>
  24. 26. Where do you find PPV? <ul><li>Table? </li></ul><ul><li>NO </li></ul><ul><li>Make new table </li></ul><ul><li>Switch to odds </li></ul>
  25. 27. Use This Table ? NO
  26. 28. Make a New Table
  27. 29. Make a New Table
  28. 30. Switch to Odds <ul><li>1000 patients. 100 have disease. 900 healthy. Who will test positive? </li></ul><ul><li>Diseased 100__X.95 =_95 </li></ul><ul><li>Healthy 900 X.08 = 72 </li></ul><ul><li>We will end with 95+72= 167 positive tests of which 95 will have the disease </li></ul><ul><li>PPV = 95/167 </li></ul>
  29. 31. From pretest to posttest odds <ul><li>Diseased 100 X .95 =_95 </li></ul><ul><li>Healthy 900 X .08 = 72 </li></ul><ul><li>100 = Pretest odds </li></ul><ul><li>900 </li></ul><ul><li>.95 = Sensitivity__ = prob. Of pos test in dis </li></ul><ul><li>.08 1-Specificity prob. Of pos test in hlth </li></ul><ul><li>95 =Posttest odds. Probability is 95/(95+72) </li></ul><ul><li>72 </li></ul>
  30. 32. <ul><li>Remember to switch back to probability </li></ul>
  31. 33. What is this second fraction? <ul><li>Likelihood Ratio Positive </li></ul><ul><li>Multiplied by any patient’s pretest odds gives you their posttest odds. </li></ul><ul><li>Comparing LR+ of different tests is comparing their ability to “rule in” a diagnosis. </li></ul><ul><li>As specificity increases LR+ increases and PPV increases (Sp P In) </li></ul>
  32. 34. Clinical Interpretation: likelihood ratios <ul><li>Likelihood ratio </li></ul><ul><li>LR+ = Sensitivity/(1-Specificity) </li></ul><ul><li>LR- = (1-Sensitivity)/Specificity </li></ul>
  33. 35. Clinical Interpretation: Positive Likelihood Ratio and PV+ O = PRE-TEST ODDS OF DISEASE POST-ODDS (+) = O x LR+ =
  34. 36. Likelihood Ratio Negative <ul><li>Diseased 100 _ X .05 =_5__ </li></ul><ul><li>Healthy 900 X .92 = 828 </li></ul><ul><li>100 = Pretest odds </li></ul><ul><li>900 </li></ul><ul><li>.05 = 1-sensitivity = prob. Of neg test in dis </li></ul><ul><li>.92 Specificity prob. Of neg test in hlth </li></ul><ul><li>(LR-) </li></ul><ul><li>Posttest odds= 5/828. Probability=5/833=0.6% </li></ul><ul><li>As sensitivity increases LR- decreases and NPV increases (Sn N Out) </li></ul>
  35. 37. Clinical Interpretation: Negative Likelihood Ratio and PV- POST-ODDS (-) = O x LR- =
  36. 38. <ul><li>Remember to switch to probability and also to use 1 minus </li></ul>
  37. 39. Post test probability given a negative test = Post odds (-)/ 1+ post odds (-)
  38. 40. Value of a diagnostic test depends on the prior probability of disease <ul><li>Prevalence (Probability) = 5% </li></ul><ul><li>Sensitivity = 90% </li></ul><ul><li>Specificity = 85% </li></ul><ul><li>PV+ = 24% </li></ul><ul><li>PV- = 99% </li></ul><ul><li>Test not as useful when disease unlikely </li></ul><ul><li>Prevalence (Probability) = 90% </li></ul><ul><li>Sensitivity = 90% </li></ul><ul><li>Specificity = 85% </li></ul><ul><li>PV+ = 98% </li></ul><ul><li>PV- = 49% </li></ul><ul><li>Test not as useful when disease likely </li></ul>
  39. 41. Clinical interpretation of post-test probability Disease ruled out Disease ruled in
  40. 42. Advantages of LRs <ul><li>The higher or lower the LR, the higher or lower the post-test disease probability </li></ul><ul><li>Which test will result in the highest post-test probability in a given patient? </li></ul><ul><li>The test with the largest LR+ </li></ul><ul><li>Which test will result in the lowest post-test probability in a given patient? </li></ul><ul><li>The test with the smallest LR- </li></ul>
  41. 43. Advantages of LRs <ul><li>Clear separation of test characteristics from disease probability. </li></ul>
  42. 44. Likelihood Ratios - Advantage <ul><li>Provide a measure of a test’s ability to rule in or rule out disease independent of disease probability </li></ul><ul><li>Test A LR+ > Test B LR+ </li></ul><ul><ul><li>Test A PV+ > Test B PV+ always! </li></ul></ul><ul><li>Test A LR- < Test B LR- </li></ul><ul><ul><li>Test A PV- > Test B PV- always! </li></ul></ul>
  43. 45. Using Likelihood Ratios to Determine Post-Test Disease Probability
  44. 47. Predictive Values <ul><li>Alternate formulations: Bayes’ Theorem </li></ul><ul><li>PV+ = </li></ul><ul><li>Se  Pre-test Prevalence </li></ul><ul><li>Se  Pre-test Prevalence + (1 - Sp )  (1 - Pre-test Prevalence) </li></ul><ul><li>High specificity to “rule-in” disease </li></ul><ul><li>PV- = </li></ul><ul><li>Sp  (1 - Pre-test Prevalence) </li></ul><ul><li>Sp  (1 - Pre-test Prevalence) + (1 - Se )  Pre-test Prevalence </li></ul><ul><li>High sensitivity to “rule-out” disease </li></ul>
  45. 48. Clinical Interpretation: Predictive Values
  46. 49. Clinical Interpretation: Predictive Values
  47. 50. If Predictive value is more useful why not reported? <ul><li>Should they report it? </li></ul><ul><li>Only if everyone is tested. </li></ul><ul><li>And even then. </li></ul><ul><li>You need sensitivity and specificity from literature. Add YOUR OWN pretest probability. </li></ul>
  48. 51. So how do you figure pretest probability? <ul><li>Start with disease prevalence. </li></ul><ul><li>Refine to local population. </li></ul><ul><li>Refine to population you serve. </li></ul><ul><li>Refine according to patient’s presentation. </li></ul><ul><li>Add in results of history and exam (clinical suspicion). </li></ul><ul><li>Also consider your own threshold for testing. </li></ul>
  49. 52. Pretest Probability: Clinical Significance <ul><li>Expected test result means more than unexpected. </li></ul><ul><li>Same clinical findings have different meaning in different settings (e.g.scheduled versus unscheduled visit). Heart sound, tender area. </li></ul><ul><li>Neurosurgeon. </li></ul><ul><li>Lupus nephritis. </li></ul>
  50. 53. What proportion of all patients will test positive? <ul><li>Diseased X sensitivity </li></ul><ul><li>+ Healthy X (1-specificity) </li></ul><ul><li>Prevalence X sensitivity + </li></ul><ul><li> (1-prevalence)(1-specificity) </li></ul><ul><li>We call this “test prevalence” </li></ul><ul><li>i.e. prevalence according to the test. </li></ul>
  51. 54. Combination tests: serial and parallel testing <ul><li>Combinations of specificity and sensitivity superior to the use of any single test may sometimes be achieved by strategic uses of multiple tests. There are two usual ways of doing this. </li></ul><ul><li>Serial testing: Use >1 test in sequence, stopping at the first negative test. Diagnosis requires all tests to be positive. </li></ul><ul><li>Parallel testing: Use >1 test simultaneously, diagnosing if any test is positive. </li></ul>
  52. 55. Serial Testing <ul><li>Doing the tests sequentially, instead of together with the same decision rule, is a cost saving measure. </li></ul><ul><li>This strategy </li></ul><ul><ul><li>increases specificity above that of any of the individual tests, but </li></ul></ul><ul><ul><li>degrades sensitivity below that of any of them singly. </li></ul></ul><ul><li>Serial test to rule-in disease </li></ul>
  53. 56. Combination tests: parallel testing <ul><li>Parallel Testing </li></ul><ul><li>Usual decision strategy diagnoses if any test positive. This strategy </li></ul><ul><ul><li>increases sensitivity above that of any of the individual tests, but </li></ul></ul><ul><ul><li>degrades specificity below that of any individual test. </li></ul></ul><ul><ul><li>Parallel test to rule-out disease </li></ul></ul>
  54. 57. Clinical settings for parallel testing <ul><li>Parallel testing is used to rule-out serious but treatable conditions (example rule-out MI by CPK, CPK-MB, Troponin, and EKG. Any positive is considered positive) </li></ul>
  55. 58. Clinical settings for serial testing <ul><li>When treatment is hazardous (surgery, chemotherapy) we use serial testing to raise specificity.(Blood test followed by more tests, followed by imaging, followed by biopsy). </li></ul>
  56. 59. Typical setting for finding Sensitivity and Specificity <ul><li>Best if everyone who gets the new test also gets “ gold standard ” </li></ul><ul><li>Doesn’t happen </li></ul><ul><li>Even reverse doesn’t happen </li></ul><ul><li>Not even a sample of each (case-control type) </li></ul><ul><li>Case series of patients who had both tests </li></ul>
  57. 60. EXAMPLE <ul><li>Patients who had both a stress test and cardiac catheterization. </li></ul><ul><li>So what if patients were referred for catheterization based on the results of the stress test? </li></ul><ul><li>Not a random or even representative sample. </li></ul><ul><li>It is a biased sample. </li></ul>
  58. 62. If the test is used to decide referral for gold standard? 1000 900 Sp 828/900 = .92 100 Sn95/100 =.95 Total 833 828 5 Test Negative 167 72 95 Test Positive Total No Disease Disease
  59. 63. If the test is used to decide referral for gold standard? 1000 900 164 Sp 99/164=.4 100 86 Sn85/86=.99 Total 833 833  100 828 99 5 1 Test Negative 167 167  150 72 65 95 85 Test Positive Total No Disease Disease
  60. 64. If the test is used to decide referral for gold standard? 250 164 Sp 99/164=.4 86 Sn85/86=.99 Total 100 99 1 Test Negative 150 65 85 Test Positive Total No Disease Disease

×