05 diagnostic tests cwq

2,608 views

Published on

Published in: Health & Medicine, Technology
0 Comments
4 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
2,608
On SlideShare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
110
Comments
0
Likes
4
Embeds 0
No embeds

No notes for slide

05 diagnostic tests cwq

  1. 1. Diagnostic Testing <ul><li>Prof. Wei-Qing Chen MD PhD </li></ul><ul><li>Department of Biostatistics and Epidemiology </li></ul><ul><li>School of Public Health </li></ul><ul><li>87332199 </li></ul><ul><li>[email_address] </li></ul>
  2. 2. OBJECTIVES OF LECTURE <ul><li>understand that making a diagnosis is not black & white </li></ul><ul><li>understand subclinical, preclinical disease </li></ul><ul><li>understand accuracy (validity) of a diagnostic test </li></ul><ul><li>understand the need for a ‘Gold Standard’ </li></ul><ul><li>understand the indices by which accuracy is assessed </li></ul><ul><li>understand reliability of a diagnostic test </li></ul>
  3. 3. Section A Diagnosis and Diagnostic Testing
  4. 4. Diagnosis <ul><li>Diagnosis is “the process determining health status and the factors responsible for producing it. </li></ul><ul><ul><li>Separating a target disease from health/other diseases </li></ul></ul><ul><ul><li>Indicating that his outcome will be different (die earlier, suffer more, develop complications) </li></ul></ul><ul><ul><li>Being indicative of treatment </li></ul></ul><ul><ul><li>Changing the patterns following treatment </li></ul></ul>
  5. 5. Way of diagnosis <ul><li>Collecting clinical data </li></ul><ul><ul><li>Symptoms: malaise, memory loss, fatigue, anxiety, etc. </li></ul></ul><ul><ul><li>Signs: pale skin, hyperactive, red in face, etc. </li></ul></ul><ul><ul><li>Special tests: cell count, x-ray, PB, cholesterolemia </li></ul></ul>
  6. 6. Way of diagnosis <ul><li>Assembling a diagnostic entity (category) </li></ul><ul><ul><li>Diagnosis based on one variable: made by an “abnormal” value of a physiological function. </li></ul></ul><ul><ul><ul><li>Hypertension: SBP  160mmHg,or/and DBP  95mmHg, </li></ul></ul></ul><ul><ul><ul><li>Anaemia: haemoglobin is less than 12mg% in adult women. </li></ul></ul></ul>
  7. 7. Way of diagnosis <ul><li>Assembling a diagnostic entity (category) </li></ul><ul><ul><li>Diagnosis based on several variable: made by an “abnormal” value of several physiological function. </li></ul></ul><ul><ul><ul><li>Metabolism syndrome: Hypertension, overweight, obesity in abdomen, high blood lipid, high blood glucose. </li></ul></ul></ul>
  8. 8. Diagnostic testing <ul><li>A “diagnostic test” originally is meant a test performed in a laboratory. </li></ul><ul><li>In this chapter, including histories of disease, signs, symptoms, physical exams, special tests (x-ray, ECG, CT, cell counts, etc) </li></ul>
  9. 9. Section B Assessment on Diagnostic Testing
  10. 10. Diagnostic Tests <ul><li>Suppose a researcher had developed a new test for diagnosing the presence of disease A </li></ul><ul><li>The new test is half the price of the current test for the same disease and can be administered during a routine checkup, as opposed to a half day hospital stay </li></ul>
  11. 11. Diagnostic Tests <ul><li>From a cost benefits perspective, this new test sounds like a winner </li></ul><ul><li>However, before it becomes part of standard medical practice, it is important to evaluate the accuracy of this test compared to the existing technology </li></ul>
  12. 12. <ul><li>Is the diagnostic test valid??? </li></ul><ul><li>Goal: Evaluate the “accuracy” of the new test </li></ul>
  13. 13. <ul><li>diagnose individuals for “disease” using both </li></ul><ul><ul><li>Gold standard (perfect) </li></ul></ul><ul><ul><li>New diagnostic test </li></ul></ul>The way to evaluate a new diagnostic test Continued
  14. 14. Selection of Gold standard <ul><li>A gold standard test is currently recognized as the most reliable test. </li></ul><ul><ul><li>Tissue diagnosis </li></ul></ul><ul><ul><li>Radiological exam </li></ul></ul><ul><ul><li>Autopsies </li></ul></ul><ul><ul><li>Prolonged follow-up </li></ul></ul>
  15. 15. Selection of subjects <ul><li>Target patients </li></ul><ul><ul><li>Patients diagnosed by “gold standard test” </li></ul></ul><ul><ul><li>Typical and untypical patients </li></ul></ul><ul><ul><li>Patients in early, middle and later period </li></ul></ul><ul><ul><li>Light, middle and serious patients </li></ul></ul><ul><ul><li>With and without complications </li></ul></ul>
  16. 16. Selection of subjects <ul><li>Patients without target disease as control group </li></ul><ul><li>Healthy persons are not suitable for as control group </li></ul>
  17. 17. <ul><li>Blindly comparing the results between the gold standard test and the new test </li></ul>
  18. 18. <ul><li> GOLD STANDARD </li></ul><ul><li>(The Truth) </li></ul><ul><li> Yes (+) No (–) </li></ul><ul><li> Yes (+) </li></ul><ul><li> No (–) </li></ul><ul><li> </li></ul><ul><li>  </li></ul>Creating a 2 x 2 Table New Test True P False P False N True N
  19. 19. Dichotomous model <ul><li>Test true from Dichotomization </li></ul><ul><li>Types of true </li></ul><ul><ul><li>True Positives = positive tests that are correct = a </li></ul></ul><ul><ul><li>True Negatives = negative tests that are correct = d </li></ul></ul>
  20. 20. Dichotomous model <ul><li>Test Errors from Dichotomization </li></ul><ul><li>Types of errors </li></ul><ul><ul><li>False Positives = positive tests that are wrong = b </li></ul></ul><ul><ul><li>False Negatives = negative tests that are wrong = c </li></ul></ul>
  21. 21. DIAGNOSTIC ACCURACY OF A TEST Definition The extent to which the results of a diagnostic test reflect true disease status Terminology Accuracy / Validity interchangeable
  22. 22. Indicators for assessing diagnostic test
  23. 23. <ul><li>Measures of diagnostic accuracy </li></ul><ul><li>Sensitivity </li></ul><ul><li>Specificity </li></ul><ul><li>Predictive values </li></ul><ul><li>Measures of reliability / reproductivity </li></ul><ul><li>Percent agreement </li></ul>
  24. 24. <ul><li>The ability of the test to detect the presence of disease (i.e. abnormality) </li></ul><ul><li>The proportion of those with the disease who test positive (positive in disease, PID) </li></ul><ul><li>True Positives </li></ul><ul><li>True Positives + False Negatives </li></ul><ul><li>a / a+c </li></ul>Sensitivity
  25. 25. Developmental characteristics: test parameters <ul><li>Sensitivity = Pr(T+|D+) = a/(a+c) </li></ul><ul><li>Sensitivity is PID (Positive In Disease) </li></ul>
  26. 26. <ul><li>The ability of the test to detect freedom from disease (i.e. normality) </li></ul><ul><li>The proportion of those without the disease who have a normal test (negative in health, NIH) </li></ul><ul><li>True Negatives </li></ul><ul><li>True Negatives + False Positives </li></ul><ul><li>d / b+d </li></ul>Specificity
  27. 27. Developmental characteristics: test parameters <ul><li>Specificity = Pr(T-|D-) = d/(b+d) </li></ul><ul><li>Specificity is NIH (Negative In Health) </li></ul>
  28. 28. Developmental characteristics: test parameters <ul><li>Pr(T+|D-) = F alse P ositive Rate ( FP rate) = </li></ul><ul><li>b/(b+d) </li></ul>
  29. 29. Developmental characteristics: test parameters <ul><li>Pr(T-|D+) = F alse N egative Rate ( FN rate) = </li></ul><ul><li>c/(a+c) </li></ul>
  30. 30. Developmental characteristics: test parameters <ul><li>Sensitivity = Pr(T+|D+) = 1 - FN rate </li></ul><ul><li>Specificity = Pr(T-|D-) = 1 - FP rate </li></ul>
  31. 31. Example <ul><li>Accuracy of an exercise test for diagnosing coronary artery disease </li></ul><ul><ul><li>Screen: A random sample of 1,442 patients with symptoms of coronary artery disease </li></ul></ul><ul><ul><li>Gold standard: Angiography </li></ul></ul><ul><ul><li>New diagnostic test: Exercise tolerance test (ECG) </li></ul></ul>
  32. 32. Resulting 2 x 2 Table <ul><li>Coronary Artery Disease </li></ul><ul><li>(based on angiography) </li></ul><ul><li> + – </li></ul><ul><li>Exercise + 800 115 915 </li></ul><ul><li>Tolerance test – 200 327 527 </li></ul><ul><li>Test 1000 442 1442 </li></ul><ul><li>Source: Weiner (1979) NEJM </li></ul>
  33. 33. Sensitivity and Specificity <ul><li>Sensitivity </li></ul><ul><ul><li>Proportion of those with disease who are positive on the new diagnostic test </li></ul></ul>Continued
  34. 34. Sensitivity and Specificity <ul><li>Specificity </li></ul><ul><ul><li>Proportion of those without disease who are negative on the new diagnostic test </li></ul></ul>
  35. 35. Positive and Negative Predictive Value <ul><li>Positive Predictive Value : Of all the people who tested positive for a disease, the proportion that actually has it </li></ul><ul><li>Negative Predictive Value : Of all the people who tested negative for a disease, the proportion that actually does not have it </li></ul><ul><li>In these patients, what you know are their test results, from which you are trying to determine whether they actually have the disease. </li></ul>
  36. 36. Positive Predictive Value <ul><li>Referring back to the exercise tolerance test: </li></ul><ul><ul><li>We want to know the chances of having coronary artery disease for someone who tests positive with the exercise tolerance test </li></ul></ul>
  37. 37. Resulting 2 x 2 Table <ul><li>Coronary Artery Disease </li></ul><ul><li>(based on angiography) </li></ul><ul><li> + – </li></ul><ul><li>Exercise + 800 115 915 </li></ul><ul><li>Tolerance test – 200 327 527 </li></ul><ul><li>Test 1000 442 1442 </li></ul><ul><li>Source: Weiner (1979) NEJM </li></ul>
  38. 38. Positive Predictive Value <ul><li>Positive predictive value (PPV) = </li></ul><ul><ul><li>The proportion of all individuals with a positive test who actually have the disease </li></ul></ul>Continued
  39. 39. Positive Predictive Value <ul><li>Positive predictive value (PPV) = </li></ul><ul><ul><li>“ Given that someone has a positive test result, what are the chances this person has the disease?” </li></ul></ul>Continued
  40. 40. Positive Predictive Value <ul><li>This is not the same as sensitivity </li></ul><ul><li>Sensitivity = </li></ul><ul><ul><li>“ Given that someone has the disease, what are the chances this person gets a positive result?” </li></ul></ul>
  41. 41. Negative Predictive Value <ul><li>Referring back to the exercise tolerance test: </li></ul><ul><ul><li>We want to know the chances of not having coronary artery disease for someone who tests negative with the exercise tolerance test </li></ul></ul>Continued
  42. 42. Negative Predictive Value <ul><li>Negative predictive value (NPV) = </li></ul><ul><ul><li>The proportion of all individuals with a negative test who do not have the disease </li></ul></ul>
  43. 43. Negative Predictive Value <ul><li>Negative predictive value (NPV) = </li></ul><ul><ul><li>“ Given that someone has a negative test result, what are the chances this person does not have the disease?” </li></ul></ul>Continued
  44. 44. Negative Predictive Value <ul><li>This is not the same as specificity </li></ul><ul><li>Specificity = </li></ul><ul><ul><li>“ Given that someone does not have the disease, what are the chances this person gets a negative result?” </li></ul></ul>
  45. 45. Notes on Interpretation <ul><li>The positive predictive value is 88% and the negative predictive value is 62% </li></ul><ul><li>The sample was from a population of patients with symptoms of coronary artery disease </li></ul>Continued
  46. 46. Notes on Interpretation <ul><li>Interpretation </li></ul><ul><ul><li>If you have symptoms of coronary disease and you have a positive exercise test, there is an 88% chance you have coronary artery disease </li></ul></ul><ul><ul><li>If you have a negative test result, there is a 62% you do not have coronary artery disease </li></ul></ul>Continued
  47. 47. Notes on Interpretation <ul><li>However, in an asymptomatic population the positive predictive value might be much lower </li></ul><ul><li>These estimates only apply to the population tested—the population of individuals with symptoms of coronary artery disease </li></ul>
  48. 48. Summary <ul><li>Sensitivity and specificity do not depend on prevalence of a disease and can always be estimated in a diagnostic test </li></ul><ul><li>PPV and NPV do depend on the population prevalence of disease </li></ul>Continued
  49. 49. Summary <ul><li>If we start with a completely random sample, we can also estimate PPV and NPV for the population from which we have sampled </li></ul><ul><li>If we want to estimate PPV and NPV for a different population we will need more machinery </li></ul>Continued
  50. 50. Summary <ul><li>If we over sample cases we will need more machinery to estimate PPV and NPV in a population with a given prevalence of disease </li></ul>
  51. 51. Likelihood Ratios and Post-Test Disease Probability
  52. 52. Likelihood Ratio(LR) <ul><li>An LR is the probability of a particular test result for a person with the disease of interest divided by the probability of that test result for a person without the disease of interest </li></ul>
  53. 53. Clinical Interpretation: likelihood ratios <ul><li>Likelihood ratio = </li></ul><ul><li>Pr{test result|disease present} </li></ul><ul><li>Pr{test result|disease absent} </li></ul><ul><li>LR+ = Pr{T+|D+}/Pr{T+|D-} = Sensitivity/(1-Specificity)=0.93/(1-0.92)=11.63 </li></ul><ul><li>LR- = Pr{T-|D+}/Pr{T-|D-} = (1-Sensitivity)/Specificity=(1-0.93)/0.92=0.08 </li></ul>
  54. 54. <ul><li>Pretest probability of disease </li></ul><ul><li>0.13 </li></ul>
  55. 55. Pretest odds of disease <ul><li>Pretest odds of disease are defined as the estimate before diagnostic testing of the probability that a patient has the disease of interest divided by the probability that the patient does not have the disease of interest. </li></ul><ul><li>Pretest odds=Pretest probability/(1- Pretest probability) </li></ul><ul><li>=0.13/(1-0.13=0.13/0.87=0.15 </li></ul>
  56. 56. Posttest odds of disease <ul><li>Posttest odds of disease are defined as the estimate after diagnostic testing of the probability that a patient has the disease of interest divided by the probability that the patient does not have the disease of interest. </li></ul><ul><li>Posttest odds=Pretest probability  LR + </li></ul><ul><li>=0.15  11.63=1.76 </li></ul>
  57. 57. Posttest probability <ul><li>Posttest probability=Posttest odds/(1+ Posttest odds) </li></ul><ul><li>=1.76/(1+1.76)=1.76/2.76=0.64 </li></ul>
  58. 58. Clinical interpretation of post-test probability Disease ruled out Disease ruled in
  59. 59. Advantages of LRs <ul><li>The higher or lower the LR, the higher or lower the post-test disease probability </li></ul><ul><li>Which test will result in the highest post-test probability in a given patient? </li></ul><ul><li>The test with the largest LR+ </li></ul><ul><li>Which test will result in the lowest post-test probability in a given patient? </li></ul><ul><li>The test with the smallest LR- </li></ul>
  60. 60. Advantages of LRs <ul><li>Clear separation of test characteristics from disease probability. </li></ul>
  61. 61. Likelihood Ratios - Advantage <ul><li>Provide a measure of a test’s ability to rule in or rule out disease independent of disease probability </li></ul><ul><li>Test A LR+ > Test B LR+ </li></ul><ul><ul><li>Test A PV+ > Test B PV+ always! </li></ul></ul><ul><li>Test A LR- < Test B LR- </li></ul><ul><ul><li>Test A PV- > Test B PV- always! </li></ul></ul>
  62. 62. Figure 1a : Likelihood Ratio Nomogram
  63. 63. Figure 1b : Likelihood Ratio Nomogram
  64. 64. Developmental characteristics: Cut-points and Receiver Operating Characteristic (ROC) Healthy
  65. 65. Developmental characteristics: Cut-points and Receiver Operating Characteristic (ROC) Healthy Sick
  66. 66. Developmental characteristics: Cut-points and Receiver Operating Characteristic (ROC) Fals pos= 20% True pos=82%
  67. 67. Developmental characteristics: Cut-points and Receiver Operating Characteristic (ROC) F pos= 100% T pos=100%
  68. 68. Developmental characteristics: Cut-points and Receiver Operating Characteristic (ROC) Fals pos= 9% True pos=70%
  69. 69. Developmental characteristics: Cut-points and Receiver Operating Characteristic (ROC) T neg= 100% F neg=100%
  70. 70. Developmental characteristics: Cut-points and Receiver Operating Characteristic (ROC) “ F pos + T pos “ is the highest
  71. 72. Developmental characteristics: Cut-points and Receiver Operating Characteristic (ROC) Receiver Operating Characteristic (ROC)
  72. 73. Developmental characteristics: Cut-points and Receiver Operating Characteristic (ROC) Receiver Operating Characteristic (ROC)
  73. 74. Receiver Operating Characteristic (ROC) <ul><li>ROC Curve allows comparison of different tests for the same condition without (before) specifying a cut-off point. </li></ul><ul><li>The test with the largest AUC (Area under the curve) is the best. </li></ul>
  74. 76. Section C <ul><li>Diagnostic strategies </li></ul>
  75. 77. Combination tests: serial and parallel testing <ul><li>Combinations of specificity and sensitivity superior to the use of any single test may sometimes be achieved by strategic uses of multiple tests. There are two usual ways of doing this. </li></ul><ul><li>Serial testing: Use >1 test in sequence, stopping at the first negative test. Diagnosis requires all tests to be positive. </li></ul><ul><li>Parallel testing: Use >1 test simultaneously, diagnosing if any test is positive. </li></ul>
  76. 78. Combination tests: serial testing <ul><li>Doing the tests sequentially, instead of together with the same decision rule, is a cost saving measure. </li></ul><ul><li>This strategy </li></ul><ul><ul><li>increases specificity above that of any of the individual tests, but </li></ul></ul><ul><ul><li>degrades sensitivity below that of any of them singly. </li></ul></ul><ul><li>However, the sensitivity of the serial combination may still be higher than would be achievable if the cut-point of any single test were raised to achieve the same specificity as the serial combination. </li></ul>
  77. 79. Combination tests: serial testing Demonstration: Serial Testing with Independent Tests <ul><li>Se SC = sensitivity of serial combination </li></ul><ul><ul><li>Sp SC = specificity of serial combination </li></ul></ul><ul><li>Se SC = Product of all sensitivities= Se 1 X Se 2 X…etc Hence Se SC < all individual Se </li></ul><ul><li>1-Sp SC = Product of all(1-Sp) </li></ul><ul><li>Hence Sp SC > all individual Sp i </li></ul><ul><li>Serial test to rule-in disease </li></ul>
  78. 80. Combination tests: parallel testing <ul><li>Parallel Testing </li></ul><ul><li>Usual decision strategy diagnoses if any test positive. This strategy </li></ul><ul><ul><li>increases sensitivity above that of any of the individual tests, but </li></ul></ul><ul><ul><li>degrades specificity below that of any individual test. </li></ul></ul><ul><li>However, the specificity of the combination may be higher than would be achievable if the cut-point of any single test were lowered to achieve the same sensitivity as the parallel combination. </li></ul>
  79. 81. Combination tests: parallel testing Demonstration: Parallel Testing with Independent Tests <ul><li>Se PC = sensitivity of parallel combination </li></ul><ul><ul><li>Sp PC = specificity of parallel combination </li></ul></ul><ul><li>1-Se PC = Product of all(1 - Se) </li></ul><ul><li>Hence Se PC > all individual Se </li></ul><ul><li>Sp PC = Product of all Sp </li></ul><ul><li>Hence Sp PC < all individual Sp i </li></ul><ul><li>Parallel test to rule-out disease </li></ul>
  80. 82. Clinical settings for parallel testing <ul><li>Parallel testing is used to rule-out serious but treatable conditions (example rule-out MI by CPK, CPK-MB, Troponin, and EKG. Any positive is considered positive) </li></ul><ul><li>When a patient has non-specific symptoms, large list of possibilities (differential diagnosis). None of the possibilities has a high pretest probability. Negative test for each possibility is enough to rule it out. Any positive test is considered positive. </li></ul>
  81. 83. <ul><li>Because specificity is low, further testing is now required (serial testing) to make a diagnosis (Sp P In). </li></ul>
  82. 84. Clinical settings for serial testing <ul><li>When treatment is hazardous (surgery, chemotherapy) we use serial testing to raise specificity.(Blood test followed by more tests, followed by imaging, followed by biopsy). </li></ul>
  83. 85. Calculate sensitivity and specificity of parallel tests <ul><li>(Serial tests in HIV CDC exercise) </li></ul><ul><li>2 tests in parallel </li></ul><ul><li>1 st test sens = spec = 80% </li></ul><ul><li>2 nd test sens = spec = 90% </li></ul><ul><li>1-Sensitivity of combination = </li></ul><ul><li>(1-0.8)X(1-0.9)=0.2X0.1=0.02 </li></ul><ul><li>Sensitivity= 98% </li></ul><ul><li>Specificity is 0.8 X 0.9 = 0.72 </li></ul>
  84. 86. Increasing the prevalence of disease <ul><li>Referral process </li></ul><ul><li>Selected demographic groups </li></ul><ul><li>Specific of the clinical situation </li></ul>
  85. 87. Effect of prevalence on predictive value(Se=70%, Sp=90%) 93.0 50,000 Clinical suspicious prostatic nodule 5.6 500 Men, age 75or greater 0.4 35 General population PPV (%) Prevalence (case/100,000) Setting
  86. 88. Lead Time 1990 1997 2000 death Diagnosis and treatment Biologic onset of disease 1990 1994 2000 death Biologic onset of disease Screening: diagnosis & treatment
  87. 89. Length Bias 1995 2000 death Biologic onset of disease 1989 1994 death Biologic onset of disease Screening: diagnosis & treatment 2002 1994 Screening:
  88. 90. Thanks!!!

×