This document discusses key concepts for evaluating diagnostic tests, including sensitivity, specificity, and predictive values. It explains how the criterion used to classify test results can affect sensitivity and specificity, as shown by receiver operating characteristic (ROC) curves. ROC curves summarize how criteria impact the trade-off between sensitivity and specificity. Likelihood ratios quantify this relationship and are used to calculate pre-test and post-test probabilities of disease. The sequential use of multiple diagnostic tests with different likelihood ratios can progressively update the probability that a patient has a disease.

1. Shrikant R. Bharadwaj
EVALUATING A
DIAGNOSTIC
TEST – PART 2
BSopt., PhD

2. Attributes of a diagnostic test
Last class…
•Accuracy & Precision
•Sensitivity & Specificity
•Positive and Negative Predictive values
This class…
•Effect of criterion on sensitivity & specificity
•Receiver Operating Characteristics (ROC) curves
•Likelihood ratios
•Pre-test and post-test probabilities

3. Sensitivity & Specificity - revisited
Disease Present
Disease Absent
Test Positive
True Positive (TP)
False Positive (FP)
Test Negative
False Negative (FN)
True Negative (TN)
•
Sensitivity: How good is the test at identifying people with the disease in
a population
i.e. Sensitivity = TP / (TP + FN)
•
Specificity: How good is the test at eliminating people without the
disease in a population
i.e. Specificity = TN / (FP + TN)

4. Diabetic Retinopathy Example
Criterion 3: Very few false negatives and false positives
Criterion 6: Very large false negatives but only few false positives

5. Effect of Criterion
The Gaussian distributions are defined by the Gold-standard test
The criterion is set by the diagnostic test
Disease
absent
Disease
present

6. Effect of Conservative Criterion
Mitt Romney’s gang!
Disease absent
Disease
present

7. Effect of Liberal Criterion
Barack Obama’s gang!
Disease absent
Disease present

8. Criterion and the 2 x 2 Contingency Table
Disease
absent
Disease
present
TP
FN
True Positive = Right side of criterion in “diseased” distribution
False Negative = Left side of criterion in “diseased” distribution

9. Criterion on the 2 x 2 Contingency Table
Disease
absent
Disease
present
TN
FP
True Negative = Left side of criterion in “non-diseased” distribution
False Positive = Right side of criterion in “non-diseased” distribution

10. Criterion on the 2 x 2 Contingency Table
Disease
absent
Disease
present
TN
TP
Sensitivity
TP / (TP + FN)
Specificity
TN / (TN + FP)
FN FP
The values in the 2 x 2 contingency table will change depending on
the criteria used

11. Conservative Criterion
Disease
absent
Disease
present
Sensitivity
TP / (TP + FN)
Specificity
TN / (TN + FP)
TN
TP
FN
FP
TP and FP decrease while TN and FN increase
Sensitivity decreases while Specificity increases

12. Liberal Criterion
Disease
absent
Disease
present
TP
TN
FN
FP
TP and FP increase while TN and FN decrease
Sensitivity increases while Specificity decreases
Sensitivity
TP / (TP + FN)
Specificity
TN / (TN + FP)

13. Receiver Operating Characteristics (ROC) Curve
*
1
Curve summarizing the
impact of criteria on
sensitivity and
specificity
.9
*
.8
.7
This curve has its origin
in World War II
Sensitivity
.6
*
.5
*
*
Y-axis: Sensitivity
X-axis: 1 – Specificity
.4
.3
Steeper the ROC curve
better is the
performance of a
diagnostic test
.2
.1
*
0
0
.1
.2
.3
.4
.5
.6
1 - Specificity
.7
.8
.9
1

14. Family of ROC Curves
Good ROC curve is
obtained with the
sensitivity is high for a
whole range of
specificities
Good
ROC
curve
Not so
good
ROC
curve
The FN are small for a
whole range of FP
The test does not miss a
single person with the
disease, even if it
includes number of false
positives

15. Area under the ROC curve
1
.9
.8
.7
Sensitivity
.6
.5
.4
.3
.2
.1
0
0
.1
.2
.3
.4
.5
.6
1 - Specificity
.7
.8
.9
1

16. Concept II: Likelihood Ratio (LR)
•
Ratio of two likelihoods
1. Likelihood of result being expected in a diseased condition
2. Likelihood of result being expected in a non-diseased
condition
•
Likelihood is synonymous to “probability” or “chance”
•
Likelihood ratio for a given criterion is therefore equal to…
Likelihood of an individual with disease have a positive test
Likelihood of an individual without disease having a positive test
•
LR = Sensitivity / (1 – Specificity)
•
Larger the LR, better is the test result; LR = 1 indicates no
diagnostic value for the test

17. Likelihood Ratio (LR) Example
•
LR = Sensitivity / (1 – Specificity)
•
Three different diagnostic parameters for the diagnosis of
glaucoma
1. IOP cut-off of 21mmHg (Sensitivity: 50%; Specificity: 92%)
LR for IOP: 0.5 / (1 – 0.92) = 0.5 / 0.08 = 6.25
2. Optic disc changes (Sensitivity: 80%; Specificity: 95%)
LR for Optic disc: 0.8 / (1 – 0.95) = 0.8 / 0.05 = 16
3. GDx Nerve Fiber score >50 (Sensitivity: 53%; Specificity: 99%)
LR for GDx: 0.53 / (1 – 0.99) = 0.53 / 0.01 = 55
•
GDx has the maximum value as a diagnostic test in detecting
patients with glaucoma.

18. LR & ROC curve
Steeper the ROC curve better is
the LR & therefore better is the
diagnostic test
LR = Sensitivity / (1 –
Specificity)
LR = 0.8 / 0.8 = 1
The test that generated this
ROC curve and LR carry no
diagnostic value
LR = 0.8 / 0.5 = 1.6
LR = 0.8 / 0.15 = 5.3
LR = 0.8 / 0.05 = 16

19. Concept III: Pre-test and Post-test probabilities
•
Pre-test probability: The probability of the disease being present in
a given individual before the given diagnostic test is applied
•
Pre-test probability can be computed from…

General prevalence of the disease in the population

Probability estimated from another diagnostic test

Guesstimate!
•
Post-test probability: The probability of the disease being present
in a given individual after the given diagnostic test is applied
•
Post-test probability can be computed using…
• Likelihood ratios
• Predictive value
• Relative risk

20. Patient Example
•
•
45-yr old patient
IOP: 24mmHg; Gonioscopy: Open angles;
HVF: Normal
•
Pre-test probability: 5%
•
LR of IOP >21mmHg: 6.25
- Post-test probability = 30%
•
LR of Optic disc: 16
- Post-test probability = 48%
•
LR of GDX: 55
- Post-test probability = 78%

21. Example of Sequential Use of LR
•
•
45-yr old patient
IOP: 24mmHg; Gonioscopy: Open angles;
HVF: Normal
Step 1: Pre-test probability: 5%
LR of IOP >21mmHg: 6.25
Post-test probability of glaucoma = 30%
2
1
Step 2: Pre-test probability: 30%
LR of Optic disc: 16
Post-test probability of glaucoma = 87%
Step 3: Pre-test probability: 87%
LR of GDX: 55
Post-test probability of glaucoma = ~100%
3

22. Summary
Diagnostic parameters
Diagnostic parameters
Accuracy / /Precision
Accuracy Precision
Sensitivity / /Specificity
Sensitivity Specificity
ROC curves
ROC curves
Ability to detect / /eliminate disease
Ability to detect eliminate disease
Positive / /Negative predictive values
Positive Negative predictive values
Likelihood ratios
Likelihood ratios
Pre- & Post-test probabilities
Pre- & Post-test probabilities
ROC curves & LR are
different avatars of
the same logic
Confidence booster
for disease
diagnosis

23. THANK YOU