Validity is the ability of a test to indicate which individuals have the disease and which do not
Sensitivity The ability of the test to identify correctly those who have the disease
Specificity The ability of the test to identify correctly those who do not have the disease
Gold standard test is the best test available. It is often invasive or expensive
A new test is, for example, a new screening test or a less expensive diagnostic test
Suspectd target condition
Gold
standard test
Positive
Negative
Diagnostic
test
Positive
Negative
Accuracy of diagnostic test compared to gold standard test
 Dichotomous test (only 2 results)
Sensibility (Sn) & Specificity (Sp)
Positive Predictive Value (PPV)
Negative Predictive Value (NPV)
Likelihood Ratios + and –ve (LR)
Diagnostic Odds Ratio (OR)
 Multilevel test ( > 2 results)
Receiver Operating haracteristic (ROC)
+ -
a
(True positives)
b
(All people without
disease)
c d
(True negatives)
New Test
+
-
DISEASE
 The ability of a diagnostic test (or procedure) to correctly classify
individuals into two categories (positive and negative) is
assessed by two parameters, sensitivity and specificity
Sensitivity
P[T +|D +]
True Positive Rate
Proportion of true positives correctly identified as such
=1-false negative rate
Specificity
P[T-|D -]
True Negative Rate
Proportion of true negatives correctly identified as such
=1- false positive rate
+ + + +
+ + + +
+ + + +
+ + + +
+ + + +
+ + + +
+ + + +
+ + + +
+ + + +
+ + + +
+ + + +
+ + + +
+ + + +
+ + + +
+ + + +
+ + + +
+ + + +
+ + + +
+ + + +
+ + + +
• 100% Sensitive
• 100% Specific
• 100% PPV
• 100% NPV
+ + + + + + + + + +
+ + + + + + + + + +
+ + + +
+ + + +
+ + + +
+ + + +
+ + + +
+ + + +
+ + + + + + + + + +
+ + + + + + + + + +
+ + + +
+ + + +
+ + + +
+ + + +
+ + + +
+ + + +
32
True positives:
test + ve
and
disease is present
+ + + + + + + + + +
+ + + + + + + + + +
+ + + +
+ + + +
+ + + +
+ + + +
+ + + +
+ + + +
False positives:
test + ve
and
disease is absent
12
+ + + + + + + + + +
+ + + + + + + + + +
+ + + +
+ + + +
+ + + +
+ + + +
+ + + +
+ + + +
True Negative
test - ve
and
disease is also absent
48
+ + + + + + + + + +
+ + + + + + + + + +
+ + + +
+ + + +
+ + + +
+ + + +
+ + + +
+ + + +
False Negative
test - ve
and
disease is present
8
+ + + + + + + + + +
+ + + + + + + + + +
+ + + + + + + + + +
+ + + + + + + + + +
+ + + + + + + + + +
- - - - - - - - - -
- - - - - - - - - -
- - - - - - - - - -
- - - - - - - - - -
- - - - - - - - - -
Test +
Test -
Disease + Disease -
Total
TRUE +
FALSE - TRUE -
FALSE +
a b
c d
TEST
+ -
a
(True positives)
b
c
d
(True negatives)
DISEASE
+
-
+ + + +
+ + + +
+ + + +
+ + + +
+ + + +
+ + + +
+ + + +
+ + + +
True + = 32
Diseased = 40
= 80%
+ + + +
+ + + +
+ + + +
+ + + +
+ + + +
+ + + +
+ + + +
+ + + +
Sensitivity = =
= Pr (T+ |D+)
+
true positives
disease +
DISEASE
+ -
a
(True positives)
b
(All people without
disease)
c
d
(True negatives)
TEST
+
-
Sensitivity
Mnemonic : Sens = PID “Positivity in Disease”
a
c
aa
c
 Sensitivity is the ability of the test to identify
correctly those who have the disease (a) from
all individuals with the disease (a+c)
True - = 48
Non-Diseased = 60
= 80%
+ + + + + +
+ + + + + +
Specificity = =
= Pr (T- |D-)
+
true negatives
disease -
DISEASE
+ -
a
(True positives)
b
c
d
(True negatives)
TEST +
- d
b
d
 Specificity is the ability of the test to
identify who do not have the disease (d) from
all individuals free from the disease (b+d)
“Sensitivity, specificity are useful to the clinician who
is trying to arrive a diagnosis”
Reality:
 No
We do not know the Gold std test result when we are seeing the
patient
 All the above assume that the GST status of the patient is known
SpPin
If a Specific test is Positive then Disease is
‘IN’
PID (Pelvic Inflammatory Disease)
Positivity in Disease
• Sackett et al
SnNout
If a Sensitive test is Negative then Disease is
‘OUT’
NIH (National Institute Of Health)
Negative in Health
• Sackett et al
“Sensitivity, Specificity: Statisticians”
“Predictive Values : Physicians”
Accuracy =
a+b
a+b+c+d
TEST
+ -
a
(True positives)
b
c
d
(True negatives)
DISEASE
+
-
The proportion of all tests
that are correct classifications
 Closeness of an estimate to the exact
or true value
+ + + + + + + +
+ + + + + + + +
+ + + + + + + +
+ + + + + + + +
+ +
+ +
+ +
+ +
+ +
+ +
True +ve + True –ve / Total i.e. 80%
+ + + + + + + +
+ + + + + + + +
+ + + + + + + +
+ + + + + + + +
+ + + + + + + +
+ + + + + + + +
+ + + + + + + +
+ + + + + + + +
+ +
+ +
+ +
+ +
+ +
+ +
Screening
Results
True characteristics in population
Total
Disease No Disease
Positive
Negative
Total
A screening test is used to identify the disease
1000100 900
80
20
100
800
180
820
Screening Results
True characteristics in population
Total
Disease No Disease
Positive
Negative
Total
Sensitivity = 80/100 = 80% Specificity = 800/900=89%
1000100 900
80
20
100
800
180
820
Screening Results
True characteristics in population
Total
Disease No Disease
Positive
Negative
Total
Sensitivity = 80/100 = 80% Specificity = 800/900=89%
1000100 900
80
20
100
800
180
820
 The clinician knows that tricuspid regurgitation is a complication of
pulmonary hypertension and wonders how accurately a single
physical sign—the presence of a holosystolic murmur at the left
lower sternal border—detects this complication.
 In this study, 42 patients have significant tricuspid regurgitation and
58 patients do not
• sensitivity is 0.52 (22/42)
• specificity is 0.95 (55/58),
• the positive LR is 10.1
[(22/42)/(3/58)], and
• the negative LR is 0.5
[(20/42)/(55/58)].
Diabetics Non-
diabetics
High
Blood
Sugar
Low
20 diabetics
20 non-diabetics
Public Health Service, Diabetes program guide No. 506 DC, US Govt Printing Office 1960
Eg.
Diabetics Non-
diabetics
High
Blood
Sugar
Low
Subjects are screened
using fasting plasma
glucose with a low
(blood sugar) cut point
Diabetics Non-
diabetics
+ 17 14
- 3 6
20 20
Sensitivity = 85% ----- (17/20)
Specificity = 30% ----- ( 6/20)
Diabetics Non-
diabetics
High
Blood
Sugar
Low
Subjects are screened
using fasting plasma
glucose with a high
(blood sugar) cut point
Diabetics Non-
diabetics
+ 5 2
- 15 18
20 20
Sensitivity = 25% ---- ( 5/20)
Specificity = 90% ---- (18/20)
Diabetics Non-
diabetics
High
Blood
Sugar
Low
In a typical population,
there is no line
separating the two
groups, and the subjects
are mixed
Diabetics Non-
diabetics
High
Blood
Sugar
Low
In a typical population,
there is no line
separating the two
groups, and the subjects
are mixed
Diabetics Non-
diabetics
High
Blood
Sugar
Low
In fact, there is no color
or label
Diabetics Non-
diabetics
High
Blood
Sugar
Low
A screening test using a
high cut-point will treat
the bottom box as
normal and will identify
the 7 subjects above the
line as having diabetes
Diabetics Non-
diabetics
High
Blood
Sugar
Low
A screening test using a
high cut-point will treat
the bottom box as
normal and will identify
the 7 subjects above the
line as having diabetes
But a low cut-point will
result in identifying 31
subjects as having
diabetes
 Different cut-points yield different sensitivities and
specificities
 The cut-point determines how many subjects will be
considered as having the disease
 The cut-point that identifies more true negatives will
also identify more false negatives
 The cut-point that identifies more true positives will
also identify more false positives
Fasting Sugar Levels 2 hr
after eating (mg/100 ml)
Sensitivity (%) Specificity (%)
70 98.6 8.8
80 97.1 25.5
90 94.3 47.6
100 88.6 69.8
110 85.7 84.1
120 71.4 92.5
130 64.3 96.9
140 57.1 99.4
150 50.0 99.6
160 47.1 99.8
170 42.9 100.0
180 38.6 100.0
200 27.1 100.0
 A receiver operating characteristic curve (ROC) is a graphical plot that
illustrates the diagnostic ability of a binary characteristic system as its
discrimination threshold is varied.
 ROC is a way to compare diagnostic tests.
 The name came from Signal Detection Theory developed during WW II for
analysis of RADAR images.
 AUC - ROC curve are performance
measurements for classification problem
at various thresholds settings.
 ROC is a probability curve and
 AUC represents degree or measure of
separability.
 Higher the AUC, better the model is at
distinguishing between patients with
disease and no disease.
• AUROC must be > 0.5 otherwise it is like
tossing a coin
• Lower end of 95% CI limit of the AUROC must
be > 0.5
• When comparing tests, the test with a higher
AUROC is better
Area of
triangle
= 0.5
The ROC curve is plotted with TPR against the FPR
where TPR is on y-axis and FPR is on the x-axis.
TPR
FPR
Curve 1:
• represents a test with high sensitivity (high TP rate) and with high
specificity (low FP rate) at all cut point values.
Curve 2:
• illustrates a test with lower sensitivity and specificity.
Curve 3:
• corresponds to a totally uninformative test: at any level of the cut
point, the test has equal chances of labelling a normal and a diseased
individual as “positive.”
TPR
• A common criterion for test selection
• optimal TP rate (i.e., optimal sensitivity) at
• an acceptable level of FP rate (i.e., at an acceptable specificity).
• Alternately, a test may be selected by the overall features of its ROC
curve: a common “omnibus” criterion for optimality is a large area
under the curve (AUC).
AUC measures the probability that a pair of individuals, one with and one
without the disease, will both be correctly identified by the test.
1. Accuracy of the test - reflected in the sensitivity and
specificity
 test’s ability to find true positives for the disorder (sensitivity)
or true negatives for the disorder (specificity).
2. Post-test probabilities and likelihood ratios
 concerns how the test performs in the population being tested
and is reflected in predictive values
SpPin
If a Specific test is Positive then Disease is
‘IN’
PID (Pelvic Inflammatory Disease)
Positivity in Disease
• Sackett et al
SnNout
If a Sensitive test is Negative then Disease is
‘OUT’
NIH (National Institute Of Health)
Negative in Health
• Sackett et al
“Sensitivity, specificity are useful to the clinician who
is trying to arrive a diagnosis”
Reality:
No
We do not know the Gold std test result when we are seeing the patient
All the above assume that the GST status of the patient is known
“Sensitivity, Specificity: Statisticians”
“Predictive Values : Physicians”
Predictive Values
“It is a mathematical formula for determining conditional probability”
 Positive predictive value (PPV): The proportion of patients who
test positive who actually have the disease
 Negative predictive value (NPV): The proportion of patients who
test negative who are actually free of the disease
 A clinician who wishes to interpret the results of a diagnostic test will want to
know the probability that a patient is truly positive if the test is positive and
similarly the probability that the patient is truly negative if the test is negative.
Positive predictive value (PPV) =
proportion of test positives that are truly positive
Negative predictive value (NPV) =
proportion of test negatives that are truly negative
+ + + + + +
+ + + + + +
+ + + +
+ + + +
+ + + +
+ + + +
+ + + +
+ + + +
+ + + +
+ + + +
+ + + + + +
+ + + + + +
True + = 32
PPV = 72.7%
+ + + +
+ + + +
+ + + +
+ + + +
+ + + +
+ + + +
+ + + +
+ + + + All + = 44
+ + + +
+ + + +
+ + + +
+ + + +
+ + + +
+ + + +
+ + + +
+ + + +
+ + + + + +
+ + + + + +
+ + + +
+ + + +
+ + + +
+ + + +
+ + + +
+ + + +
+ + + +
+ + + +
+ + + + + +
+ + + + + +
+ + + +
+ + + +
+ + + +
+ + + +
+ + + +
+ + + +
+ + + +
+ + + +
NPV = 86%
True -ve = 48
All –ve = 56
PPV = = Pr (D + |T+)
+
TEST
+ -
a
(True positives)
b
(All people without
disease)
c
d
(True negatives)
DISEASE
+
-
(probability of disease
given positive test)
aa
b
NPV = = Pr (D - |T-)
+
TEST
+ -
a
(True positives)
b
(All people without
disease)
c
d
(True negatives)
DISEASE
+
-
(probability of no disease
given negative test)
c
dd
Screening
Results
True characteristics in
population Total
Disease No Disease
Positive
Negative
Total 1000100 900
80
20
100
800
180
820
Screening Results
True characteristics in population
Total
Disease No Disease
Positive
Negative
Total
Positive Predictive value= 80/180 = 44%
1000100 900
80
20
100
800
180
820
Negative Predictive value= = 800/820=98%
 The prevalence of the disease in the population tested, and
the test itself (sensitivity and specificity).
 In general, it depends more on the specificity (and less on
the sensitivity) of the test (if the disease prevalence is low)
Disease
Prevalence
Test
Results Sick Not Sick Totals
Positive
Predictive
Value
1 %
+ 99 495 594
99 / 594
= 17 %
- 1 9,405 ,
Totals 100 9,900 10,000
5 %
+ 495 475 970
495 / 970
= 51 %- 5 9025 9,303
Totals 500 9,500 10,000
Example: Sensitivity = 99% ; Specificity = 95%
• Mausner JS, Kramer S. Epidemiology: an Introductory Text. Philadelphia, WB Saunders 1985, p221.
Disease
+ -
250 250
250 250
Test
+
-
Prevalence = 50%
Sensitivity = 50%
Specificity = 50%
PPV = 250/500 = 50%
500
500
1000500 500
Disease
+ -
100 400
100 400
Test
+
-
Prevalence = 20%
Sensitivity = 50%
Specificity = 50%
PPV = 100/500 = 20%
500
500
1000200 800
Change
Prevalence
Disease
+ -
180 400
20 400
Test
+
-
Prevalence = 20%
Sensitivity = 90%
Specificity = 50%
PPV = 180/580 = 31%
580
420
1000200 800
Change
Sensitivity
Disease
+ -
100 80
100 720
Test
+
-
Prevalence = 20%
Sensitivity = 50%
Specificity = 90%
PPV = 100/180 = 56%
180
820
1000200 800
Restore Sensitivity
Change Specificity
“We can get the PPV and NPV of a test derived from
another setting and start using it in our own setting”
Reality: PPV and NPV change depending on disease
prevalence.
As prevalence increases, PPV increases & NPV decreases
Examples:
 Mantoux test has higher PPV in India than USA
 Sweat Chloride has higher PPV in USA than India
PRETEST PROBABILITY
• Pretest probability is the probability of disease (i.e., prevalence)
before application of the results of a physical finding.
• It is the starting point for all clinical decision
• Eg. Clinician may know that a certain physical finding shifts the probability of
disease upward 40%
• If the pretest probability for the particular diagnosis was 50%, the finding is
diagnostic (i.e., post-test probability 50% + 40% = 90%);
• if the pretest probability was only 10%, the finding is less helpful, because the
probability of disease is still the flip of a coin (i.e., post-test probability 10% +
40% = 50%).
Likelihood ratios
 One of the best measures of diagnostic accuracy
 Interpreting them requires a calculator to convert back and forth
between probability of disease (a term familiar to all clinicians) and
odds of disease
Probability of finding in persons with disease
Probability of same finding in persons with out disease
=
sensitivity
False positive rate
=
sensitivity
1 - specificity
P(T+ |D +)
P(T+ |D -)
Specificity (TNR) = P(T- |D-)
FPR = (1-Specificity) = P(T+ |D -)
P(T- |D+)
P(T- | D-)
=
1- sensitivity
specificity
TPR = Sensitivity = P[T +|D+]
TNR = (1-Sensitivity) = P(T- |D -)
=
False negative rate
Specificity
Likelihood ratios Eg.
 LRs may range from 0 to infinity.
 LRs greater than 1 argue for the diagnosis of interest
 The bigger the number, the more convincingly the finding suggests
that disease
 Findings whose LRs lie between 0 and 1 argue against the diagnosis
of interest; the closer the LR is to 0, the less likely the disease.
 Findings whose LRs equal 1 lack diagnostic value.
Likelihood ratios Eg.
 Among patients with abdominal distension who undergo
ultrasonography - the physical sign “bulging flanks” is present in 80%
of patients with confirmed ascites and in 40% without ascites (i.e., the
distension is from fat or gas).
 The LR for “bulging flanks” in detecting ascites = 2 (i.e., 80% divided by 40%)
 The finding of “flank tympany” is present in 10% of patients with
ascites but in 30% with distension from other causes.
 The LR for “flank tympany” in detecting ascites is 0.3 (i.e., 10% divided
by 30%).
 How much does the finding of bulging flanks (LR = 2.0) argue for
ascites, and
 How much does the finding of flank tympany (LR = 0.3) argue
against it?
 To answer these questions using traditional methods, clinicians
must first identify the pretest probability (or prevalence) of
ascites in their practice and then perform 3 calculations.
Three ways:
(1) using graphs or other easy-to-use nomograms ;
(2) using bedside approximations, or
(3) using formulas.
 Only the curves for seven
likelihood ratios are
depicted
(from LR = 0.1 to LR = 10)
(2) Likelihood Ratios and Bedside Estimates
Likelihood Ratio Approximate Change in
Probability
0.1 -45%
0.2 -30%
0.3 -25%
0.4 -20%
0.5 -15%
1 No change
2 +15%
3 +20%
4 +30%
5 +35%
6
7
8 +40%
9
10 +45%
• McGee S. Simplifying likelihood ratios. J Gen Intern Med.
2002;17:646-64
• Simply remembering 3 specific LRs—2, 5, and 10—and the first
3 multiples of 15 (i.e., 15, 30, and 45)
• A LR of 2 increases probability 15%, one of 5 increases it
30%, and one of 10 increases it 45%
• For those LRs between 0 and 1, the clinician simply inverts
2, 5, and 10 (i.e., 1/2 = 0.5, 1/5 = 0.2, 1/10 = 0.1).
• LR of 2.0 increases probability 15%, its inverse, 0.5,
decreases probability 15%.
• Similarly, an LR of 0.2 (the inverse of 5) decreases
probability 30%, and a LR of 0.1 (the inverse of 10)
decreases it 45%.
(3) using formulas
CONVENTIONAL 3 calculations.
1. to then convert pretest probability (Ppre) to pretest odds (Opre),
using Opre = Ppre/(1 − Ppre),
2. then multiply the pretest odds (Opre) by the LR for the finding to
derive the posttest odds (i.e., Opost = LR × Opre), and
3. then convert posttest odds back to posttest probability, using
Ppost = Opost/(1 + Opost).
Finding of bulging flanks (LR = 2.0)
 If about 2 out of every 5 patients with abdominal distension have
ascites, the pretest probability is 40%.
1. Pretest odds = 0.4/(1 − 0.4) = 0.667
2. posttest odds = 0.667 × 2.0 = 1.333;
3. posttest probability = 1.333/(1 + 1.333) = 0.57 or 57%)
 Similar calculations reveal that the finding of flank tympany
decreases the probability of ascites from 40% to 17%.
 Simplicity
 If the LR of a finding is large, disease is likely, and if the LR of a finding is close to zero,
disease is doubtful.
 Accuracy
 superior to using sensitivity and specificity because the earlier described mnemonics,
SpPin and SnNout, are sometimes misleading.
 Levels of Findings
 A physical sign measured on an ordinal scale (e.g., 0, 1+, 2+, 3+) or a continuous scale
(e.g., blood pressure) can be categorized into different levels to determine the LR for
each level, thereby increasing the accuracy of the finding.
 Combining Findings
 Individual LRs can be combined, however, only if the findings are “independent.”
•
•
(= SENSITIVITY)
(1 – SPECIFICITY).
• SIX
LR POSITIVE TEST
•
• Findings with LRs greater than 1 argue for the specific disease (the greater the value of the LR,
the more the probability of disease increases).
• Findings with LRs less than 1 argue against the disease (the closer the number is to zero, the
more the probability of disease decreases). LRs that equal 1 do not change probability of
disease at all.
•
•
•
•
•
•
•
•
•
•
After having the results of the test, the
posttest probability of disease
if the test is normal, the posttest probability is
c/(c+d), &
if it is abnormal, the posttest probability is
a/(a+b)
The last is the same as the
PREDICTIVE VALUE OF A POSITIVE TEST.
DISEASE
+ -
a
(True positives)
b
(All people without
disease)
c d
(True negatives)
Test
+
-
EXAMPLE
•
•
FEVER+ COUGH+ EXPECTORATION : 4 WEEKS
(N=100)
TB No TB
Prevalence = Diseased / Total = 40/100 = 40 %
Hence pre-test probability of disease is
40 %
ESR
+ + + + + + + + +
+ + + + + + + + +
+ + + + + + + + +
+ + + + + + + + +
+ + + + +
+ + + + +
+ + + + +
+ + + + +
+ + + + +
+ + + + +
TB+
No TB
Sensitivity = a/(a+c) =
Specificity = d/(b+d) =
PPV = a/(a+b) =
NPV = d/(c+d) =
+ LR = Sn/(1-Sp) =
- LR = (1-Sn)/Sp =
Accuracy = (a+d)/(a+b+d+c) =
ESR+ ESR-
True positives = 36
False + ve = 30
False – ve = 4
True – ve = 30
a
36
b
30
c
4
d
30
ESR
+ + + + + + + + + +
+ + + + + + + + + +
+ + + + + + + + + +
+ + + + + + + + + +
+ + + +
+ + + +
+ + + +
+ + + +
+ + + +
TB+ No TB
Sensitivity = a/(a+c) = 90%
Specificity = d/(b+d) = 50%
PPV = a/(a+b) = 55%
NPV = d/(c+d) = 88%
+ LR = Sn/(1-Sp) = 1.8
- LR = (1-Sn)/Sp = 0.2
Accuracy = (a+d)/(a+b+d+c) = 66%
ESR+
ESR-
True positives = 36
False + ve = 30
False – ve = 4
True – ve = 30
a
36
b
30
c
4
d
30
SPUTUM AFB
+ + + +
+ + + +
+ + + +
+ + + +
+
+
TB+
No TB
AFB+ AFB -
True + ve = 16
False – ve = 24
False + ve = 2
True –ve = 58
Total = 100
Sensitivity = a/(a+c) =
Specificity = d/(b+d) =
PPV = a/(a+b) =
NPV = d/(c+d) =
+ LR = Sn/(1-Sp) =
- LR = (1-Sn)/Sp=
Accuracy = (a+d)/(a+b+d+c) =
a
16
c
24
b
2
d
58
Sensitivity = a/(a+c) = 40%
Specificity = d/(b+d) = 97%
PPV = a/(a+b) = 89%
NPV = d/(c+d) = 70%
+ LR = Sn/(1-Sp) = 12
- LR = (1-Sn)/Sp= 0.62
Accuracy = (a+d)/(a+b+d+c) = 74 %
 Is the test available, affordable, and accurate in our setting?
 Can a clinically sensible estimate of the pre-test probabilities of the
patient be made from personal experience, prevalence statistics,
practice databases, or primary studies?
 Are the study patients similar to the patient in question?
 How current is the study we are analysing — has evidence moved on
since the publication of the study?
 Could the result move the clinician across a test-treatment
threshold
 Will the patient be willing to have the test carried out?
 Will the results of the test help the patient reach their goals?
 Sens, spec are useful ways of describing a test; but do not help
in clinical decision making
 PPV & NPV enable diagnosis, when a test result is available;
but are influenced by prevalence
 LR allow post-test prob to be calculated and is uninfluenced by
prevalence
 For continuous variables, level of cut-off inversely related to
sensitivity
 Greater the area under ROC better is test

Diagnostic Tests for PGs

  • 2.
    Validity is theability of a test to indicate which individuals have the disease and which do not Sensitivity The ability of the test to identify correctly those who have the disease Specificity The ability of the test to identify correctly those who do not have the disease Gold standard test is the best test available. It is often invasive or expensive A new test is, for example, a new screening test or a less expensive diagnostic test
  • 3.
    Suspectd target condition Gold standardtest Positive Negative Diagnostic test Positive Negative Accuracy of diagnostic test compared to gold standard test
  • 4.
     Dichotomous test(only 2 results) Sensibility (Sn) & Specificity (Sp) Positive Predictive Value (PPV) Negative Predictive Value (NPV) Likelihood Ratios + and –ve (LR) Diagnostic Odds Ratio (OR)  Multilevel test ( > 2 results) Receiver Operating haracteristic (ROC)
  • 5.
    + - a (True positives) b (Allpeople without disease) c d (True negatives) New Test + - DISEASE
  • 6.
     The abilityof a diagnostic test (or procedure) to correctly classify individuals into two categories (positive and negative) is assessed by two parameters, sensitivity and specificity Sensitivity P[T +|D +] True Positive Rate Proportion of true positives correctly identified as such =1-false negative rate Specificity P[T-|D -] True Negative Rate Proportion of true negatives correctly identified as such =1- false positive rate
  • 7.
    + + ++ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + • 100% Sensitive • 100% Specific • 100% PPV • 100% NPV
  • 8.
    + + ++ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
  • 9.
    + + ++ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + 32 True positives: test + ve and disease is present
  • 10.
    + + ++ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + False positives: test + ve and disease is absent 12
  • 11.
    + + ++ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + True Negative test - ve and disease is also absent 48
  • 12.
    + + ++ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + False Negative test - ve and disease is present 8
  • 13.
    + + ++ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Test + Test - Disease + Disease - Total TRUE + FALSE - TRUE - FALSE + a b c d
  • 14.
  • 15.
    + + ++ + + + + + + + + + + + + + + + + + + + + + + + + + + + + True + = 32 Diseased = 40 = 80% + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
  • 16.
    Sensitivity = = =Pr (T+ |D+) + true positives disease + DISEASE + - a (True positives) b (All people without disease) c d (True negatives) TEST + - Sensitivity Mnemonic : Sens = PID “Positivity in Disease” a c aa c  Sensitivity is the ability of the test to identify correctly those who have the disease (a) from all individuals with the disease (a+c)
  • 17.
    True - =48 Non-Diseased = 60 = 80% + + + + + + + + + + + +
  • 18.
    Specificity = = =Pr (T- |D-) + true negatives disease - DISEASE + - a (True positives) b c d (True negatives) TEST + - d b d  Specificity is the ability of the test to identify who do not have the disease (d) from all individuals free from the disease (b+d)
  • 19.
    “Sensitivity, specificity areuseful to the clinician who is trying to arrive a diagnosis” Reality:  No We do not know the Gold std test result when we are seeing the patient  All the above assume that the GST status of the patient is known
  • 20.
    SpPin If a Specifictest is Positive then Disease is ‘IN’ PID (Pelvic Inflammatory Disease) Positivity in Disease • Sackett et al
  • 21.
    SnNout If a Sensitivetest is Negative then Disease is ‘OUT’ NIH (National Institute Of Health) Negative in Health • Sackett et al
  • 22.
  • 23.
    Accuracy = a+b a+b+c+d TEST + - a (Truepositives) b c d (True negatives) DISEASE + - The proportion of all tests that are correct classifications  Closeness of an estimate to the exact or true value
  • 24.
    + + ++ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + True +ve + True –ve / Total i.e. 80% + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
  • 25.
    Screening Results True characteristics inpopulation Total Disease No Disease Positive Negative Total A screening test is used to identify the disease 1000100 900 80 20 100 800 180 820
  • 26.
    Screening Results True characteristicsin population Total Disease No Disease Positive Negative Total Sensitivity = 80/100 = 80% Specificity = 800/900=89% 1000100 900 80 20 100 800 180 820
  • 27.
    Screening Results True characteristicsin population Total Disease No Disease Positive Negative Total Sensitivity = 80/100 = 80% Specificity = 800/900=89% 1000100 900 80 20 100 800 180 820
  • 28.
     The clinicianknows that tricuspid regurgitation is a complication of pulmonary hypertension and wonders how accurately a single physical sign—the presence of a holosystolic murmur at the left lower sternal border—detects this complication.  In this study, 42 patients have significant tricuspid regurgitation and 58 patients do not
  • 29.
    • sensitivity is0.52 (22/42) • specificity is 0.95 (55/58), • the positive LR is 10.1 [(22/42)/(3/58)], and • the negative LR is 0.5 [(20/42)/(55/58)].
  • 30.
    Diabetics Non- diabetics High Blood Sugar Low 20 diabetics 20non-diabetics Public Health Service, Diabetes program guide No. 506 DC, US Govt Printing Office 1960 Eg.
  • 31.
    Diabetics Non- diabetics High Blood Sugar Low Subjects arescreened using fasting plasma glucose with a low (blood sugar) cut point Diabetics Non- diabetics + 17 14 - 3 6 20 20 Sensitivity = 85% ----- (17/20) Specificity = 30% ----- ( 6/20)
  • 32.
    Diabetics Non- diabetics High Blood Sugar Low Subjects arescreened using fasting plasma glucose with a high (blood sugar) cut point Diabetics Non- diabetics + 5 2 - 15 18 20 20 Sensitivity = 25% ---- ( 5/20) Specificity = 90% ---- (18/20)
  • 33.
    Diabetics Non- diabetics High Blood Sugar Low In atypical population, there is no line separating the two groups, and the subjects are mixed
  • 34.
    Diabetics Non- diabetics High Blood Sugar Low In atypical population, there is no line separating the two groups, and the subjects are mixed
  • 35.
  • 36.
    Diabetics Non- diabetics High Blood Sugar Low A screeningtest using a high cut-point will treat the bottom box as normal and will identify the 7 subjects above the line as having diabetes
  • 37.
    Diabetics Non- diabetics High Blood Sugar Low A screeningtest using a high cut-point will treat the bottom box as normal and will identify the 7 subjects above the line as having diabetes But a low cut-point will result in identifying 31 subjects as having diabetes
  • 38.
     Different cut-pointsyield different sensitivities and specificities  The cut-point determines how many subjects will be considered as having the disease  The cut-point that identifies more true negatives will also identify more false negatives  The cut-point that identifies more true positives will also identify more false positives
  • 39.
    Fasting Sugar Levels2 hr after eating (mg/100 ml) Sensitivity (%) Specificity (%) 70 98.6 8.8 80 97.1 25.5 90 94.3 47.6 100 88.6 69.8 110 85.7 84.1 120 71.4 92.5 130 64.3 96.9 140 57.1 99.4 150 50.0 99.6 160 47.1 99.8 170 42.9 100.0 180 38.6 100.0 200 27.1 100.0
  • 40.
     A receiveroperating characteristic curve (ROC) is a graphical plot that illustrates the diagnostic ability of a binary characteristic system as its discrimination threshold is varied.  ROC is a way to compare diagnostic tests.  The name came from Signal Detection Theory developed during WW II for analysis of RADAR images.
  • 41.
     AUC -ROC curve are performance measurements for classification problem at various thresholds settings.  ROC is a probability curve and  AUC represents degree or measure of separability.  Higher the AUC, better the model is at distinguishing between patients with disease and no disease.
  • 42.
    • AUROC mustbe > 0.5 otherwise it is like tossing a coin • Lower end of 95% CI limit of the AUROC must be > 0.5 • When comparing tests, the test with a higher AUROC is better Area of triangle = 0.5 The ROC curve is plotted with TPR against the FPR where TPR is on y-axis and FPR is on the x-axis. TPR FPR
  • 43.
    Curve 1: • representsa test with high sensitivity (high TP rate) and with high specificity (low FP rate) at all cut point values. Curve 2: • illustrates a test with lower sensitivity and specificity. Curve 3: • corresponds to a totally uninformative test: at any level of the cut point, the test has equal chances of labelling a normal and a diseased individual as “positive.” TPR • A common criterion for test selection • optimal TP rate (i.e., optimal sensitivity) at • an acceptable level of FP rate (i.e., at an acceptable specificity). • Alternately, a test may be selected by the overall features of its ROC curve: a common “omnibus” criterion for optimality is a large area under the curve (AUC). AUC measures the probability that a pair of individuals, one with and one without the disease, will both be correctly identified by the test.
  • 44.
    1. Accuracy ofthe test - reflected in the sensitivity and specificity  test’s ability to find true positives for the disorder (sensitivity) or true negatives for the disorder (specificity). 2. Post-test probabilities and likelihood ratios  concerns how the test performs in the population being tested and is reflected in predictive values
  • 45.
    SpPin If a Specifictest is Positive then Disease is ‘IN’ PID (Pelvic Inflammatory Disease) Positivity in Disease • Sackett et al
  • 46.
    SnNout If a Sensitivetest is Negative then Disease is ‘OUT’ NIH (National Institute Of Health) Negative in Health • Sackett et al
  • 47.
    “Sensitivity, specificity areuseful to the clinician who is trying to arrive a diagnosis” Reality: No We do not know the Gold std test result when we are seeing the patient All the above assume that the GST status of the patient is known
  • 48.
  • 49.
    Predictive Values “It isa mathematical formula for determining conditional probability”  Positive predictive value (PPV): The proportion of patients who test positive who actually have the disease  Negative predictive value (NPV): The proportion of patients who test negative who are actually free of the disease
  • 50.
     A clinicianwho wishes to interpret the results of a diagnostic test will want to know the probability that a patient is truly positive if the test is positive and similarly the probability that the patient is truly negative if the test is negative. Positive predictive value (PPV) = proportion of test positives that are truly positive Negative predictive value (NPV) = proportion of test negatives that are truly negative
  • 51.
    + + ++ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
  • 52.
    + + ++ + + + + + + + + True + = 32 PPV = 72.7% + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + All + = 44 + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
  • 53.
    + + ++ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
  • 54.
    + + ++ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + NPV = 86% True -ve = 48 All –ve = 56
  • 55.
    PPV = =Pr (D + |T+) + TEST + - a (True positives) b (All people without disease) c d (True negatives) DISEASE + - (probability of disease given positive test) aa b
  • 56.
    NPV = =Pr (D - |T-) + TEST + - a (True positives) b (All people without disease) c d (True negatives) DISEASE + - (probability of no disease given negative test) c dd
  • 57.
    Screening Results True characteristics in populationTotal Disease No Disease Positive Negative Total 1000100 900 80 20 100 800 180 820
  • 58.
    Screening Results True characteristicsin population Total Disease No Disease Positive Negative Total Positive Predictive value= 80/180 = 44% 1000100 900 80 20 100 800 180 820 Negative Predictive value= = 800/820=98%
  • 59.
     The prevalenceof the disease in the population tested, and the test itself (sensitivity and specificity).  In general, it depends more on the specificity (and less on the sensitivity) of the test (if the disease prevalence is low)
  • 60.
    Disease Prevalence Test Results Sick NotSick Totals Positive Predictive Value 1 % + 99 495 594 99 / 594 = 17 % - 1 9,405 , Totals 100 9,900 10,000 5 % + 495 475 970 495 / 970 = 51 %- 5 9025 9,303 Totals 500 9,500 10,000 Example: Sensitivity = 99% ; Specificity = 95%
  • 61.
    • Mausner JS,Kramer S. Epidemiology: an Introductory Text. Philadelphia, WB Saunders 1985, p221.
  • 62.
    Disease + - 250 250 250250 Test + - Prevalence = 50% Sensitivity = 50% Specificity = 50% PPV = 250/500 = 50% 500 500 1000500 500
  • 63.
    Disease + - 100 400 100400 Test + - Prevalence = 20% Sensitivity = 50% Specificity = 50% PPV = 100/500 = 20% 500 500 1000200 800 Change Prevalence
  • 64.
    Disease + - 180 400 20400 Test + - Prevalence = 20% Sensitivity = 90% Specificity = 50% PPV = 180/580 = 31% 580 420 1000200 800 Change Sensitivity
  • 65.
    Disease + - 100 80 100720 Test + - Prevalence = 20% Sensitivity = 50% Specificity = 90% PPV = 100/180 = 56% 180 820 1000200 800 Restore Sensitivity Change Specificity
  • 66.
    “We can getthe PPV and NPV of a test derived from another setting and start using it in our own setting” Reality: PPV and NPV change depending on disease prevalence. As prevalence increases, PPV increases & NPV decreases Examples:  Mantoux test has higher PPV in India than USA  Sweat Chloride has higher PPV in USA than India
  • 67.
    PRETEST PROBABILITY • Pretestprobability is the probability of disease (i.e., prevalence) before application of the results of a physical finding. • It is the starting point for all clinical decision • Eg. Clinician may know that a certain physical finding shifts the probability of disease upward 40% • If the pretest probability for the particular diagnosis was 50%, the finding is diagnostic (i.e., post-test probability 50% + 40% = 90%); • if the pretest probability was only 10%, the finding is less helpful, because the probability of disease is still the flip of a coin (i.e., post-test probability 10% + 40% = 50%).
  • 68.
    Likelihood ratios  Oneof the best measures of diagnostic accuracy  Interpreting them requires a calculator to convert back and forth between probability of disease (a term familiar to all clinicians) and odds of disease
  • 70.
    Probability of findingin persons with disease Probability of same finding in persons with out disease
  • 71.
    = sensitivity False positive rate = sensitivity 1- specificity P(T+ |D +) P(T+ |D -) Specificity (TNR) = P(T- |D-) FPR = (1-Specificity) = P(T+ |D -)
  • 72.
    P(T- |D+) P(T- |D-) = 1- sensitivity specificity TPR = Sensitivity = P[T +|D+] TNR = (1-Sensitivity) = P(T- |D -) = False negative rate Specificity
  • 73.
    Likelihood ratios Eg. LRs may range from 0 to infinity.  LRs greater than 1 argue for the diagnosis of interest  The bigger the number, the more convincingly the finding suggests that disease  Findings whose LRs lie between 0 and 1 argue against the diagnosis of interest; the closer the LR is to 0, the less likely the disease.  Findings whose LRs equal 1 lack diagnostic value.
  • 74.
    Likelihood ratios Eg. Among patients with abdominal distension who undergo ultrasonography - the physical sign “bulging flanks” is present in 80% of patients with confirmed ascites and in 40% without ascites (i.e., the distension is from fat or gas).  The LR for “bulging flanks” in detecting ascites = 2 (i.e., 80% divided by 40%)  The finding of “flank tympany” is present in 10% of patients with ascites but in 30% with distension from other causes.  The LR for “flank tympany” in detecting ascites is 0.3 (i.e., 10% divided by 30%).
  • 75.
     How muchdoes the finding of bulging flanks (LR = 2.0) argue for ascites, and  How much does the finding of flank tympany (LR = 0.3) argue against it?  To answer these questions using traditional methods, clinicians must first identify the pretest probability (or prevalence) of ascites in their practice and then perform 3 calculations.
  • 76.
    Three ways: (1) usinggraphs or other easy-to-use nomograms ; (2) using bedside approximations, or (3) using formulas.
  • 77.
     Only thecurves for seven likelihood ratios are depicted (from LR = 0.1 to LR = 10)
  • 79.
    (2) Likelihood Ratiosand Bedside Estimates Likelihood Ratio Approximate Change in Probability 0.1 -45% 0.2 -30% 0.3 -25% 0.4 -20% 0.5 -15% 1 No change 2 +15% 3 +20% 4 +30% 5 +35% 6 7 8 +40% 9 10 +45% • McGee S. Simplifying likelihood ratios. J Gen Intern Med. 2002;17:646-64 • Simply remembering 3 specific LRs—2, 5, and 10—and the first 3 multiples of 15 (i.e., 15, 30, and 45) • A LR of 2 increases probability 15%, one of 5 increases it 30%, and one of 10 increases it 45% • For those LRs between 0 and 1, the clinician simply inverts 2, 5, and 10 (i.e., 1/2 = 0.5, 1/5 = 0.2, 1/10 = 0.1). • LR of 2.0 increases probability 15%, its inverse, 0.5, decreases probability 15%. • Similarly, an LR of 0.2 (the inverse of 5) decreases probability 30%, and a LR of 0.1 (the inverse of 10) decreases it 45%.
  • 80.
  • 81.
    CONVENTIONAL 3 calculations. 1.to then convert pretest probability (Ppre) to pretest odds (Opre), using Opre = Ppre/(1 − Ppre), 2. then multiply the pretest odds (Opre) by the LR for the finding to derive the posttest odds (i.e., Opost = LR × Opre), and 3. then convert posttest odds back to posttest probability, using Ppost = Opost/(1 + Opost).
  • 82.
    Finding of bulgingflanks (LR = 2.0)  If about 2 out of every 5 patients with abdominal distension have ascites, the pretest probability is 40%. 1. Pretest odds = 0.4/(1 − 0.4) = 0.667 2. posttest odds = 0.667 × 2.0 = 1.333; 3. posttest probability = 1.333/(1 + 1.333) = 0.57 or 57%)  Similar calculations reveal that the finding of flank tympany decreases the probability of ascites from 40% to 17%.
  • 83.
     Simplicity  Ifthe LR of a finding is large, disease is likely, and if the LR of a finding is close to zero, disease is doubtful.  Accuracy  superior to using sensitivity and specificity because the earlier described mnemonics, SpPin and SnNout, are sometimes misleading.  Levels of Findings  A physical sign measured on an ordinal scale (e.g., 0, 1+, 2+, 3+) or a continuous scale (e.g., blood pressure) can be categorized into different levels to determine the LR for each level, thereby increasing the accuracy of the finding.  Combining Findings  Individual LRs can be combined, however, only if the findings are “independent.”
  • 84.
    • • (= SENSITIVITY) (1 –SPECIFICITY). • SIX LR POSITIVE TEST
  • 85.
    • • Findings withLRs greater than 1 argue for the specific disease (the greater the value of the LR, the more the probability of disease increases). • Findings with LRs less than 1 argue against the disease (the closer the number is to zero, the more the probability of disease decreases). LRs that equal 1 do not change probability of disease at all.
  • 88.
  • 89.
  • 90.
  • 91.
    After having theresults of the test, the posttest probability of disease if the test is normal, the posttest probability is c/(c+d), & if it is abnormal, the posttest probability is a/(a+b) The last is the same as the PREDICTIVE VALUE OF A POSITIVE TEST. DISEASE + - a (True positives) b (All people without disease) c d (True negatives) Test + -
  • 92.
  • 93.
    FEVER+ COUGH+ EXPECTORATION: 4 WEEKS (N=100) TB No TB Prevalence = Diseased / Total = 40/100 = 40 % Hence pre-test probability of disease is 40 %
  • 94.
    ESR + + ++ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + TB+ No TB Sensitivity = a/(a+c) = Specificity = d/(b+d) = PPV = a/(a+b) = NPV = d/(c+d) = + LR = Sn/(1-Sp) = - LR = (1-Sn)/Sp = Accuracy = (a+d)/(a+b+d+c) = ESR+ ESR- True positives = 36 False + ve = 30 False – ve = 4 True – ve = 30 a 36 b 30 c 4 d 30
  • 95.
    ESR + + ++ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + TB+ No TB Sensitivity = a/(a+c) = 90% Specificity = d/(b+d) = 50% PPV = a/(a+b) = 55% NPV = d/(c+d) = 88% + LR = Sn/(1-Sp) = 1.8 - LR = (1-Sn)/Sp = 0.2 Accuracy = (a+d)/(a+b+d+c) = 66% ESR+ ESR- True positives = 36 False + ve = 30 False – ve = 4 True – ve = 30 a 36 b 30 c 4 d 30
  • 96.
    SPUTUM AFB + ++ + + + + + + + + + + + + + + + TB+ No TB AFB+ AFB - True + ve = 16 False – ve = 24 False + ve = 2 True –ve = 58 Total = 100 Sensitivity = a/(a+c) = Specificity = d/(b+d) = PPV = a/(a+b) = NPV = d/(c+d) = + LR = Sn/(1-Sp) = - LR = (1-Sn)/Sp= Accuracy = (a+d)/(a+b+d+c) = a 16 c 24 b 2 d 58 Sensitivity = a/(a+c) = 40% Specificity = d/(b+d) = 97% PPV = a/(a+b) = 89% NPV = d/(c+d) = 70% + LR = Sn/(1-Sp) = 12 - LR = (1-Sn)/Sp= 0.62 Accuracy = (a+d)/(a+b+d+c) = 74 %
  • 98.
     Is thetest available, affordable, and accurate in our setting?  Can a clinically sensible estimate of the pre-test probabilities of the patient be made from personal experience, prevalence statistics, practice databases, or primary studies?  Are the study patients similar to the patient in question?  How current is the study we are analysing — has evidence moved on since the publication of the study?
  • 99.
     Could theresult move the clinician across a test-treatment threshold  Will the patient be willing to have the test carried out?  Will the results of the test help the patient reach their goals?
  • 100.
     Sens, specare useful ways of describing a test; but do not help in clinical decision making  PPV & NPV enable diagnosis, when a test result is available; but are influenced by prevalence  LR allow post-test prob to be calculated and is uninfluenced by prevalence  For continuous variables, level of cut-off inversely related to sensitivity  Greater the area under ROC better is test