Evaluation of Diagnostic Test

1. Evaluation of
Diagnostic Tests
Dr. Amit Bhondve
ASSISTANT PROFESSOR
DEPARTMENT OF COMMUNITY MEDICINE
SETH G.S. MEDICAL COOLEGE &K.E.M. HOSPITAL

3. Reasons for Ordering a Test
Diagnosis Monitoring
Screening Research

4. Seven question to evaluate the utility of a diagnostic
test
Can the test be reliably
performed?
Was the test evaluated
on an appropriate
population?
Was an appropriate gold
standard used?
Was an appropriate cut-
off value chosen to
optimize sensitivity and
specificity?

5. Seven question to evaluate the utility of a diagnostic
test
What are the positive and
negative likelihood ratios?
How well does the test
perform in specific
populations?
What is the balance between
cost of the disease and cost of
the test?

6. Test Sensitivity% Specificity%
ANA 99 80
dsDNA 70 95
ssDNA 80 50
Histone 30-80 50
Nucleoprotein 58 50
Sm 25 99
RNP 50 87-94
PCNA 5 95
6
Which one of these test is the best for SLE
Dx?

7. The Challenge of
Clinical
Measurement
• Diagnoses are based on information,
from formal measurements and/or
from your clinical judgment.
• This information is seldom perfectly
accurate:
• Random errors can occur
(machine not working?)
• Biases in judgment or
measurement can occur (“this kid
doesn’t look sick”)
• Due to biological variability, this
patient may not fit the general
rule
• Diagnosis (e.g., hypertension)
involves a categorical judgment;
this often requires dividing a
continuous score (blood pressure)
into categories. Choosing the
cutting-point is challenging.

8. One needs to be aware …
• Diagnostic judgments are based on
probabilities;
• That using a quantitative approach is better
than just guessing!
• That you will gradually become familiar
with the typical accuracy of measurements
in your chosen clinical field;
• That the principles apply to both diagnostic
and screening tests;
• Of some of the ways to describe the
accuracy of a measurement.

9. Why choose one test and
not another?
• Reliability: consistency or reproducibility;
this considers chance or random errors (which
sometimes increase, sometimes decrease,
scores).
“Is it measuring something?”
• Validity: “Is it measuring what it is supposed to
measure?” By extension, “what diagnostic
conclusion can I draw from a particular score
on this test?”
Validity may be affected by bias, which refers
to systematic errors (these fall in a certain
direction)
• Safety, Acceptability, Cost, etc.

12. • Validity tells you how accurately a method
measures something. If a method measures what
it claims to measure, and the results closely
correspond to real-world values, then it can be
considered valid

13. Types Of Validity
CONSTRUCT FACE
CONTENT CRITERION

14. Ways of
Assessing
Validity
• Content or “Face” validity: does it make clinical or biological
sense? Does it include the relevant symptoms?
• Criterion: comparison to a “gold standard” definitive measure
(e.g., biopsy, autopsy)
• Expressed as sensitivity and specificity
• Construct validity

15. Criterion validation: “Gold Standard”
The criterion that your clinical observation or
simple test is judged against:
more definitive (but expensive or invasive) tests, such as a
complete work-up, or
the clinical outcome (for screening tests, when workup of well
patients is unethical).
Sensitivity and specificity are calculated
from a research study comparing the
test to a gold standard.

16. Validation of methods
•Definition : Validation is the
confirmation by examination
and the provision of objective
evidence that the particular
requirements for a specific
intended use are fulfilled.

17. Steps in method validation
• Precision checks:
1. Inter assay
2. Intra assay
• Inter method comparison (reproducibility)
• Linearity (detection limit)
• Reference range verification
• Inter instrumental comparison

18. Accuracy
- How well a
measurement agrees
with an accepted value
Precision
- How well a series of
measurements agree
with each other

19. Accuracy
vs.
Precision

22. Gold Standard
• In medicine and statistics, gold standard
test refers to a diagnostic
test or benchmark that is the best
available under reasonable conditions.
• It does not have to be necessarily the best
possible test for the condition in absolute
terms.
• For example, in medicine, dealing with
conditions that require an autopsy to have
a perfect diagnosis, the gold standard test
is normally less accurate than the autopsy.

23. Gold Standard
• A hypothetical ideal "gold standard" test
has a sensitivity of 100% with respect to
the presence of the disease (it identifies
all individuals with a well defined
disease process; it does not have any
false-negative results) and
a specificity of 100% (it does not falsely
identify someone with a condition that
does not have the condition; it does not
have any false-positive results). In
practice, there are sometimes no true
"gold standard" tests. Sometimes they
are called "perfect" and "alloyed" gold
standard

28. Diagnostic Tests Characteristics
Sensitivity Specificity
Predictive
Value
Likelihood
Ratio
28

29. Validity of Screening Tests
29
a
d
c
b
True Disease Status
+ -
+
-
Sensitivity: The probability of testing
positive if the disease is truly present
Sensitivity = a / (a + c)

30. Validity of Screening Tests
30
a
d
c
b
True Disease Status
+ -
+
-
Specificity: The probability of screening
negative if the disease is truly absent
Specificity = d / (b + d)

31. • Two-by-two tables can also be used for calculating the false positive
and false negative rates.
• The false positive rate = false positives / (false positives + true
negatives). It is also equal to 1- specificity.

32. • The false negative rate = false negatives / (false negatives + true
positives). It is also equal to 1 – sensitivity.
• An ideal test maximizes both sensitivity and specificity, thereby
minimizing the false positive and false negative rates.

33. Validity of Screening Tests
33
132
63650
45
983
Breast Cancer
+ -
Physical Exam
and Mammo-
graphy +
-
Sensitivity: a / (a + c)
Sensitivity =
Specificity: d / (b + d)
Specificity =

34. Validity of Screening Tests
34
132
63650
45
983
Breast Cancer
+ -
Physical Exam
and Mammo-
graphy +
-
Sensitivity: a / (a + c)
Sensitivity = 132 / (132 + 45) = 74.6%
Specificity: d / (b + d)
Specificity = 63650 / (983 + 63650) = 98.5%

35. 2 X 2 table
Disease
Test
+ -
+
-
Sensitivity
Positive
predictive
value

36. Natural Frequencies Tree
Population
100

37. In Every 100 People, 4 Will Have The Disease
Disease +
4
Disease -
96
Population
100
If these 100 people are representative of the population at
risk, the assessed rate of those with the disease (4%)
represents the PREVALENCE of the disease – it can also be
considered the PRE-TEST PROBABILITY of having the disease

38. OF THE 4 PEOPLE WITH THE DISEASE, THE TEST WILL DETECT
3
Disease +
4
Disease -
96
Test +
3
Test -
1
Population
100
In other words, the
sensitivity is 75%

39. AMONG THE 96 PEOPLE WITHOUT THE DISEASE, 7 WILL TEST
POSITIVE
Disease +
4
Disease -
96
Test +
7
Test -
89
Test +
3
Test -
1
Population
100
In other words, the
specificity is 93%

40. POSITIVE
PREDICTIVE
VALUE = 30%
AMONG THOSE WHO TEST POSITIVE, 3 IN 10 WILL ACTUALLY
HAVE THE DISEASE
Disease +
4
Disease -
96
Test +
7
Test -
89
Test +
3
Test -
1
Population
100
This is also the
POST-TEST PROB-
ABILITY of having
the disease

41. NEGATIVE
PREDICTIVE
VALUE = 99%
AMONG THOSE WHO TEST NEGATIVE, 89 OF 90 WILL NOT
HAVE THE DISEASE
Disease +
4
Disease -
96
Test +
7
Test -
89
Test +
3
Test -
1
Population
100

42. CONVERSELY, IF SOMEONE TESTS NEGATIVE, THE CHANCE OF
HAVING THE DISEASE IS ONLY 1 IN 90
Disease +
4
Disease -
96
Test +
7
Test -
89
Test +
3
Test -
1
Population
100

43. PREDICTIVE VALUES AND CHANGING PREVALENCE
Disease +
4
Disease -
996
Population
1000
Prevalence reduced by an order
of magnitude from 4% to 0.4%

44. PREDICTIVE VALUE AND CHANGING PREVALENCE
Disease +
4
Disease -
996
Test +
70
Test -
926
Test +
3
Test -
1
Population
1000
Sensitivity and
Specificity
unchanged

45. POSITIVE
PREDICTIVE
VALUE = 4%
POSITIVE PREDICTIVE VALUE AT LOW PREVALENCE
Disease +
4
Disease -
996
Test +
70
Test -
926
Test +
3
Test -
1
Population
1000
Previously, PPV
was 30%

46. NEGATIVE
PREDICTIVE
VALUE >99%
NEGATIVE PREDICTIVE VALUE AT LOW PREVALENCE
Disease +
4
Disease -
996
Test +
70
Test -
926
Test +
3
Test -
1
Population
1000
Previously, NPV
was 99%

47. Prediction Of Low Prevalence Events
• Even highly specific tests, when applied to low prevalence events,
yield a high number of false positive results
• Because of this, under such circumstances, the Positive Predictive
Value of a test is low
• However, this has much less influence on the Negative Predictive
Value

48. Relationship Between Prevalence and Predictive
Value
0
0.2
0.4
0.6
0.8
1
0.05 0.2 0.4 0.6 0.8 0.95
Pre-test Probability (Prevalence)
Predictive
Value
PPV
NPV
Based on a test with 90% sensitivity and 82% specificity
Difference between
PPV and NPV
relatively small
Difference between
PPV and NPV
relatively large

49. Relationship Between Prevalence And Predictive
Value
0
10
20
30
40
50
60
70
80
90
100
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
0.45
0.5
0.55
0.6
0.65
0.7
0.75
0.8
0.85
0.9
0.95
PPV
NPV
Based on a test with 75% sensitivity and 93% specificity
Prevalence
Predictive
Value

50. Performance of A Test With Changing
Prevalence
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
0 0.2 0.4 0.6 0.8 1
PRE-TEST PROBABILITY
POST-TEST
A (90%)
B (70%)
C (50%)
A : Sensitivity =
Specificity = 0.9
LR+ = 9.0
B : Sensitivity =
Specificity = 0.7
LR+ = 3.0
C : Sensitivity =
Specificity = 0.5
LR+ = 1.0
POST-TEST
PROBABILITY

51. 2 X 2 table
DISEASE
Yes No Total
3 7
Yes
a b
D
10
a+b
c d
No 1 89 90
c+d
4 96 100
TEST
Total
a+c b+d a+b+c+d

52. Sensitivity
DISEASE
Yes No Total
3 7
Yes
a b
D
10
a+b
c d
No 1 89 90
c+d
4 96 100
TEST
Total
a+c b+d a+b+c+d
Sensitivity
The proportion of people with the diagnosis (N=4) who are
correctly identified (N=3)
Sensitivity = a/(a+c) = 3/4 = 75%
FALSE
NEGATIVES

53. Specificity
DISEASE
Yes No Total
3 7
Yes
a b
D
10
a+b
c d
No 1 89 90
c+d
4 96 100
TEST
Total
a+c b+d a+b+c+d
Specificity
The proportion of people without the diagnosis (N=96) who
are correctly identified (N=89)
Specificity = d/(b+d) = 89/96 = 93%
FALSE
POSITIVES

54. Value of a diagnostic test depends on the
prior probability of disease
• Prevalence (Probability) = 5%
• Sensitivity = 90%
• Specificity = 85%
• PV+ = 24%
• PV- = 99%
• Test not as useful when disease
unlikely
• Prevalence (Probability) = 90%
• Sensitivity = 90%
• Specificity = 85%
• PV+ = 98%
• PV- = 49%
• Test not as useful when disease
likely
54

55. A Test With Normally Distributed Values
Negative Positive
Degree of ‘positivity’ on test
%
of
Group
DISEASED
NON-DESEASED
Test cut-off
Assessing the performance
of the test assumes that
these two distributions
remain constant. However,
each of them will vary
(particularly through
spectrum or selection bias)

56. CASES
NON-CASES
Performance of A Diagnostic Test
Negative Positive
Degree of ‘positivity’ on test
%
of
Group
DISEASED
NON-DESEASED
Test cut-off
FALSE
NEGATIVES
FALSE
POSITIVES

57. Minimising False Negatives: A Sensitive Test
Negative Positive
Degree of ‘positivity’ on test
%
of
Group
DISEASED
NON-
DESEASED
Test cut-off
Cut-off shifted to minimise
false negatives ie to
optimise sensitivity
CONSEQUENCES:
- Specificity reduced
- A Negative result from a
seNsitive test rules out the
diagnosis - snNout
CASES
NON-CASES

58. Minimising False Positives: A Specific Test
Negative Positive
Degree of ‘positivity’ on test
%
of
Group
DISEASED
NON-DESEASED
Test cut-off
Cut-off shifted to
minimise false positives
ie to optimise specificity
CONSEQUENCES:
- Sensitivity reduced
- A Positive result from
a sPecific test rules in
the diagnosis - spPin

59. ROC curves: simplest case
• Consider diagnostic test for a disease
• Test has 2 possible outcomes:
• ‘postive’ = suggesting presence of disease
• ‘negative’
• An individual can test either positive or negative
for the disease

60. Non-diseased Diseased
Evaluation Result Value
Or
Subjective Judgment Of Likelihood That Case Is Diseased
Threshold
Receiver Operating Characteristics (ROC)

61. Non-diseased
Centers
Diseased
Centers
Test result value
or
subjective judgment of likelihood that case is diseased
Threshold

62. Non-diseased
Centers
Diseased
Centers
Cutoff point
more typically:

63. Non-diseased
cases
Diseased
cases
FP rate
more typically:

64. Non-diseased
Centers
Diseased
Centers
TP rate
more typically:

65. Threshold
TPF,
sensitivity
FPF, 1-specificity
less aggressive
mindset
Non-diseased
Centers
Diseased
Centers

66. Threshold
TPF,
sensitivity
FPF, 1-specificity
moderate
mindset
Non-diseased
cases
Diseased
cases

67. Threshold
TPF,
sensitivity
FPF, 1-specificity
more
aggressive
mindset
Non-diseased
cases
Diseased
cases

68. Threshold
Non-diseased
cases
Diseased
cases
TPF,
sensitivity
FPF, 1-specificity
Entire ROC curve

69. TPF,
sensitivity
FPF, 1-specificity
Entire ROC curve

70. Problems with AUC
• No clinically relevant meaning
• A lot of the area is coming from the range of large
false positive values, no one cares what’s going on in
that region (need to examine restricted regions)
• The curves might cross, so that there might be a
meaningful difference in performance that is not
picked up by AUC

71. Likelihood Ratios

72. Pre-test & post-test probability
• Pre-test probability of disease can be compared with the estimated
later probability of disease using the information provided by a
diagnostic test.
• The difference between the previous probability and the later
probability is an effective way to analyze the efficiency of a diagnostic
method.

73. • It tells you how much a positive or negative result changes the likelihood
that a patient would have the disease.
• The likelihood ratio incorporates both the sensitivity and specificity of
the test and provides a direct estimate of how much a test result will
change the odds of having a disease

74. • The likelihood ratio for a positive result (LR+) tells you how much the
odds of the disease increase when a test is positive.
• The likelihood ratio for a negative result (LR-) tells you how much the
odds of the disease decrease when a test is negative.

75. Positive & Negative Likelihood Ratios
• We can judge diagnostic tests: positive and negative likelihood ratios.
• Like sensitivity and specificity, are independent of disease prevalence.

76. Likelihood Ratios (Odds)
• The probability of a test result in those with the disease divided by
the probability of the result in those without the disease.
• How many more times (or less) likely a test result is to be found in
the disease compared with the non-diseased.
76

77. Positive Likelihood Ratios
• This ratio divides the probability that a diseased patient will test
positive by the probability that a healthy patient will test positive.
• The positive likelihood ratio
+LR = sensitivity/(1 – specificity)

78. False Positive Rate
• The false positive rate = false positives / (false positives + true
negatives). It is also equal to 1- specificity.
• The false negative rate = false negatives / (false negatives + true
positives). It is also equal to 1 – sensitivity.

79. Positive Likelihood Ratios
• It can also be written as the
true positive rate/false positive rate.
• Thus, the higher the positive likelihood ratio, the better the test (a
perfect test has a positive likelihood ratio equal to infinity).

80. Negative Likelihood Ratio
• This ratio divides the probability that a diseased patient will test
negative by the probability that a healthy patient will test negative.
• The negative likelihood ratio
–LR = (1 – sensitivity)/specificity.

81. False Negative Rate
• The false negative rate = false negatives / (false negatives + true
positives).
• It is also equal to 1 – sensitivity.

82. Negative Likelihood Ratio
• It can also be written as the
false negative rate/true negative rate.
• Therefore, the lower the negative likelihood ratio, the better the test
(a perfect test has a negative likelihood ratio of zero).

83. Positive & Negative Likelihood Ratios
• Although likelihood ratios are independent of disease prevalence, their
direct validity is only within the original study population.

84. Probability of Disease
• Pre-test probability of disease = disease prevalence
• Post-test probability of disease =
• If normal, c/(c+d)
• If negative, a/(a+b)
84
Disease present, gold
standard
Disease absent, gold
standard
False positives (b)
True positives (a)
Test result positive
True negatives (d)
False negatives (c)
Test result negative

85. Bayes Theorem
Post-test Odds =
Likelihood Ratio X Pre-test Odds

86. Using Likelihood Ratios to Determine Post-Test
Disease Probability
86
Pre-test
probability
of disease
Pre-test
odds of
disease
Likelihood
ratio
Post-test
odds of
disease
Post-test
probability
of disease

87. Pre-test & post-test probability
• “Post-test probability” depends on the accuracy of the diagnostic test
and the pre-test probability of disease
• A test result cannot be interpreted without some knowledge of the
pre-test probability

88. Where does “pre-test probability” come
from?
• Clinical experience
• Epidemiological data
• “Clinical decision rules”
• Guess

89. what is the likelihood that this patient has the
disease?
• A disease with a prevalence of 30% must be diagnosed.
• There is a test for this disease.
• It has a sensitivity of 50% and a specificity of 90%.

90. Likelihood Ratios
Sensitivity
1 – Specificity
= 0.88 / (1 – 0.82)
= 4.89
This means that Anne’s positive FNA biopsy will be approx. 5 times as likely to be seen
with, as opposed to without, thyroid cancer.
Sensitivity Specificity
FNA Biopsy 88% 82%
From: J Clin End & Metab. 2006;
91(11):4295-4301.

91. Prevalence of 30%
Sensitivity of 50%
Specificity of 90%
30
70
15
70 – 63 = 7
100
22 positive
tests in
total of
which 15
have the
disease
About 70%
Disease +ve
Disease -ve
63
15

92. Likelihood
Disease +
4
Test +
3
Test -
1
Population
100
The likelihood that
someone with the
disease will have a
positive test is ¾ or
75%
This is the same as
the sensitivity

93. Likelihood II
Disease -
96
Test +
7
Test -
89
Population
100
The likelihood that
someone without
the disease will
have a positive test
is 7/96 or 7%
This is the same as
the (1-specificity)

94. Likelihood Ratio
Likelihood of Positive Test
in the Absence of the Disease
Sensitivity
1- Specificity
= = 10.7
Likelihood of Positive Test Given
The Disease
=
Likelihood Ratio
A Likelihood Ratio of 1.0 indicates an uninformative test
(occurs when sensitivity and specificity are both 50%)
The higher the Likelihood Ratio, the better the test
(other factors being equal)
0.75
0.07
=

95. Diagnostic Odds Ratio
DISEASE
Yes No Total
3 7
Yes
a b
D
10
a+b
c d
No 1 89 90
c+d
4 96 100
TEST
Total
a+c b+d a+b+c+d
The Diagnostic Odds Ratio is
the ratio of odds of having the
diagnosis given a positive test
to those of having the
diagnosis given a negative test
2
.
38
011
.
0
429
.
0
89
1
7
3
DOR



Potentially useful as an
overall summary
measure, but only in
conjunction with other
measures (LR,
sensitivity, specificity)

96. Is there an
easier way?

97. Likelihood Ratio And Pre- And Post-test
Probabilities
For a given test with a
given likelihood ratio, the
post-test probability will
depend on the pre-test
probability (that is, the
prevalence of the condition
in the sample being
assessed)

98. Sensitivity Analysis of A Diagnostic Test
Value 95% CI
Pre-test
probability
35%
26% to
44%

99. Sensitivity Analysis of A Diagnostic Test
Applying the 95% confidence
intervals above to the
nomogram, the post-test
probability is likely to lie in the
range 55-85%
Value 95% CI
Pre-test
probability
35% 26% to 44%
Likelihood
ratio
5.0 3.0 to 8.5

100. Applying A Diagnostic Test In Different
Settings
• The Positive Predictive Value of a test will vary (according to the prevalence of the
condition in the chosen setting)
• Sensitivity and Specificity are usually considered properties of the test rather than the
setting, and are therefore usually considered to remain constant
• However, sensitivity and specificity are likely to be influenced by complexity of
differential diagnoses and a multitude of other factors (cf spectrum bias)

101. Likelihood Ratios (Odds)
•This is an alternative way of describing the
performance of a diagnostic test. Similar to S and S,
and can be used to calculate the probability of
disease after a positive or negative test (predictive
value). Advantage of this is that it can be used at
multiple levels of test results.
101

102. What is this second fraction?
• Likelihood Ratio Positive
• Multiplied by any patient’s pretest odds gives you their posttest odds.
• Comparing LR+ of different tests is comparing their ability to “rule in”
a diagnosis.
• As specificity increases LR+ increases and PPV increases (Sp P In)
102

103. Clinical interpretation of post-test probability
103
Don't
treat for
disease
Do further
diagnostic
testing
Treat for
disease
Probability of disease:
0 1
Testing
threshold
Treatment
threshold
Disease
ruled out
Disease
ruled in
If you are here, Test
will help you to go
toward one end of
this probability, either
0 or 1 to get the final
decision.

104. Values of Positive and Negative
Likelihood Ratios (LR)
LR Poor-fair Good Excellent
Positive
likelihood
ratio
2.1-5 5.1-10 >10
Negative
likelihood
ratio
0.5-0.2 0.19-0.1 <0.1

105. Likelihood Ratios & You
•Allows us to determine the accuracy with which a test
identifies the target disorder
•As the LR becomes larger, the likelihood of the target
disease increases:
Likelihood ratio Interpretation
>10 Strong evidence to rule in disease
5-10 Moderate evidence to rule in disease
2-5 Weak evidence to rule in disease
0.5-2 No significant change in the likelihood of disease
0.2-0.5 Weak evidence to rule out disease
0.1-0.2 Moderate evidence to rule out disease
<0.1 Strong evidence to rule out disease

106. Advantages of LRs
•The higher or lower the LR, the higher or lower the
post-test disease probability
•Which test will result in the highest post-test
probability in a given patient?
•The test with the largest LR+
•Which test will result in the lowest post-test
probability in a given patient?
•The test with the smallest LR-
106

107. Advantages of LRs
• Clear separation of test characteristics from disease probability.
107

108. Likelihood Ratios - Advantage
• Provide a measure of a test’s ability to rule in or rule out disease
independent of disease probability
• Test A LR+ > Test B LR+
• Test A PV+ > Test B PV+ always!
• Test A LR- < Test B LR-
• Test A PV- > Test B PV- always!
108

109. Predictive Values
Alternate formulations:Bayes’ Theorem
PV+ =
Se  Pre-test Prevalence
Se  Pre-test Prevalence + (1 - Sp)  (1 - Pre-test Prevalence)
High specificity to “rule-in” disease
PV- =
Sp  (1 - Pre-test Prevalence)
Sp  (1 - Pre-test Prevalence) + (1 - Se)  Pre-test Prevalence
High sensitivity to “rule-out” disease
109

110. Clinical Interpretation: Predictive Values
110
PV+ And PV-1
Of Electrocardiographic Status2
For Angiographically Verified3
Coronary Artery
Disease, By Age And Sex Of Patient
Sex Age PV+ (%) PV- (%)
F <40 32 88
F 40-50 46 80
F 50+ 62 68
M <40 62 68
M 40-50 75 54
M 50+ 85 38
1. Based on statistical smoothing of results from 78 patients referred to NC
Memorial Hospital for chest pain. Each value has a standard error of 6-7%.
2. At least one millivolt horizontal st segment depression.
3. At least 50% stenosis in one or more main coronary vessels.

111. If Predictive value is more useful why not
reported?
• Should they report it?
• Only if everyone is tested.
• And even then.
• You need sensitivity and specificity from literature. Add YOUR OWN
pretest probability.
111

112. So how do you figure pretest probability?
•Start with disease prevalence.
•Refine to local population.
•Refine to population you serve.
•Refine according to patient’s presentation.
•Add in results of history and exam (clinical
suspicion).
•Also consider your own threshold for testing.
112

113. Pretest Probability: Clinical Significance
• Expected test result means more than unexpected.
• Same clinical findings have different meaning in different settings
(e.g.scheduled versus unscheduled visit). Heart sound, tender area.
• Neurosurgeon.
• Lupus nephritis.
113

114. What proportion of all patients will test positive?
• Diseased X sensitivity
+ Healthy X (1-specificity)
• Prevalence X sensitivity +
(1-prevalence)(1-specificity)
• We call this “test prevalence”
• i.e. prevalence according to the test.

115. Some Examples
Diabetes mellitus (type 2)
• Check out this:

116. Some Examples from
Essential Evidence Plus
Disease Link Address
Diabetes Mellitus (type
2)
http://www.essentialevidenceplus.com/content/eee/127
Deep Vein Thrombosis http://www.essentialevidenceplus.com/content/eee/28
Arrhythmia (Atrial
Fibrillation & Flutter)
http://www.essentialevidenceplus.com/content/eee/13
http://www.essentialevidenceplus.com/

117. Test Sensitivity Specificity LR(+)
ANA 99 80 4.95
dsDNA 70 95 14
ssDNA 80 50 1.6
Histone 30-80 50 1.1
Nucleoprotein 58 50 1.16
Sm 25 99 25
RNP 50 87-94 3.8-8.3
PCNA 5 95 1
Which one of these test is the best for SLE
Dx?

118. Determining Usefulness of a Medical Test
Question Possible Designs Statistics for
Results
1. How
reproducible
is the test?
Studies of:
- intra- and inter
observer &
- intra- and inter
laboratory
variability
Proportion
agreement,
coefficient of
variance, mean &
distribution of
differences (avoid
correlation
coefficient)

119. Determining Usefulness of a Medical Test
Question Possible Designs Statistics for
Results
2. How
accurate is
the test?
Cross-sectional, case-
control, cohort-type
designs in which test
result is compared with
a “gold standard”
Sensitivity,
specificity,
PV+, PV-,
ROC curves,
LRs

120. Determining Usefulness of a Medical Test
Question Possible
Designs
Statistics for Results
3. How
often do
test results
affect
clinical
decisions?
Diagnostic
yield studies,
studies of pre-
& post test
clinical
decision
making
Proportion abnormal,
proportion with
discordant results,
proportion of tests
leading to changes in
clinical decisions; cost
per abnormal result or
per decision change

121. Determining Usefulness of a Medical Test
Question Possible
Designs
Statistics for Results
4. What are
the costs,
risks, &
acceptability
of the test?
Prospective or
retrospective
studies
Mean cost, proportions
experiencing adverse
effects, proportions
willing to undergo the
test

122. Determining Usefulness of a Medical Test
Question Possible Designs Statistics for
Results
5. Does
doing the
test
improve
clinical
outcome,
or having
adverse
effects?
Randomized trials, cohort
or case-control studies in
which the predictor
variable is receiving the
test & the outcome
includes morbidity,
mortality, or costs related
either to the disease or to
its treatment
Risk ratios, odd
ratios, hazard
ratios, number
needed to treat,
rates and ratios
of desirable
and
undesirable
outcomes

123. Was it clear enough !

124. Key References
Sedlmeier P and Gigerenzer G. Teaching Bayesian reasoning in
less than two hours. Journal of Experimental Psychology:
General. 130 (3):380-400, 2001.
Knotternus JA (ed). The Evidence Base of Clinical Diagnosis.
London: BMJ Books, 2002.
Sackett DL, Haynes RB, Guyatt G, and Tugwell P. Clinical
Epidemiology : A Basic Science for Clinical Medicine. Boston,
Mass: Little, Brown & Co, 1991.
Loong TW. Understanding sensitivity and specificity with the
right side of the brain. BMJ 2003: 327: 716-19.