Diagnosing Diagnostic describes the statistical properties of a diagnostic in an understandable way. Included in the slide are also a motivating example.

1. 1
Diagnosing a
Diagnostic
Athula Herath, Ph.D
Statistical Director,
Research Statistics/Translational Sciences
Cambridge, UK
08th
April 2015

2. 2
Contents
✔
Introduction to the
statistical concepts of
diagnostics

3. 3
Statistical Properties of a Diagnostic
➢
Sensitivity : the probability that the test will be
positive given a patient with a condition.
➢
Specificity: the probability that the test will be
negative given a patient without a condition.
➢
Positive Predictive Value (PPV): the probability that the
patient will have a condition given a positive test result.
➢
Negative Predictive Value (NPV): the probability that patient
will not have a condition given a negative test result.

4. oo
ooooo
o
oo
x
xxxx
xx
xxx
V
50
100
150
x – responder to treatment
o – non responder
Biomarker
Expression
Sensitivity?
Specificity?
of the Marker
Single variable classifier

5. xo
o
ooo
x
o
ox
xx
o
xxx
o
oxx
V
50
100
150
x – non responder to the treatment
o – responder
Biomarker
Expression
Level
Sensitivity?
Specificity?
of the Marker
Cut-of

6. o
o
ooo
o
o
xx
xxx
xx
TRUE
50
100
150
Biomarker B
o
o
o
x
x
x
FALSE
x – non responder to the treatment
o – responder
Biomarker A
Expression
Level

7. 7
xx
xxx
xx
TRUE
50
100
150
o
o
ooo
o
o
o
o
o
x
x
x
FALSE
Sensitivity?
Specificity?
of the Joint Marker
Biomarker P
x – non responder to the treatment
o – responderBiomarker A
Expression
Level
Biomarker B

8. 8
Summary – Classifiers
(single vs. multiple)
➢
If we use a single biomarker, we use
a threshold on the single value
➢
In multi-dimensional (multivariate)
we define a combination to form an
index (e.g.: linear combination with
appropriate weights) and then
define a threshold on the index.

9. 9
Thresholds (cut-off values)
• Diferent thresholds have diferent sensitivity specificity values.
o
o
xooo
x
o
ox
o
xx o
o
xxx
xx
A
50
100
150
x – non responder to treatment
o –responder
Biomarker A
Expression Level
Sensitivity?
Specificity?
of the MarkerThreshold Sensitivity Specificity
50
100
150

10. 10
Diagnostics Tests
➢
A good diagnostic test has small
false positive and false negative
rates across a reasonable range of
cut of values
➢
A bad diagnostic test will have low
false positive rates while having high
false negative rates (and vice versa)

11. 11
ROC Curve
• A ROC curve is a graphical representation of the
trade ofs between the false negative and the
false positive rates for every possible cut-off.
• The ROC curve shows the sensitivity on the X
axis and 1-speficity on the Y axis
1-Specificity
S
e
n
s
i
t
i
v
i
t
y

12. 12
ROC Curves
➢
A good diagnostic test shows a ROC curve
which climbs rapidly towards the upper left
hand corner of the graph.
➢
This means simultaneous high specificity
and sensitivity
➢
The diagonal of the ROC curve shows for
every improvement of the sensitivity is
matched by a corresponding decline of the
specificity (a line of poor decision)
➢
A poor diagnostic test will follow (overlaps)
the diagonal of the ROC plot

13. 13
ROC Curves (Good/Poor)
Good Diagnostic Poor Diagnostic

14. 14
AUC – Area under the curve of the ROC plot
➢
How quickly the rock curve rise to the
upper left hand is measured by the AUC.
➢
If the AUC 1 we have an ideal test (100%
sensitivity and 100% specificity).
➢
If the AUC is 0.5 we have a diagnostic test
with 50% sensitivity and 50% specificity.
➢
So a better diagnostic test will have an AUC
closer to 1 while a poor diagnostic test will
have AUC closer to 0.5

15. 15
AUC of ROC Plots
Good Diagnostic (closer to 1) Poor Diagnostic (0.5)

16. 16
AUC values of ROC Plots
➢
0.6 to 0.75 --- Fair
➢
0.75 to 0.92 --- Good
➢
0.92 to 0.97 --- Good
➢
0.97 to 1.00 --- Excellent

17. 17
Molecular indices are making their way
into clinical (development) decision
making – Oncotype Recurrence Index (ORI)
http://www.nejm.org/doi/full/10.1056/NEJMoa041588

18. 18
Mapping of the ORI to clinical outcome
(thresholds)
http://www.nejm.org/doi/full/10.1056/NEJMoa041588

19. 19
Not only that they are making a real difference in
decision making for appropriate treatment

20. 20
Thank You!