validity of diagnostic test

1. Dr. Alaa Hassan - MB BCh, MSc (PH), MDDr. Alaa Hassan - MB BCh, MSc (PH), MD
Criterion based Validity
Dr. Alaa El Dine Hassan
MB BCh, MSc (PH), MD
Assist. Professor -
Community and Family
Medicine

2. Dr. Alaa Hassan - MB BCh, MSc (PH), MDDr. Alaa Hassan - MB BCh, MSc (PH), MD
The Policeman’sThe Policeman’s
capability of detectingcapability of detecting
thieves is in challengethieves is in challenge
3 thieves3 thieves
6 Good People6 Good People

3. Dr. Alaa Hassan - MB BCh, MSc (PH), MDDr. Alaa Hassan - MB BCh, MSc (PH), MD
ThievesThieves
GoodGood
PeoplePeople
33 66
We Have:
– 3 thieves
– 6 good People

4. Dr. Alaa Hassan - MB BCh, MSc (PH), MDDr. Alaa Hassan - MB BCh, MSc (PH), MD
4 In Prison4 In Prison
5 Free5 Free

5. Dr. Alaa Hassan - MB BCh, MSc (PH), MDDr. Alaa Hassan - MB BCh, MSc (PH), MD
ThievesThieves
GoodGood
PeoplePeople
InIn
PrisonPrison
44
FreeFree
55
33 66

6. Dr. Alaa Hassan - MB BCh, MSc (PH), MDDr. Alaa Hassan - MB BCh, MSc (PH), MD
4 In Prison4 In Prison
5 Free5 Free

7. Dr. Alaa Hassan - MB BCh, MSc (PH), MDDr. Alaa Hassan - MB BCh, MSc (PH), MD
ThievesThieves
GoodGood
PeoplePeople
InIn
PrisonPrison
44
FreeFree
55
33 66
22
44
How Many good
People Free?
How Many Thieves
in Prison?
22
11
False
Classification?

8. Dr. Alaa Hassan - MB BCh, MSc (PH), MDDr. Alaa Hassan - MB BCh, MSc (PH), MD
4 In Prison4 In Prison
5 Free5 Free
The policeman Correctly Identified:
– 2 out of 3 thieves (66.66%)
– 4 out of 6 Good People (66.66%)

9. Dr. Alaa Hassan - MB BCh, MSc (PH), MDDr. Alaa Hassan - MB BCh, MSc (PH), MD

10. Dr. Alaa Hassan - MB BCh, MSc (PH), MDDr. Alaa Hassan - MB BCh, MSc (PH), MD
• A new instrument is released in the market
which is claimed to identify people with lung
cancers.
• The instrument is supposed to give:
– “Positive reading” with diseased individuals
and
– “Negative reading” with Non Diseased individuals

11. Dr. Alaa Hassan - MB BCh, MSc (PH), MDDr. Alaa Hassan - MB BCh, MSc (PH), MD
• A young motivated physician wishes to
evaluate the performance of the instrument.
• He applies the instrument to 100 persons of
whom
– 30 are known to be diseased (Diagnosed lung
cancer Cases)
– 70 are known to be not diseased

12. Dr. Alaa Hassan - MB BCh, MSc (PH), MDDr. Alaa Hassan - MB BCh, MSc (PH), MD
DiseasedDiseased
NotNot
DiseasedDiseased
3030 7070

13. Dr. Alaa Hassan - MB BCh, MSc (PH), MDDr. Alaa Hassan - MB BCh, MSc (PH), MD
• The instrument gave Positive reading with 35
subjects and Negative readings with 65
Subjects.

14. Dr. Alaa Hassan - MB BCh, MSc (PH), MDDr. Alaa Hassan - MB BCh, MSc (PH), MD
DiseasedDiseased
NotNot
DiseasedDiseased
TestTest
PositivePositive
3535
TestTest
NegativeNegative
6565
3030 7070

15. Dr. Alaa Hassan - MB BCh, MSc (PH), MDDr. Alaa Hassan - MB BCh, MSc (PH), MD
• Out of the 35 subjects who were labeled
Positive, 25 were actually Diseased.
• Out of the 65 subjects who were labeled
Negative , 60 were truly disease free.

16. Dr. Alaa Hassan - MB BCh, MSc (PH), MDDr. Alaa Hassan - MB BCh, MSc (PH), MD
DiseasedDiseased
NotNot
DiseasedDiseased
TestTest
PositivePositive
3535
TestTest
NegativeNegative
6565
3030 7070
2525
6060
True Positives (TP)
True Negatives (TN)

17. Dr. Alaa Hassan - MB BCh, MSc (PH), MDDr. Alaa Hassan - MB BCh, MSc (PH), MD
• By Simple subtraction , we can deduce that 5
diseased subjects were falsely labeled
Negative
• And 10 Non Diseased Subjects were falsely
labeled Positive

18. Dr. Alaa Hassan - MB BCh, MSc (PH), MDDr. Alaa Hassan - MB BCh, MSc (PH), MD
DiseasedDiseased
NotNot
DiseasedDiseased
TestTest
PositivePositive
3535
TestTest
NegativeNegative
6565
3030 7070
2525
6060
True Positives (TP)
True Negatives (TN)
1010
55
False Positives (FP)
False Negatives (FN)

19. Dr. Alaa Hassan - MB BCh, MSc (PH), MDDr. Alaa Hassan - MB BCh, MSc (PH), MD
• So the test was able to correctly identify 25
out of 30 Diseased individuals (83.33%)
• 60 out of 70 Non Diseased individuals (85.7%)

20. Dr. Alaa Hassan - MB BCh, MSc (PH), MDDr. Alaa Hassan - MB BCh, MSc (PH), MD
Sensitivity
• The sensitivity is how good the test is at
picking out patients with disease.
• = the proportion of patients with disease who
test positive.
• = the proportion of cases labeled Positive by
the test, relative to all cases that actually have
the disease:
= TP / All Diseased
= TP / (TP + FN).

21. Dr. Alaa Hassan - MB BCh, MSc (PH), MDDr. Alaa Hassan - MB BCh, MSc (PH), MD
DiseasedDiseased
NotNot
DiseasedDiseased
TestTest
PositivePositive
TP+FPTP+FP
TestTest
NegativeNegative
TN+FNTN+FN
TP+FNTP+FN
(All Diseased)(All Diseased)
TN+FPTN+FP
(All Disease Free)(All Disease Free)
TPTP
TNTN
FPFP
FNFN
Sensitivity = TP / (TP + FN)

22. Dr. Alaa Hassan - MB BCh, MSc (PH), MDDr. Alaa Hassan - MB BCh, MSc (PH), MD
Specificity
• Specificity is the ability of the test to pick out
patients who do NOT have the disease.
• is the proportion of patients without disease
who test negative.
• = TN / All Non Diseased
• = TN / (TN + FP).

23. Dr. Alaa Hassan - MB BCh, MSc (PH), MDDr. Alaa Hassan - MB BCh, MSc (PH), MD
DiseasedDiseased
NotNot
DiseasedDiseased
TestTest
PositivePositive
TP+FPTP+FP
TestTest
NegativeNegative
TN+FNTN+FN
TP+FNTP+FN
(All Diseased))(All Diseased))
TN+FPTN+FP
(All Disease Free)(All Disease Free)
TPTP
TNTN
FPFP
FNFN
Specificity = TN / (TN + FP)

24. Dr. Alaa Hassan - MB BCh, MSc (PH), MDDr. Alaa Hassan - MB BCh, MSc (PH), MD

25. Dr. Alaa Hassan - MB BCh, MSc (PH), MDDr. Alaa Hassan - MB BCh, MSc (PH), MD
In real Life practice
• Tests have false positive and false negative results,
so……
• If a patient who has undergone an ELISA test for HIV,
tested positive…….
• How confident will you be that he is really infected?
• What is the probability of being REALLY infected with
HIV?

26. Dr. Alaa Hassan - MB BCh, MSc (PH), MDDr. Alaa Hassan - MB BCh, MSc (PH), MD
Predictive value of a positive test
• Is the probability that a person is Diseased when a
positive test result is observed.
• Is the proportion of patients with positive tests who
have disease:
== TP / All Test PositivesTP / All Test Positives
= TP / (TP + FP).
• This is the same thing as posttest probability of
disease given a positive test.
• It measures how well the test rules in disease.

27. Dr. Alaa Hassan - MB BCh, MSc (PH), MDDr. Alaa Hassan - MB BCh, MSc (PH), MD
DiseasedDiseased
NotNot
DiseasedDiseased
TestTest
PositivePositive
TP+FPTP+FP
(All Test Positive)(All Test Positive)
TestTest
NegativeNegative
TN+FNTN+FN
(All Test Negative)(All Test Negative)
TP+FNTP+FN TN+FPTN+FP
TPTP
TNTN
FPFP
FNFN
PPV = TP / (TP + FP)

28. Dr. Alaa Hassan - MB BCh, MSc (PH), MDDr. Alaa Hassan - MB BCh, MSc (PH), MD
Lung Cancer Example Revisited

29. Dr. Alaa Hassan - MB BCh, MSc (PH), MDDr. Alaa Hassan - MB BCh, MSc (PH), MD
DiseasedDiseased
NotNot
DiseasedDiseased
TestTest
PositivePositive
3535
TestTest
NegativeNegative
6565
3030 7070
2525
6060
True Positives (TP)
True Negatives (TN)
1010
55
False Positives (FP)
False Negatives (FN)
Predictive value of Positive Test

30. Dr. Alaa Hassan - MB BCh, MSc (PH), MDDr. Alaa Hassan - MB BCh, MSc (PH), MD
Predictive value of Positive test
= TP / All test Positives
= 25 / 35
= 71.4%

31. Dr. Alaa Hassan - MB BCh, MSc (PH), MDDr. Alaa Hassan - MB BCh, MSc (PH), MD
So if a patient of unknown status undergoes the
test and gets a Positive Reading, There is a
71.4% Possibility of being truly Diseased.

32. Dr. Alaa Hassan - MB BCh, MSc (PH), MDDr. Alaa Hassan - MB BCh, MSc (PH), MD
• Predictive value of a negative test is the
proportion of patients with negative tests who
do not have disease:
= TN / All Test Negatives= TN / All Test Negatives
= TN / (TN + FN).
• It measures how well the test rules out
disease.

33. Dr. Alaa Hassan - MB BCh, MSc (PH), MDDr. Alaa Hassan - MB BCh, MSc (PH), MD
DiseasedDiseased
NotNot
DiseasedDiseased
TestTest
PositivePositive
TP+FPTP+FP
(All Test Positive)(All Test Positive)
TestTest
NegativeNegative
TN+FNTN+FN
(All Test Negative)(All Test Negative)
TP+FNTP+FN
(All Diseased)(All Diseased)
TN+FPTN+FP
(All Disease Free)(All Disease Free)
TPTP
TNTN
FPFP
FNFN
NPV = TN / (TN + FN)

34. Dr. Alaa Hassan - MB BCh, MSc (PH), MDDr. Alaa Hassan - MB BCh, MSc (PH), MD
Lung Cancer Example Revisited

35. Dr. Alaa Hassan - MB BCh, MSc (PH), MDDr. Alaa Hassan - MB BCh, MSc (PH), MD
DiseasedDiseased
NotNot
DiseasedDiseased
TestTest
PositivePositive
3535
TestTest
NegativeNegative
6565
3030 7070
2525
6060
True Positives (TP)
True Negatives (TN)
1010
55
False Positives (FP)
False Negatives (FN)
Predictive value of Negative Test

36. Dr. Alaa Hassan - MB BCh, MSc (PH), MDDr. Alaa Hassan - MB BCh, MSc (PH), MD
Predictive value of Negative test
= TN / TN + FN
= 60 / 65
= 92.3%

37. Dr. Alaa Hassan - MB BCh, MSc (PH), MDDr. Alaa Hassan - MB BCh, MSc (PH), MD
So if a patient of unknown status undergoes the
test and gets a Negative Reading, There is a
92.3% Possibility of being truly Disease free.

38. Dr. Alaa Hassan - MB BCh, MSc (PH), MDDr. Alaa Hassan - MB BCh, MSc (PH), MD

39. Dr. Alaa Hassan - MB BCh, MSc (PH), MDDr. Alaa Hassan - MB BCh, MSc (PH), MD
Positive Likelihood Ratios
• likelihood ratios are not yet commonly
reported in peer-reviewed literature or in
marketing information provided by test
manufacturers.
• They can be a valuable tool for comparing the
accuracy of several tests to the gold standard,
and
• They are NOT dependent upon the prevalence
of disease.

40. Dr. Alaa Hassan - MB BCh, MSc (PH), MDDr. Alaa Hassan - MB BCh, MSc (PH), MD
The positive LR represents :
the odds of a positive test in an infected population
compared to
the odds of a positive test a non infected population.
TP/ All DiseasedTP/ All Diseased
————--------—————--------—
FP/ All Disease freeFP/ All Disease free
Or it can also be expressed as :Or it can also be expressed as :
sensitivitysensitivity
——————————
1-specificity1-specificity

41. Dr. Alaa Hassan - MB BCh, MSc (PH), MDDr. Alaa Hassan - MB BCh, MSc (PH), MD
DiseasedDiseased
NotNot
DiseasedDiseased
TestTest
PositivePositive
TP+FPTP+FP
(All Test Positive)(All Test Positive)
TestTest
NegativeNegative
TN+FNTN+FN
(All Test Negative)(All Test Negative)
TP+FNTP+FN
(All Diseased)(All Diseased)
TN+FPTN+FP
(All Disease Free)(All Disease Free)
TPTP
TNTN
FPFP
FNFN
Positive Likelihood Ratio

42. Dr. Alaa Hassan - MB BCh, MSc (PH), MDDr. Alaa Hassan - MB BCh, MSc (PH), MD
• Useful tests have larger positive LR
• Less useful tests will have smaller positive LR.

43. Dr. Alaa Hassan - MB BCh, MSc (PH), MDDr. Alaa Hassan - MB BCh, MSc (PH), MD
Example:
A test with a positive LR of 5.0 :
– For every
• 1 non-infected subjects that test positive
• 5 of the infected subjects will test positive.
– For every
• 1 false positive
• 5 true positives
– True positives are 5 times False Positives

44. Dr. Alaa Hassan - MB BCh, MSc (PH), MDDr. Alaa Hassan - MB BCh, MSc (PH), MD
The Negative LR represents :
the odds of athe odds of a Negative testNegative test in anin an infected populationinfected population
compared tocompared to
the odds of athe odds of a Negative testNegative test in ain a non infectednon infected populationpopulation.
FN/ All DiseasedFN/ All Diseased
------------—————------------—————
TN/ All Disease freeTN/ All Disease free
Or it can also be expressed as :Or it can also be expressed as :
1- Sensitivity1- Sensitivity
——----------------——----------------——
SpecificitySpecificity

45. Dr. Alaa Hassan - MB BCh, MSc (PH), MDDr. Alaa Hassan - MB BCh, MSc (PH), MD
DiseasedDiseased
NotNot
DiseasedDiseased
TestTest
PositivePositive
TP+FPTP+FP
(All Test Positive)(All Test Positive)
TestTest
NegativeNegative
TN+FNTN+FN
(All Test Negative)(All Test Negative)
TP+FNTP+FN
(All Diseased)(All Diseased)
TN+FPTN+FP
(All Disease Free)(All Disease Free)
TPTP
TNTN
FPFP
FNFN
Negative Likelihood Ratio

46. Dr. Alaa Hassan - MB BCh, MSc (PH), MDDr. Alaa Hassan - MB BCh, MSc (PH), MD
• Useful tests have negative LR close to 0
• Less useful tests have higher negative LR.