1. E P I D E M I C A L E R T A N D R E S P O N S ELaboratory Training for Field Epidemiologists
Sensitivity and specificity
Predictive values positive and negative
Interpretation of results
Sep 2007
2. E P I D E M I C A L E R T A N D R E S P O N S ELaboratory Training for Field Epidemiologists
OutlineOutline
• Intrinsic characteristics of a test
– Sensitivity
– Specificity
• Performance of a test in a population
– Predictive value of a positive test
– Predictive value of a negative test
3. E P I D E M I C A L E R T A N D R E S P O N S ELaboratory Training for Field Epidemiologists
SensitivitySensitivity
Sensitivity = True positives / Affected persons
Estimate the 95% confidence interval
Persons testing positive
(True positives)
Persons testing negative
(False negatives)
Affected persons
(Positive by gold standard)
The sensitivity of a test in the ability of the test to identify
correctly affected individuals
Proportion of persons testing positive among affected individuals
4. E P I D E M I C A L E R T A N D R E S P O N S ELaboratory Training for Field Epidemiologists
Estimating the sensitivity of a testEstimating the sensitivity of a test
• Identify affected individuals with a gold standard
• Obtain a wide panel of samples that are representative of the
population of affected individuals
– Recent and old cases
– Severe and mild cases
– Various ages and sexes
• Test the affected individuals
• Estimate the proportion of affected individuals that are
positive with the test
5. E P I D E M I C A L E R T A N D R E S P O N S ELaboratory Training for Field Epidemiologists
Estimating the sensitivity of a rapid testEstimating the sensitivity of a rapid test
for leishmaniasisfor leishmaniasis
• Identify persons with leishmaniasis with a gold standard
– Parasitologically proven infection
• Obtain a wide panel of samples that are representative of the
population of individuals with leishmaniasis
– Recent and old cases
– Severe and asymptomatic cases
– Various ages and sexes
• Test the persons with leishmaniasis
• Estimate the proportion of persons with leishmaniasis that
are positive with the rapid test
6. E P I D E M I C A L E R T A N D R E S P O N S ELaboratory Training for Field Epidemiologists
Sensitivity of a rapid test forSensitivity of a rapid test for
leishmaniasisleishmaniasis
Patients with
leishmaniasis
Rapid test
True positive 148
False negative 2
150
Sensitivity = 148 / (150) = 98%
95% confidence interval: 94%-99%
7. E P I D E M I C A L E R T A N D R E S P O N S ELaboratory Training for Field Epidemiologists
What factors influenceWhat factors influence
the sensitivity of a test?the sensitivity of a test?
• Characteristics of the affected persons?
YES: Antigenic characteristics of the pathogen in the area
(e.g., if the test was not prepared with antigens reflecting the
population of pathogens in the area, it will not pick up infected
persons in the area)
• Characteristics of the non-affected persons?
NO: The sensitivity is estimated on a population of affected persons
• Prevalence of the disease?
NO: The sensitivity is estimated on a population of affected persons
Sensitivity is an INTRINSIC characteristic of the test
8. E P I D E M I C A L E R T A N D R E S P O N S ELaboratory Training for Field Epidemiologists
SpecificitySpecificity
Specificity = True negatives / Non-affected persons
Estimate the 95% confidence interval
Persons testing negative
(True negatives)
Persons testing positive
(False positives)
Non-affected persons
(Negative by gold standard)
The specificity of a test in the ability of the test to identify
correctly non-affected individuals
Proportion of person testing negative among non affected individuals
9. E P I D E M I C A L E R T A N D R E S P O N S ELaboratory Training for Field Epidemiologists
Estimating the specificity of a testEstimating the specificity of a test
• Identify non affected individuals
– Negative with a gold standard
– Unlikely to be infected
• Obtain a wide panel of samples that are representative of the
population of non-affected individuals
– Diverse unaffected population: Difficult to find. Ideally, those
individuals that would need to be tested but not infected
• Test the non-affected individuals
• Estimate the proportion of non-affected individuals that are
negative with the test
10. E P I D E M I C A L E R T A N D R E S P O N S ELaboratory Training for Field Epidemiologists
Estimating the specificity of a rapid testEstimating the specificity of a rapid test
for leishmaniasisfor leishmaniasis
• Identify persons without leishmaniasis
– Persons without sign and symptoms of the infection
– Persons at low risk of infection, negative with gold standard
• Obtain a wide panel of samples that are representative of the
population of individuals without leishmaniasis
– Persons from neighbouring villages having similar characteristics
but with no transmission and no infections
• Test the persons without leishmaniasis
• Estimate the proportion of persons without leishmaniasis
that are negative with the rapid test
11. E P I D E M I C A L E R T A N D R E S P O N S ELaboratory Training for Field Epidemiologists
Specificity of a rapid test forSpecificity of a rapid test for
leishmaniasisleishmaniasis
Individuals
without
leishmaniasis
Test
False positive 12
True negative 188
200
Specificity = 188 / 200 = 94%
95% confidence interval: 90%-96%
12. E P I D E M I C A L E R T A N D R E S P O N S ELaboratory Training for Field Epidemiologists
What factors influenceWhat factors influence
the specificity of a test?the specificity of a test?
• Characteristics of the affected persons?
NO: The specificity is estimated on a population of non affected
• Characteristics of the non-affected persons?
YES: The diversity of antibodies to various other antigens in the
population may affect cross reactivity (e.g., If malaria is endemic,
polyclonal hyper gammaglobulinemia may increase the proportion
of false positives)
• Prevalence of the disease?
NO: The specificity is estimated on a population of non affected
Specificity is an INTRINSIC characteristic of the test
13. E P I D E M I C A L E R T A N D R E S P O N S ELaboratory Training for Field Epidemiologists
Identifying the cut-off to use with a testIdentifying the cut-off to use with a test
on the basis of panel analysis: Ideal caseon the basis of panel analysis: Ideal case
Cut-off
0
5
10
15
20
25
1 2 3 4 5 6 7 8 9 10 11 12 13 14
Possible values of the test
Numberoftests
Sick
Well
14. E P I D E M I C A L E R T A N D R E S P O N S ELaboratory Training for Field Epidemiologists
Identifying the cut-off to use with a testIdentifying the cut-off to use with a test
on the basis of panel analysis: Real caseon the basis of panel analysis: Real case
Cut-off
0
5
10
15
20
25
1 2 3 4 5 6 7 8 9 10 11 12 13 14
Possible values of the test
Numberoftests
Sick
Well
True
negatives
False
negatives
True
positives
False
positives
15. E P I D E M I C A L E R T A N D R E S P O N S ELaboratory Training for Field Epidemiologists
To whom sensitivity and specificityTo whom sensitivity and specificity
matters most?matters most?
• Look at denominators!
– Panels of affected individuals
– Panels of negative individuals
• To laboratory specialists!
16. E P I D E M I C A L E R T A N D R E S P O N S ELaboratory Training for Field Epidemiologists
OutlineOutline
• Intrinsic characteristics of a test
– Sensitivity
– Specificity
• Performance of a test in a population
– Predictive value of a positive test
– Predictive value of a negative test
17. E P I D E M I C A L E R T A N D R E S P O N S ELaboratory Training for Field Epidemiologists
How is the test doing in a real population?How is the test doing in a real population?
Status of persons
Affected Non-affected
Test
Positive True + False + A+B
Negative False - True - C+D
A + C B+D A+C+B+D
• The test is now used in a real population
• This population is made of
– Affected individuals
– Non-affected individuals
• The proportion of affected individuals is the prevalence
18. E P I D E M I C A L E R T A N D R E S P O N S ELaboratory Training for Field Epidemiologists
Predictive value of a positive testPredictive value of a positive test
Persons affected
(True positives)
Persons not affected
(False positives)
Persons testing positive
(Positive by test)
Predictive value of a positive test =
True positives / Persons testing positive
Estimate the 95% confidence interval
The predictive value of a positive test is the probability that
an individual testing positive is truly affected
Proportion of affected persons among those testing positive
19. E P I D E M I C A L E R T A N D R E S P O N S ELaboratory Training for Field Epidemiologists
Predictive value of a positive testPredictive value of a positive test
Status of persons
Affected Non-affected
Test
Positive A B A+B
Negative C D C+D
A + C B+D A+C+B+D
PVP = A / (A+B)
This is only valid for the sample of specimens tested
20. E P I D E M I C A L E R T A N D R E S P O N S ELaboratory Training for Field Epidemiologists
What factors influenceWhat factors influence
the predictive value positive of a test?the predictive value positive of a test?
• Sensitivity?
YES: To some extend.
• Specificity?
YES: The more the test is specific, the more it will be negative for
non affected persons. Thus, when the test is positive, it is probably
truly positive (All non affected were correctly identified as testing
negative).
• Prevalence of the disease?
YES: Low prevalence: The test will pick up more false positives
YES: High prevalence: The test will pick up more true positives
21. E P I D E M I C A L E R T A N D R E S P O N S ELaboratory Training for Field Epidemiologists
Predictive value positive of a testPredictive value positive of a test
according to prevalence and specificityaccording to prevalence and specificity
0
10
20
30
40
50
60
70
80
90
100
0
10
20
30
40
50
60
70
80
90
100
Prevalence (%)
PVP % 70%
80%
90%
95%
Specificity
22. E P I D E M I C A L E R T A N D R E S P O N S ELaboratory Training for Field Epidemiologists
Predictive value of a negative testPredictive value of a negative test
Persons non affected
(True negatives)
Persons affected
(False negatives)
Persons testing negative
(Negative by test)
Predictive value of a negative test =
True negatives / Persons testing negative
Estimate the 95% confidence interval
The predictive value of a negative test is the probability that
an individual testing negative is truly non-affected
Proportion of non-affected persons among those testing negative
23. E P I D E M I C A L E R T A N D R E S P O N S ELaboratory Training for Field Epidemiologists
Predictive value of a negative testPredictive value of a negative test
Status of persons
Affected Non-affected
Test
Positive A B A+B
Negative C D C+D
A + C B+D A+C+B+D
PVN = D / (C+D)
This is only valid for the sample of specimens tested
24. E P I D E M I C A L E R T A N D R E S P O N S ELaboratory Training for Field Epidemiologists
What factors influenceWhat factors influence
the predictive value negative of a test?the predictive value negative of a test?
• Sensitivity?
YES: The more the test is sensitive, the more it captures affected
persons. Thus, when the test is negative, it is probably truly
negative (all affected were captured among the positive).
• Specificity?
YES: But to a lesser extend.
• Prevalence of the disease?
YES: Low prevalence: The test will pick up more true negatives
YES: High prevalence: The test will pick up more false negatives
25. E P I D E M I C A L E R T A N D R E S P O N S ELaboratory Training for Field Epidemiologists
Predictive value negative of a testPredictive value negative of a test
according to prevalence and sensitivityaccording to prevalence and sensitivity
Sensitivity
0
10
20
30
40
50
60
70
80
90
100
0
10
20
30
40
50
60
70
80
90
100
Prevalence (%)
PVN %
70%
80%
90%
95%
26. E P I D E M I C A L E R T A N D R E S P O N S ELaboratory Training for Field Epidemiologists
Relation between predictive values andRelation between predictive values and
(1) sensitivity and (2) specificity(1) sensitivity and (2) specificity
PVP =
Se.P
Se.P+(1−Sp)(1−P)
PVN=
Sp(1-P)
Sp(1-P)+(1−Se).P
27. E P I D E M I C A L E R T A N D R E S P O N S ELaboratory Training for Field Epidemiologists
Positive (PPV) and negative (NPV) predictivePositive (PPV) and negative (NPV) predictive
values of a test according to the prevalencevalues of a test according to the prevalence
(95% sensitivity and specificity)(95% sensitivity and specificity)
0
20
40
60
80
100
Prevalence (%)
Predictivevalue(%)
PVP
PVN
0 50 75 10025
28. E P I D E M I C A L E R T A N D R E S P O N S ELaboratory Training for Field Epidemiologists
To whom predictive values matters most?To whom predictive values matters most?
• Look at denominators!
– Persons testing positive
– Persons testing negative
• To clinicians and epidemiologists!
29. E P I D E M I C A L E R T A N D R E S P O N S ELaboratory Training for Field Epidemiologists
SummarySummary
• Sensitivity and specificity matter to laboratory specialists
– Studied on panels of positives and negatives
– Look into the intrinsic characteristics of the test:
• Capacity to pick affected
• Capacity to pick non affected
• Predictive values matter to clinicians and epidemiologists
– Studied on homogeneous populations
– Look into the performance of the test in real life:
• What to make of a positive test
• What to make of a negative test
30. E P I D E M I C A L E R T A N D R E S P O N S ELaboratory Training for Field Epidemiologists
Developed by:
The Department of Epidemic and Pandemic Alert
and Response of the World Health Organization
with the assistance of:
European Program for Field Epidemiology
Training
Canadian Field Epidemiology Programme
Thailand Ministry of Health
Institut Pasteur
Interpretation of results