Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Sensitivity, Specificity
and Likelihood Ratios
K.S. Chew
Faculty of Medicine and Health Sciences
Universiti Malaysia Saraw...
Sensitivity
• Proportion of patients with disease who are tested positive
with a test
• A 100% sensitive test will not hav...
Specificity
• Proportion of patients without disease who are tested
negative with a test
• A 100% specific test will not h...
Sensitivity
Disease +ve Disease -ve
Test +ve a (TP) b (FP)
Test –ve c (FN) d (TN)
1/25/2016 4
TP = True positive
FN = Fals...
Positive Predictive Value
Disease +ve Disease -ve
Test +ve a (TP) b (FP)
Test –ve c (FN) d (TN)
1/25/2016 5
TP = True posi...
Specificity
Disease +ve Disease -ve
Test +ve a (TP) b (FP)
Test –ve c (FN) d (TN)
1/25/2016 6
TP = True positive
FN = Fals...
Negative Predictive Value
Disease +ve Disease -ve
Test +ve a (TP) b (FP)
Test –ve c (FN) d (TN)
1/25/2016 7
TP = True posi...
Sensitivity and Specificity
Image taken from: http://library.med.utah.edu/WebPath/TUTORIAL/BIOSTATS/BIOSTATS.html
1/25/201...
Sensitivity and Specificity
Image taken from: http://library.med.utah.edu/WebPath/TUTORIAL/BIOSTATS/BIOSTATS.html
1/25/201...
Example: Troponin assays
• First generation assay: cut-off 0.5 microgm/l
• 3rd generation assay: 0.05 – 0.10 microgm/l
• H...
Causes of elevated troponins
• Myocardial ischemic conditions
• ACS
• Myocardial ischemic conditions other than ACS
• Syst...
Sensitivity and Specificity
• A trade-off
• When sensitivity increases, specificity decreases
• When specificity increases...
Receiver Operating Characteristics Curve
• When sensitivity increases, specificity decreases
• Therefore, when sensitivity...
Area under curve (AUC)
• Specificity is a measure of true negative; therefore (1 –
specificity) is a measure of false posi...
Example:
1/25/2016 15
Reichlin T, Hochholzer W, Bassetti S,
Steuer S, Stelzig C, Hartwiger S, et al.
Early Diagnosis of My...
Methods
• Multi-center, n = 718, symptoms suggestive of MI
• Diagnostic accuracy of different troponin assays
• Abbott–Arc...
Results
1/25/2016 17
Reichlin et al 2009
Results
• AUC significantly higher for:
• Abbott–Architect Troponin I, 0.96 (95% CI 0.94 to 0.98)
• Roche High-Sensitive T...
Results
1/25/2016 19
Reichlin et al 2009
Likelihood Ratios
• Positive likelihood ratio refers to the likelihood of a patient
with the disease to be tested as posit...
Likelihood Ratios
• LR are more helpful than sensitivity and specificity because
sensitivity and specificity are derived f...
Likelihood Ratios
• Positive likelihood ratio refers to the likelihood of a patient
with the disease to be tested as posit...
Likelihood ratios
• Negative likelihood ratio refers to the likelihood of patient
with the disease to be tested negative a...
Usefulness of LRs
• To choose a diagnostic test
• E.g. which test would be the best to RULE IN a disease?
• Which test wou...
Example:
1/25/2016 25
Collins et al, J Cardiac Failure 2015:21(1)
Fagan Nomogram
1/25/2016 26
Example:
• Why PERC score should only be used when Well’s criteria is
in the low risk category?
• LR (-) of PERC is 0.17 (...
PERC Score
• PERC score is a rule-out criteria for pulmonary embolism
where if none of the 8 PERC criteria are present in ...
Determine your point of equipoise?
• Point of equipoise is the balance point when the risk-
benefit of investigating furth...
Expolarating a LR (-) of 0.17
and a Post-test probability of 1.8%
Therefore, the pre-test probability
must be below 10%
1/...
Wells criteria
Only in the low risk category
of Wells Criteria where the
probability of PE is below
10%. Therefore, PERC s...
Recommended Video Tutorials
• 6 short video series on sensitivity and specificity:
• https://www.youtube.com/watch?v=U4_3f...
Upcoming SlideShare
Loading in …5
×

Sensitivity, specificity and likelihood ratios

7,034 views

Published on

A short tutorial on sensitivity, specificity and likelihood ratios. In this presentation, I demonstrate why likelihood ratios are better parameters compared to sensitivity and specificity in real world setting.

Published in: Health & Medicine

Sensitivity, specificity and likelihood ratios

  1. 1. Sensitivity, Specificity and Likelihood Ratios K.S. Chew Faculty of Medicine and Health Sciences Universiti Malaysia Sarawak Email: kschew@unimas.my1/25/2016 1
  2. 2. Sensitivity • Proportion of patients with disease who are tested positive with a test • A 100% sensitive test will not have any false negative results (although it may have a high rate of false positive results) • Therefore, a negative result of a highly sensitive test means it is likely to be a true negative (it rules out the disease) “SN-OUT” 1/25/2016 2
  3. 3. Specificity • Proportion of patients without disease who are tested negative with a test • A 100% specific test will not have false positive results (although it may have high rate of false negative results) • Therefore, a positive result of a highly specific test means it is likely to be true positive (it rules in the disease) “SP-IN” 1/25/2016 3
  4. 4. Sensitivity Disease +ve Disease -ve Test +ve a (TP) b (FP) Test –ve c (FN) d (TN) 1/25/2016 4 TP = True positive FN = False negative FP = False positive TN = True negative Sensitivity = (a)/(a+c)
  5. 5. Positive Predictive Value Disease +ve Disease -ve Test +ve a (TP) b (FP) Test –ve c (FN) d (TN) 1/25/2016 5 TP = True positive FN = False negative FP = False positive TN = True negative Positive PV = (a)/(a+b)
  6. 6. Specificity Disease +ve Disease -ve Test +ve a (TP) b (FP) Test –ve c (FN) d (TN) 1/25/2016 6 TP = True positive FN = False negative FP = False positive TN = True negative Specificity = (d)/(b+d)
  7. 7. Negative Predictive Value Disease +ve Disease -ve Test +ve a (TP) b (FP) Test –ve c (FN) d (TN) 1/25/2016 7 TP = True positive FN = False negative FP = False positive TN = True negative Negative PV = (d)/(c+d)
  8. 8. Sensitivity and Specificity Image taken from: http://library.med.utah.edu/WebPath/TUTORIAL/BIOSTATS/BIOSTATS.html 1/25/2016 8 To increase sensitivity, shift to the left (purple line) But by shifting to the left, it increases proportion of false positive, which means reduced specificity
  9. 9. Sensitivity and Specificity Image taken from: http://library.med.utah.edu/WebPath/TUTORIAL/BIOSTATS/BIOSTATS.html 1/25/2016 9 To increase specificity, shift to the right (purple line) But by shifting to the right, it increases proportion of false negative, which means reduced sensitivity
  10. 10. Example: Troponin assays • First generation assay: cut-off 0.5 microgm/l • 3rd generation assay: 0.05 – 0.10 microgm/l • High-sensitive troponin (hsTn): 0.0030 microgm/l • High-sensitive Roche Elecsys: 0.0014 microgm/l • The diagnostic sensitivity of hsTn assays (ability to rule-out MI) are of the order of 90–95% when tested at the point of admission (still misses 5 - 10% of cases) 1/25/2016 10 Ref: Gamble et al, Br J Cardiol. 2013;20(4)
  11. 11. Causes of elevated troponins • Myocardial ischemic conditions • ACS • Myocardial ischemic conditions other than ACS • Systemic conditions • Myocardial injury without ischemic insults • Systemic conditions – renal failure, sepsis • Specific identifiable precipitants – cardiac contusion, burns >30% BSA 1/25/2016 11
  12. 12. Sensitivity and Specificity • A trade-off • When sensitivity increases, specificity decreases • When specificity increases, sensitivity decreases 1/25/2016 12 Image taken from: http://groups.csail.mit.edu/cb/struct2net/webserver/about.html
  13. 13. Receiver Operating Characteristics Curve • When sensitivity increases, specificity decreases • Therefore, when sensitivity increases, (1 – specificity) increases • AUC – represents how good a test is 1/25/2016 13
  14. 14. Area under curve (AUC) • Specificity is a measure of true negative; therefore (1 – specificity) is a measure of false positive • While AUC of 1 represents a perfect test; AUC of 0.5 is a worthless test (a.k.a for every one true positive, there is an equal chance of getting one false positive) • Interpretation: • 0.90 -1 = excellent • 0.80 - 0.90 = good • 0.70 - 0.80 = fair • 0.60 - 0.70 = poor • 0.50 - 0.60 = fail 1/25/2016 14
  15. 15. Example: 1/25/2016 15 Reichlin T, Hochholzer W, Bassetti S, Steuer S, Stelzig C, Hartwiger S, et al. Early Diagnosis of Myocardial Infarction with Sensitive Cardiac Troponin Assays. N Eng J Med 2009;361(9):858-67.
  16. 16. Methods • Multi-center, n = 718, symptoms suggestive of MI • Diagnostic accuracy of different troponin assays • Abbott–Architect Troponin I • Roche High-Sensitive Troponin T • Roche Troponin I, and Siemens Troponin I Ultra) • vs standard assay (Roche Troponin T). • Final diagnosis determined by 2 independent cardiologists: reviewing clinical history, physical findings, labs, ECG, echo, angio findings, etc 1/25/2016 16 Reichlin et al 2009
  17. 17. Results 1/25/2016 17 Reichlin et al 2009
  18. 18. Results • AUC significantly higher for: • Abbott–Architect Troponin I, 0.96 (95% CI 0.94 to 0.98) • Roche High-Sensitive Troponin T, 0.96 (95% CI 0.94 to 0.98) • Roche Troponin I, 0.95 (95% CI, 0.92 to 0.97) • Siemens Troponin I Ultra 0.96 (95% CI, 0.94 to 0.98) • standard assay, 0.90 (95% CI, 0.86 to 0.94) 1/25/2016 18 Reichlin et al 2009
  19. 19. Results 1/25/2016 19 Reichlin et al 2009
  20. 20. Likelihood Ratios • Positive likelihood ratio refers to the likelihood of a patient with the disease to be tested as positive compared to a patient without the disease • Negative likelihood ratio refers to the likelihood of patient with the disease to be tested negative as compared to a patient without the disease • Every test has both LR (+) and LR (-) 1/25/2016 20
  21. 21. Likelihood Ratios • LR are more helpful than sensitivity and specificity because sensitivity and specificity are derived from population where we already know whether they have or do not have the disease • Whereas LRs tell us prospectively how a positive or negative test results affect the likelihood of patient to have a disease when we do not know whether they have it or not • Likelihood ratios have factored in the sensitivity, specificity of the test (the TP, TN, FP, FN) 1/25/2016 21
  22. 22. Likelihood Ratios • Positive likelihood ratio refers to the likelihood of a patient with the disease to be tested as positive compared to a patient without the disease • LR (+) • = (True positive)/(False positive) • = (sensitivity)/(1-specificity) • The higher LR (+), the better the test to RULE IN the disease 1/25/2016 22
  23. 23. Likelihood ratios • Negative likelihood ratio refers to the likelihood of patient with the disease to be tested negative as compared to a patient without the disease • LR (-) = (False Negative)/(True Negative) • = (1 – sensitivity)/(specificity) • The smaller the LR (-), the better the test TO RULE OUT the disease 1/25/2016 23
  24. 24. Usefulness of LRs • To choose a diagnostic test • E.g. which test would be the best to RULE IN a disease? • Which test would be the best to RULE out a disease? • To calculate a post-test probability (use Fagan Normogram) 1/25/2016 24
  25. 25. Example: 1/25/2016 25 Collins et al, J Cardiac Failure 2015:21(1)
  26. 26. Fagan Nomogram 1/25/2016 26
  27. 27. Example: • Why PERC score should only be used when Well’s criteria is in the low risk category? • LR (-) of PERC is 0.17 (95% CI: 0.11 – 0.25) • Ref: Carpenter CR, et al (2009). Differentiating low-risk and no-risk PE patients: the PERC score. J Emerg Med, 36 (3), 317-22 1/25/2016 27
  28. 28. PERC Score • PERC score is a rule-out criteria for pulmonary embolism where if none of the 8 PERC criteria are present in a patient, PE can be ruled out clinically • B = Blood in sputum (hemoptysis) • R = Room air O2 Sat>95% • E = estrogen or homonal use • A = Age >50 years • T = Thrombotic events (DVT, PE) or its possibility • H = HR >/= 100/min • S = surgery past 4 weeksl 1/25/2016 28
  29. 29. Determine your point of equipoise? • Point of equipoise is the balance point when the risk- benefit of investigating further for PE vs risk-benefit of NOT investigating further for PE. • Kline et al (2004) – point of equipoise for PE is 1.8%. 1/25/2016 29
  30. 30. Expolarating a LR (-) of 0.17 and a Post-test probability of 1.8% Therefore, the pre-test probability must be below 10% 1/25/2016 30 Determine that your post-test probability is no more than 1.8% (point of equipoise) LR (-) for PERC
  31. 31. Wells criteria Only in the low risk category of Wells Criteria where the probability of PE is below 10%. Therefore, PERC score should be used only when the Wells score is in the low risk category 1/25/2016 31
  32. 32. Recommended Video Tutorials • 6 short video series on sensitivity and specificity: • https://www.youtube.com/watch?v=U4_3fditnWg&list=PL41c kbAGB5S2PavLIXUETzAmi5reIod23 • On likelihood ratios: • https://www.youtube.com/watch?v=TzPvCSFZUSQ 1/25/2016 32

×