Successfully reported this slideshow.
Upcoming SlideShare
×

# Sensitivity, specificity and likelihood ratios

7,034 views

Published on

A short tutorial on sensitivity, specificity and likelihood ratios. In this presentation, I demonstrate why likelihood ratios are better parameters compared to sensitivity and specificity in real world setting.

Published in: Health & Medicine
• Full Name
Comment goes here.

Are you sure you want to Yes No
• I like it very much

Are you sure you want to  Yes  No

### Sensitivity, specificity and likelihood ratios

1. 1. Sensitivity, Specificity and Likelihood Ratios K.S. Chew Faculty of Medicine and Health Sciences Universiti Malaysia Sarawak Email: kschew@unimas.my1/25/2016 1
2. 2. Sensitivity • Proportion of patients with disease who are tested positive with a test • A 100% sensitive test will not have any false negative results (although it may have a high rate of false positive results) • Therefore, a negative result of a highly sensitive test means it is likely to be a true negative (it rules out the disease) “SN-OUT” 1/25/2016 2
3. 3. Specificity • Proportion of patients without disease who are tested negative with a test • A 100% specific test will not have false positive results (although it may have high rate of false negative results) • Therefore, a positive result of a highly specific test means it is likely to be true positive (it rules in the disease) “SP-IN” 1/25/2016 3
4. 4. Sensitivity Disease +ve Disease -ve Test +ve a (TP) b (FP) Test –ve c (FN) d (TN) 1/25/2016 4 TP = True positive FN = False negative FP = False positive TN = True negative Sensitivity = (a)/(a+c)
5. 5. Positive Predictive Value Disease +ve Disease -ve Test +ve a (TP) b (FP) Test –ve c (FN) d (TN) 1/25/2016 5 TP = True positive FN = False negative FP = False positive TN = True negative Positive PV = (a)/(a+b)
6. 6. Specificity Disease +ve Disease -ve Test +ve a (TP) b (FP) Test –ve c (FN) d (TN) 1/25/2016 6 TP = True positive FN = False negative FP = False positive TN = True negative Specificity = (d)/(b+d)
7. 7. Negative Predictive Value Disease +ve Disease -ve Test +ve a (TP) b (FP) Test –ve c (FN) d (TN) 1/25/2016 7 TP = True positive FN = False negative FP = False positive TN = True negative Negative PV = (d)/(c+d)
8. 8. Sensitivity and Specificity Image taken from: http://library.med.utah.edu/WebPath/TUTORIAL/BIOSTATS/BIOSTATS.html 1/25/2016 8 To increase sensitivity, shift to the left (purple line) But by shifting to the left, it increases proportion of false positive, which means reduced specificity
9. 9. Sensitivity and Specificity Image taken from: http://library.med.utah.edu/WebPath/TUTORIAL/BIOSTATS/BIOSTATS.html 1/25/2016 9 To increase specificity, shift to the right (purple line) But by shifting to the right, it increases proportion of false negative, which means reduced sensitivity
10. 10. Example: Troponin assays • First generation assay: cut-off 0.5 microgm/l • 3rd generation assay: 0.05 – 0.10 microgm/l • High-sensitive troponin (hsTn): 0.0030 microgm/l • High-sensitive Roche Elecsys: 0.0014 microgm/l • The diagnostic sensitivity of hsTn assays (ability to rule-out MI) are of the order of 90–95% when tested at the point of admission (still misses 5 - 10% of cases) 1/25/2016 10 Ref: Gamble et al, Br J Cardiol. 2013;20(4)
11. 11. Causes of elevated troponins • Myocardial ischemic conditions • ACS • Myocardial ischemic conditions other than ACS • Systemic conditions • Myocardial injury without ischemic insults • Systemic conditions – renal failure, sepsis • Specific identifiable precipitants – cardiac contusion, burns >30% BSA 1/25/2016 11
12. 12. Sensitivity and Specificity • A trade-off • When sensitivity increases, specificity decreases • When specificity increases, sensitivity decreases 1/25/2016 12 Image taken from: http://groups.csail.mit.edu/cb/struct2net/webserver/about.html
13. 13. Receiver Operating Characteristics Curve • When sensitivity increases, specificity decreases • Therefore, when sensitivity increases, (1 – specificity) increases • AUC – represents how good a test is 1/25/2016 13
14. 14. Area under curve (AUC) • Specificity is a measure of true negative; therefore (1 – specificity) is a measure of false positive • While AUC of 1 represents a perfect test; AUC of 0.5 is a worthless test (a.k.a for every one true positive, there is an equal chance of getting one false positive) • Interpretation: • 0.90 -1 = excellent • 0.80 - 0.90 = good • 0.70 - 0.80 = fair • 0.60 - 0.70 = poor • 0.50 - 0.60 = fail 1/25/2016 14
15. 15. Example: 1/25/2016 15 Reichlin T, Hochholzer W, Bassetti S, Steuer S, Stelzig C, Hartwiger S, et al. Early Diagnosis of Myocardial Infarction with Sensitive Cardiac Troponin Assays. N Eng J Med 2009;361(9):858-67.
16. 16. Methods • Multi-center, n = 718, symptoms suggestive of MI • Diagnostic accuracy of different troponin assays • Abbott–Architect Troponin I • Roche High-Sensitive Troponin T • Roche Troponin I, and Siemens Troponin I Ultra) • vs standard assay (Roche Troponin T). • Final diagnosis determined by 2 independent cardiologists: reviewing clinical history, physical findings, labs, ECG, echo, angio findings, etc 1/25/2016 16 Reichlin et al 2009
17. 17. Results 1/25/2016 17 Reichlin et al 2009
18. 18. Results • AUC significantly higher for: • Abbott–Architect Troponin I, 0.96 (95% CI 0.94 to 0.98) • Roche High-Sensitive Troponin T, 0.96 (95% CI 0.94 to 0.98) • Roche Troponin I, 0.95 (95% CI, 0.92 to 0.97) • Siemens Troponin I Ultra 0.96 (95% CI, 0.94 to 0.98) • standard assay, 0.90 (95% CI, 0.86 to 0.94) 1/25/2016 18 Reichlin et al 2009
19. 19. Results 1/25/2016 19 Reichlin et al 2009
20. 20. Likelihood Ratios • Positive likelihood ratio refers to the likelihood of a patient with the disease to be tested as positive compared to a patient without the disease • Negative likelihood ratio refers to the likelihood of patient with the disease to be tested negative as compared to a patient without the disease • Every test has both LR (+) and LR (-) 1/25/2016 20
21. 21. Likelihood Ratios • LR are more helpful than sensitivity and specificity because sensitivity and specificity are derived from population where we already know whether they have or do not have the disease • Whereas LRs tell us prospectively how a positive or negative test results affect the likelihood of patient to have a disease when we do not know whether they have it or not • Likelihood ratios have factored in the sensitivity, specificity of the test (the TP, TN, FP, FN) 1/25/2016 21
22. 22. Likelihood Ratios • Positive likelihood ratio refers to the likelihood of a patient with the disease to be tested as positive compared to a patient without the disease • LR (+) • = (True positive)/(False positive) • = (sensitivity)/(1-specificity) • The higher LR (+), the better the test to RULE IN the disease 1/25/2016 22
23. 23. Likelihood ratios • Negative likelihood ratio refers to the likelihood of patient with the disease to be tested negative as compared to a patient without the disease • LR (-) = (False Negative)/(True Negative) • = (1 – sensitivity)/(specificity) • The smaller the LR (-), the better the test TO RULE OUT the disease 1/25/2016 23
24. 24. Usefulness of LRs • To choose a diagnostic test • E.g. which test would be the best to RULE IN a disease? • Which test would be the best to RULE out a disease? • To calculate a post-test probability (use Fagan Normogram) 1/25/2016 24
25. 25. Example: 1/25/2016 25 Collins et al, J Cardiac Failure 2015:21(1)
26. 26. Fagan Nomogram 1/25/2016 26
27. 27. Example: • Why PERC score should only be used when Well’s criteria is in the low risk category? • LR (-) of PERC is 0.17 (95% CI: 0.11 – 0.25) • Ref: Carpenter CR, et al (2009). Differentiating low-risk and no-risk PE patients: the PERC score. J Emerg Med, 36 (3), 317-22 1/25/2016 27
28. 28. PERC Score • PERC score is a rule-out criteria for pulmonary embolism where if none of the 8 PERC criteria are present in a patient, PE can be ruled out clinically • B = Blood in sputum (hemoptysis) • R = Room air O2 Sat>95% • E = estrogen or homonal use • A = Age >50 years • T = Thrombotic events (DVT, PE) or its possibility • H = HR >/= 100/min • S = surgery past 4 weeksl 1/25/2016 28
29. 29. Determine your point of equipoise? • Point of equipoise is the balance point when the risk- benefit of investigating further for PE vs risk-benefit of NOT investigating further for PE. • Kline et al (2004) – point of equipoise for PE is 1.8%. 1/25/2016 29
30. 30. Expolarating a LR (-) of 0.17 and a Post-test probability of 1.8% Therefore, the pre-test probability must be below 10% 1/25/2016 30 Determine that your post-test probability is no more than 1.8% (point of equipoise) LR (-) for PERC
31. 31. Wells criteria Only in the low risk category of Wells Criteria where the probability of PE is below 10%. Therefore, PERC score should be used only when the Wells score is in the low risk category 1/25/2016 31
32. 32. Recommended Video Tutorials • 6 short video series on sensitivity and specificity: • https://www.youtube.com/watch?v=U4_3fditnWg&list=PL41c kbAGB5S2PavLIXUETzAmi5reIod23 • On likelihood ratios: • https://www.youtube.com/watch?v=TzPvCSFZUSQ 1/25/2016 32