More Related Content
What's hot
What's hot (11)
Similar to Screening and diagnostic testing
Similar to Screening and diagnostic testing (20)
More from amitakashyap1
More from amitakashyap1 (20)
Recently uploaded
Recently uploaded (20)
Screening and diagnostic testing
- 1. Screening and Diagnostic Testing Amita kashyap Sr Prof nPSM SMS Med College, Jaipur
- 2. Assessing the Validity and Reliability of Diagnostic and Screening Tests • Screening and diagnostic tests - To distinguish between people who have the disease and those who do not. • Hence quality of screening and diagnostic tests is a critical issue. • In using a test to so distinguish, it is important to understand how characteristics are distributed in human populations.
- 3. 0 5 10 15 20 25 30 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 Number of Subjects Induration in mm Distribution of tuberculin reaction.
- 4. 0 20 40 60 80 100 120 <110 110 120 130 140 150 160 170 180 >180 Number in thousands systolic blood pressure in mm Hg Distribution of systolic blood pressure
- 5. Variability and Bias • A 45 yr old man’s BP was 140/86 during routine check up for job, he was obese. • His father died of MI at 65 yrs of age • Total S Cholesterol (non fasting) was 242 mg/ dl • No other abnormality • Physician asked him to come after 2 weeks; fasting, for further testing • Repeat total S Cholesterol (fasting) was 198 mg/ dl • Physician’s decision to treat by drugs changed!!
- 7. Levels features Individual Individual variability Measurement variability Population Genetic variability btw individuals Environmental variability Measurement variability Sample Manner of Sampling Size of Sample Measurement variability Levels of Variability
- 8. Sources of variability features Individual characteristics Diurnal variation Factors like Age, diet and exercise Environmental like season and temperature Measurement characteristics Poor calibration of instrument Inherent lack of precision of the instrument Observers misreading or recording Potential Sources of Variability
- 11. Validity • The degree to which a measurement or study reaches a correct conclusion 1. Internal Validity – the extent to which the results of an accurately reflect the true situation. To improve on it we decrease the impact of factors extraneous to the study question by i. restricting the type of subjects and ii. the environment in which the study is performed 2. External Validity - generalizability
- 12. Bias – a threat to validity • The systematic error in a study that leads to a distortion of the results • Randomization reduces the chance difference between the groups – Selection Bias – Information Bias – Confounding : (can be quantified) otherwise evaluation of bias is subjective • Likelihood of 1) the presence of bias and 2) its potential magnitude of effect
- 13. Total Population Sample Frame Sampling scheme Eligible Subjects Inclusion Criteria Exclusions Subjects asked to participate Informed consent Non Participants Participants Lost to Follow Up Participants complete the study
- 14. Information Bias Unacceptability Bias Recall and interviewer Bias
- 15. Oob Obesity Mayocardial Infarction Total Cholesterol Confounding A potential confounder must satisfy two conditions: 1. Association with the disease of interest in the absence of exposure 2. Association with the exposure but not as a result of being exposed
- 16. Confounding
- 17. Practice of clinical Medicine is the artful application of Science. Variability is the law of life. No two individuals react or behave alike – probability is the guide of Life!
- 18. Diagnostic Testing • Patient Profile : A 54 year old school teacher got her physical examination for insurance. She had no complaints; (she had hot flashes a year ago but had resolved without treatment). Physical examination, including breast, pelvic (PAP smear), and rectal examination; NAD. • Physician recommended mammogram. (?) • Mammogram was not normal, hence she was referred to a surgeon who also found Breast normal but • Based on mammographic abnormality however; both surgeon and radiologist agreed for FNA under radiologic guidance for abnormal breast. • FNA specimen revealed cancer cells and patient was scheduled for further surgery next week.
- 19. 0.3 13 20 40 60 80 100 64 After positive FNA result 54 yr old women Before Mamogram After positive mammogram Probability of Breast Cancer (Percent) palpable lump Prior to mamogram Tests are performed to detect the disease, assess its severity, predict outcome, or to monitor response to therapy Schematic Diagram of the estimated Probability of Breast Cancer in a 54 yr old women without palpable Breast Mass, after A positive mammogram and following a positive FNA test result 1% H/o Br Ca In mother
- 20. Total a + c b + d
- 21. Sensitivity and Specificity Surgical Biopsy (Gold Standard) FNA results positive Disease No Disease Total 14 8 22 negative 1 91 92 TOTAL 15 99 114 Sensitivity = 14 15 (14+1) X 100 = 93% Specificity = 91 99 (91+8) X 100 = 92% PV+ = 14 14 + 8 X 100 = 64% PV – = 91 91 + 1 X 100 = 99% •Accuracy (validity)- determining the ‘True Status’ of the disease •Descriptors of test accuracy - Sensitivity and Specificity- •the validity of the test assessed relative to gold standard Pre FNA P(Br Ca) =15/114 =0.13 Pre FNA P(No Br Ca) =99/114 =0.87
- 22. Post FNA probability of disease for +ve or –ve test result guide further action Whether the probability of Br. Ca is 13% or 64%; further workup is required, A –ve test result would reduce the probability that Br. Ca is present to 1% (100% minus PV-ve) So! Now no Biopsy…but keep watch The greater the sensitivity, the more likely the test will detect the persons with the disease Predictive value (+ve and -ve)-estimation of the probability of the presence or absence of disease if test is positive or negative Predictive value of a test is affected by the prevalence of the disease.
- 23. Surgical biopsy FNA results Positive Cancer No Cancer Total 14 8 22 Negative 1 91 92 Total 15 99 114 Surgical biopsy FNA results Positive Cancer No Cancer Total 113 15 128 Negative 8 181 189 Total 121 196 317 Effect of Prevalence on Predictive value of a test: For Patients without palpable masses For Patients With palpable masses Prevalence= 13% Sensitivity = 14/15=93% Specificity = 91/99 =92% PV + =14/22= 64% PV - = 91/92= 99% Prevalence= 38% Sensitivity = 93% Specificity = 92% PV + = 88% PV - = 96%
- 24. Specific Example Test Result Pts with disease Pts without the disease
- 25. Test Result Call these patients “negative” Call these patients “positive” Threshold
- 26. Test Result Call these patients “negative” Call these patients “positive” without the disease with the disease True Positives Some definitions ...
- 27. Test Result Call these patients “negative” Call these patients “positive” without the disease with the disease False Positives
- 28. Test Result Call these patients “negative” Call these patients “positive” without the disease with the disease True negatives
- 29. Test Result Call these patients “negative” Call these patients “positive” without the disease with the disease False negatives
- 30. Test Result without the disease with the disease ‘‘-’’ ‘‘+’’ Moving the Threshold: left e.g. Suspicious FNA results considered positive
- 31. Test Result without the disease with the disease ‘‘-’’ ‘‘+’’ Moving the Threshold: right e.g. Suspicious FNA results considered negative
- 32. Surgical biopsy FNA results positive Cancer No Cancer Total 113 15 128 negative 8 181 189 Total 121 196 317 Effect of cut off value: Suspicious FNA results considered positive Prevalence= 38% Sensitivity = 93% Specificity = 92% PV + = 88% PV - = 96% Surgical biopsy FNA results positive Cancer No Cancer Total 91 0 91 negative 30 196 226 Total 121 196 317 Suspicious FNA results considered negative Prevalence= 38% Sensitivity = 75% Specificity = 100% PV + = 100% PV - = 87%
- 33. Likelihood Ratios (LR) – in interpretation of Dx tests • Definition: An LR is the probability of a particular test result for a persons with the disease divided by the probability of that test result in non-diseased persons LR+ - Probability of +ve test result for a person with disease (true positive/ total diseased) Probability of +ve test result for a person without disease (false positive/ total Non-diseased) Sensitivity / 1-specificity = (14/15)/(8/99)=.93/.08= 11.63 Sensitivity and specificity are expressed as proportion An LR+ve of 1 indicates?
- 34. LR¯ - Probability of -ve test result for a person with the disease (false positive/ total diseased) Probability of -ve test result for a person without disease (true negatives/ total Non-diseased) i.e. 1-Sensitivity)/Specificity Surgical Biopsy (Gold Standard) FNA results positive Disease No Disease Total 14 8 22 negative 1 91 92 TOTAL 15 99 114 LR+ = Sensitivity / 1-specificity = 0.93/1-0.92 =0.93/0.08=11.63 LR¯ - 1-Sensitivity)/Specificity = 1-0.93/0.92 = 0.07/0.92=0.08 In contrast to PV, LR does not vary as a function of Prevalence
- 35. Receiver Operating Characteristic (ROC) Curve • Diagnostic tests giving quantitative outcome e.g. serum levels of enzymes, there are many options about where to set a cut off point – as the cut off point rises (from 200 to 250mg/dl for total cholesterol) the sensitivity will increase with a corresponding decrease in specificity. •At each cutoff point, sensitivity and (1- specificity) is calculated and plotted on ‘y’ and ‘x’ axis respectively along the full range of cutoff points
- 37. Say 1 Say 0
- 38. Say 1 Say 0
- 39. Say 1 Say 0
- 40. Say 0 Say 1
- 41. Say 1 Say 0
- 42. Say 1 Say 0
- 43. Say 0 Say 1
- 44. True Positive Rate (sensitivity) 0% 100% False Positive Rate (1-specificity) 0% 100% ROC curve LR+ = 1, +ve test is equally likely in persons with or without the disease Signal Noise Substantial gain in sensitivity with only modest reduction in specificity AUC - summary Index Highest possible value = 1 Area under diagonal line=0.5
- 45. True Positive Rate 0% 100% False Positive Rate 0 % 100% True Positive Rate 0% 100% False Positive Rate 0% 100% A good test: A poor test: ROC curve comparison
- 46. Best Test: Worst test: True Positive Rate 0 % 100% False Positive Rate 0 % 100 % True Positive Rate 0 % 100% False Positive Rate 0 % 100 % The distributions don’t overlap at all The distributions overlap completely ROC curve extremes
- 48. Screening Test • Identify individuals with a disease before it is detected by routine diagnosis (survival may remain same but appear more-lead time bias) • Treatment initiated after screening (early than routine) will improve chance of survival • Length biased sampling occurs when a screening program detects a less aggressive (…slow progressing) disease only • To overcome these biases – age specific mortality rates are calculated in entire population (screened and not screened). It is important to identify false negative results
- 49. • High FP rate and low PV+ is due to low prevalence of the disease in general population • Criteria for Screening Test – – morbidity & motality must be sufficient concern – A high risk population must exist – Test should be sensitive and specific with minimal risk & acceptable – Effective intervention known Disease Status Mammography positive Cancer No Cancer Total 132 985 1117 negative 47 62,295 62,342 Total 179 63,280 63,459 Prevalence= 0.3% Sensitivity = 73.7% Specificity = 98.4% PV + = 11.8% PV - = 99.9% Usefulness of Mammography
- 50. The process of making an objective and systematic analysis of information from all the randomized controlled trials
Editor's Notes
- To understand how a disease is transmitted and develops and to provide appropriate and effective health care, it is necessary to distinguish between people in the population who have the disease and those who do not. This is an important challenge, both in the clinical arena, where patient care is the issue, and in the public health arena, where secondary prevention programs that involve early disease detection and intervention are being considered and where etiologic studies are being conducted to provide a basis for primary prevention. Thus, the quality of screening and diagnostic tests is a critical issue. Regardless of whether the test is a physical examination, a chest X-ray, an electrocardiogram, or a blood or urine assay, the same issue arises: How good is the test in separating populations of people with and without the disease in question? This chapter addresses the question of how we assess the quality of newly available screening and diagnostic tests to make reasonable decisions about their use and interpretation.
- A large group centers on the value of 0 mm—no induration—and another group centers near 20 mm of induration. This type of distribution, in which there are two peaks, is called a bimodal curve. The bimodal distribution permits the separation of individuals who had no prior experience with tuberculosis (people with no induration, seen on the left) from those who had prior experience with tuberculosis (those with about 20 mm of induration, seen on the right). Although some individuals fall into the “gray zone” in the center, and may belong to either curve, most of the population can be easily distinguished using the two curves. Thus, when a characteristic has a bimodal distribution, it is relatively easy to separate most of the population into two groups (e.g., ill and not ill, having a certain condition or abnormality and not having that condition or abnormality).
- In general, however, most human characteristics are not distributed bimodally. Figure shows the distribution of systolic blood pressures in a group of men. In this figure there is no bimodal curve; what we see is a unimodal curve—a single peak. Therefore, if we want to separate those in the group who are hypertensive from those who are not hypertensive, a cutoff level of blood pressure must be set above which people are designated hypertensive and below which they are designated normotensive. No obvious level of blood pressure distinguishes normotensive from hypertensive individuals. Although we could choose a cutoff for hypertension based on statistical considerations, we would ideally like to choose a cutoff on the basis of biologic information; that is, we would want to know that a pressure above the chosen cutoff level is associated with increased risk of subsequent disease, such as stroke, myocardial infarction, or subsequent mortality. Unfortunately, for many human characteristics, we do not have such information to serve as a guide in setting this level. In either distribution—unimodal or bimodal—it is relatively easy to distinguish between the extreme values of abnormal and normal. With either type of curve, however, uncertainty remains about cases that fall into the gray zone.
- According to National cholesterol education program S Ch > 240 mg/dl is an indication for drug tt, 200 – 239 is boderline where diet and lifestyle corrections are considered
- To minimize individual variability – repeat measure and take average, measure 24 hr, for measurement variability – standardize instrument-caliberation, repeat measure and take average, technique (fasting or not); same laboratory type of analyser
- Berkson’s bias – hospital patients usually has more than one disease therefore false associations can be registered
- Unacceptability Bias e.g. – in a case control study regardless of disease status participants may under report eating high fat diet thinking its not good making it difficult for the researchers to identify an association- bias towards the null hypothesis
- There are two accepted methods for dealing with potential confounder Consider them in design by matching on potential confounder or by restricting the sample to limited levels of potential confounders Evaluate confounder in analysis by stratification or by using multivariate analysis (multiple logistic regression)
- Clinical decision making is weighing of probabilities… The purpose of a diagnostic test is to move the estimated probability of the presence of a disease toward either end of the probability scale based on new meaningful information that will alter subsequent treatment / diagnostic plans If the patient’s sister or mother had been previously diagnosed with CA Br patient’s likelihood of having breast cancer prior to any test could have been as high as 1%, if palpable lump would have been there probability would raise to 20-40% Different mammographic and FNA characteristics eg appearance of the nucleus or nuclear/ cytoplasmic ratio change the the estimate of br ca probability …raised/ lower. Further more different pathologists may have different opinion – definite cancer cell/ suspicious
- Sensitivity and specificity are descriptors of the accuracy of a test. Sensitivity – defined as the percentage of persons with the disease of interest having positive test results. Tests with great sensitivity are useful clinically to rule out the presence of a disease. Specificity – defined as the percentage of the persons without the disease of interest having negative test results Positive Predictive Value (PV+) :– defined as percentage of the persons with the positive test result actually having the disease Negative Predictive Value (PV –) :– defined as percentage of the persons with the positive test result actually having the disease
- Before the FNA was performed, the average likelihood of not having breast cancer among the sample women was 87% (99 out of 114). After a negative FNA test the probability of not having cancer is raised to 99% - (91/92=99%) Usefulness of the FNA test For a patient without a palpable lump is considered: - a +ve result increased the probability of Br. Ca from 13% to 64% (but further workup is still required since we cannot take chance of missing a case) a negative test result, however, would reduce the probability that breast cancer is present to 1% (100-99=1%) - now decision could be made to defer surgical biopsy and repeat mamographic and physical examination in several months for women with abnormal mamogram but normal FNA accepting a 1 in 100 risk of mistakenly delaying treatment of an existing cancer. An aggressive approach could be to perform FNA in all cases considering 1% of the total population is a lot of women Although the FNA test has identical sensitivity and specificity in pts with and without lump but PV+ increased from 64% in women without lump to 88% in women with the palpable lump (high prevalence)– therefore its easier to confirm the presence of breast cancer Av. Likelihood of not having Ca Br. = 99/114 =87% Likelihood of having Ca Br. after a -ve FNA test =(100-99= 1%)
- Moving the cut off point changes the sensitivity, specificity and +ve and –ve predictive values and hence the way the test is used! With a cutoff point set btw the categories of benign and suspicious, a –ve FNA test would reduce the probability of Br. Ca by 96% but with 4% chance of Br. Ca. biopsy may still be warranted. A +ve FNA result indicate 88% likelihood of having CA Br. But still would not confirm the diagnosis absolutely. Alternatively, by setting the cutoff point btw suspicious and malignant (a more stringent requirement to consider test +ve) the PV + = 100%. This could be useful clinically as now women with +ve FNA would require no further testing prior to definitive treatment.
- The smallest possible value of LR+ve occurs when numerator is minimized (sensitivity =0) and maximum when denominator is minimized i.e. specificity is 100% (so 1-1=0) resulting in LR of positive infinite. An LR+ve of 1 indicates a test with no value in sorting out persons with and without the disease of interest, since the probability of a +ve test result is equally likely for affected and unaffected persons The larger the value of LR+VE the stronger the association btw having a positive test result and having the disease.
- The sizes of the two likelihood ratios indicate the strength of association btw a test result and likelihood of the disease A diagnostic test with a large LR+ value increases the suspicion of disease for patients with positive results – larger the size better is the diagnostic value of the test…arbitrarily a value of 10 is perceived as an indication of a test of high value for LR+ and 0.08 for LR-
- The ROC plot of a given test is obtained by calculating the sensitivity and specificity of every observed value, and then plotting sensitivity (on the Y axis) against 1 - specificity (on the X axis). A test that does not discriminate between normal and abnormal would give a diagonal straight line from the bottom left corner to the top right corner. All points on such a line represent a 1:1 ratio of true to false positives. An ideal test would give a rectangular plot passing from the origin at the bottom left hand corner towards top left hand corner at first and thence to the top right hand corner. In reality the ROC curves of many of the tests in common use fall in between these extremes. The cut-off point for deciding between normal and abnormal is selected arbitrarily where the ROC curve changes direction from being vertical to horizontal. The more the ROC curve arches into the upper left hand corner away from the diagonal, the better the test.
- Breast cancer is an important public health problem with sufficiently high mortality and morbidity. Early detection allows less extensive surgical treatment and reduces mortality and morbidity. Since the incidence increases steadily with advancing age a high risk group can be constructed - > 50 yrs or so; recommending screening above 50 routinely.
- The overall odds ratio is then calculated by pooling the data of all the studies. This is also called "typical" odds ratio. It is calculated by the difference between number of deaths in the treatment group (i.e. the number observed) and the number of deaths in this group if the treatment were ineffective (i.e. number expected). This gives the Observed minus the Expected statistic. The confidence interval of O E is also calculated The outcome in the case of each study can be estimated separately by calculating the O E value. If the observed number (O) differs systematically from the expected number (E), there is clear evidence of effect. The totaled O E gives a measure of the overall statistical significance and the effect size