This document discusses the validity and reliability of diagnostic and screening tests. It defines validity as a test's ability to accurately distinguish those with a disease from those without. Validity has two components: sensitivity and specificity. Reliability refers to a test's ability to produce consistent results regardless of who performs it. A test must be both valid and reliable to be considered good. Factors like cutoff points, disease prevalence, and multiple tests can impact validity and predictive values. Reliability is affected by intra- and inter-observer variations and can be measured using percent agreement and kappa statistics. Both validity and reliability are important for a test to provide useful information.

1. Assessing the Validity & Reliability Of
DiagnosticAnd Screening Tests
DR. MD ZAFAR EQUEBAL
JUNIOR RESIDENT
JNMCH,AMU, ALIGARH
1

2.  To provide appropriate and effective health care, it is necessary to distinguish
between people in the population who are suffering from the disease and those
who don’t.
 How one can say someone is having disease or not , it is based on results of
different diagnostic or screening tests which may be :
Physical examination
Chest X-ray , CT scan , MRI
ECG, EEG
Blood or urine assay
Histopathology
Refractive Error examination
2

3.  Characteristics of a Good Test
Accuracy / Validity
Reliability
VALIDITY
The validity of a test is defined as its ability to distinguish between who have
disease and who do not i.e. If a test measure what it is supposed to measure and
how much test result close to real value.
(Real value/ True value – Result obtained from Gold standard test )
3

4. Validity has 2 components: Sensitivity and Specificity
SENSITIVITY
The sensitivity of the test is defined as the ability of a test to identify correctly
those who are actually suffering from the disease.
SPECIFICITY
The specificity of the test is defined as the ability of a test to identify correctly
those who do not have the disease.
4

5. Test with Dichotomous Result –
Result given by the test is like that not every person with disease is test positive
also not every person who are free from disease is test negative
True Positive(A) = Those actually suffering from disease and test positive
True Negative (D) = Those free from disease and test negative
False Positive (B) = who do not have the disease and test positive
False Negative (C)= who actually suffering from disease and test negative
Diseased Non Diseased
Test + A B
Test - C D
5

6. Calculation of Validity :
Sensitivity = A / A+C (True positive/ True positive+ False negative)
Specificity = D / B+D (True negative/ False positive +True negative )
Example:
Diseased Non
Diseased
Test + 80 100
Test - 20 800
Sensitivity= 80/100 = 80 %
Specificity = 800/900 = 89%
6
Solution:

7. Test of Continuous Variables
Not every time we get directly result of a test to be positive or negative, for
continuous variables we have to set a cutoff point (above the cutoff level it is said to
be positive and below it is negative), such as for Diabetes plasma sugar level, for
Hypertension blood pressure status.
Example: population of 20 diabetics and 20 non diabetics who are being
screened using a blood sugar test
Actual Scenario After High Cutoff for Diabetes
After Low Cutoff for Diabetes
Sensitivity ↓ 25%
Specificity ↑ 90%
Sensitivity ↑ 85%
Specificity ↓ 30%7

8. When we ↓ Cutoff level = ↓ Specificity but ↑
Sensitivity and also more False Positives i.e.
those actually free from disease became Positive by
our test and become emotional and financial burden
to Health care system.
When we ↑ Cutoff level = ↓ Sensitivity but ↑
Specificity and also more False negatives i.e.
those actually suffering from disease became
negative by our test and could be diagnosed if
appropriate test applied, hence would be
treatable at early stage of disease.8

9. Test in Combination:
I. Sequential / Two stage test
Test in Series :
Test A = positive → Test B → also positive ⇒ Declare it Positive
= Negative ⇒ Declare it Negative (assume true negatives) ⇒ Gain in
Specificity
II. Simultaneous test
Test in Parallel :
Test A + Test B = Either test positive ⇒ Declare it positive ⇒ Gain in Sensitivity
= Both test negative ⇒ Declare it negative
Ex. A patient admitted to a hospital may have a number of tests performed at the time of
admission. When multiple tests are used simultaneously to detect a specific disease, the
individual is generally considered to have tested “positive” if he or she has a positive result on
any one or more of the tests. The individual is considered to have tested “negative” if he or she
tests negative on All of the tests ,hence there is gain in sensitivity.
9

10. Predictive value of a Test
(Diagnostic power)
Positive predictive value (PPV)
Proportion of the patients who test positive, actually suffering from the disease
OR diagnostic power of a test to correctly detect the disease.
PPV = A /A+B
Negative predictive value (NPV)
Proportion of the patients who test negative, actually free from the disease OR
diagnostic power of a test to correctly exclude the disease.
NPV = D /C+D
Predictive value of a test depends on
1. Prevalence of the Disease
2. Specificity > Sensitivity
Diseased Non Diseased
Test + A B
Test - C D
10

11. RELIABILITY
Reliability (also termed reproducibility or repeatability) refers to the stability or
consistency of information, i.e. the extent to which similar information is obtained
when a measurement is performed more than once.
 Regardless of the Validity i.e. sensitivity and specificity of a test, if the test
results cannot be reproduced, results are having minimum value.
 Following factors may be responsible for variation in test results :-
1.INTRA SUBJECT VARIATION -variation within same individual subjects
2.INTRA OBSERVER VARIATION -variation in the reading of the same test by
the same observer
11

12. 3.INTER OBSERVER VARIATION - variation between the reading the same test
results by different observer.
 Extent to which two or
more observers agree or
disagree on a test result
can evaluated by
percent agreement .
KAPPA STATISTIC
 Kappa expresses the extent to which the observed agreement exceeds that
which would be expected by chance alone i.e. to what extent does the
agreement between the two observers exceed the level of agreement that
would result just from chance.
 Concept of kappa statistic, proposed by Cohen in 1960.
12

13. Numerator=percent agreement observed − percent agreement expected by chance
alone
Denominator = maximum that the observers could hope to improve their agreement
(i.e., 100% − percent agreement expected by chance alone
13

14.  Possible values of kappa range from –1 to +1, although values less than 0 are
not realistic in practice (the observed agreement would be worse than by chance
alone).
 Landis and Koch suggested that a kappa greater than 0.75 represents
excellent agreement beyond chance, a kappa below 0.40 represents poor
agreement, and a kappa of 0.40 to 0.75 represents intermediate to good
agreement.
 like percent agreement, kappa is primarily used for the assessment of
reliability—that is, when there is no clear-cut standard and it is appropriate
to give equal weight to both sets of readings (observed and by chance).
14

15. RELATIONSHIP BETWEEN VALIDITY AND RELIABILITY
Graph of hypothetical
test results that are
Reliable, but not Valid
Graph of hypothetical
test results that are both
Valid and Reliable
Graph of hypothetical
test results that are
Valid, but not Reliable
15

16. REFRENCES :-
1. GORDIS EPIDEMIOLOGY , SIXTH EDITION
2. RESEARCH METHODS IN COMMUNITY MEDICINE-ABRAMSON, SIXTH EDITION.
3. EPIDEMIOLOGY BEYOND THE BASICS-MYOSES SZKLO, FOURTH EDITION
16