Validity and reliability expressesions means as to how measurements and diagnostic approaches can more efficiently and maintaning the accuracy with many repeated tests. In validity we basically speak of specificity and sensitivity of tests, which can be affected by prevalence.

1. VALIDITY OF DIAGNOSTIC AND
SCREENING TESTS
By Konje Eveline

2. Introduction
 To understand how a disease is transmitted
and develops
 To provide appropriate and effective health
care
 The quality of screening and diagnostic test is
a critical issue
 Validity and reliability

3. Validity of a test
 The validity of a test is ability to do what is
supposed to do
 distinguish between who has a disease and who
does not
 The validity has 2 components sensitivity
and specificity
 Sensitivity is the probability of testing positive if
the disease is truly present
 Specificity is the probability of testing negative if
the disease is truly absent

4. Quality of a test
Test Results
Disease Status
Diseased Non diseased
Positive True positive
(a)
False positive
(b)
Negative False negative
(c)
True negative
(d)

5. Sensitivity
 The formula for sensitivity is
 The limits of a test’s sensitivity are presented by false
negative results
 The false negative error rate is
 
c
a
a
y
sensitivit


c
a
c
FNrate



6. Specificity
 It is the ability of a test to indicate non
diseased when disease is absent
 The formula is
 The limits of a test’s specificity are false positive
results
 False positive error rate is given by
d
b
d
specificy


d
b
b
ate
F


Pr

7. Example One
 Suppose we have a population of 1000
people of whom 100 have a certain
disease, 80 were correctly identified as
positive by the test. Of the 900 who
didn’t have the disease, the 800 were
correctly identified as negative. How
good was the test?

8. Prevalence
 The proportion of people who have a
given disease or condition of interest at
a specified point in time.
 Sensitivity and specificity are
independent of disease prevalence
 Sensitivity includes only subjects with the
disease
 Specificity includes only subjects without
the disease

9. Why are FP and FN important
False positives
 False positive results pose burden on
the health care system (retest)
 Worry is induced in persons
False negative
 If the disease is a serious one for which
effective intervention is available in its
early stage (cancer) a FN result could be
a virtual death sentence

10. Predictive value of a test
 If we screen the disease free population what
is the proportion of people who have the
disease who will be correctly identified
 If the result is +ve, what is the probability that
s/he has the disease being tested (ie positive
predictive value)
 If the result is –ve, what is the probability that
the patient doesn’t have the disease (ie negative
predictive value)

11. Relationship between predictive
value & disease prevalence
 The higher the prevalence in the
screened population tend to increase
value of positive predictive value
 Screening program is most productive
and efficient if its directed to a high risk
target population

12. Question One
 A physical examination was used to screen for
breast cancer in 2500 women with biopsy-
proven adeno-carcinoma of the breast and in
5000 age and race matched control women.
The results of the physical examination were
positive in 1800 cases and in 800 control
women, all of whom showed no evidence of
cancer at biopsy.
 Determine the validity of the physical examination and
predictive values.

13. THIS IS A 2ND OPINION, AT
1ST I THOUGHT U HAD
SOMETHING ELSE
RELIABILITY OF TESTS

14. Introduction
 Aspect of assessing diagnostic and screening
tests
 It asks whether a test is reliable or repeatable
 Regardless of the sensitivity & specificity
 if the test results cant be reproduced, the value and
usefulness of the test are minimal
 Intra-subject variation and inter-observer
variation do cause the variation between test
results

15. Intra-subject variation
 The values obtained in measuring many
human characteristics often vary over
time, even during a short time
 Blood pressure, weight? Height?
 In evaluating any test result, its
important to consider conditions under
which the test was performed including
the time of the day

16. Intra-observer variation
 Sometimes variation occurs between
two observations made by the same
observer
 Tests & examinations differ in the
degree to which subjective factors
enter into the observer’s conclusions
 The greater the subjective element in
the reading, the greater the intra-
observer variation in readings is likely
to be

17. Inter-observer variation
 This is variation between observers
 Two examiners often don’t derive the same
results
 The extent to which observers agree or disagree
is an important issue
 We need to be able to express the extent of
agreement in quantitative terms
 Overall % agreement is used to measure extent
of variation

18. Overall percent agreement
 In 2by2 table
 High chance for observers to agree on “d”
 Tends to inflate the agreement percent
 Disregard the subjects who were labelled
negative by both observers (ie cell d)
 Therefore the overall % agreement will be
100
% x
c
b
a
a
agreement




19. KAPPA statistic
 Reading the results can agree by chance
alone
 To what extent the readings agree
beyond what we would expect by chance
alone
 Kappa statistics will provide the answer
 Numerator: % agreement observed minus
% agreement expected by chance alone
 Denominator: 100%-%agreement by
chance

20. Historical classification of non
small cell carcinoma by two
pathologists
Grade II Grade III Total by B
Grade II 41 3 44
Grade III 4 27 31
Total by A 45 30 75
Pathologist
B
Pathologist A

21. Interpretation of kappa
statistics
 Above 0.75 represents Excellent agreement
beyond chance
 Kappa of 0.40 to 0.75 represents
intermediate to good agreement
 Below 0.40 represents poor agreement

22. Relationship btn validity &
reliability
 Curve for test results that are reliable
 Curve for test results that are valid
 Curve for both validity and reliability

23. THANKS