2. Learning Objectives
Define and discuss reliability and validity
Define the term screening
Define and discuss sensitivity and specificity
3. Screening for Disease
Screening--the presumptive identification
of unrecognized disease or defects by the
application of tests, examinations, or
other procedures that can be applied
rapidly
Positive screening results are followed by
diagnostic tests to confirm actual disease
4. Multiphasic Screening
Defined as the use of two or more
screening tests together among large
groups of people
Information obtained on risk factor
status, history of illness, and health
measurements
Commonly used by employers and
health maintenance organizations
5. Mass Screening and Selective
Screening
Mass screening--screening on a large scale of
total population groups regardless of risk
status
Selective screening--screens subsets of the
population at high risk for disease
◦ More economical, and likely to yield more true
cases.
6. Mass Health Examinations
Population or epidemiologic surveys--purpose
is to gain knowledge regarding the
distribution and determinants of diseases
in selected populations
7. Mass Health Examinations (cont’d)
Epidemiologic surveillance--aims at the
protection of community health
through case detection and
intervention
Case finding (opportunistic screening)--the
utilization of screening tests for
detection of conditions unrelated to
the patient’s chief complaint
9. Social
The health problem should be important
for the individual and the community
Diagnostic follow-up and intervention
should be available to all who require
them
There should be a favorable cost-benefit
ratio
Public acceptance must be high
10. Scientific
Natural history of the condition should
be adequately understood
◦ This knowledge permits identification of early
stages of disease and appropriate biologic
markers of progression
Prevalence of the disease or condition is
high
11. Ethical
The program can alter the natural
history of the condition in a significant
proportion of those screened
Suitable, acceptable tests for screening
and diagnosis of the condition as well as
acceptable, effective methods of
prevention are available
12. Characteristics of a Good
Screening Test
Simple
Rapid
Inexpensive
Safe
Acceptable
14. Reliability (Precision)
The ability of a measuring instrument to
give consistent results on repeated trials
Repeated measurement reliability--the
degree of consistency among repeated
measurements of the same individual on
more than one occasion
15. Reliability (cont’d)
Internal consistency reliability--evaluates the
degree of agreement or homogeneity
within a questionnaire measure of an
attitude, personal characteristic, or
psychological attribute.
Interrater reliability--reliability assessments
derived from agreement among trained
experts
16. Validity (Accuracy)
The ability of a measuring instrument to
give a true measure
Can be evaluated only if an accepted and
independent method for confirming the
test measurement exists
17. Validity (cont’d)
Content validity--the degree to which a
measure covers the range of meanings
included within the concept
Criterion-referenced validity--found by
correlating a measure with an external
criterion of the entity being assessed
18. Validity (cont’d)
Two types of criterion-referenced
validity:
◦ Predictive validity--denotes the ability of a
measure to predict some attribute or
characteristic in the future
◦ Concurrent validity--obtained by correlating a
measure with an alternative measure of the
same phenomenon taken at the same point in
time
20. Interrelationships Between
Reliability and Validity
It is possible for a measure to be highly
reliable but invalid
It is not possible for a measure to be
valid but unreliable
22. Sources of Unreliability and
Invalidity
Measurement bias--constant errors that
are introduced by a faulty measuring
device and tend to reduce the reliability
of measurements
23. Sources of Unreliability and
Invalidity (cont’d)
Halo effect--bias that affects the validity of
questionnaire measurements
24. Example of Halo
All items of a checklist evaluation of an
employee may be filled out in the same general
direction based on the supervisor’s opinion of
the individual
25. Sources of Unreliability and
Invalidity (cont’d)
Social desirability effects
◦ Respondent answers questions in a manner
that agrees with desirable social norms
26. Contingency (2 by 2)Table
Disease
Present Absent Total
Screen Positive a b a+b
Result
Negative c d c+d
Total a+c b+d
27. Sensitivity and Specificity
Sensitivity = the probability that a person
with the disease will test positive
P(+ | D)
Specificity = the probability that a person
who does not have the disease will test
negative
27
29. Predictive Values
Positive Predictive value = the probability
that a person who tests positive has the
disease
Negative Predictive value = The probability
that a person who tests negative does not
have the disease
29
33. Effects of Disease Prevalence on
the Predictive Value of a
Screening Test
When the prevalence of a disease falls,
the predictive value (+) falls, and the
predictive value (-) rises.
34. Relationship Between Sensitivity
and Specificity
To improve sensitivity, the cut point
used to classify individuals as diseased
should be moved farther in the range of
the nondiseased (normals)
To improve specificity, the cut point
should be moved farther in the range
typically associated with the disease
36. Procedures to Improve Sensitivity
and Specificity
Retrain screeners--reduces the amount
of misclassification in tests that require
human assessment
Recalibrate screening instrument--
reduces the amount of imprecision
Utilize a different test
Utilize more than one test
37. Evaluation of Screening Programs
Randomized control trials
◦ Subjects receive either the new screening
test or usual care
Case-control studies
◦ Cases--fatal cases of the disease
◦ Controls--nonfatal cases
◦ Exposure--screening program