The document discusses the concepts of reliability and validity in measurement, emphasizing that reliability refers to consistency while validity pertains to how well an indicator truly measures the intended concept. It explains the relationship between the two, noting that reliability is necessary for validity but does not guarantee it, alongside various types and threats to validity. Additionally, it outlines levels of measurement, along with specific types of validity such as face, content, criterion, predictive, and construct validity.

1. RELIABILITY
VERSUS
VALIDITY

2. Reliability & Validity
 Reliability
Reliability
 “
“dependability”
dependability”
 is the indicator consistent?
is the indicator consistent?
 same result every time?
same result every time?
 Does not necessary measure what you think it measures
Does not necessary measure what you think it measures
(ex. May consistently measure something other than the
(ex. May consistently measure something other than the
concept)
concept)
 Validity
Validity
 measurement validity - how well the conceptual and
measurement validity - how well the conceptual and
operational definitions mesh with each other
operational definitions mesh with each other
 does measurement tool measure what we think ?
does measurement tool measure what we think ?

3. Relationship between Measurement
Reliability and Validity
 reliability necessary for validity but does not
reliability necessary for validity but does not
guarantee it (
guarantee it (“necessary but not sufficient”)
“necessary but not sufficient”)
 measure can be reliable but invalid
measure can be reliable but invalid

4. Examples of Types of
Reliability
 stability
stability
 over time
over time
 representative
representative
 across different subgroups of a population
across different subgroups of a population
 (ex. of problems: young people may exaggerate
(ex. of problems: young people may exaggerate
their ages, older people may reduce theirs)
their ages, older people may reduce theirs)
 equivalence
equivalence
 Comparable results from multiple indicators
Comparable results from multiple indicators
 intercoder reliability
intercoder reliability

5. Improving Reliability
 clearly conceptualize constructs
clearly conceptualize constructs
 increase level of measurement
increase level of measurement
 use pretests, pilot studies
use pretests, pilot studies
 e.g. use multiple indicators :
e.g. use multiple indicators :
Dependent
Variable Measure
Independent
Variable Measure
Empirical
Association?
a2 a3
a1 b1 b2
A B
Specific Indicators
Specific Indicators
Measurement Using
Multiple Indicators
Neuman (2000: 167)

6. Selected Types of Validity: Face
Validity
 judgement by the scientific community that indicator
judgement by the scientific community that indicator
measures the concept
measures the concept
Construct/concept
Measure
Scientific Community

7. Content Validity
 measure represents all the aspects of conceptual definition of
measure represents all the aspects of conceptual definition of
construct.
construct.
 judgement concerning how adequately a test samples behavior
judgement concerning how adequately a test samples behavior
representative of the universe of behavior the test was
representative of the universe of behavior the test was
designed to sample
designed to sample.
.
Love
Beliefs
& Values

8.  Criterion
Criterion
 The validity of indicator is verified by comparing it with another
The validity of indicator is verified by comparing it with another
measure of the same construct
measure of the same construct
 Predictive
Predictive
 Relies on occurrence of future event or behavior for
Relies on occurrence of future event or behavior for
verification of indicator
verification of indicator
 Concurrent
Concurrent
 relies on pre-existing accepted measure to verify indicator
relies on pre-existing accepted measure to verify indicator
 Construct Validity
Construct Validity
 A type of measurement validity that uses multiple indicators
A type of measurement validity that uses multiple indicators
 convergent and discriminate
convergent and discriminate
Some types of Measurement Validity

9. Other Dimensions
of Validity
 Internal Validity
Internal Validity
 no error of logic internal to research design
no error of logic internal to research design
 External Validity
External Validity
 results can be generalized
results can be generalized
 Statistical validity
Statistical validity
 correct statistical methodology chosen ?
correct statistical methodology chosen ?
 assumptions fully met
assumptions fully met

10. More terms & concepts for assessing
validity (usually in experimental research)
 Threats to Internal Validity
Threats to Internal Validity
 Selection bias
Selection bias
 History effects (something in context changes)
History effects (something in context changes)
 Maturation
Maturation
 Testing effect
Testing effect
 Instrumentation
Instrumentation
 Mortality
Mortality
 Statistical regression
Statistical regression
 Contamination
Contamination
 Compensatory behaviour
Compensatory behaviour
 Experimenter expectanc
Experimenter expectanc
 Threats to External Validity
Threats to External Validity
 Realism
Realism
 Reactivity
Reactivity

11. Another Aspect of Measurement
Design: Levels of Measurement
 1-Nominal (ex. Mother tongue)
1-Nominal (ex. Mother tongue)
 different categories (names, labels, images)
different categories (names, labels, images)
 not ranked
not ranked
 2-Ordinal (county fair prizewinners ranked by first, second & third prize)
2-Ordinal (county fair prizewinners ranked by first, second & third prize)
 different categories
different categories
 rank-ordered
rank-ordered
 attributes indicate relatively more or less of that variable
attributes indicate relatively more or less of that variable
 distance between the attributes of a variable is imprecise
distance between the attributes of a variable is imprecise

12. Levels of Measurement (cont’d)
 3- Interval Measures (age measured by 5 year age groups)
3- Interval Measures (age measured by 5 year age groups)
 different categories
different categories
 ranked in order
ranked in order
 Can tell amount of difference between categories
Can tell amount of difference between categories
 Usually no true zero
Usually no true zero
 4- Ratio Measures (age measured by date of birth)
4- Ratio Measures (age measured by date of birth)
 different categories
different categories
 ranked in order
ranked in order
 amount of difference between categories
amount of difference between categories
 also possible to state proportion (have a true zero)
also possible to state proportion (have a true zero)
 Relations between levels --
Relations between levels --can collapse from higher
can collapse from higher
into lower, not vice versa
into lower, not vice versa

13. Babbie (1995: 101)
The Research
Process