2. Errors in measurement
Reliability
If you measure the same thing twice do you get the same
values?
Validity
Does your measure really measure what it is supposed to
measure??
reliable
valid
reliable
invalid
unreliable
invalid
3. Reliability
True score + measurement error
A reliable measure will have a small amount of
error
Multiple “kinds” of reliability
• Test-retest
• Internal consistency
• Inter-rater
4. Reliability
Test-restest reliability
Test the same participants more than once
• Measurement from the same person at two different times
• Should be consistent across different administrations
Reliable Unreliable
5. Reliability
Internal consistency reliability
Multiple items testing the same construct
Extent to which scores on the items of a measure
correlate with each other
• Cronbach’s alpha (α)
• Split-half reliability
• Correlation of score on one half of the measure with
the other half (randomly determined)
6. Reliability
At least 2 raters observe behavior
Inter-rater reliability
Extent to which raters agree in their observations
• Are the raters consistent?
Requires some training in judgment
7. Validity
Does your measure really measure what it is
supposed to measure?
There are many “kinds” of validity
10. Construct Validity
Usually requires multiple studies, a large body
of evidence that supports the claim that the
measure really tests the construct
11. Face Validity
At the surface level, does it look as if the
measure is testing the construct?
“This guy seems smart to me,
and
he got a high score on my IQ measure.”
12. Internal Validity
Did the change in the
DV result from the
changes in the IV or
does it come from
something else?
The precision of the results
13. Threats to internal validity
History – an event happens the experiment
Maturation – participants get older (and other changes)
Selection – nonrandom selection may lead to biases
Mortality – participants drop out or can’t continue
Testing – being in the study actually influences how the
participants respond
The precision of the results
14. External Validity
Are experiments “real life” behavioral situations,
or does the process of control put too much
limitation on the “way things really work?”
15. External Validity
Variable representativeness
Relevant variables for the behavior studied
along which the sample may vary
Setting representativeness
Are the properties of the research setting
similar to those outside the lab (Ecological validity)
Subject representativeness
Characteristics of sample and target
population along these relevant variables
16. Sampling
Why do we do we use sampling methods?
Typically don’t have the resources to test everybody,
so we test a subset
19. Sampling
Why do we do we use sampling methods?
Goals of “good” sampling:
– Maximize Representativeness:
– To what extent do the characteristics of
those in the sample reflect those in the
population
– Reduce Bias:
– A systematic difference between those in
the sample and those in the population
20. Sampling Methods
Probability sampling
Simple random sampling
Systematic sampling
Stratified sampling
Non-probability sampling
Convenience sampling
Quota sampling
Have some element of
random selection
Susceptible to biased
selection
21. Simple random sampling
Every individual has a equal and independent
chance of being selected from the population
