threats to validity and control issues

1. Developmental Research
Methods
Topic 5: Testing Hypotheses
10/1/2014 CEDP321 Ryan Sain, Ph.D. 1

2. Variation
 Remember, we are interested in two
types of varaiation
 Systematic and unsystematic (chance)
 There are two sources of systematic
variance
◦ Ones due to the IV
◦ Ones due to confounds

3. Types of Error
The Null Hypothesis Is …..
True False
Based on the Test,
We either…
Fail to Reject the
Null
RESEARCH
OBJECTIVE
TYPE II
ERROR
β
Reject the Null
TYPE I
ERROR
α
RESEARCH
OBJECTIVE

4. Threats to Validity
 Internal:
◦ History
◦ Maturation
◦ Instrumentation
◦ Selection
◦ Testing effects
 Pretest sensitization
◦ Regression toward
the mean
◦ Mortality
10/1/2014 CEDP 596-04 Ryan Sain, Ph.D.

5. More Threats
 Reactance
◦ To experiment –
Hawthorne
◦ Expectancy Effect
◦ To testing (sensitized)
 Multiple treatment
◦ Cannot control for prior
treatments
10/1/2014 CEDP 596-04 Ryan Sain, Ph.D.

6. Even more threats
 Subject –
Experimenter
Effects
◦ Hypothesis guessing
◦ Diffusion of
treatment
◦ Good subject effect
◦ Evaluation
apprehension
◦ Obedience
 Demand
Characteristics
◦ Acting out the role of
subject 10/1/2014 CEDP 596-04 Ryan Sain, Ph.D.

7. Control
 Controlling for
confounds
◦ Constancy of
conditions
 THOUSANDS of
designs
 Experimental
 Quasi-experimental
 Varying levels of
internal validity
 Do what you can
10/1/2014 CEDP 596-04 Ryan Sain, Ph.D.

8. Controlling for threats
 Random
assignment
 Pilot studies
◦ Quasi-control
subjects
 Blinding
 Distracter
items
 Standardizin
g
10/1/2014 CEDP 596-04 Ryan Sain, Ph.D.

9. Credits
 www.sxc.hu
◦ Padlocks and faces – emospada
◦ Steering Nature – johnnyberg
◦ Spices – pape2000
10/1/2014 CEDP 596-04 Ryan Sain, Ph.D.