Research Methods in Psychology Quasi-Experimental Designs 1 Characteristics of True Experiments Manipulate Independent Variable (IV) Treatment, comparison conditions High degree of control Choice of the DVs Random assignment to conditions Unambiguous outcome regarding effect of IV on DV Internal validity 2 Applied Research Goals Test external validity of lab findings Improve conditions in which people live and work (natural settings) Quasi-experiments Procedures that approximate the conditions of highly controlled laboratory experiments 3 Permission Difficult to gain permission to conduct true experiments in natural settings Difficult to gain access to participants Random assignment perceived as unfair People want a “treatment” Random assignment is best way to determine whether a treatment is effective Use “waiting-list” control group or alternate treatments Tablets in English and science classes example Obstacles to Conducting True Experiments in Natural Settings 4 Advantage of True Experiments Threats to internal validity are controlled 8 general threats to internal validityhistoryregressionmaturationselectiontestingsubject attritioninstrumentationadditive effects with selection 5 Threats to Internal Validity History When an event occurs at the same time as the treatment and changes participants’ behavior Participants’ “history” includes events other than treatment Difficult to infer treatment has an effect 6 History Threat, continued Does a campus recycling awareness campaign influence the amount of paper, plastic, and cans in campus bins? History threat: Suppose at week 4 (X = treatment) a popular celebrity also starts to promote recycling in the media. Can you conclude the campus campaign was effective? 7 Series 1 1 2 3 4 X 5 6 7 8 30 35 30 35 40 55 55 60 55 Week Recycling (Kg) Threats to Internal Validity, continued Maturation Participants naturally change over time. These maturational changes, not treatment, may explain any changes in participants during an experiment. 8 Maturation Threat, continued Does a new reading program improve 2nd graders’ reading comprehension? Reading comprehension improves naturally as children mature over the year. Can you conclude the reading program was effective? 9 Series 1 Pre Post 25 70 Reading Comprehension Threats to Internal Validity, continued Testing Taking a test generally affects subsequent testing. Participants’ performance on a measure at the end of a study may differ from an initial testing because of their familiarity with the measure. 10 Testing Threat, continued Does teaching a new problem solving strategy influence people’s ability to solve problems quickly? If similar problems are used in the pretest, faster problem solving at post-test may be due to familiarity with the test. Can we conclude the new strategy improves problem-solving ability? 11 Series 1 Pre Post 12 4 Minutes (Mean) Threats to Internal Validity, continued Ins ...

1. Research Methods in Psychology
Quasi-Experimental Designs
1
Characteristics of True Experiments
Manipulate Independent Variable (IV)
Treatment, comparison conditions
High degree of control
Choice of the DVs
Random assignment to conditions
Unambiguous outcome regarding effect of IV on DV
Internal validity
2
Applied Research
Goals
Test external validity of lab findings
Improve conditions in which people live and work (natural
settings)
Quasi-experiments
Procedures that approximate the conditions of highly controlled
laboratory experiments
3

2. Permission
Difficult to gain permission to conduct true experiments in
natural settings
Difficult to gain access to participants
Random assignment perceived as unfair
People want a “treatment”
Random assignment is best way to determine whether a
treatment is effective
Use “waiting-list” control group or alternate treatments
Tablets in English and science classes example
Obstacles to Conducting True Experiments in Natural Settings
4
Advantage of True Experiments
Threats to internal validity are controlled
8 general threats to internal
validityhistoryregressionmaturationselectiontestingsubject
attritioninstrumentationadditive effects with selection
5
Threats to Internal Validity
History
When an event occurs at the same time as the treatment and
changes participants’ behavior
Participants’ “history” includes events other than treatment
Difficult to infer treatment has an effect

3. 6
History Threat, continued
Does a campus recycling awareness campaign influence the
amount of paper, plastic, and cans in campus bins?
History threat: Suppose at week 4 (X = treatment) a popular
celebrity also starts to promote recycling in the media.
Can you conclude the campus campaign was effective?
7
Series 1 1 2 3 4 X 5 6 7 8 30 35
30 35 40 55 55 60 55
Week
Recycling (Kg)
Threats to Internal Validity, continued
Maturation
Participants naturally change over time.
These maturational changes, not treatment, may explain any
changes in participants during an experiment.
8
Maturation Threat, continued
Does a new reading program improve 2nd graders’ reading
comprehension?

4. Reading comprehension improves naturally as children mature
over the year.
Can you conclude the reading program was effective?
9
Series 1 Pre Post 25 70
Reading Comprehension
Threats to Internal Validity, continued
Testing
Taking a test generally affects subsequent testing.
Participants’ performance on a measure at the end of a study
may differ from an initial testing because of their familiarity
with the measure.
10
Testing Threat, continued
Does teaching a new problem solving strategy influence
people’s ability to solve problems quickly?
If similar problems are used in the pretest, faster problem
solving at post-test may be due to familiarity with the test.
Can we conclude the new strategy improves problem-solving
ability?
11

5. Series 1 Pre Post 12 4
Minutes (Mean)
Threats to Internal Validity, continued
Instrumentation
Instruments used to measure participants’ performance may
change over time
Example: observers may become bored or tired
Changes in participants’ performance may be due to changes in
instruments used to measure performance, not to a treatment.
12
Instrumentation, continued
Suppose a police protection program is implemented to decrease
incidence of assault.
At the same time the program is implemented (X), reporting
laws change such that what constitutes assault is broadened.
Can we conclude the program was effective (or ineffective)?
13
Series 1 1 2 3 4 X 5 6 7 8 25 20
22 24 35 45 40 35 38 Series 2 1 2 3
4 X 5 6 7 8 Series 3 1 2 3 4
X 5 6 7 8
Week

6. Assaults
Threats to Internal Validity, continued
Regression
Individuals sometimes perform very well or very poorly because
of chance (e.g., luck).
Chance factors are not likely present during 2nd testing, so
scores will not be as extreme.
Scores will “regress” (go toward) the mean.
Regression effects, not treatment, may account for changes in
participants’ performance over time.
14
Regression, continued
Suppose students are selected for an enrichment program
because of their very high scores on a brief test.
Regression: to the extent the test is an unreliable measure of
ability, we can expect their scores to regress to the mean at the
2nd testing.
Can we conclude the enrichment program was effective (or
ineffective)?
15
Series 1 Pre Post 90 70
Test Score (Mean)

7. Threats to Internal Validity, continued
Subject attrition
When participants are lost from the study (attrition), the group
equivalence formed at the start of the study may be destroyed.
Differences between treatment and control groups at the end of
the study may be due to natural differences in those who remain
in each group.
16
Subject Attrition, continued
Suppose an exercise program is offered to employees who
would like to lose weight.
At Time 1, N = 50
M weight = 225 pounds
At Time 2, N = 25 (25 drop out of study)
Suppose the 25 who stayed in program weighed, on average,
150 pounds at Time 1
Did the exercise program help people to lose weight?
17
East Time1 Time2 0 225 150 0
MeanWeight
Threats to Internal Validity, continued
Selection
Occurs when differences exist between individuals in treatment

8. and control groups at the start of a study
These differences become alternative explanations for any
differences observed at the end of the study
Random assignment controls the selection threat
18
Selection, continued
Suppose a community recycling program is tested. Individuals
who are interested in recycling are encouraged to participate.
Evaluation: Compare the weight of garbage (i.e., not recycled)
from participants in the program with weight of garbage from
those not in the new program.
Can we tell if the new recycling program is effective?
19
East In Not In 0 15 35 0
Recycle Program
mean lbs/week
Threats to Internal Validity, continued
Additive effects with selection
When one group of participants in an experiment
Responds differently to an external event (history)
Matures at a different rate
Is measured more sensitively by a test (instrumentation)
These threats (rather than treatment) may account for any group
differences at the end of a study.

9. 20
Additive effects with selection, continued
Suppose School A starts a program (X) to prevent alcohol abuse
on campus (Week 4). The DV is number of alcohol-related
infractions in student residences.
School B is a comparison.
During Week 4 the newspaper at School A reports a student
death due to intoxication (“local history effect”).
Is the program effective?
School A 1 2 3 4 X 5 6 7 8 9 8
8 9 1 3 2 3 2 School B 1 2 3
4 X 5 6 7 8 8 10 9 7 8 7
10 9 9 Column1 1 2 3 4 X 5 6
7 8
Week
# Infractions
Threats to Internal Validity, continued
With no comparison group, must rule out:
history, maturation, testing, instrumentation, regression, subject
attrition, selection
When there is a comparison group, you must rule out these
threats:
selection, additive effects with selection
Adding a comparison group helps rule out many threats to

10. internal validity.
22
Stretching Exercise, page 323
Threats to Internal Validity, continued
Threats even true experiments may not eliminate
Contamination
resentment, rivalry, diffusion of treatments
Experimenter expectancy effects
Novelty effects (including Hawthorne effect)
Threats to external validity
Treatment effects may not generalize
Best way to assess external validity: replication
24
Quasi-Experiments
“Quasi-” (resembling) experiments
Important alternative when true experiments are not possible
Lack the high degree of control found in true experiments
Often no random assignment
Researchers must seek additional evidence to eliminate threats
to internal validity
25

11. The One-Group Pretest-Posttest Design
“Bad experiment” or “pre-experimental design”
Intact group is selected to receive a treatment
e.g., a classroom of children, a group of employees
Pretest is 1st Observation (O1)
Treatment is implemented (X)
Posttest is 2nd Observation (O2)
O1 X O2
26
One-Group Pretest-Posttest Design, cont.
O1 X O2
None of the threats to internal validity are controlled.
Any change between pretest (O1) and posttest (O2) may be due
to treatment (X) or
History
Maturation
Testing
Or instrumentation, regression, subject attrition, selection
27
Quasi-Experimental Designs
Nonequivalent Control Group Design
A group similar to the treatment group serves as a comparison
group
Obtain pretest and posttest measures for individuals in both

12. groups
Random assignment to groups is not used
Pretest scores are used to determine whether the groups are
equivalent
Equivalent only on this dimension
28
Nonequivalent Control Group Design,
continuedTreatment↓O1XO2← treatment group-------------------
-------------------------------O1O2← nonequivalent control
grouppretestposttest
Nonequivalent Control Group Design, continued
Example: Does taking a research methods course improve
reasoning ability?
Compare students in research methods and developmental
psychology courses
DV: 7-item test of methodological and statistical reasoning
ability
Suppose group differences are observed at the posttest
30
Nonequivalent Control Group Design, continued
By adding a comparison group, rule out these threats to internal
validity:
history
maturation
testing
instrumentation

13. regression
Assume that these threats happen the same to both groups,
therefore, can’t be used to explain posttest differences.
31
Methods Pre Post 3 5 Developmental Pre Post 2.5
2.75
Mean Reasoning Score
Nonequivalent Control Group Design, continued
What threats are not ruled out?
Selection
Without random assignment to conditions, the two groups are
probably not equivalent on many dimensions.
These preexisting differences may account for group differences
at the posttest.
32
Nonequivalent Control Group Design, continued
Additive effects with selection
The two groups
May have different experiences (selection X history or “local
history effect”)
May mature at different rates (selection X maturation)
May be measured more or less sensitively by the instrument
(selection X instrumentation)

14. May drop out of the study (courses) at different rates
(differential subject attrition)
May differ in terms of regression to the mean (differential
regression)
33
Quasi-Experiments, continued
Simple Interrupted Time Series Design
Observe a DV for some time before and after a treatment is
introduced.
Archival data are often used.
Look for clear discontinuity in the time-series data for evidence
of treatment effectiveness.
O1 O2 O3 O4 X O5 O6 O7 O8
34
Simple Interrupted Time-Series Design, cont.
Example: Study habits
Intervention: An instructional course to change students’ study
habits
Implemented during summer following the sophomore year
(after semester 4)
DV: semester GPA
Suppose a discontinuity is observed when the treatment (X) is
introduced

15. 35
Simple Interrupted Time-Series Design, cont.
What threats can be ruled out?
Maturation: assume maturational changes are gradual, not
abrupt
Testing (GPA): if testing influences performance, these effects
are likely to show up in initial observations (before X)
Testing effects less likely with archival data
Regression: if scores regress to the mean, they will do so in
initial observations
discontinuity
36
Series 1 1 2 3 4 5 6 7 8 1.9 2
1.75 2 3 3.25 3 3.5
Semester
Mean GPA
Quasi-Experiments, continued
Time Series with Nonequivalent Control Group Design
Add a comparison group to the simple time-series design
O1 O2 O3 O4 X O5 O6 O7 O8
--------------------------------------------------------------
O1 O2 O3 O4 O5 O6 O7 O8

16. 37
Time Series with Nonequivalent Control Group Design,
continued
Example: Study habits
Suppose a nonequivalent control group is added—these students
don’t participate in the study habits course
Who could be in the comparison group?
What threats would you be able to rule out?
38
Treatment 1 2 3 4 5 6 7 8 1.9 2
1.75 2 3 3.25 3 3.5 Control 1 2 3 4
5 6 7 8 2 1.9 2.1 2 2 2.1 2.25 2
Semester
Mean GPA
An Example
Study to determine if well being is increased if nursing home
residents are given the opportunity to make daily personal
decisions (how their room is arranged, visits, movie choices)
Two groups: choice group and no-choice group
Assignment to groups was done by floor in a nursing home

17. These floors were chosen due to similarity in the residents’
physical and psychological health and prior SES
Questionnaires administered 1 week before and three weeks into
the study
Staff members rated residents before and after treatment
(alertness, sociability, and activity)
Contest—guess the number of jelly beans in a jar.
What is the independent variable?
What is/are the dependent variable(s)?
What type of quasi-experimental design?
Which threats to internal validity are controlled?
Which threats are not controlled?