Experimental and quasi-experimental designs involve manipulating an independent variable and observing its effects on a dependent variable. Key aspects include establishing experimental and control groups, pretesting and posttesting to measure the dependent variable both before and after exposure to the independent variable, and using random assignment or matching to make the groups as equivalent as possible. Threats to internal and external validity must be considered, and variations can be created by manipulating the number of groups, stimuli, measurements, and subject selection/assignment procedures.

1. 1
Experimental and
Quasi-Experimental
Designs

2. •Experimentation is an approach to research
best suited for explanation and evaluation
•An experiment is “a process of observation, to
be carried out in a situation expressly brought
about for that purpose”
•Experiments involve:
•Taking action
•Observing the consequences of that action
•Especially suited for hypothesis testing
2

3. •Variables, time order, measures, and groups
are the central features of the classical
experiment
•Involves three major pairs of components:
•Independent and dependent variables
•Pretesting and posttesting
•Experimental and control groups
3

4. • The Independent Variable takes the form of a
dichotomous stimulus that is either present or
absent
• It varies (i.e., is independent) in our experimental
process
• “The Cause”
4

5. • The outcome, the effect we expect to see
• Depends on the Independent Variable
• Might be physical conditions, social behavior,
attitudes, feelings, or beliefs
• “The Effect”
5

6. • Subjects are initially measured in terms of the
Dependent Variable prior to association with
the Independent Variable (pretested)
• Then, they are exposed to the Independent
Variable
• Then, they are re-measured in terms of the
Dependent Variable (posttested)
• Differences noted between the measurements
on the Dependent Variable are attributed to
influence of the Independent Variable
6

7. • Experimental group – Exposed to whatever
treatment, policy, initiative we are testing
• Control group – Very similar to experimental
group, except that they are NOT exposed
• If we see a difference, we want to make sure it
is due to the Independent Variable, and not to
a difference between the two groups
7

8. • Pointed to the necessity of control groups
• Independent Variable: improved working
conditions (better lighting)
• Dependent Variable: improvement in employee
satisfaction and productivity
• Workers were responding more to the attention
than to the improved working conditions
8

9. • We often don’t want people to know if they are
receiving treatment or not
• We expose our control group to a “dummy”
Independent Variable just so we are treating
everyone the same
• Medical research: Participants don’t know what
they are taking
• Ensures that changes in Dependent Variable
actually result from Independent Variable and
are not psychologically based
9

10. • Experimenters may be more likely to “observe”
improvements among those who received drug
• In a Double-Blind experiment, neither the
subjects nor the experimenters know which is
the experimental group and which is the
control group
• Broward County Florida and Portland, Oregon
domestic violence policing units study: “keeping
safe” strategies
10

11. • First, must decide on target population – the
group to which the results of your experiment
will apply
• Second, must decide how to select particular
members from that group for your experiment
• Cardinal rule – ensure that Experimental and
Control groups are as similar as possible
• Randomization purposes towards this
11

12. • “Randomization”
• Central feature of the classical experiment
• Produces experimental and control groups that are
statistically equivalent
• Farrington and associates:
• “Randomization insures that the average unit in
the treatment group is approximately equivalent
to the average unit in another group before the
treatment is applied”
• “All Other Things are Equal”
12

13. • Experiments potentially control for many
threats to the validity of causal inference
• Experimental design ensures:
• Cause precedes effect via taking posttest
• Empirical correlation exists via comparing pretest
to posttest
• No spurious 3rd
variable influencing correlation via
posttest comparison between experimental and
control groups, and via randomization
13

14. • Conclusions drawn from experimental results
may not reflect what went on in experiment
• History: External events may occur during
the course of the experiment
1. Maturation: People constantly are growing
2. Testing: The process of testing and retesting
14

15. 4. Instrumentation: Changes in the
measurement process
5. Statistical regression: Extreme scores regress
to the mean
6. Selection biases: The way in which subjects
are chosen (use random assignment)
7. Experimental mortality: Subjects may drop
out prior to completion of experiment
15

16. 8. Causal time order: Ambiguity about order of
stimulus and Dependent Variable – which
caused which?
9. Diffusion/Imitation of treatments:
Experimental group may pass on elements to
Control group when communicating
10. Compensatory treatment: Cgroup is deprived
of something considered to be of value
16

17. 11. Compensatory Rivalry: Control group
deprived of the stimulus may try to
compensate by working harder
12. Demoralization: Feelings of deprivation
among control group result in subjects giving
up
17

18. • Potential threats to internal validity are only
some of the complications faced by
experimenters; they also have the problem
of generalizing from experimental findings to
the real world
• Two dimensions of generalizability:
• Construct Validity
• External Validity
18

19. • Concerned with generalizing from experiment
to actual causal processes in the real world
• Link construct and measures to theory
• Clearly indicate what constructs are
represented by what measures
• Decide how much treatment is required to
produce change in Dependent Variable
19

20. • Significant for experiments conducted under
carefully controlled conditions rather than more
natural conditions
• Reduces internal validity threats
• John Eck (2002): "diabolical dilemma."
• Suggestion:
• explanatory studies  internal validity
• applied studies  external validity
20

21. • Becomes an issue when findings are based on
small samples
• More cases allows you to reliably detect small
differences; less cases result in detection of
only large differences
• Finding cause-and-effect relationships through
experiments depends on two related factors:
• Number of Subjects
• Magnitude of posttest differences between the
experimental and control groups
21

22. • Four basic building blocks present in
experimental designs:
1.The number of experimental & control groups
2.The number & variation of experimental stimuli
3.The number of pretest & posttest measurements
4.The procedures used to select subjects and assign
them to groups
• Variations on the classical experiment can be
produced by manipulating the building blocks
of experiments
22

23. • When randomization isn’t possible for legal or
ethical reasons
• Renders them subject to Internal Validity
threats
• Quasi = “to a certain degree”
• Two categories:
• nonequivalent-groups designs
• time series designs
23

24. • When we cannot randomize, we cannot
assume equivalency; hence the name
• We take steps to make groups as comparable
as possible
• Match subjects in Experimental and Control
groups using important variables likely related
to Dependent Variable under study
• Aggregate matching – comparable average
characteristics
24

25. • Cohort – Group of subjects who enter or leave
an institution at the same time
• Ex: A class of police officers who graduate from a
training academy at the same time, All
persons who were sentenced to probation in May
• Necessary to ensure that two cohorts being
examined against one another are actually
comparable
25

26. • Longitudinal Studies
• Examine a series of observations over time
• Interrupted – Observations compared before
and after some intervention
(used in cause-and-effect studies)
• Instrumentation threat to internal validity is
likely because changes in measurements may
occur over a long period of time
• Often use measures produced by CJ organizations
26

27. • A large number of variables are studied for a
small number of cases or subjects
• Case-oriented research: Many cases are
examined to understand a small number of
variables (e.g., Boston Gun Project)
• Variable-oriented research: A large number of
variables are studied for a small number of
cases or subjects
• Case Study Design: Centered on an in-depth
examination of one or a few cases on many
dimensions
27