Experimental Quasi Designs and bacon bites

1
Chapter 7:
Experimental
and Quasi-
Experimental
Designs
© 2018 Cengage Learning. All Rights Reserved.

2
Learning Objectives
• Recognize that experiments are well suited for the controlled
testing of causal processes and for some evaluation studies
• Describe how the classical experiment tests the effect of an
experimental stimulus on some dependent variable through the
pretesting and posttesting of experimental and control groups
• Understand that a group of experimental subjects need not be
representative of some larger population but that experimental
and control groups must be similar to each other
• Describe how random assignment is the best way to achieve
comparability in the experimental and control groups
• Describe how the classical experiment with random assignment
of subjects guards against most of the threats to internal
invalidity

3
Learning Objectives, cont.
• Understand that the controlled conditions under which experiments
take place may restrict our ability to generalize results to real-world
constructs or to other settings
• Recognize how the classical experiment may be modified by
changing the number of experimental and control groups, the
number and types of experimental stimuli, and the number of
pretest or posttest measurements
• Know the reasons that quasi-experiments are conducted when it is
not possible or desirable to use an experimental design, and be
able to describe different categories of quasi-experiments
• Understand the differences between case-oriented and variable-
oriented research
• Be able to describe how experiments and quasi-experiments can
be customized by using design building blocks to suit particular
research purposes

4
Introduction
• Experimentation is an approach to research
best suited for explanation and evaluation
• An experiment is “a process of observation,
to be carried out in a situation expressly
brought about for that purpose”
• Experiments involve:
• Taking action
• Observing the consequences of that action
• Especially suited for hypothesis testing

5
The Classical Experiment
• Variables, time order, measures, and
groups are the central features of the
classical experiment
• Involves three major pairs of
components:
• Independent and dependent variables
• Pretesting and posttesting
• Experimental and control groups

6
Independent Variables
• The Independent Variable takes the form of a
dichotomous stimulus that is either present or
absent
• It varies (i.e., is independent) in our
experimental process
• “The Cause”

7
Dependent Variables
• The outcome, the effect we expect to see
• Depends on the Independent Variable
• Might be physical conditions, social behavior,
attitudes, feelings, or beliefs
• “The Effect”

8
Pretesting and Posttesting
• Subjects are initially measured in terms of the
Dependent Variable prior to association with the
Independent Variable (pretested)
• Then, they are exposed to the Independent
Variable
• Then, they are remeasured in terms of the
Dependent Variable (posttested)
• Differences noted between the measurements on
the Dependent Variable are attributed to influence
of the Independent Variable

9
Discussion Question 1
What if you took part in a social science
experiment? What assurances would you
expect from the administrators of the
experiment, if any?

10
Experimental and Control Groups
• Experimental group: Exposed to whatever
treatment, policy, or initiative we are testing
• Control group: Very similar to experimental
group, except that they are NOT exposed
• If we see a difference, we want to make
sure it is due to the Independent Variable,
and not due to a difference between the two
groups

11
Hawthorne Effect
• Pointed to the necessity of control groups
• Independent Variable: improved working
conditions (better lighting)
• Dependent Variable: improvement in
employee satisfaction and productivity
• Workers were responding more to the
attention than to the improved working
conditions

12
Placebo
• We often don’t want people to know if they are
receiving treatment or not
• We expose our control group to a “dummy”
Independent Variable just so we are treating
everyone the same
• Medical research: Participants don’t know what
they are taking
• Ensures that changes in Dependent Variable
actually result from Independent Variable and
are not psychologically based

13
Double-Blind Experiment
• Experimenters may be more likely to
“observe” improvements among those who
received drug
• In a Double-Blind experiment, neither the
subjects nor the experimenters know which
is the experimental group and which is the
control group
• Broward County Florida and Portland, Oregon domestic
violence policing units study: “keeping safe” strategies

14
Would you ever participate in a double-blind
experiment? Why or why not?

15
Selecting Subjects
• First, must decide on target population, the
group to which the results of your
experiment will apply
• Second, must decide how to select particular
members from that group for your
experiment
• Cardinal rule: ensure that Experimental and
Control groups are as similar as possible
• Randomization aims to achieve this

16
Random Assignment
• “Randomization”
• Central feature of the classical experiment
• Produces experimental and control groups that are
statistically equivalent
• Farrington and associates:
• “Randomization insures that the average unit in the
treatment group is approximately equivalent to the
average unit in another group before the treatment is
applied”
• “All Other Things Are Equal”

17
How difficult is it to randomize an experiment?
Is it costly? Can any researcher do it?

18
Experiments and Causal Inference
• Experiments potentially control for many
threats to the validity of causal inference
• Experimental design ensures:
• Cause precedes effect via taking posttest
• Empirical correlation exists via comparing pretest to
posttest
• No spurious third variable influencing correlation via
posttest comparison between experimental and control
groups, and via randomization

19
Threats to Internal Validity
• Conclusions drawn from experimental
results may not reflect what went on in
experiment
• History: External events may occur during
the course of the experiment
• Maturation: People constantly are growing
• Testing: The process of testing and
retesting

20
Threats to Internal Validity, cont.
• Instrumentation: Changes in the
measurement process
• Statistical regression: Extreme scores
regress to the mean
• Selection biases: The way in which subjects
are chosen (use random assignment)
• Experimental mortality: Subjects may drop
out prior to completion of experiment

21
Threats to Internal Validity, slide 3
• Causal time order: Ambiguity about order of
stimulus and Dependent Variable—which
caused which?
• Diffusion/Imitation of treatments:
Experimental group may pass on elements
to Control group when communicating
• Compensatory treatment: Control group is
deprived of something considered to be of
value

22
Threats to Internal Validity, slide 4
• Compensatory Rivalry: Control group
deprived of the stimulus may try to
compensate by working harder
• Demoralization: Feelings of deprivation
among control group result in subjects
giving up

23
Generalizability and Threats to Validity
• Potential threats to internal validity are
only some of the complications faced by
experimenters; they also have the
problem of generalizing from
experimental findings to the real world
• Two dimensions of generalizability:
• Construct Validity
• External Validity

24
Threats to Construct Validity
• Concerned with generalizing from
experiment to actual causal processes in
the real world
• Link construct and measures to theory
• Clearly indicate what constructs are
represented by what measures
• Decide how much treatment is required to
produce change in Dependent Variable

25
Threats to External Validity
• Significant for experiments conducted under
carefully controlled conditions rather than
more natural conditions
• Reduces internal validity threats
• John Eck (2002): "diabolical dilemma."
• Suggestion:
• explanatory studies -> internal validity
• applied studies -> external validity

26
Threats to Statistical Conclusion Validity
• Becomes an issue when findings are based on
small samples
• More cases allows you to reliably detect small
differences; less cases result in detection of
only large differences
• Finding cause-and-effect relationships through
experiments depends on two related factors:
• Number of Subjects
• Magnitude of posttest differences between the experimental
and control groups

27
Variations in the Classical Experimental Design
• Four basic building blocks present in
experimental designs:
• The number of experimental and control groups
• The number and variation of experimental stimuli
• The number of pretest and posttest measurements
• The procedures used to select subjects and assign them to
groups
• Variations on the classical experiment can
be produced by manipulating the building
blocks of experiments

28
Quasi-Experimental Designs
• When randomization isn’t possible for
legal or ethical reasons
• Renders them subject to Internal Validity
threats
• Quasi = “to a certain degree”
• Two categories:
• nonequivalent-groups designs
• time-series designs

29
Nonequivalent-Groups Designs
• When we cannot randomize, we cannot
assume equivalency; hence the name
• We take steps to make groups as
comparable as possible
• Match subjects in Experimental and Control
groups using important variables likely related
to Dependent Variable under study
• Aggregate matching: comparable average
characteristics

30
Cohort Designs
• Cohort: Group of subjects who enter or
leave an institution at the same time
• Ex: A class of police officers who graduate from a
training academy at the same time; all persons who
were sentenced to probation in May
• Necessary to ensure that two cohorts
being examined against one another are
actually comparable

31
Time-Series Designs
• Longitudinal Studies
• Examine a series of observations over time
• Interrupted: Observations compared before
and after some intervention (used in cause-
and-effect studies)
• Instrumentation threat to internal validity is
likely because changes in measurements
may occur over a long period of time
• Often use measures produced by CJ organizations

32
Scientific Realism
• A large number of variables are studied for
a small number of cases or subjects
• Case-oriented research: Many cases are
examined to understand a small number of
variables (e.g., Boston Gun Project)
• Variable-oriented research: A large number
of variables are studied for a small number
of cases or subjects
• Case Study Design: Centered on an in-depth examination
of one or a few cases on many dimensions

Experimental Quasi Designs and bacon bites

Recommended

Recommended

More Related Content

Similar to Experimental Quasi Designs and bacon bites

Similar to Experimental Quasi Designs and bacon bites (20)

More from pq5jnhdws9

More from pq5jnhdws9 (7)

Recently uploaded

Recently uploaded (20)

Experimental Quasi Designs and bacon bites