2. Introduction for experimental designs
Pre-experimental Design
True experimental Design
Quasi experimental Design
Lecture Outline
Dr. Eman M.Mortada
3. Experimental Research Design
Research Design
➢ It is the researcher’s overall plan for:
❖ Answering the research question
❖ Testing the research hypotheses
➢ Research design can be thought of as the structure of research --
it is the "glue" that holds all of the elements in a research project
together.
Dr. Eman M.Mortada
5. Major components of the Classic Experiment
Independent and dependent variables
◼ Effects of Treatment or Independent Variable
on some outcome variable (Dependent
Variable)
Experimental and control groups
◼ Control group Very similar to experimental
group, except that they do not receive a
treatment in an experiment
Pretesting and posttesting
◼ Ability to assess change before and after
manipulation Dr. Eman M.Mortada
IV DV
Experimen
tal group
Control
group
Pretesting posttesting
Random
Assignment
6. Independent Variables Dependent Variables
This variable is manipulated
(controlled) by the
experimenter
Known factor
“The Cause”
Treatment or intervention
Recap:
1.Independent vs dependent Variables
This variable is measured” is
used to determine the effect of
the independent variable
Unknown factor
“The outcomes ”
Responses or Endpoints
Dr. Eman M.Mortada
Does I.V. → D.V.????
7. Recap:
2. Experimental vs Control Groups
• Experimental group: receives treatment
• Control group: does not receive treatment
Dr. Eman M.Mortada
8. Recap:
3. Pretesting vs Posttesting
❑ Pre-test is the measurement of the
Dependent Variable prior to
treatment or intervention
❑ Post-test is measurement of the
Dependent Variable after the
treatment/ independent variable has
been applied
❑ Differences noted between the
measurements on the Dependent
Variable are attributed to influence
of the Independent Variable
Pretest
Posttest
Dr. Eman M.Mortada
9. Random Assignment
3 things to remember about Random Assignment
1) takes place before the experiment begins
2) is a process of assigning or distributing individuals to groups
3) forms groups that are equivalent and differ only by chance
Dr. Eman M.Mortada
10. Random selection Random assignment
Random assignment vs Random
selection of subjects
11. Random assignment vs Random
selection of subjects
Random selection of subjects
◼ Every member of a population has an equal chance of
being selected to be a member of the sample
◼ Participants are chosen from a population using random
procedures
◼ Purpose: create a representative sample
◼ This ensures generalizability to the population from
which the participants were selected (i.e., external
validity)
Random assignment
◼ Every individual participating in the experiment has an
equal chance of being assigned to any of the experimental
or control groups.
◼ Purpose: create equivalent groups
Dr. Eman M.Mortada
12. Recap:
Notation for experimental designs
R =Random assignment to the treatment or the comparison group
X = Exposure of a group to an experimental treatment
O=Observation or measurement of the dependent variable (pretest,
posttest, etc.)
❖ O1= pre-test (measurement)
❖ O2= post-test (measurement)
Dr. Eman M.Mortada
Research notation
R O1 X O2
Random
assignment
Observation 1 intervention Observation 2
13. Recap:
Types of Designs
Randomized or
True experiment
Yes
No
Is random assignment used?
Is there a control group or
multiple measures?
Yes No
Quasi-experiment Non-experiment
Dr. Eman M.Mortada
1
2
14. Categories of Experimental Research Designs
True or Classical Experimental Design:
Includes random selection, random
assignment, a control group and experimental
group
Quasi-Experimental Design
Some have control group
do not use random selection and assignment to
treatment conditions.
Pre-Experimental Design:
Lack control group and random assignment
Weaker than classical experiment little or no
control of extraneous variables
Substitude classical experiments when they are
not possible
Dr. Eman M.Mortada
Randam
ization
Control
True or
Classica
l
Experi
mental
√√ √√
Quasi-
Experi
mental
--- √√
Pre-
Experi
mental
--- ---
15. *Experimental Research Designs
1. Pre-
experimental
Design
One-shot Case
Study Design
One-group Pretest
Posttest Design
Static Group
Comparison
2. True
Experimental
Design
Posttest Only
Control Group
Design
Pretest Posttest
Control Group
Design
Solomon Four
Group Design
3. Quasi-
experimental
Design
Non-equivalent
Control Group
Design
Time Series Design
Dr. Eman M.Mortada
17. I. Pre-experimental Design
Pre-experimental designs represent the simplest form of research
designs.
Cannot be classified as true experiments
➢ Conditions are not randomly assigned
May provide a basis for continued testing of an intervention
Often used in exploratory research, “pre-,” indicating they are
preparatory or prerequisite to true experimental designs
Designs that do not control for many extraneous variables and
provide weak evidence of cause and effect
Only use when a stronger design cannot be used to answer the
research question
Dr. Eman M.Mortada
18. Three Examples of Pre-Experimental
Designs
Types Of
Pre-
experiment
al Design
One-shot Case Study Design
One-group Pretest Posttest Design
Static Group Comparison
Dr. Eman M.Mortada
19. T O
1. One-Shot Case Study
Key elements:
A single group of participants receives a
treatment or intervention
Then a dependent variable is observed to
assess the effect of the treatment
✓ No random assignment
✓ No pre-measures
impossible to determine change from
pre- treatment (no baseline measure)
Measurement taken only after
intervention-No baseline measure taken
✓ No comparison
Study lacks any comparison or control of
extraneous influences
A.K.A. – after-only design
Diagrammed as:
X O
treatment observation
e.g.: a training program is implemented
and participants are given a posttest at
the conclusion of the training
X O2
Dr. Eman M.Mortada
20. Disadvantages
Weakest design for cause-
effect
Controls none of the
threats to internal or
external validity
Hard to determine if
change has occurred is
due to intervention
Advantages
Good for studying
effect of new program
One-Shot Case Study (cont’d)
To remedy this design, Multiple one-shot designs begin to build a
case
X O
treatment observation
Dr. Eman M.Mortada
21. 2. One-Group Pretest-Posttest Design
O1 T
A.K.A. – Pre/post designs
The difference between O1 and O2 is used to infer
an effect due to X.
O1 X O
Pretest treatment Posttest
O1 X O2
O2
Key elements:
A pre-test is added for the
experimental group
followed by a treatment or
intervention, followed by a
posttest.
No control group
Offers comparison of the
same individuals before and
after the treatment (e.g.,
training) has been
administered to the group
Diagrammed as:
Dr. Eman M.Mortada
22. Disadvantages
a weak design —no control
group
Cannot rule out threats to
internal or external validity
(therefore, cannot be sure
about conclusions)
Advantages
Better than one shot case
study
Can determine if change
occurred
*Pretest Intervention Posttest
One-Group Pretest-Posttest Design (cont’d)
▪ To remedy this design, a comparison group could be added.
23. Threats to internal validity
1. History
Many change-producing events may have occurred between O1
and O2 . History is more viable the longer the lapse between the
pretest and posttest.
2. Maturation
During the time between O1 and O2 the individuals may have grown
older, wiser. If time between 1st & 2nd measurements is extended,
may suffer maturation
3. Testing
The fact that the participants in the study were exposed to a
pretest may, by itself, influence performance on the posttest.
4. Instrumentation
If O1 and O2 are obtained from judges (or raters), for example,
than the judges may become more skillful between the two sets
of observations.
Dr. Eman M.Mortada
One-Group Pretest-Posttest Design (cont’d)
•History
•Maturation
•Testing
•Instrumentation
24. A.K.A., after-only design with control group
3. Static-Group Comparison Design
Experimental Group X O2
Control Group O2
T O
O
Key elements:
Use of 2 groups.
Experimental group is measured after
being exposed to treatment.
Control group is measured without
been exposed to the treatment.
No pre-measure is taken
= Data gathered at one time point
after the intervention
Diagrammed as: X O
treatment observation
- O
Dr. Eman M.Mortada
25. Static-Group Comparison Design (cont’d)
shows that change occurred
Determines the influence of a
variable on one group and not
on another.
1- Major weakness is lack of
assurance that the groups were
equal on variables of interest
prior to the treatment
2- still not a good design as the
possibility of other threats
Advantages Disadvantages
Dr. Eman M.Mortada
26. Static-Group Comparison Design (cont’d)
Threats to internal validity
1. Selection
It is possible that the treatment group was already prepared to do
better (or worse) than the comparison group on O; hence the
treatment group might have performed differently from the
comparison group even in the absence of X.
2. Mortality
It is possible that differences are due to the fact that the nature of
the treatment is such that participants drop out at higher rates
than do participants in the comparison group.
Dr. Eman M.Mortada
•Selection
•Mortality
27. ❑ One shot case study:
X O2
❑ Before and After Design
O1 X O2
❑ Static Group Comparison
X O2
O2
# Pre experimental design(cont’d)
Dr. Eman M.Mortada
28. Pre experimental design(cont’d)
❑ Advantages ❑ Disadvantages
❑ Very simple
❑ Convenient to conduct in
natural settings
❑ It should be considered a pilot
test at best and followed up
with a better research design.
❑ Weak design to establish
casual relationship between
independent and dependent
variable
❑ Very little control over the
research
❑ Higher threat to internal
validity
Dr. Eman M.Mortada
30. II. True or classical Experimental Designs
“true” experimental design is one that truly isolates the effects of the
independent variable on the dependent variable while controlling
for effects of any extraneous variables.
Dr. Eman M.Mortada
31. II. True or classical Experimental Designs
Dr. Eman M.Mortada
Three Features of a True Experiment:
(1) Randomization – random assignment – no systematic bias
(2) Manipulation –Independent Variable(s)
(treatment/intervention)
(3) Control – control group/experimental group
design
Randomi
zation
Manipul
Contr
33. Diagrammed as
R O1 X O2
R O1 O2
1. Pretest-Posttest Control Group Design
Key elements:
Subjects randomly assigned to
groups
Pretest given to both groups
Experimental group receives
experimental treatment.
Comparison group receives
routine treatment or no
treatment.
Both groups are measured or
observed twice
A.K.A., Before-After
with Control
Most frequently used True experimental design
Dr. Eman M.Mortada
1 2 5
3
4
35. The best design to use for
cause-effect questions
because it rules out most
other possible explanations
for the results obtained.
Controls for all internal
validity threats
Any differences between
groups that might have
existed prior to X are
(assumed to be) controlled
through random assignment.
Can only be generalized to
situations in which pretest is
administered
External threat of reactive
effects of the pretest
Advantages Disadvantages
Pretest-Posttest Control Group Design
(cont’d)
Dr. Eman M.Mortada
36. Use in situations when cannot be pretested
2. Posttest-Only Control Group Design
A.K.A., After-Only with Control Key elements:
Identical to the pretest-posttest control
group design except that the pretest is
not administered to either of the two
groups
Subjects randomly assigned to groups
Experimental group receives the
experimental treatment.
Comparison group receives routine
treatment or no treatment.
Posttest given to both groups
Diagrammed as
R X O2
R X O2
Dr. Eman M.Mortada
37. T O2
P
R
O2
Earliest recorded example of random group
allocation as recent as 1928 (Forsetlund et al. 2007)
Posttest only Control Group Design
Random
assignment
Experimental
group
Control
group
Treatment Post-test
Post-test
Control
Dr. Eman M.Mortada
38. 3. Solomon Four-Group Design
Key elements:
Combines pretest-posttest with control
group design and the posttest-only
with control group design
Subjects are randomly assigned to one
of four different groups
➢ two of the groups receive the
treatment (IV)
➢ two of the groups do not receive the
treatment
only one of the control groups is
administered the pre-test
Posttest given to all four groups
Diagrammed as:
R O1 X O2
R O1 O2
R X O2
R O2
Dr. Eman M.Mortada
39. Solomon Four-Group Design (cont’d)
O1
T
O2
R
O2
P
O3
P O2
T
O2
Exp group 1
Control group 1
Exp group 2
Control group 2
Pretest
Pretest
Treatment
Treatment
Post-test
Post-test
Post-test
Post-test
Random
assignment
Dr. Eman M.Mortada
The combination of random assignment and the
presence of a pretest and a control group serve
to control for all threats to internal validity.
40. Solomon Four-Group Design (cont’d)
Best control of threats.
Provides means for controlling
the interactive testing effect and
other sources of extraneous
variation
Minimizes threats to both
internal and external validity**
Differences between groups
can be associated with the
experimental treatment
Requires a large sample
Statistical analysis is
complicated.
Strengths Weaknesses
Dr. Eman M.Mortada
41. Experimental Design
This is the strongest set of
designs in terms of internal and
external validity
Randomization Controls the
influence of confounding
variables, providing more
conclusive answers
The investigators have control
over variables such as the dose
or degree of intervention.
The blinding process reduces
distortion in assessment.
Costly in time and money
increased likelihood that
patients will be lost to follow-
up
Artificiality (hinders
generalizability)
Ethical concerns
Strengths Weaknesses
Dr. Eman M.Mortada
43. Dr. Eman M.Mortada
The word "quasi"
Latin word means (apparently, as if)
because they are variations of the
classical design
44. Characteristics of True Designs
Manipulation (treatment)
Randomization
Control group
Less control
More real-world
❑ Quasi-experiments “A research design
involving an experimental approach but
where random assignment to treatment and
comparison groups has not been used”
(Campbell & Stanley, 1963).
➢ Involves manipulation of an independent
variable
➢ Lacks either randomization or control-group
or both
❑ Do not properly control for the effects of
extraneous variables on dependent variables.
45. Dr. Eman M.Mortada
❑ Quasi-experimental design is applicable when the subjects
are not able to be randomized
❑ Some variables cannot ethically be randomized
❑ Typically employ naturally occurring groups: Classrooms,
clinics, organizations, geographic areas, etc.
❑ Some Control exerted by researcher to increase validity by
Observing participants at additional times both before and
after the program
Quasi-Experimental Designs(Q-E)(cont’d)
47. Key elements:
It is structured like a pretest-
posttest randomized
experiment, but it lacks the
key feature of the randomized
designs random assignment.
No randomization
➢ As a result, the groups can
not be assumed to be
equivalent
➢ In other words, the groups
may be different prior to the
study
➢ Collects data before and after
intervention
Diagrammed as:
O1 X O2
O1 O2
1. The Nonequivalent Control Group
Design
Most frequently used quasi-
experimental design
Dr. Eman M.Mortada
The term means that assignment to group
was not random
48. The Nonequivalent Control Group Design (cont’d)
Threats to internal
validity controlled
History
Maturation
Instrumentation change
Threats to internal
validity that remain
Selection bias
Advantages Disadvantages
Dr. Eman M.Mortada
49. Diagrammed as:
O1O2 O3 X O4O5O6
2. Time-Series Design
T
O1 O2 O3 O4 O5 O6
Key elements:
Has neither a control group nor
randomization
Involves the collection of data over an
extended period of time and the
introduction of the intervention during
that period
Treatment effect is assessed by comparing
the pattern of pre- and posttest scores
for a single group of research participants
Dr. Eman M.Mortada
50. Time-Series Design (cont’d)
Use of multiple pretest
and posttest
measurements determine
treatment effect &
demonstrates reliability
of effect
Primary weakness is
Internal validity threats:
no control of history
and testing effect
Advantages Disadvantages
Dr. Eman M.Mortada
51. Quasi-experimental Design
❑ Q-E designs are a reasonable
alternative to RCT
❑ Useful where pre-selection
and randomization of groups is
difficult
❑ Greater external validity (more
like real world conditions)
❑ Saves time and resources vs.
experimental designs
❑ Lack randomization or control
group features
❑ Cause-and-effect inferences
cannot be made easily – may
alternative explanations for
results
Strengths Weakness
Dr. Eman M.Mortada
52. The selection of Good Research design
No single design can be applied to every research question
because every situation is unique.
Goal of research design choose the strongest design that is
possible, ethical, and feasible
Sometimes there really is not much choice because the
situation itself is very limited.
In these cases, researchers state the limitations of the design
and present their conclusions within the context of those
limitations
Dr. Eman M.Mortada
RCTs are not always possible or practical.
53. Choice of Appropriate Experimental
Design
Many factors to consider
use of control group
number of comparison groups
pretest(s)
within-participants or between-participants
number of independent and dependent variables
Dr. Eman M.Mortada
55. Randomized pretest-posttest control group design
R Pre T Post
R Pre C Post
Pre T Post
Pre C Post
Non-equivalent control group pretest-
posttest design
57. Randomized posttest control only design
R T Post
R C Post
T Post
One group posttest-
only design
(One shot study)
58. Types of Experimental Design
Design Random
assignment
Experimen
tal group
Control
group
Pretest Post test
Classical √ √ √ √ √
One shot
case study
X √ X X √
One group
pretest/postt
est
X √ X √ √
Static
comparison
group
X √ √ X √
Time series X √ X √ √
Non
