Experimental designs

RESEARCH DESIGN
Intervention/ Experimental Design
Prabesh Ghimire

Interventional Research
• Studies in which investigator assigns the exposure
• Often used to determine the effectiveness of an intervention or
effectiveness of a health service delivery.
• Can also be used to establish the safety, cost-effectiveness and
acceptability of an intervention.
Prabesh Ghimire, MPH 2

Types
• Pre-Experimental
• Quasi-Experimental
• True Experimental

Before we begin…
Experimental designs involve:
• Groups: Intervention and Control
• One-group design: only intervention group, no controls
• Two-group design: both intervention group and control group
• Four-groups design
• Tests/Assessments: Pre-test and Post-test
• Post-test only design
• Pre-test post-test design
• Randomization: Randomized and Non-randomized
• Equivalent design: groups are randomly allocated
• Non-equivalent design: groups are not allocated randomly

Before we begin…
Baseline/ Pre-test Intervention Endline/Post-test
Intervention Group
(I1)
X Intervention Group
(I2)
Control Group (C1) Control Group (C2)
Randomization

Before we begin…
Understanding the notations used in this presentation:
• I1 – Pre-test in the intervention group
• I2 – Post-test in the intervention group
• C1 – Pre-test in the control group
• C2 – Post-test in the control group
• X – Intervention

Pre-Experimental Study Design
• In a pre-experiment, either a single group or multiple groups are
observed subsequent to some agent or treatment presumed to
cause change.
• Pre-experimental designs either fail to include a pretest, a
control or comparison group, or both
• In addition, no randomization procedures are used to control for
extraneous variables.
• When true experiments and quasi-experiments are not possible,
researchers may turn to a pre-experimental design

• Useful in cases where a researcher cannot control or predict
whether, when, or how the stimulus is administered, as in the case of
natural disasters (chemical poisoning, earthquake)
• They are considered “pre-,” indicating they are preparatory or
prerequisite to true experimental designs.
• Often, researchers want to see if their interventions will have an effect on a
small group of people before they seek funding and dedicate time to conduct
a true experiment.
• Usually conducted as a first step towards establishing the evidence
for or against an intervention.

Types of Pre-Experimental Design
1. One-Shot Case Study (One group post-test design)
2. Static Group Comparison (Post-test only non-equivalent
group design)
3. One-group pre-test/ post-test design

1. One-Shot Case Study (Ex post facto design)
• One group only post-test design
• A single group of people is measured on some dependent variable after
intervention has taken place.
• A group is administered a covid-19 vaccine and then followed up for
certain period to check if they present a COVID infection.
• Useful in cases where the administration of the stimulus is quite costly or
otherwise not possible (example disaster)
• In this instance, no pretest is administered, nor is a control group present.
X  O2
(Intervention) (Post-Test)

1. One-Shot Case Study
• In the study of the impact of earthquake (natural intervention),
• Researcher using this design would test the impact of earthquake only among
a community that was hit by earthquake
• Would not seek a comparison group from a community that did not
experience the earthquake.
I1  X  I2 (earthquake hit)
(No pre-test) (Earthquake) (Post-Test)
• Researchers using a one-shot case study design must be extremely
cautious when making claims about the effect of the stimulus,

2. Static Group Comparison (Post-test only non-equivalent group design)
• In the study of the impact of earthquake, researcher using this
design
• Identifies an experimental group from a community that experienced the
earthquake; and
• Control group from a similar community that had not been hit by the
earthquake
I1  X  I2 (hit by earthquake)
C1   C2 (not hit by earthquake)
(No pre-test) (Earthquake) (Post-Test)

3. Static Group Comparison
• Has the advantage of including a comparison group that did not
experience the stimulus (earthquake)
• It is difficult to be sure that the groups are truly comparable
because the experimental and control groups were determined
by factors other than random assignment.
• The design would only allow for posttests

• Research designs in which
• a single group of research participants or subjects is pretested,
• given some treatment/ intervention,
• then post-tested.
• In this instance, pre- and posttests are both taken, but there is no
control group to compare the experimental group to.
I1 (Pre-test)  X  I2 (vaccinated)- Post-test
C1 (No pre-test)   C2 (no control/ no post-test)
(COVID Vaccine)

• If the pretest and posttest scores differ significantly, then the
difference may be attributed to the intervention.
• But because the research design is not strictly experimental and
there is no control group, this inference is uncertain,
• Useful when
• a researcher cannot identify a sample that is large enough to split into
control and experimental groups
• Researcher do not have access to a control group

Strengths
• Pre-experiments can be a cost-effective way to discern whether
a potential explanation is worthy of further investigation.
• Useful in cases where a researcher cannot control or predict
whether, when, or how the stimulus is administered, as in the
case of natural disasters
Limitations
• Subject to numerous threats to their validity.

Quasi-Experimental Design

• Experimental studies that lack random assignment to
experimental and control groups.
• Useful in the cases where experimental and control groups
already exist. For example:
• a researcher might conduct research at two different agency sites, one
of which receives the intervention and the other does not.
• The researcher does not need to assigned participants to treatment or
comparison groups because the groupings already existed prior to the
study.

• Quasi-experiments are most likely to be conducted in field settings in
which random assignment is difficult or impossible.
• While this method is more convenient for real-world research, researchers
cannot be sure that the groups are comparable.
• Often used to evaluate the effectiveness of a treatment (psychotherapy) or
an educational intervention.
• Useful tool in situations where true experiments cannot be used for ethical
or practical reasons.
• Example: it would be unethical to randomly provide some people with covid vaccine
but purposely prevent others from receiving it solely for the purpose of research.

Types of Quasi-Experimental Design
• Pre-test post-test non-equivalent control group design
• Interrupted time series design
• Regression discontinuity analysis

Types of Quasi-Experimental Design
• Pre-Test Post-test Non-equivalent Control Group Design
Group Baseline/ Pre-
test
Intervention Endline/Post-
test
Intervention
Group
Pre-Test X Post-Test
Control Group Pre-Test Post-Test
No
Randomization

TIMES SERIES DESIGN

Time Series Design
• Type of quasi experimental design
• Time series: set of measurements taken at intervals over a
period of time.
• In this design, a series of periodic measurements is taken from
one group of study units, followed by treatment, then another
series of measurements.
• Time Series Design collects data on the same variable at
regular intervals (weeks, months, years, etc.) in the form of
aggregate measures of a population.
• Example: Unemployment rates, accident rates, fatality rate

Time Series Design
March (T-3) April (T-2) May (T-1) Intervention July (T+1) August (T+2) September
(T+3)
Fatality rate Fatality rate Fatality rate Crackdown Fatality rate Fatality rate Fatality rate
Province March (T-
3)
April (T-2) May (T-1) Interventi
on
July (T+1) August
(T+2)
Septembe
r (T+3)
Lumbini Fatality
rate
Fatality
rate
Fatality
rate
Crackdow
n
Fatality
rate
Fatality
rate
Fatality
rate
Bagmati Fatality
rate
Fatality
rate
Fatality
rate
Fatality
rate
Fatality
rate
Fatality
rate
One group - Interrupted Time Series Design
Interrupted Time Series Design with comparison group

Time Series Design
Time Series Design are useful for:
• Establishing a baseline measure
• Describing changes over time
• Keeping track of trends
• Forecasting future trends

• Time series data are nearly always presented in the form of a
chart or graph.
• The horizontal (or x) axis is divided into time intervals
• The vertical (y) axis shows the values of the dependent variable
as they fluctuate over time.
Time Series Design

Time Series
Design Example
https://open.lib.umn.edu/psychologyresearchmethods/
chapter/7-3-quasi-experimental-research/
• The dependent variable is the number of
student absences per week in a research
methods course.
• The treatment is that the instructor begins
publicly taking attendance each day so that
students know that the instructor is aware of
who is present and who is absent.

True Experimental Design
• Design used to refer to any randomized experiments
• Carried out with or without a pretest/ baseline on at least 2
randomly assigned subjects.
• For the true experimental design, following criteria should be
met:
• Control group must be present
• A variable that can be manipulated by the researcher (e.g. dose)
• Randomization

Techniques of random selection and
participant assignment
• Referring to a random number table
• Computer generated random number
• Coin tossing
• Shuffling cards or envelopes
• Throwing dice
• Sequentially numbered drug containers of identical appearance
• Sequentially numbered, opaque, sealed envelopes

Techniques of random selection and
participant assignment
High risk of bias if
• Sequence generated by odd or even date of birth, day of visit,
etc.
• Allocation by judgment of clinician, participant
• Allocation based on results of laboratory test
• Case record number
• Unsealed or non-opaque envelopes

Types of True Experimental Design
1. Post-test only control group design
2. Pre-test Post-test control group design
3. Solomon four group design

1. Post-Test Only Control Group Design
• Study participants are randomly selected and assigned to the 2
groups (control and experimental), and only the experimental group
is treated/intervened.
• Neither group is assessed/ pre-tested before intervention
• After close observation, both groups are post-tested, and a
conclusion is drawn from the difference between these group.
I1  X  I2
C1   C2
(No pre-test) (Dexamethasone) (Post-test)
Randomization

2. Pre-Test Post-Test Control Group Design
• Subjects are randomly assigned to the 2 groups
• Both are pre-tested
• Only the intervention group is treated.
• After close observation, both groups are post-tested to measure
the degree of change in each group.
I1  X  I2
C1   C2
(Pre-test) (Dexamethasone) (Post-test)
Randomization

3. Solomon Four Group Design
• In this design, the sample is divided into two treatment groups
and two control groups.
• One treatment group and one control group receive the pretest,
and the other two groups do not.
• This design represents a combination of posttest-only and
pretest-posttest control group design, and is intended to test for
the potential biasing effect of pretest measurement on posttest
measures that tends to occur in pretest-posttest designs but not
in posttest only designs.

3. Solomon Four Group Design
I1 (No pre-test)  X  I2 (Post-test)
C1 (No pre-test)   C2 (Post-test)
I1 (Pre-test)  X  I2 (Post-test)
C1 Pre-test)   C2 (Post-test)

Assignment
Review each of the following study designs:
• One-shot case study (one-group post-test only design)
• Static group comparison (post-test only, non-equivalent group
design)
• One group pre-test post-test design
• Non-equivalent group, pre-test post-test design
• Post-test only, control group design (equivalent group)
• Pre-test Post-test control group design (non-equivalent group)
• Pre-test post-test control group design

CLINICAL TRIAL/ RCT

Clinical Trial
• Research design that studies new tests/therapies and
treatments and evaluates their effects on human health
outcomes.
• The purpose of the clinical trial is assessment of efficacy, safety,
or risk benefit ratio.
• Clinical trials are carefully designed, reviewed and completed,
and need to be approved before they can start.
• Interventions may be prophylactic, therapeutic or diagnostic:
• Novel vaccines, drugs, dietary choices, dietary supplements, medical
devices

Research Question in Clinical Trial
• Types of questions
• Assessing efficacy of an intervention
• Assessing the effectiveness of an intervention

Phases in Clinical Trial
• There are 4 phases of biomedical clinical trials:
• Phase I studies usually test new drugs for the first time in a small
group of people to evaluate a safe dosage range and identify side effects.
• Phase II studies test treatments that have been found to be safe in phase I
but now need a larger group of human subjects to monitor for any
adverse effects.
• Phase III studies are conducted on larger populations and in different
regions and countries, and are often the step right before a new
treatment is approved.
• Phase IV studies take place after country approval and there is a need for
further testing in a wide population over a longer timeframe.
Click here for further reading

Design of Clinical Trial

Types of Clinical Trial
• Uncontrolled Trials
• Controlled Trials
• Non-randomized controlled trial
• Randomized controlled trial/ Randomized clinical trial
Control arm options in controlled trials
• Placebo concurrent control
• “No treatment” concurrent control
• Active treatment concurrent control
• Dose-comparison concurrent control

Uncontrolled Trial
• This design incorporates no control arm.
• This design is usually utilized to determine pharmacokinetic
properties of a new drug (Phase 1 trials).
• Uncontrolled trials are known to produce greater mean effect
estimates than a controlled trial, thereby inflating the
expectations from the intervention.
• There is a threat of inherent bias and results are considered
less valid than RCT.

Uncontrolled Trial
Nair B. (2019). Clinical Trial Designs. Indian dermatology online journal, 10(2), 193–201.
https://doi.org/10.4103/idoj.IDOJ_475_18

Randomized Controlled Trial
• A part of clinical trial
• In RCTs the patients are randomly assigned to the different
study groups.
• This is intended to ensure that all potential confounding factors
are divided equally among the groups that will later be
compared

Strengths and Limitations
Strengths
• Often provides the strongest evidence in support of cause-effect
relationships
• Basis for clinical and public health policy
• Excellent internal validity; removes validity threats
• Provides precise measures of efficacy and acute toxicity of new
therapies under ideal conditions.
• Because of randomization, measurement of effect size is less
prone to bias.

Strengths and Limitations
Limitations
• Limited external validity
• Patients with co-morbidity are under-represented in RCTs.
• Have limited ability to detect rare and chronic toxicities, especially
those that occur in patients after the completion of the trial.
• Chances of experimental mortality (attrition) cannot be ruled out.
• Might be costly
• Some research problems cannot be studied using an experiment
because of ethical or technical reasons.

Types of RCT Designs
• Parallel group trial design
• Cross-over design
• Cluster Randomized Controlled Trial
• Factorial Randomized Controlled Trial

Parallel Group Trial Design
• Most commonly used study design/ classical RCT
• Study participants are randomized to one of two groups
• The two group, usually comprise an interventional group and a
comparator group, which are followed forward in time.
• The comparator group may receive placebo or standard of care
• After randomization each participant will stay in their assigned
treatment arm throughout the study.

Parallel Group Trial Design
Group
A
Group
B
Tinmouth A, Hebert P. Interventional trials: an overview of design alternatives. Transfusion. 2007
Apr;47(4):565-7. doi: 10.1111/j.1537-2995.2007.01202.x. PMID: 17381612.

Parallel Groups
• Multiple concurrent experimental arms
• Different treatments
• Different doses
• Control arms
• Placebo
• Active control (known effective treatment)

Parallel Group Design
Strengths
• Can be applied to many diseases and allows running
experiments simultaneously in a number of groups, and groups
can be in separate locations.
• Simplest design to plan, implement, analyze, and interpret
Limitations
• People dislike the possibility of receiving placebo, so it could be
a deterrent for them to sign up to participate.

Cross-Over Design
• This approach randomly assigns participants to one group, who
then “crossover" to another treatment arm during the course of
the trial.
• Uses individual as their own controls
• This means that even if they are initially put into a placebo
group, they will also eventually receive the study drug or
standard of care during the trial.

Cross-Over Design
Group
A
Group
B
Group
B
Group
A
Tinmouth A, Hebert P. Interventional trials: an overview of design alternatives. Transfusion. 2007
Apr;47(4):565-7. doi: 10.1111/j.1537-2995.2007.01202.x. PMID: 17381612.

Cross-Over Design
• Patients are assigned to receive two treatments in a random
order.
• Each treatment is given a defined period of time with a washout
period between the two treatments
• The washout period between the two intervention phases is
included to reduce carryover effects from the previous
treatments
• helps researchers determine whether the outcome of the study is due
to the effects of the study drug.

Cross-Over Design
• In this design, some participants start with drug A and then
switch to drug B (AB sequence) in one trial arm,
• While participants in other trial arm start with drug B and then
switch to drug A (BA sequence).

Cross-Over Design
Strengths
• Require fewer patients than a parallel study since each patient acts as his or her
own control.
• Minimizes between subject variability
• Ethical- opportunity to receive both treatments
• Best suited to patients with chronic conditions with stable symptoms.
Limitations
• Take longer to complete since patients will receive multiple treatments during the
trial.
• Many patients may withdraw due to longer study period
• Carryover effects from treatments may impact results
• Period effects are likely (progression of disease, dropouts)

Other Designs
• Factorial Design
• Evaluates different interventions alone and in combination.
• In 2*2 factorial design four groups are compared
• Therapy A
• Therapy B
• Therapy A+B in combination
• No therapy

Other Designs
• Cluster Design (Cluster RCT)
• Groups or clusters are randomly assigned, not individuals E.g. classes,
hospital wards, quarantine centers, platoons….
• Useful in educational research.
Ward
A
Ward
B

For further reading
• Tinmouth A, Hebert P. Interventional trials: an overview of design
alternatives. Transfusion. 2007 Apr;47(4):565-7. doi: 10.1111/j.1537-
2995.2007.01202.x. PMID: 17381612.
• Kabisch, M., Ruckes, C., Seibert-Grafe, M., & Blettner, M. (2011).
Randomized controlled trials: part 17 of a series on evaluation of scientific
publications. Deutsches Arzteblatt international, 108(39), 663–668.
https://doi.org/10.3238/arztebl.2011.0663
• Nair B. (2019). Clinical Trial Designs. Indian dermatology online
journal, 10(2), 193–201. https://doi.org/10.4103/idoj.IDOJ_475_18

Community Trial
• Community trials, also called community intervention studies,
are (mostly preventive) experimental studies with whole
communities (such as cities or states) as experimental units
• Interventions are assigned to all members in each of a number
of communities.
• Carried out “on the ground”.

COMMUNITY TRIAL

Community Trial
• Use to measure the effect of community level interventions
• Vaccine effectiveness studies
• Nutritional interventions (micro-nutrient supplementation)
• Educational campaign
• Measures to control vector (Indoor residual spray)
• Prophylaxis
• Subsidies (cash transfers, vouchers)
• Screening

Rationale for community trial
• Environmental change may be easier that voluntary behaviour
(cigarette tax vs. stop smoking)
• Some interventions are not selective (e.g. water fluoridation,
IRS)
• Individual randomization may not be feasible because all
members of group are treated same.
• Individual randomization, although feasible, may result in
substantial contamination.

Community Trial Designs
• Single community
• Before-after: O1 X O2
• Single (interrupted) time series: O1 O2 O3 X O4 O5 O6
• One intervention and one control community
O x O
O O
• One intervention and multiple control communities
• Multiple intervention and control communities

BLINDING

Concept of Blinding
• Concealment of group allocation from one or more individuals
involved in a research study
• Most commonly a RCT
• Also called masking
• Blinding is used in combination with randomization to limit the
occurrence of conscious and unconscious bias
• in the conduct of clinical trials (performance bias) and
• interpretation of outcomes (ascertainment bias).

Concept of Blinding
• This is important because bias can affect recruitment and
allocation, care, attitudes, assessments, etc.
• Minimizes the likelihood of differential treatment or assessments
of outcomes
• Used to ensure the objectivity of trial results

Whom to Blind?
1. Study Participants
2. Data Collectors and Outcome Assessors
3. Clinicians administering the treatment
4. Data Analyst

Whom to Blind?
1. Study Participants
• If participants are not blinded, knowledge of group assignment
may affect their behaviour in the trial and their responses to
subjective outcome measures.
• Blinded patients may report symptoms differently from
unblinded patients
• For example, a participant who is aware that he is not receiving
active treatment may be
• less likely to comply with the trial protocol,
• more likely to seek additional treatment outside of the trial and
• more likely to leave the trial without providing outcome data.

Whom to Blind?
2. Data collectors/ Outcome assessors
• Crucial to ensure unbiased ascertainment of outcomes.
• Helps to reduce detection bias.
• Outcome assessors (study nurses or investigators) who are
aware of the actual treatment may unconsciously or
intentionally alter their assessment.
• Particularly, in case of soft endpoints, e.g. pain blinding of
outcome assessors is important.
• For hard comparators like mortality detection bias is irrelevant

Whom to Blind?
3. Clinicians administering the treatment
• Blinded clinicians are much less likely to transfer their attitudes
to participants or to provide differential treatment to the active
and placebo groups than are unblinded clinicians

Whom to Blind?
4. Data analyst
• Bias may also be introduced during the statistical analysis of the
trial through the selective use and reporting of statistical tests.
• This may be a subconscious process spurred by investigators
eager to see a positive result.
• The best method to avoid this potential bias is blinding of the
data analyst until the entire analysis has been completed.

Biases due to lack of blinding
• Performance bias
• Detection/ bias
• Participant’s expectation bias
• Data analysts
• Observer bias

Types of Blinding
• Unblinded
• All parties are aware of the treatment the participant receives
• Single blind
• Only the participant is unaware of the treatment they receive
• Double blind
• The participant and the clinicians / data collectors are unaware of the
treatment the participant receives
• Triple blind
• Participant, clinicians and data collectors / outcome assessors / data analysts
are all unaware of the treatment the participant receives

How to blind?
• Drug trial: matching the placebo in color, taste and dosing
schedule.
• Not informing patients of what group they are in
• Using independent outcome assessors
• Not disclosing allocation to data analysts:
• Example using variable names as GrpA, GrpB instead of EXP & CTRL
groups

For further reading
• Karanicolas, P. J., Farrokhyar, F., & Bhandari, M. (2010). Practical tips for
surgical research: blinding: who, what, when, why, how?. Canadian journal
of surgery. Journal canadien de chirurgie, 53(5), 345–348.
• Boutron, I., Estellat, C., Guittet, L., Dechartres, A., Sackett, D. L.,
Hróbjartsson, A., & Ravaud, P. (2006). Methods of blinding in reports of
randomized controlled trials assessing pharmacologic treatments: a
systematic review. PLoS medicine, 3(10), e425.
https://doi.org/10.1371/journal.pmed.0030425

THANK YOU

Experimental designs

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Experimental designs

Similar to Experimental designs (20)

More from Prabesh Ghimire

More from Prabesh Ghimire (20)

Recently uploaded

Recently uploaded (20)

Experimental designs