This document discusses experimental design and observational studies. It defines an experiment as a controlled study that establishes cause and effect by varying factors and comparing treatment groups to a control group. An observational study merely observes or collects existing data without influencing variables and cannot prove causation. The document provides guidelines for planning experiments and describes randomized, matched pairs, and rigorously controlled experimental designs. It also discusses placebos, replication, blinding, and examples of an experimental drug study and observational study types.

1. Section 1.3: Experimental Design
& Observational Studies
• Experiment = A controlled study conducted to determine the effect of
varying one or more factors (explanatory variables) has on a response
variable.
– Intended to establish cause-and-effect.
– Treatment = Any combination of the values of the factors.
– A Control group serves as a baseline treatment that can be used to compare to
other treatments.
• Observational Study – when the investigator collects data merely by
watching or asking questions or uses existing data, such as medical or sports
records.
– The investigator doesn’t influence the data.
– The investigator doesn’t change anything.
– An observational study generally can’t establish cause-and-effect .
1

2. Guidelines for Planning an Experiment
1. Specify the variable (or variables).
– The variable must be something you cam measure.
2. Specify the population.
3. Identify the individuals or items of interest.
– These are sometimes called the “experimental units”
4. Specify the method for measuring or observing.
5. Determine the sampling method.
6. Specify the hypothesis that you want to test and the significance level you
will use. (See chapter 7)
7. Collect the data.
8. Use inference (a statistical hypothesis test) and make a conclusion.
– E.g. Did the treatment make a significant difference.
9. Note any concerns you might have about your data collection methods
and list any recommendations for future.
2

3. Experimental Options
Randomized 2-Treatment Design
The experimental group gets the treatment of interest.
The control group gets either no treatment (or a “placebo” ) or is given the
standard treatment (the usual teaching method, medicine, or fertilizer).
Randomization reduces differences between the 2 groups.
Randomized Block Design
Example: Block 1 = full-time students. Block 2 = part-time students.
Randomly choose some from each block . Then randomly assign ½ of those
chosen in each block to get the treatment ½ to be in the control group.
Matched Pairs Design
The two groups that are matched with each other in some way. Examples:
Twins
Same subject before and after a treatment is applied (weight loss)
Left & right halves of a field for an agricultural experiment
Rigorously Controlled Design
Group similar subjects by characteristics that are important to the
experiment (e.g. age, health, gender). Assign equal numbers of each
group to each treatment group, so that the different treatments are the
only significant difference. 3

4. Other aspects of experimentation
• Replication
– Nobody is going to make major changes based on a
single study. That what graduate students are for.
• “Blind” Experiments
– Single-Blind: subjects does not know if treatment or
placebo
– Double-Blinded: an experiment in which both
the subject and the provider do not know which
treatment is being applied.
• This is the Gold Standard for medical studies
4
Placebo:
A pill or treatment that
has no effect, but looks,
tastes, and smells like the
experimental medication.

5. Experiment Example:
Lipitor: a Cholesterol-Lowering Drug
Study of 2838 people, ages 40 to 75, with Type 2 Diabetes
Double blind experiment: people randomly assigned to 2 groups
& tracked for 4 years.
1428 received Lipitor: 83 cardio events. 61 deaths.
1410 received a placebo: 127 cardio events. 82 deaths.
Cardio event = heart attack, stroke, etc.

6. Observational Study Types
• Cross-sectional study – data observed, measured, or collected at
one point in time.
• Retrospective (or case-control) study – data collected from the past
using records, interviews, and other similar artifacts.
• Prospective (or longitudinal or cohort) study – data collected in the
future from groups sharing common factors.
6
An Observational Study does not try to change or influence the
individuals or items.
It tries to survey or measure things they way they are.
It may research existing data.

7. Types of Survey Questions
Open questions allow free responses
• Allows the respondent to state exactly how they feel.
• Harder to process the data, since there may be many possible answers
• Can be useful to do a small open-question survey, in order to get possible answers
for a closed-question survey
Example: What do you consider to be the most important issue in the next election?
Closed Questions have only a pre-determined list of possible answers:
• Simpler to analyze, but may not have the best selection of answers.
• In a closed-question survey, use multiple versions of poll with the order of the
possible answers randomized.
Example: Which of the following do you consider to be the most important issue in
the next election?
1. Safe train crossings
2. Crime
3. Housing growth
4. New water sources

8. Review Questions
1. In a study of upper respiratory tract infections, 337
Children were given Echinacea, & 370 were given
placebos.
Is this an observational study or an experiment?
2. A student collected measurements of arm lengths from
her family members.
Is this random, systematic, convenience, stratified, or
cluster sampling?
3. ABC conducted exit polls. Specific polling stations were
randomly selected & all voters were surveyed.
Is this random, systematic, convenience, stratified, or
cluster sampling?
4. At a sobriety check-point, every 5th driver was stopped.
Is this random, systematic, convenience, stratified, or
cluster sampling?
Experiment
Convenience
Cluster
Systematic
End of Section