Bias,confounding, causation and experimental designs

Bias, confounding and causation
Professor Tarek Tawfik Amin
Epidemiology and Public Health, Faculty of Medicine, Cairo University
Geneva Foundation for Medical Education and Training
Asian Pacific Organization for Cancer Prevention
amin55@myway.com dramin55@gmail.com
Basic Research Competency Program for Research Coordinators
August 2015, MEDC, Faculty Of Medicine, Cairo University, Cairo, Egypt.

Types of variables
Causal model Study design Unit of measurement
Independent
Intervening
Extraneous
Dependent
Active Attribute
Quantitative Qualitative
Continuous
Categorical
Constants
Dichotomous
Polytomies
Can be manipulated
Changed or controlled
Characteristics
Age, gender, genetics
Only one value or category
5/7/2016 2Professor Tarek Tawfik

Smoking
Assumed cause
Independent variable Dependent variable
Assumed effect
Cancer
Affect the relationship
Age of the person
Extent of smoking
Duration of smoking
Exercise
Extraneous (undesirable) variables- Modulate the cause-effect
relationship (random error)
Intervening variables - Confounders
A confounding variable is associated with the exposure and it affects the
outcome, but it is not an intermediate link in the chain of causation
between exposure and outcome. (systematic error)
Occupation

5/7/2016 Professor Tarek Tawfik 4
Bias in observational designs
Bias in research denotes deviation from the truth.
(when there is systematic difference between theresults from
study and the truth).
All observational studies and badly done randomized controlled
trials have built-in bias.
The most often used classification of bias includes:
I. Selection bias,
II. Information bias,
III. Confounding.

I- Selection Bias
Are the groups similar in all important respects?
Selection bias stems from absence of comparability
between groups being studied.
In a cohort study, are participants in the exposed and
unexposed groups similar in all important respects except for
exposure?
In case-control study, are cases and controls, similar in all
respects except for the disease in questions?

Selection Bias
Bias accompanying case-control study:
Berkson bias (admission-rate bias): knowledge of the
exposure of interest might lead to an increased rate of
admission to hospital. Admission preference of disease of
interest.
Neyman bias (an incidence-prevalence bias): arises when a
gap in time occurs between exposure and selection of study
subjects. This bias crops up in studies of diseases that are
quickly fatal, transient, or sub-clinical.
Myocardial infarction and its relation to snow shoveling.

Selection Bias
Unmasking bias:
An exposure might lead to provoking of an outcome.
Estrogen replacement therapy and symptomless endometrial
cancer.
Non-respondent bias:
In observational studies, non-respondents are different from
respondents.
Smokers are less likely to return questionnaires than are non-
smokers or pipe and cigar smokers.

II- Information Bias
Has the information been gathered in the same way?
Also known as observation, classification
or measurement bias, results from
incorrect determination of exposure or
outcome or both.
Information should be gathered in the
same way in any comparative study.

II- Information Bias
Has the information been gathered in the same way?
Sources:
 Differentials in information gathering:
(bedside for cases while using telephone for control).
 Diagnostic suspicion bias:
(intensive search for HIV in drug addicts).
 Family history bias:
Medical information flows differently to affected and non-
affected family members (rheumatoid arthritis).

Information Bias
Recall bias: cases are motivated to search their
memories in order to identify the cause of their
illness than the healthy people.
Observer bias: one observer consistently under or
over reports a particular variable. Meticulous
observation of those who are exposed than the non-
exposed.

Information Bias control
Observer and data gatherer should be
blinded.
Using a standardized instruments for
data collection,
Proper selection of the subjects are the
possible maneuvers to lower the
information bias.

III- Confounding.
Is an external factor blurring the effect?
A confounding variable is associated with the exposure and it
affects the outcome, but it is not an intermediate link in the
chain of causation between exposure and outcome.
Myocardial infarctionOral contraceptive
Smoking
IUD insertion
STDs
Salpingitis

Confounding ‘Control’
 Restriction (exclusion or specification):
Enrollment with restricted selection criteria, including non-
smokers.
 Matching:
A pair wise matching (for every case who smokes, a control who
smokes is found).
 Stratification:
Used after completion of the study. Results can be stratified by
the levels of the confounding factor.
 Multivariate analysis techniques:
logistic regression, proportional hazard regression, and others.

Judgment of Associations
Bogus, indirect, or real?
Statistical associations do not imply causal associations.
Types of associations:
 Bogus or spurious associations:
Results of selection, information bias and chance.
 Indirect association:
Stems from confounding.
 Real associations.

Hill’s Criteria for Real Associations
Temporal sequence:
Did exposure precede outcome? the cause must antedate the
outcome.
Strength of association:
How strong is the effect, measured as relative risk (>3 ) or odds
ratio (> 1)?
Consistency of association:
Has effect been seen by others? In different populations with
different study designs.

Biological gradient (dose-response
relationship):
Does increased exposure result in more of the outcome?
Lung cancer and years of cigarette smoking.
Specificity of association:
Does exposure lead only to outcome?
“weak criterion, few exposure will only lead to the outcome”.
Biological plausibility: biological
experimentation
Does the association make sense?
“weak criterion, limited by our lack of knowledge”.

Coherence with existing knowledge:
Is the association consistent with available evidence?
The effect of cigarette smoke on the bronchial epithelium of
animals is coherent with an increased risk of caner in human.
Experimental evidence:
Has a randomized controlled study been done?
Analogy:
Is the association similar to others?

Post test
1- Confounder:
a. Associated with the exposure
b. If known matching minimize it occurrence
c. Directly linked the exposure and the outcome
d. Modulate the cause-effect relationship
2- Extraneous (prognostic) factors.
a. Cause variation in cause effect relationship
b. A form of systematic error
c. Increase by increasing sample size
d. None of the above.
3- Bias
a. Affects all observational designs
b. Can affect sound randomized experiment
c. A built-in error in all research design

For each suggest a descriptor
Case-control
Cohort
Cross-sectional
Hill’s criteria
Experimental evidence

Experimental design
Validity of experimental research
Professor Tarek Tawfik Amin
Epidemiology and Public Health, Faculty of Medicine, Cairo University
Geneva Foundation for Medical Education and Training
Asian Pacific Organization for Cancer Prevention
International Osteoporosis Foundation
Wiley Innovative Panel
amin55@myway.com dramin55@gmail.com
Basic Research Competency Program for Research Coordinators
August 2015, MEDC, Faculty Of Medicine, Cairo University, Cairo, Egypt.

Experimental designs

Experimental designs Observational designs
1. The research
question.
What if? What is?
2. Description - The researcher manipulates independent
variables (e.g., type of treatment,
teaching method) and measures
dependent variables (Efficiency, disease
control, symptoms, scales) in order to
establish cause-and-effect relationships
between them.
- The independent variables controlled or
set by the researcher.
- The dependent variables measured by
the researcher
- Just observation of the both
dependent variables
(outcome) and the
predictors (exposures)
without interfering
- Both are only measured
(no control)
3. Errors and biases A “good” experiment is one that confines
the variation of measurement scores to
variation caused by the treatment itself.
- Many
Differences between observational and experimental designs

True experimental designs
Experimental research is the only type which can
establish cause-and-effect relationships between
variables.
Two purposes: (especially the true experimental)
1- Provide answer to research question
2- Control the difference (covariances)

• Experimental designs should be developed
to ensure internal and external validity of
the study.

The triad of tension (researcher quest)
Maximizing variation
(effect, impact) of
independent variable
(treatment) and the
outcome.
Controlling
extraneous variables
(unwanted)
Limit factor other
than treatment on the
outcome
Minimizing random
errors
[unreliable instrument
and assessment)
Researcher
Elimination
Stratification
Randomization
Matching
Statistics
(ANCOVA)
Controlled conditions
Increase reliability
measures

The ultimate question: Internal
validity• Are the changes (improvement, variation) of the dependent variable
(outcome) caused by the independent variable (intervention) or they
are caused by other factors (extraneous variables), which were not
part of the study?

Validity Internal validity External validity
The ultimate
question
The changes in the dependent
variable influenced only by
intervention and not by other
influences?
“How confidently can I
generalize my experimental
findings to the world?”
Causes Extraneous sources of
variation not controlled.
Non representativeness).
Threats - History,
- Maturation,
- Testing,
- Instrumentation,
- Statistical regression,
- Differential selection,
- Experimental mortality,
- Selection-maturation interaction
- The John Henry effect
- Experimental treatment diffusion
- The reactive effects of testing,
- The interaction of treatment and subject,
- The interaction of testing and subject,
- Multiple treatment interference.

I- History
• Events other than the treatment during the course of an experiment may
influence treatment effect.
• Its influence must occur during the experiment.
• In conducting an experiment both groups are statistically similar in
exposure to historical events.
II- Maturation
• Subjects change over the course of an experiment: physical, mental,
emotional, or spiritual.
• Perspective can change.
• The natural process of human growth result in changes in post-test
scores quite apart from the treatment.
Internal validity threats

III- Testing
• In pre-test, treatment, and post-test design.
• Using the same test both times, the group may show an improvement
simply because of their experience with the test. This is especially true
when the treatment period is short and the tests are given within a short
time.
• It is better to only give a post-test and randomly assign equal groups.
IV-Instrumentation
• Using different tests for pre- and post-measurements, then the change in
pre- and post-scores may be due to differences between the tests rather
than the treatment.
• The best remedy is randomization and a post-test only design.

V- Statistical regression [regression towards the
mean]
• Statistical regression refers to the tendency of extreme scores,
whether low or high, to move toward the average on a second
testing.
• Subjects who score very high or very low on one test will
probably score less high or low when they take the test again.
That is, they regress toward the mean.
• Do not study groups formed from extreme scores. Study the full
range of scores.

VI- Differential selection
• If we select groups for “treatment” and
“control” differently, then the results may be
due to the differences between groups before
treatment.
• Randomization solves this problem

VII- Experimental mortality
• Also called “attrition,” refers to the loss of subjects from the
experiment.
• If there is a systematic bias in the subjects who drop out,
then posttest scores will be biased.
• If subjects drop out because they are aware that they’re not
improving as they should, then the post-test scores of those
completed the treatment will be positively biased.
• Your results will appear more favorable than they really are.

VIII- John Henry Effect
• John Henry, the legendary “steel driving’ man,” set himself to prove he could drive
railroad spikes faster and better than the newly invented steam-powered machine
driver. He exerted himself so much in trying to outdo the "experimental"
condition that he died of a ruptured heart.
• If subjects in a control group find out they are in
competition with those in an experimental
treatment, they tend to work harder.
• The differences between control and treatment
groups are decreased, minimizing the perceived
treatment effect.

IX- Treatment diffusion
• If subjects in the control group perceive the treatment as
very desirable, they may try to find out what’s being done.
• Over the course of the experiment, some of the materials of
the treatment group may be borrowed by the control group
members. Over time, the treatment “diffuses” to the control
group, minimizing the treatment effect.
• This often happens when the groups are in close proximity.
• Both the John Henry Effect and Treatment Diffusion can be
controlled if experimental and control groups are isolated.

External validity threats
I- Reactive effects of testing
• Subjects may respond differently to experimental treatments
merely because they are being tested.
• Since the population at large is not tested, experimental effects
may be due to the testing procedures rather than the treatment
itself. This reduces generalizability.
• Pretest sensitization: Subjects who take a pre-test are sensitized
to the following treatment (creating a different population
compared to the untested population): DO NOT USE
PRETEST.
• Post-test sensitization: Posttest can be a learning experience
that helps subjects to “put all the pieces together.”

II- Treatment and Subject Interaction
• Subjects in a sample may react to the experimental
treatment in ways that are hard to predict.
• This limits the ability of the researcher to
generalize findings outside the experiment itself.
• If there is a systematic bias in a sample, then
treatment effects may be different when applied to
a different sample.

III- Testing and Subject Interaction
• Subjects may react to testing in ways that are hard to predict.
• Test anxiety or “test-wiseness” in a sample, the treatment
effects will be different when applied to a different sample.
IV- Multiple Treatment Effect
• An experiment exposes subjects to several treatments and
test scores show that treatment X produced the best results;
one cannot declare treatment X the best. It may be the
combination of the treatments that led to the results.
• Treatment X, given alone, may produce different results.

True experimental designs
Experimental research designs

Pretest-posttest control group
Interventio
n group
Control
group
Randomization
Baselineassessment
Pretesting
Treatment
Intervention
Placebo
Standard care
Interventio
n group
Control
group
Postintervention
assessment
Posttesting
t-test independent
(or equivalent)
t-paired or equivalent
Pretest sensitization
Possible interaction (pretest and treatment) 39

Posttest ONLY control group
Interventio
n group
Control
group
Randomization
Treatment
Intervention
Placebo
Standard care
Interventio
n group
Control
group
Postintervention
assessment
Posttesting
t-test independent
(or equivalent)

Solomon Four Groups Design
Interventio
n 1
Control 1
Interventio
n 2
Control 2
Randomization
Baselineassessment
Pretesting
Interventio
n 1
Control 1
Interventio
n 2
Control 2
Treatment
Intervention
Treatment
Intervention
Placebo
standard
Placebo
standard
Postinterventionassessment
Posttesting
t-test: effect of the pre-test Effect of treatmen
ANOV
A
Several ways to analyze data
Control extraneous variables
Large sample size
41Professor Tarek Tawfik

The quasi experimental designs
• When randomization is difficult or
can’t be done

Time
series
Group
Baseline Assessment
Test 1 Test 2 Test 3 Test 4
Test 5 Test 7 Test 8
Group
Outcome Assessment
Test 6
Intervention
Treatment
Compare mean scores (pre-post intervention)
-No control group
- Complex data analysis
-Instrumental problem
- Possible Reactive Effect of Repeated Testing.

Non equivalent Control Group Design
Group 1
Group 2
Baseline
data
Pretesting
Intervention
Treatment Group 1
Group 2
Outcome
data
Post
testing
Non randomized
Non-equal groups
ANCOVA is applicable
No control for:
-Selection-Maturation interaction
-Statistical regression
-Suffers from pre-test sensitization 44Professor Tarek Tawfik

Counter balanced design
Group 2
Group 1
Treatment 1
Assessment 1
Treatment 1
Assessment 1
Treatment 2
Assessment 2
Treatment 2
Assessment 2
Group 2
Group 1
Time
Latin Square Analysis
Selection Maturation Interaction
Multiple treatments (interventions) effect [external validity]

Pre-experimental design
• Pre-experimental designs should not be considered
true experiments, and are not appropriate for formal
research.
• Data collected with these designs is highly suspect.

One shot case study
Group
Intervention
Treatment
Group
Post test
Assessment
- None of the sources of internal or external invalidity are controlled.
- Suffers history, maturation, regression, and differential selection.
- Suffers from the external source of “treatment and subject.”
- The design is useless for most practical purposes
Only descriptive analysis (no comparison group)

One-Group Pretest/Posttest
Group Group
Intervention
Treatment
Baseline assessment
Pretesting
Outcome assessment
Post-testing
t-Paired or Wilcoxin Rank
- History, maturation, testing, instrumentation, and selection-maturation interaction.
- The reactive effects of pre-and post- tests and treatment-subject are external sources
of invalidity

Static-Group comparison
Group 1
Group 2 Group 2
Group 1
Intervention
Treatment
Outcome
Assessment
Post testing
The results of statistical analysis are meaningless since there is no as
that groups were the same at the beginning of the treatment.
- This design suffers most from selection, attrition, and
selection-maturation
interaction problems.
- It fails to control the external invalidity source of
treatment and subject.5/7/2016 49Professor Tarek Tawfik

Post testing
1- In experimental design, the researcher tries to:
a. Control independent variable
b. Measure extraneous variables
c. Observe confounders
2- Internal validity is violated through
a. Sound methodology and design
b. Change in population geography
c. Lack of representativeness
3- Long duration experimentation may suffer from the following type of
internal validity threats:
a. History
b. Maturation
c. Selection

Multiple interventions
Pre-post design
Quasi randomized designs
Post test design
Time series
Counter balanced design
State the name of validity threats

Group activity: criticize validity of the use design
• Hydrocortisone Cream to Reduce Perineal Pain after Vaginal Birth: A Randomized Controlled Trial.
Abstract
• PURPOSE: To determine if the use of hydrocortisone cream decreases perineal pain in the immediate
postpartum period.
• STUDY DESIGN AND METHODS: This was a randomized controlled trial (RCT), crossover study
design, with each participant serving as their own control. Participants received three different
methods for perineal pain management at three sequential perineal pain treatments after birth: two
topical creams (corticosteroid; placebo) and a control treatment (no cream application). Treatment
order was randomly assigned, with participants and investigators blinded to cream type. The primary
dependent variable was the change in perineal pain levels (posttest minus pretest pain levels)
immediately before and 30 to 60 minutes after perineal pain treatments. Data were analyzed with
analysis of variance, with p < 0.05 considered significant.
• RESULTS:A total of 27 participants completed all three perineal pain treatments over a 12-hour period.
A reduction in pain was found after application of both the topical creams, with average perineal pain
change scores of -4.8 ± 8.4 mm after treatment with hydrocortisone cream (N = 27) and -6.7 ± 13.0 mm
after treatment with the placebo cream (N = 27). Changes in pain scores with no cream application
were 1.2 ± 10.5 mm (N = 27). Analysis of variance found a significant difference between treatment
groups (F2,89 = 3.6, p = 0.03), with both cream treatments having significantly better pain reduction
than the control, no cream treatment (hydrocortisone vs. no cream, p = 0.04; placebo cream vs. no
cream, p = 0.01). There were no differences in perineal pain reduction between the two cream
treatments (p = .54).
• CLINICAL IMPLICATIONS: This RCT found that the application of either hydrocortisone cream or
placebo cream provided significantly better pain relief than no cream application. 52

• Intramyocardial injection of hydrogel with high interstitial spread does not impact action potential
propagation.
Abstract
• Injectable biomaterials have been evaluated as potential new therapies for myocardial infarction (MI) and
heart failure. These materials have improved left ventricular (LV) geometry and ejection fraction, yet there
remain concerns that biomaterial injection may create a substrate for arrhythmia. Since studies of this risk
are lacking, we utilized optical mapping to assess the effects of biomaterial injection and interstitial spread
on cardiac electrophysiology. Healthy and infarcted rat hearts were injected with a model poly(ethylene
glycol) hydrogel with varying degrees of interstitial spread. Activation maps demonstrated delayed
propagation of action potentials across the LV epicardium in the hydrogel-injected group when compared
to saline and no-injection groups. However, the degree of the electrophysiological changes depended on
the spread characteristics of the hydrogel, such that hearts injected with highly spread hydrogels showed
no conduction abnormalities. Conversely, the results of this study indicate that injection of a hydrogel
exhibiting minimal interstitial spread may create a substrate for arrhythmia shortly after injection by
causing LV activation delays and reducing gap junction density at the site of injection. Thus, this work
establishes site of delivery and interstitial spread characteristics as important factors in the future design
and use of biomaterial therapies for MI treatment.STATEMENT OF SIGNIFICANCE:Biomaterials for treating
myocardial infarction have become an increasingly popular area of research. Within the past few years, this
work has transitioned to some large animals models, and Phase I & II clinical trials. While these materials
have preserved/improved cardiac function the effect of these materials on arrhythmogenesis, which is of
considerable concern when injecting anything into the heart, has yet to be understood. Our manuscript is
therefore a first of its kind in that it directly examines the potential of an injectable material to create a
substrate for arrhythmias. This work suggests that site of delivery and distribution in the tissue are
important criteria in the design and development of future biomaterial therapies for myocardial infarction.

• Protective effects of fish oil, allopurinol, and verapamil on hepatic ischemia-reperfusion
injury in rats.
Abstract
• BACKGROUND:The major aim of this work was to study the protective effects of fish oil (FO),
allopurinol, and verapamil on hepatic ischemia-reperfusion (IR)-induced injury in
experimental rats.MATERIALS AND METHODS:Sixty male Wistar albino rats were randomly
assigned to six groups of 10 rats each. Group 1 served as a negative control. Group 2 served
as hepatic IR control injury. Groups 3, 4, 5, and 6 received N-acetylcysteine (standard), FO,
allopurinol, and verapamil, respectively, for 3 consecutive days prior to ischemia. All animals
were fasted for 12 h, anesthetized and underwent midline laparotomy. The portal triads were
clamped by mini-artery clamp for 30 min followed by reperfusion for 30 min. Blood samples
were withdrawn for estimation of serum alanine transaminase (ALT) and aspartate
transaminase (AST) activities as well as hepatic thiobarbituric acid reactive substances,
reduced glutathione, myeloperoxidase, and total nitrate/nitrite levels, in addition to
histopathological examination.RESULTS:Fish oil, allopurinol, and verapamil reduced hepatic IR
injury as evidenced by significant reduction in serum ALT and AST enzyme activities. FO and
verapamil markedly reduced oxidative stress as compared to control IR injury. Levels of
inflammatory biomarkers in liver were also reduced after treatment with FO, allopurinol, or
verapamil. In accordance, a marked improvement of histopathological findings was observed
with all of the three treatments.CONCLUSION:The findings of this study prove the benefits of
FO, allopurinol, and verapamil on hepatic IR-induced liver injury and are promising for further
clinical trials.

Thank you

Bias,confounding, causation and experimental designs

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (14)

Similar to Bias,confounding, causation and experimental designs

Similar to Bias,confounding, causation and experimental designs (20)

Recently uploaded

Recently uploaded (20)

Bias,confounding, causation and experimental designs