Nicholas Jewell MedicReS World Congress 2014

Statistical Methods for Observational Drug
Studies
Nicholas P. Jewell
Departments of Statistics &
School of Public Health (Biostatistics)
University of California, Berkeley
October 17, 2014

What is the Primary Scientific
Question?
• What is the one “number” you want to know
(what is the measure of effect of interest)?
• What will be the “newspaper headline”?
• What is the parameter of interest?
• What would be a meaningful effect? (range of
parameter values of interest)
© Nicholas P. Jewell, 2014

Types of Studies
• Administrative databases (FDA Adverse Event Reporting
Ststem (FAERS), medical insurance/claims databases,
registeries
• Often no denominators, thereby compromising incidence
estimates
• Exposures only have proxies available
• Temporality/causation
• Observational studies
• Confounding/causation
• Selection bias
• Information bias (misclassification, detection bias etc)
• Randomized Clinical Trials
• Meta-Analyses
• apples & oranges

Observational Data Collection--
Sampling
• What is the target population?
• What is the Study Population?
• How will individuals be sampled (case-control,
cohort, longitudinal)?
• How is exposure assigned to individuals?
• What is now my parameter of interest? 4

Can I Draw a Prototype DAG to
address Causation and
Confounding?
• Direct acyclic graphs (DAG) to related
variables, including exogeneous and
selection variables if necessary
• Is the parameter of interest identifiable
from the design and under what
assumptions?
5

Pre-specified Analysis Plan
• Multiple Outcomes (Which one is
primary)?
• Confounders?
• Subgroups of Interest?
• Effect Modification of Interest?
• Mediation?
6

7
Confounding Variables
• Classic Conditions for confounding
– C must cause D
– C must cause E
C
E ? D

8
Directed Acyclic Graphs
• A node A can be a collider on a specific pathway if the path
entering and leaving A both have arrows pointing into A. A path
is blocked if it contains a collider.
F B
D
C
A
– D is a collider on the pathway C-D-A-F-B; this path is
blocked

9
Using Causal Graphs to Detect Confounding
• Delete all arrows from E that point to any
other node
• Is there now any unblocked backdoor
pathway from E to D?
– Yes—confounding exists
– No—no confounding

Using Causal Graphs to Detect Confounding
10
C F
E D
C F
E D
C F
E D
C F
E D

11
Vaccination & Autism Example
Medical Care
Access
Vaccination Autism
Family
History
SES

12
Checking for Residual Confounding
• After stratification on one or more factors,
has confounding been removed?
– Cannot simply remove stratification factors
and relevant arrows and check residual DAG
– Have to worry about colliders

13
Controlling for Colliders
• Stratification on a collider can induce an association
that did not exist previously
Rain
Sprinkler Wet Pavement
Diet sugar (B)
Fluoridation (A) Tooth Decay
(D) © Nicholas P. Jewell, 2014

Checking for Residual Confounding
• Delete all arrows from E that point to any other
node
• Add in new undirected edges for any pair of
nodes that have a common descendant in the
set of stratification factors S
• Is there still any unblocked backdoor path from E
to D that doesn’t pass through S ? If so there is
still residual confounding, not accounted for by
S .
14

15
Vaccination & Autism Example
Medical Care
Access
Vaccination Autism
Family
History
SES

Vaccination & Autism Example: Stratification
16
on Medical Care Access
Vaccination Autism
Family
History
SES
Still confounding: need to stratify additionally
on SES or Family History, or both

17
Caution: Stratification Can Introduce
Confounding!
C
E D
F
No Confounding
Stratification on C introduces confounding!

Return to Randomized Studies
• What is the Right Parameter to Estimate and
How do We Interpret It?
• Adjustment for Baseline Factors (Pre-randomization)?
• Adjustment for Post-Randomization Factors?
18

What is the Right Parameter?
• Two randomized treatment groups, continuous
response
• Interested in the difference in means
• Use group sample averages and take differences
• No model assumption underlying estimation/
inference (we don’t need to assume Normal
distributions)
• Parameter being estimated (difference in group
means) has a causal effect (both marginally and for
specific subjects (i.e. conditionally))
19

• Two randomized treatment groups, time to event data (usually
right censored)
• Interested in the difference in survival experience (hazards?)
• Use Kaplan-Meier estimates and log rank test?
• Summarize with estimated hazard ratio based on PH
assumption?
– Suggested by CONSORT guidelines and COCHRANE handbook
• Parameter being estimated (difference in group means) has no
causal interpretation if PH model is wrong (which it always is)
• Parameter being estimated also depends on the censoring
distribution!
20

Proportional Hazards
21
(courtesy of Hajime Uno) © Nicholas P. Jewell, 2014

Non-Proportional Hazards
22
(courtesy of Hajime Uno) © Nicholas P. Jewell, 2014

Alternatives with Survival Data
• Ratio or difference in survival functions at time t
• Ratio or difference in survival percentiles
• Restricted (up to time t) mean of survival time
– Difference in integrals of survival curves up to time t
– (Weighted) difference in Survival Curves
– Pre-specify t?
• Differences in cumulative hazard
– Aalen’s additive hazard model
– test statistic: time-weighted estimates of integrated
hazard differences
23
h(t, x, (t)) = 0(t) + 1(t)x
Z t
0
1(u)du

Non-PH Can Make a Difference!
Ascot Clinical Trial (Statins vs. Placebo)
24

25

26

It Makes a Difference!
The assumption of statin benefit to women within
ASCOT is based on a lack of heterogeneity test
within a model that is incorrect. The results become
clear and transparent when an alternative (less
restrictive model is used)
27

• Two randomized treatment groups, binary outcomes
• Interested in the difference in “success”
proportions
• Use logistic regression to estimate treatment effect
adjusting for important confounders/baseline
predictors
• Parameter being estimated (relative difference in
group odds of success) in subgroups has a
conditional interpretation but not as a marginal
assessment of the treatment effect (this is typically
smaller)
28

Should I adjust for baseline predictors
in randomized studies
• Two randomized treatment groups, binary outcomes
• The answer depends on which parameter you are
interested in: marginal or conditional
– Marginally: can gain precision by adjsutment
– Conditionally: always lose precision by adjustment
(Robinson Jewell) with simple logistic regression (is there
a better estimator?)
– Always can gain efficiency when testing the null hypothesis
• It is important to construct estimates that leverage
baseline predictors that do not depend on the model
specification being right
29

Adjustment for Post-Randomization
Factors
• Beware! You are conditioning or selecting on a
factor that is itself not randomized.
• MIRA trial
• Guarantee or Immortal Time Bias
– Nobel prize winners have longer lives than the rest of us (as
do Oscar winners etc).
• Only considering those with complete data (eg
ignoring early drop outs)
30

Mediation: The MIRA Trial
• Gates Foundation study to determine the
effectiveness of a latex diaphragm in the reduction
of heterosexual acquisition of HIV among women
• Two arm, randomized, controlled trial
• Primary intervention: diaphragm and gel provision
to diaphragm arm (nothing to control arm).
• Secondary Intervention: Intensive condom provision
and counseling given to both arms, plus treatment
of STIs
• Trial is not blinded
• 5000 women seen for 18 months in three sites in
Zimbabwe and South Africa
31

MIRA Trial: Basic Intention to
Treat Results
• Basic Intent-to-Treat Analysis:
– 158 new HIV infections in Diaphragm Arm
– 151 new HIV infections in Control Arm
• ITT estimate of Relative Risk is 1.05 with a
95% CI of (0.84, 1.30)
• End of story . . . . .?
32

Estimating Direct Effects: MIRA Trial
HIV
Infection
Treatment Group
(Diaphragm use)
Condom
Use
Confounders
after stratification on condom use
HIV
Infection
Treatment Group
(Diaphragm use)
Confounders
randomization hasn’t ruled out confounding of direct effect!
Now have to adjust for confounders (but we are still ITT)
33

Results of Direct Effects Analysis
• Relative Risk of HIV infection between Diaphragm
arm and Control arm by end of Trial, with Condom
Use Fixed at “Never”: 0.59 (95% CI: 0.26, 4.56)
• Relative Risk of HIV infection between Diaphragm
arm and Control arm by end of Trial, with Condom
Use Fixed at “Always”: 0.96 (95% CI: 0.59, 1.45)
Conclusion: No definitive evidence from direct effects
analysis that diaphragms prevent (or don’t prevent)
HIV.
34

Meta-Analyses
• Random or Fixed Effects
– heterogeneity
– Measure of association (RR or ER)
• Zero-event trials/use of all evidence
– Drug safety studies (Vioxx, Celebrex, Avandia, etc)
– Continuity corrections
– Study duration
35

Meta-Analyses and No Event
Studies
• One study: 30 events under Tx, 3 under
placebo (5000 individuals in both arms)
– Risk difference of 0.0054 with exact 95%
confidence interval of (0.0032, 0.0076), p 0.0001
• Now add three (short duration) studies,all with
5000 in both arms, no events in either arms in
all cases, use Tian et al. method
– exact 95% confidence interval of (-0.0044, 0.0001),
p = 0.46
36

Overview
• Opportunistic Data Torturing
– Multiple comparisons
– Multiple analyses
– Subgroup analyses
• Procustean Data Torturing
– Subgroup analyses (or lack thereof)
– Meta-analyses
• Sensitivity Analyses/Proxies/Assumptions/Honesty
• Blinding the Statistician (who funds the statistician?)
• Use of surrogate outcomes/exposures (similar to blank or spiked
samples) for comparisons of interest (HSV in MIRA trial)
• Pre-specified Data Driven Analyses
• Dissemination and Publication
37

References
• Jewell, N. P. Statistics for Epidemiology Chapman 7 Hall, 2003.
• Umo, H., Claggett, B., Tian, L., Inoue, E., Gallo, P., Miyata, T., Schrag, D., Takeuchi, M., Uyama,
Y., Zhao, L., Skali, H., Solomon, S., Jaconus, S., Hughes, M., Packer, M. Wei, L.J. “Moving
beyond the hazard ratio in quantifying the between-group difference in survival analysis,” J.
Clinical Oncology, 32, 2014, 2380-2385.
• Hosmer, D.W. Royston, P. “Using Aalen’s linear hazards model to investigate time-varying
effects in the proportional hazards regression model,” Stata Journal, 2, 2002, 331-350.
• Sever, P.S., Dahlöf, B., Poulter, N.R., Wedel, H., Beevers, G., Caulfield, M., Collins, R. et al.
“Prevention of coronary and stroke events with atorvastatin in hypertensive patients who have
average or lower-than-average cholesterol concentrations, in the Anglo-Scandinavian Cardiac
Outcomes Trial—Lipid Lowering Arm (ASCOT-LLA): a multicentre randomised controlled trial,”
Lancet, 361, 2003, 1149-1158.
• Rosenblum, M., Jewell, N.P., van der Laan, M., Shiboski, S., van der Straten, A. Padian, N.,
“Analysing direct effects in randomized trials with secondary interventions: an application to
human immunodeficiency virus prevention trials,” J. R. Statist. Soc. A, 172, 2009, 443–465.
• Robinson, L.D. Jewell, N.P.” Some surprising results about covariate adjustment in logistic
regression models,” International Statistical Review 59, 1991, 227-240.
• Tian, L., Cai, T., Pfeffer, M. A., Piankov, N., Cremieux, P.-Y. Wei, L. “Exact and efficient
inference procedure for meta-analysis and its application to the analysis of independent 2× 2
tables with all available data but without artificial continuity correction,” Biostatistics, 10, 2009,
275–281.
• Bailar, J.C. III., ”Science. Statistics, and deception, Annals of Internal Medicine, 104, 1986,
259-260.
• Mills. J.S., “Data torturing,” New England Journal of Medicine, 329, 1993, 1196-1199.
38

Nicholas Jewell MedicReS World Congress 2014

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (7)

Similar to Nicholas Jewell MedicReS World Congress 2014

Similar to Nicholas Jewell MedicReS World Congress 2014 (20)

More from MedicReS

More from MedicReS (20)

Recently uploaded

Recently uploaded (20)

Nicholas Jewell MedicReS World Congress 2014