1. The document discusses moving beyond conventional practices in translational statistics to obtain more robust and clinically meaningful results from clinical studies. 2. Several methodology issues are discussed, including how to define primary endpoints when there are multiple outcomes, how to handle dropouts and competing risks, and how to quantify treatment contrasts in a model-free way. 3. Alternative approaches are proposed for various types of studies, such as using restricted mean survival times instead of hazard ratios for survival analyses and performing meta-analyses for evaluating safety issues using large amounts of data.

1. Moving beyond the comfort zone inMoving beyond the comfort zone in
practicing translational statistics
L.J. Wei
H d U i itHarvard University

3. Why are we staying in a
“Comfort Zone” ?“Comfort Zone” ?
 Generally following a fixed pattern for conducting
studies
 Are we like lawyers?
 Avoiding delay of review processes? Avoiding delay of review processes?

4. What is the goal of a clinical
study?study?
 Use efficient and reliable procedures to obtain
robust, clinically interpretable results with respect
to risk-benefit perspectives at the patient’s level.

5. What are the problems?What are the problems?
 The conventional way to conduct trials gives us The conventional way to conduct trials gives us
fragmentary information
 Lack of clinically meaningful totality evidence
 Difficult to use the trial results for future patient’s
management

6. A Few Methodology IssuesA Few Methodology Issues
1. Estimation vs. testing
 P-value provides little clinical information about
treatment effectiveness
Th i f th ff t ( ffi d t i it ) The size of the effects (efficacy and toxicity)
matters
 Design using interval estimates is quite flexible Design using interval estimates is quite flexible
 Almost everything we want to know via testing,
we can get from estimatione ca ge o es a o

7. TREAT study for EPO CV safetyTREAT study for EPO CV safety
 If we follow the patients up to 48 month,
the control arm's average stroke-free time is
46.9 months and the Darb arm's is 46 months.
The difference is 0 9 month (0 95 CI: 0 4m 1 4m)The difference is 0.9 month (0.95 CI: 0.4m, 1.4m)
with p<0.001 (very significant).

8. 2. How do we define a primary endpoint with
multiple outcomes?
 What is current practice?
C t/ it l Component/composite analyses
 Efficacy and toxicity (how to connect them
together?)together?)
 Disease burden measure?
 Competing risks problem?Competing risks problem?
 Informative dropout?

9. Example : Beta-Blocker Evaluation of
S i l (BEST) T i l (NEJM 2001)Survival (BEST) Trial (NEJM, 2001)
 Study
B i d l l l b Bucindolol vs. placebo
 patients with advanced chronic heart failure
-- n = 2707
f Average follow-up: 2 years
 Primary endpoint: overall survival
H d ti f d th 0 90 ( l 0 1) Hazard ratio for death = 0.90 (p-value = 0.1)

10. BEST TrialBEST Trial

11. Possible solutions?Possible solutions?
 Using the patient’s disease burden or progression
information during the entire followup to define
the “responder”
 Creating more than one response categories:
ordinal categorical responseordinal categorical response
 Brian Claggett’s thesis papera C agge s es s pape

12. BEST Example: 8 CategoriesBEST Example: 8 Categories
 1: No events1: No events
 2: Alive, non-HF hospitalization only
 3: Alive, 1 HF hosp., p
 4: Alive, >1 HF hosp.
 5: Late non-CV death (>12 months)( )
 6: Late CV death (>12 months)
 7: Early non-CV death (<12 months)
 8: Early CV death (<12 months)

13. 3. How to handle dropouts or competing risks?
 LOCF? BOCF?
 MMRM (model based)
 Pattern mixture model (cannot handle non-
random missing)
U i d l i ith diff t t Using responder analysis with different ways to
define informative dropouts for sensitivity analysis

14. 4. Analysis of Covariance
 Compare two treatments with baseline
adjustments via regression models
F li d l diff t dj t t For nonlinear model, different adjustments may
lead to incoherent results
 The inadequacy of the Cox ANCOVA The inadequacy of the Cox ANCOVA

15. Possible solutions?
 Using the augmentation method by Tsiatis et al;
Tian et al.
N d t if th b li i t No need to pre-specify the baseline covariates,
but a set of potential covariates in the adjustment
processprocess

16. 5. Data monitoring
 Heavily utilizing p-value or conditional power
 A low conditional power may indicate that the
l i i t ll th i lsample size is too small or there is no real
treatment difference
 Using estimation and prediction for monitoring? Using estimation and prediction for monitoring?

17. 6. Stratified medicine (personalized medicine)?
 A negative trial does not mean the treatment is no
good for anyone
A iti t i l d t it k f A positive trial does not mean it works for
everyone
 The usual subgroup analysis is not adequate to The usual subgroup analysis is not adequate to
address this issue
 Need a built-in pre-specified procedure foreed a bu p e spec ed p ocedu e o
identifying patients who benefit from treatment

18. 7. Identify patients who respond the new
therapy (predictive enrichment)

19. 8. How to monitor safety?
 What is the conventional way?
 Component-wise tabulation or analysis?
 No information about multiple AE events at the
patient level
G hi l th d? Graphical method?

21. 9. Quantifying treatment contrast (difference)?
 Should be model-free parameter
 Using difference of means, median, etc.
 For censored data, using a constant hazard ratio
(heavily model-based)?
M d l b d i diffi lt t i t t Model-based measure is difficult to interpret or
validate

22. Issues for the hazard ratio
estimateestimate
 Hazard ratio estimate is routinely used for
designing, monitoring and analyzing clinical
studies in survival analysis

23. Model Free Parameter for Treatment
ContrastContrast
* Considering a two treatment comparison study in
“survival analysis”
* How do we quantify the treatment difference?
M di f il ti ( t b ti bl )• Median failure time (may not be estimable);
• t-year survival rate (not an overall measure)?
A t t h d ti ti ith th l• A constant hazard ratio over time with the log-
rank test

24. Eastern Cooperative Oncology
Gro pGroup
 E4A03 trial to compare low- and high-dose
dexamethasone for naïve patients with multiple
myeloma
 The primary endpoint is the survival time The primary endpoint is the survival time
 n=445
 The trial stopped early at the second interim The trial stopped early at the second interim
analysis; the low dose was superior.
 Patients on high-dose arm were then receiveda e s o g dose a e e e ece ed
low-dose and follow-up for overall survival were
continued.

25. A Cancer Study ExampleA Cancer Study Example
1.00.8
Group 1
Group 2
0.6
Probability
Group 2
0.20.4
P
0.0
0 10 20 30 40
Month

26.  The proportional hazards assumption is not valid
 The PH estimator is estimating a quantity which
cannot be interpreted and, worse, depends on the
study specific censoring distributionsstudy-specific censoring distributions
 Any model-based treatment contrast has such
issues (need a model-free parameter)issues (need a model free parameter)
 The logrank test is not powerful

27.  Conventional analysis:
 Log-rank test: p=0.47
 Hazard Ratio: HR=0.87 (0.60, 1.27)

28. What is the alternative way for
s r i al anal sis?survival analysis?
 Using the area under the curve of Kaplan-Meier
estimate up to a fixed time point
 Restricted mean survival time Restricted mean survival time
 Model-free and a global measure of efficacyModel free and a global measure of efficacy
 Can be estimated even under heavy censoring

29. Cancer Study ExampleCancer Study Example
Restricted Mean (up to 40 months):
 35.4 months vs. 33.3 months
 ∆ = 2 1 (0 1 4 2) months; p=0 04 ∆ = 2.1 (0.1, 4.2) months; p=0.04
 Ratio of Survival time = 35.4/33.3 = 1.06 (1.00,
1.13)
 Ratio of time lost = 6.7/4.6 = 1.46 (1.02, 2.13)

30. 10. Post-marketing/safety studies ?
 It is not appropriate to use an event driven
procedure to conduct a safety study.
Th t t i l th ti tt The event rate is low, the exposure time matters
 Requires lot of resources (large or long-term
study)study)
 Meta analysis; observational studies

31. CV safety study for anti-diabetes
dr gsdrugs
 Event driven studies, that is, we need to have a
pre-specified # of events so the resulting
confidence interval for the treatment difference is
“narrow”narrow
 For example, the upper bound of 95% confidenceFor example, the upper bound of 95% confidence
interval is less than 1.3

32. The EXAMINE trial (alogliptin)The EXAMINE trial (alogliptin)
NEJM, October 3, 2013

33. RMST (24 months):
Placebo 21.9 (21.7, 22.2)
Alogliptin 22 0 (21 8 22 3)
RMST (30 months):
Placebo 27.1 (26.7, 27.4)
Alogliptin 27 2 (26 9 27 5)Alogliptin 22.0 (21.8, 22.3)
Difference -0.08 (-0.39, 0.24)
Ratio 1.00 (0.98, 1.01)
Alogliptin 27.2 (26.9, 27.5)
Difference -0.12 (-0.56, 0.33)
Ratio 1.00 (0.98, 1.01)

34. Whole data
N=5380
1/2 of data
N=2690
1/3 of data
N=1793
1/4 of data
N=1345N 5380 N 2690 N 1793 N 1345
Hazard Ratio 0.824, 1.129 0.775, 1.213 0.742, 1.283 0.711, 1.338
Difference in RMST
(30m)
-0.558, 0.325 -0.721,
0.500
-0.852,
0.647
-0.980, 0.753

35.  11. Meta analysis for safety issues

36. ( ) f Nissen and Wolski (2007) performed a meta analysis
to examine whether Rosiglitazone (Avandia, GSK), a
drug for treating type 2 diabetes mellitus significantlydrug for treating type 2 diabetes mellitus, significantly
increases the risk of MI or CVD related death.

37. Example
Eff t f R i lit MI CVD D thEffect of Rosiglitazone on MI or CVD Deaths
 Avandia was introduced in 1999 and is widely used
as monotherapy or in fixed-dose combinations with
ith A d t A d leither Avandamet or Avandaryl.
 The original approval of Avandia was based on its
ability in reducing blood glucose and glycated
hemoglobin levels.
 Initial studies were not adequately powered to
determine the effects of this agent on micro- or
macro- vascular complications of diabetes, including
cardiovascular morbidity and mortality.

38. Example
Eff t f R i lit MI CVD D thEffect of Rosiglitazone on MI or CVD Deaths
 However the effect of any anti-diabetic therapy on However, the effect of any anti diabetic therapy on
cardiovascular outcomes is particularly important
because more than 65% of deaths in patients with
di b t f di ldiabetes are from cardiovascular causes.
 Of 116 screened studies, 48 satisfied the inclusion
criteria for the analysis proposed in Nissen and
Wolski (2007).
42 studies were reported in Nissen and Wolski (2007) the42 studies were reported in Nissen and Wolski (2007), the
remaining 6 studies have zero MI or CVD death
10 studies with zero MI events
25 t di ith CVD l t d d th25 studies with zero CVD related deaths

39.  Event Rates from 0% to 2.70% for MI
 Event Rates from 0% to 1.75% for CVD Death

40. MI CVD Death
?????? ??????
Log Odds Ratio
95% CI: (1 03 1 98); p value = 0 03 95% CI: (0 98 2 74); p value = 0 06
Log Odds Ratio
95% CI: (1.03, 1.98); p-value = 0.03
(in favor of the control)
95% CI: (0.98, 2.74); p-value = 0.06

41. QuestionsQuestions
 Rare events?
 How to utilize studies with 0/0 events?
f f ? Validity of asymptotic inference?
 Exact inference?
 Choice of effect measure? Choice of effect measure?
 Between Study Heterogeneity?
 Common treatment effect or study specific treatment Common treatment effect or study specific treatment
effect?
 The number of studies not large?g

42. Exact Inference Asymptotic Inference
MIM
%18.0ˆ  %19.0ˆ 
95% CI: (-0.08,
0.38)%
P-value = 0.27
95% CI: (0.02, 0.42)%
P-value = 0.03

43. Exact Inference Asymptotic Inference
DeathCVDD
%063.0ˆ  %11.0ˆ 
95% CI: (-0.13,
0.23)%
P-value = 0.83
95% CI: (0.00, 0.31)%
P-value = 0.05

44. SummarySummary
C ld dif t ti ti l t i i ? Could we modify our statistical training?
 Teaching young generations “how, where and what to
learn”
 Learning from doing a project with mentoring?
 Could we have a coherent approach from the
beginning to the end for a research project?beginning to the end for a research project?
 George Box: Instead of figuring out the optimalg g g p
solution to a wrong problem, try to get A solution to a
right problem.
 Asking ourselves “What is the question?”Asking ourselves What is the question?