[Webinar] How to reduce sample size...ethically and responsibly | In this free webinar, you will learn various design strategies to help reduce the sample size of your study in an ethical and responsible manner. Practical examples will be used throughout.

1. July 2019 Webinar
Draft

2.  Head of Statistics
 nQuery Lead Researcher
 FDA Guest Speaker
 Guest Lecturer
Webinar Host
HOSTED BY:
Ronan
Fitzpatrick

3. Webinar Overview
Study Design & Sample Size
Good Design for Lower Sample Size
Efficiency and Adaptive Design
Conclusions and Q&A

4. Two Sample t-test
Two Sample Chi-Squared test
ANOVA/ANCOVA
Sample Size Re-Estimation for Survival
Worked Examples

5. In 2018, 91% of organizations with clinical trials approved
by the FDA used nQuery for sample size and power calculation
About nQuery

6. PART 1 Study Design and Sample Size

7. Sample Size Determination (SSD)
SSD finds the appropriate sample size for your study
 Common metrics: Statistical power, interval width, cost
SSD seeks to balance ethical and practical issues
 A standard design requirement for regulatory purposes
SSD is crucial to arrive at valid conclusions in a study
 High incidence of non-replicable results, Type M/S errors
Yet many studies have insufficient sample size
 But others rejected due to unrealistic sample size needs?

8. Important Design Questions
What is the primary outcome of the study?
What type of hypothesis test will be used?
What kind of grouping structure will the study have?
What question/s do you want to answer?

9. To get the sample size, you must know what “success”
would mean in your trial!
To get the sample size, you must know what the study
outcome(s) will be in you trial!
To get the sample size, you must know what statistical
method will be used in your trial!
Sample Size emerges from design so think about design
choices before asking for a sample size!
Sample Size Follows Design!

10. Design choices will affect power
•Choice of model
•Choice of endpoint
•Choice of hypothesis
•Choice of Covariates/adjustments
Thus can use power to compare
choices
Sample Size Determination can
help highlight bad choices!
Design, Design, Design!
Source: S. Senn (2005)

11. PART 2 Good Design for Lower N

12. Design Choices and Sample Size
Every design choice will have some effect on the required
sample size
Good designs are efficient and thus reduce sample size
 e.g. randomization level, cross-over, covariate choice etc
Focus on two common choices that can reduce sample
size: choice of endpoint, including covariates
Will show value of using data as is rather than simplifying

13. Choice of Endpoint
How to choose the right endpoint(s) has been neglected
Things improving with time e.g. E9 Addendum on Estimands
In general, closer endpoint is to actual measure, the better
Minimizes the amount of information “wasted” in analysis
But endpoint (and resulting model) has large effect on N
Inefficient “standard” binary defaults often used e.g. “responders”
SSD can help illustrate true cost of unneeded simplifications
e.g. dichotomisation, treating TTE as binary, recurrent events as TTE

14. “Assuming a mean (±SD) number of
ventilator-free days of 12.7±10.6, we
estimated that a sample of 524
patients would need to be enrolled in
order for the study to have 80%
power, at a two-tailed significance
level of 0.05, to detect a mean
between-group difference of 2.6
ventilator-free days. On the basis of
data from the PAC-Man trial, we
estimated that the study-withdrawal
rate would be 3% and we therefore
calculated that the study required a
total of 540 patients.” Source: NEJM (2014)
Parameter Value
Significance Level (Two- Sided) 0.05
Mean Difference 2.6
Standard Deviation 10.6
n per Group (post-dropout) 262
Target Power 80%
Dichotomisation Example

15. Take previous example but
create “responders” for subjects
with >=14 (i.e. mid-point)
ventilator free-days post-
intervention. Assume will
analyse P(Respond) per group
using chi-squared test. What will
new sample size requirement be
and what would power be with
original N (262 per group)
Parameter Value
Significance Level (Two-Sided) 0.05
Proportion Control 0.4512
Proportion Treatment 0.5488
Power 80%
To get equivalent proportions, we can use
normal CDF (i.e. our Z-statistic tables). In
this case, take P((12.7-14)/10.6<0) &
P((15.3-14)/10.6<0), which is approx.
0.4512 & 0.5488.
Dichotomisation Example

16. Covariate Selection
Covariate selection is often misapplied process
e.g. stepwise regression, selecting using baseline significance
In general, should use covariates with prognostic value
Covariate selection should be part of study design and protocol
Many approaches to adjust for covariates but use ANCOVA here
ANCOVA best in randomized design vs post ANOVA and pre-post ANOVA
SSD can help show value of including covariates and reduce N
Shows covariate/outcome relationship is key not covariate/treatment

17. “Sample size estimation was based
on … a week 6 mean change from
baseline of 0.75 UI episodes, an SD
of 0.85 UI episodes,α = 0.05 and β =
0.20. To detect a difference of 0.75
between treatment groups in the
mean change from baseline in the
number of UI episodes and assuming
a 20% dropout rate it was necessary
to enrol 56 patients, that is 28 per
group. The calculation assumed a 2-
sample procedure using 2-sided
statistical testing.”
Source: Journal of Urology (2011)
Parameter Value
Significance Level (2-sided) 0.05
Placebo Mean (baseline) 5.6
Treatment Mean 4.85
Standard Deviation (Common) 0.85
Power 80%
N per group (before 20% dropout) 28
N per group after dropout 22
Covariate Example

18. Previous example was analysed
by ANCOVA using baseline as
covariate. How much would
including effect of baseline
reduce expected sample size and
decrease N?
Let’s look at R2 ranging from 0 – 1
in increments of 0.25
Parameter Value
No Correlation (Baseline vs Post) 0
Minimal Correlation 0.25
Medium Correlation 0.5
High Correlation 0.75
Perfect Correlation 1
Baseline is a common covariate even if
parallel slope assumption violated.
ANCOVA robust in randomized context.
Note correlation coefficient will equal
square root of R2 for a single covariate.
Covariate Example

19. PART 3 Efficient Adaptive Design

20. Adaptive Design Overview
Adaptive designs are any trial where a change or decision
is made to a trial while still on-going
Encompasses a wide variety of potential adaptions
 e.g. Early stopping, SSR, enrichment, seamless, dose-finding
Adaptive trials seek to give control to trialist to
improve trial based on all available information
Adaptive trials can decrease costs & better inferences

21. Adaptive Design Review
Advantages
1. Earlier Decisions
2. Reduced Potential Cost
3. Higher Potential Success
4. Greater Control
5. Better Seamless Designs
Disadvantages
1. More Complex
2. Logistical Issues
3. Modified Test Statistics
4. Greater Expertise
5. Regulatory Approval

22. Group Sequential Design
Group Sequential Designs (GSD) facilitate interim analyses
Interim analyses are those which occur while a trial is on-going
In a GSD, accrued data is analysed at pre-specified times
E.g. After half the subjects have been measured
At an interim analysis, can either stop for benefit or futility
If neither found, continue trial until end/next interim analysis
However, need to account for effect of multiple analyses
Do this by “spending” errors using error spending function

23. Group Sequential Design for Survival
Parameter Value
Significance Level (One-Sided) 0.025
Placebo Median Survival (months) 6
Everolimus Median Survival (months) 9
Hazard Ratio 0.66667
Accrual Period (Weeks) 74
Minimum Follow-Up (Weeks) 39
Power (%) 92.6
Parameter Value
Number of Looks 3
Efficacy Bound O’Brien Fleming
Futility Bound Non-Binding
Beta Spending Function Hwang-Shih-DeCani
HSD Parameter -1.25
Source: NEJM (2011)
Extend everolimus (left) example to
group sequential design with 2 interim
analyses with O’Brien Fleming efficacy
bound and non-binding Hwang-Shih-
DeCani futility bound with gamma = -1.25
GSD Survival Example

24. Sample Size Re-estimation (SSR)
Will focus here on specific adaptive design of SSR
Adaptive Trial focused on higher sample size if needed
 Obvious adaption target due to intrinsic SSD uncertainty
 Could also adaptively lower N but not encouraged
Two Primary Types: 1) Unblinded SSR; 2) Blinded SSR
 Differ on whether decision made on blinded data or not
 Both target different aspects of initial SSD uncertainty

25. Unblinded SSR
SSR suggested when interim effect size is “promising” (Chen et al)
 “Promising” user-defined but based on unblinded effect size
 Extends GSD with 3rd option: continue, stop early, increase N
Power for optimistic effect but increase N for lower relevant effects
 Updated FDA Guidance: Design which “can provide efficiency”
Common criteria proposed for unblinded SSR is conditional power (CP)
 Probability of significance given interim data
2 methods here: Chen, DeMets & Lan; Cui, Hung & Wang
 1st uses GSD statistics but only penultimate look & high CP
 2nd uses weighted statistic but allowed at any look and CP

26. Assume previous group sequential design
with added SSR option
Assume interim HR= 0.8 (from 0.666) and
inherit total E of 309 (interim E of 103 and
206) and final look alpha of 0.23 from GSD
example.
What will required E for SSR for Chen-
Demets-Lan/Cui-Hung-Wang assuming
maximum events multiplier of 3?
Parameter Value
Nominal Final Look Sig. Level 0.0231
Initial HR 0.667
Interim HR 0.8
Initial Expected Events (E) 309
Interim Events (2nd Look) 206
Maximum Events 927
Lower CP Bound (CDL/CHW) Derived/40%
Upper CP Bound 92.6%
Unblinded SSR Survival Example

27. Adaptive Survival Complications
Unknown follow-up means more interim planning uncertainty
 Can be difficult to predict time when interim analysis will occur
 Higher numbers likely in active cohort when interim analysis occurs
Adaptive designs for survival often come with new assumptions
 Usually strong assumption of constant treatment effect for GSD and SSR
Two dimensions for increasing interim events: Sample Size & Time
 N increase biases for early trends, time increase for later trends?
 Freidlin and Korn (2017) show ability to pick best treatment period

28. PART 4 Discussion and Q&A

29. Bad design choices still commonly made in research
Dichotomisation, stepwise regression, pre-post differences
SSD and power analysis can show cost of these choices
e.g. Easy to explain why 100 extra subjects would cost more
Side-by-side comps easy to show in nQuery or similar
Show value of using data “as is” & considering factors a priori
Adaptive design gives chance to get near “ideal” design
But can be less efficient if not considered carefully
Conclusions

30. Q&A
For further details,
email
info@statsols.com
Thanks for
listening!
Questions?

31. Statsols.com/trial

32. References
Senn, S. S. (2008). Statistical issues in drug development. John Wiley & Sons.
Senn, S. (2005). Dichotomania: an obsessive compulsive disorder that is badly affecting the
quality of analysis of pharmaceutical trials. Proceedings of the International Statistical
Institute, 55th Session, Sydney.
McAuley, Daniel F., et al. "Simvastatin in the acute respiratory distress syndrome." New
England Journal of Medicine 371.18 (2014): 1695-1703.
Senn, S. (2006). Change from baseline and analysis of covariance revisited. Statistics in
medicine, 25(24), 4334-4344.
Darren Dahly ANCOVA Tweetorial:
https://threadreaderapp.com/thread/1115902270888128514.html

33. References
Herschorn, S., Gajewski, J., Ethans, K., Corcos, J., Carlson, K., Bailly, G., ... & Radomski, S.
(2011). Efficacy of botulinum toxin A injection for neurogenic detrusor overactivity and
urinary incontinence: a randomized, double-blind trial. The Journal of urology, 185(6), 2229-
2235.
Jennison, C., & Turnbull, B. W. (1999). Group sequential methods with applications to clinical
trials. CRC Press.
US Food and Drug Administration. (2018) Adaptive design clinical trials for drugs and
biologics (Draft guidance). Retrieved from https://www.fda.gov/media/78495/download
Chen, Y. J., DeMets, D. L., & Gordon Lan, K. K. (2004). Increasing the sample size when the
unblinded interim result is promising. Statistics in medicine, 23(7), 1023-1038.
Cui, L., Hung, H. J., & Wang, S. J. (1999). Modification of sample size in group sequential
clinical trials. Biometrics, 55(3), 853-857.

34. References
Mehta, C.R. and Pocock, S.J., (2011). Adaptive increase in sample size when interim results
are promising: a practical guide with examples. Statistics in medicine, 30(28), 3267-3284.
Chen, Y. J., Li, C., & Lan, K. G. (2015). Sample size adjustment based on promising interim
results and its application in confirmatory clinical trials. Clinical Trials, 12(6), 584-595
Liu, Y., & Lim, P. (2017). Sample size increase during a survival trial when interim results are
promising. Communications in Statistics-Theory and Methods, 46(14), 6846-6863.
Freidlin, B., & Korn, E. L. (2017). Sample size adjustment designs with time-to-event
outcomes: a caution. Clinical Trials, 14(6), 597-604.
Yao, J. C., Shah, M. H., Ito, T., Bohas, C. L., Wolin, E. M., Van Cutsem, E., ... & Tomassetti, P.
(2011). Everolimus for advanced pancreatic neuroendocrine tumors. New England Journal of
Medicine, 364(6), 514-523.