The document summarizes adaptive trial designs, where accumulating data is used to modify aspects of an ongoing study without undermining its validity. It discusses sample size re-estimation, group sequential designs, adaptive randomization, and adaptive dose escalation to make trials more efficient and ethical. Seamless phase II/III designs that combine learning and confirmation phases into one protocol are also described. The FDA accepts some adaptive designs for approval but is cautious of other novel approaches. Case studies demonstrate how adaptations can substantially reduce sample sizes.

1. Economic and Statistical Analysis of Healthcare Technology Teresa Nelson, MS Principal Statistician ©Technomics Research 2009

2. Teresa Nelson Technomics Research, LLC Ryan Wilson Symbios Clinical, Inc. Adaptive Trial Design

3. Agenda Define adaptive designs Discuss where and why they are used Regulatory acceptance and efficiencies to be gained BREAK Recap Logistical and Ethical Considerations Business Rationale Conclusion and time for questions

4. Outline/Goals Define adaptive designs Discuss where they are used why they are used regulatory acceptance efficiencies to be gained

5. Adaptive Designs are Any design that uses accumulating data to decide how to modify aspects of the study as it continues, without undermining the validity and integrity of the trial. Changes are made by “by design” [prospectively] and not on an ad hoc basis ; Adaptation is not a remedy for inadequate planning .

6. Adaptive Designs include Sample Size Re-estimation Group Sequential Designs The new kids on the block Adaptive Randomization Adaptive Dose Escalation Adaptive Seamless Phase II/III designs

7. Why use adaptive design? Reflects medical practice in the real world Ethical with respect to both efficacy and safety of the test treatment under investigation Not only flexible, but efficient too

8. Sample Size Re-Estimation (SSR) Pivotal trial planning includes a power calculation usually based on pilot study results The power of a trial is based on both the clinically meaningful difference and the variability Variability estimates are not precisely estimated in small pilot trials, e.g. N=10 to 15

9. What should be done? Option 1: Plan a fixed trial using a conservative sample size , i.e. highest variability estimate and the lowest effect size. Option 2: Use SSR via an administrative look at the accumulated trial data Plan based on median variability and effect size estimate Use blinded look to estimate within trial variability Increase the sample size, if necessary based on administrative look Option 3: Use a group sequential design (GSD)

10. SSR – Efficient Use of Resources Feature Conservative Fixed SSR GSD Size Likely over-powered “ Right-sized” with respect to variability “ Right-sized” with respect to variability and effect size Use of resources Least efficient Moderately efficient Most efficient Type I error spent NA None Yes

11. SSR – Regulatory Acceptance Option 1, 2 & 3 are accepted by FDA as valid options to sponsors when designing pivotal trials Option 3 (Group Sequential Designs) begs the question what are the implications to looking at the accumulated data in an ongoing clinical trial?

12. Access to What? Always have access to (OPEN Session): Rates of recruitment, ineligibility, noncompliance, protocol violations and dropouts overall and by study site Completeness and timeliness of data Balance between study arms on important prognostic variables Accrual within important subsets Consider Extent of Access to (Closed Session): Interim unblinded comparative safety and efficacy endpoint results

13. What to consider when looking Regulatory – FDA Guidance “On the Establishment and Operation of Clinical Trial Data Monitoring Committees”, March 2006 Risk to scientific validity Sample Size Increased to account for inflation of Type I error Necessary for studies intended to support FDA market approval using either Frequentist or Bayesian methods

14. Continuum of Options for Looking BROAD ACCESS Broad Sponsor access to interim comparative endpoint results AND dissemination into the public domain or to personnel conducting the trial NO ACCESS Independent DSMB with NO sponsor access to interim comparative endpoint results LIMITED ACCESS Charter instructs DSMB to unblind sponsor only if futility may be an issue or access to interim power FIREWALL ACCESS Small group within Sponsor has access to all interim comparative endpoint results

15. Continuum of Options for Looking Broad Firewall Limited No Access Feasibility Market Approval Unblinded Unblinded or Blinded Public dissemination OK NO public dissemination or sharing with personnel involved in trial conduct No FDA issue Acceptable to FDA Looking for endpoints for market approval trial Endpoint change due to external changes unlikely to be okay Endpoint change due to external changes more okay Endpoint change due to external changes okay

16. Continuum of Options for Looking Broad Firewall Limited No Access Feasibility Market Approval Unblinded Unblinded or Blinded Public dissemination OK NO public dissemination or sharing with personnel involved in trial conduct No FDA issue Acceptable to FDA Looking for endpoints for market approval trial Endpoint change due to external changes unlikely to be okay Endpoint change due to external changes more okay Endpoint change due to external changes okay

17. Continuum of Options for Looking Broad Firewall Limited No Access Feasibility Market Approval Unblinded Unblinded or Blinded Public dissemination OK NO public dissemination or sharing with personnel involved in trial conduct No FDA issue Acceptable to FDA Looking for endpoints for market approval trial Endpoint change due to external changes unlikely to be okay Endpoint change due to external changes more okay Endpoint change due to external changes okay

18. Dr. Burns, are you sure this is what statisticians call a double blind experiment?

19. Under-utilized tool - CP and PD Frequentist and Bayesian tool to answer the question – “ Given the data observed to date in the trial, what is the probability (or interim power) to successfully demonstrating the endpoints at a future look (or the end of the trial)? The output is a conditional power estimate, i.e. 80%, or a probability, i.e. 0.80, knowledge that trial is ‘on track with assumptions’ Powerful tool for use in trials with interim looks at the data

20. CP and PD - Uses DSMB/Sponsor can use CP and/or PD to make decisions about trial (more in Ryan’s presentation) Sponsor can include language in an Independent DSMB’s charter to unblind sponsor if CP and/or PD get below some threshold or for sponsor to have access to CP and PD only (“Limited Access”) Buzdar et al. “Significantly higher pathologic complete remission rate after neoadjuvant therapy with transtuzumab, paclitaxel, and epirubicin chemotherapy: results of a randomized trial in human epidermal growth factor receptor 2-positive operable breast cancer. J Clin Oncol. 2005 Jun 1; 23(16): 3676-85.

21. Group Sequential Designs (GSD) Data is analyzed a fixed number of times or according to an error-spending rule while the trial is ongoing Acceptable and often suggested pivotal trial design by FDA for studies destined to support market approval Given small pilot trials and uncertainty about effect size…. Note: Last look can be set based on practical limitations, e.g. available budget First Look Last Look Variability Smallest Largest Effect Size Biggest Smallest Power 80% Higher?

22. GSD – Potential Operational Bias Knowledge that a trial designed for market approval of a device continues past interim looks at the data is knowledge that the results look promising However, this small amount of information does not greatly increase the risk of operational bias Personnel conducting the trial do not know specifics, therefore unlikely to change recruitment patterns, administration of treatment, or safety reporting At some point, whether or not the patients in the trial are treated in the most ethical manner comes into question, which has led calls for further adaptation

23. Call for Further Adaptations The new kids on the block Adaptive Randomization Adaptive Dose Escalation Adaptive Seamless Phase II/III designs

24. Where are the new designs used? Mostly in phase I or II or “feasibility” research Predominantly in pharmaceutical research Review worthwhile to see if pharmaceutical successes have a place in medical device research Not embraced / accepted by FDA for trials destined for a marketing application

25. Call for Further Adaptations FDA Critical Path Initiative – http://www.fda.gov/oc/initiatives/criticalpath 6 broad topic areas, one of which is “Topic 2: Streamlining Clinical Trials” Develop FDA guidance on advanced clinical trial design Explore and advocate innovative trial design that utilizes prior data and accumulating within trial data

26. Adaptive Randomization Conventional Covariate Response Fixed vs. modified Fixed Modified based on ongoing trial results Modification none Baseline covariates and trt assignment Response of the previous patients, e.g. ‘play the winner’ Goal Balance trt and control groups Reduce covariate imbalance Most ethical, pts receive better trt based on “up to the patient” results

27. Adaptive Randomization Conventional Covariate Response Fixed vs. modified Fixed Modified based on ongoing trial results Modification none Baseline covariates and trt assignment Response of the previous patients, e.g. ‘play the winner’ Goal Balance trt and control groups Reduce covariate imbalance Most ethical, pts receive better trt based on “up to the patient” results

28. Adaptive Randomization Summary Ethical, but opens the trial up to bias Accrual bias – patients may want to be randomized later in a trial, rather than at the beginning Accidental bias – conventional randomization balances both measured and unmeasured covariates, adaptive randomization does not. Selection bias – physicians able to select “best” patients for a treatment assignment. Operational bias – intimate knowledge of ongoing trial results, i.e. randomization becomes unbalanced in favor of the winning treatment. Since subjects are not independent, non-parametric tests may have to be used.

29. Adaptive Dose Escalation (ADE) Phase I or Phase II in the pharmaceutical research process Designed to answer questions like Safety: What is the maximum tolerated dose? Efficacy Is there any evidence of an effect? What is the nature of the dose response? What is the optimal dose? For devices – could these questions be applied to device programming?

30. Case Study #1: Migraine Headaches Trial Objective: To test whether a new mechanism of action would effectively treat migraine headaches AND to select a dose range for further investigation. Reference for Study Design: Hall D, Meier U, Diener H, “A group sequential adaptive treatment assignment design for proof of concept and dose selection in headache trials”. Contemporary Clinical Trials 26 (2005) 349 – 364.

31. CS #1: Motivation Limited information about where across the range of seven doses to focus attention A need to limit sample size for a complicated inpatient treatment A desire to reduce exposure of patients to ineffective treatment

32. CS #1: Trial Design Group sequential, adaptive, placebo-controlled trial design, Phase IIa Primary Outcome: Headache response at 2 hours after treatment Broad Efficacy Goal: To identify a narrow range of doses for further investigation from a wider dose range. Specific Efficacy Goal: To identify the lowest dose under study that has at least a 60% response rate and to compare this dose group to placebo.

33. CS #1: Trial Design Safety: Monitored by a safety officer at the central randomization center and by a safety committee at the sponsor’s site. If necessary for safety reasons, could deviate from dose assignment rules. Block size: 4 patients at selected dose, 2 placebo patients (1 placebo patient after the first 20 groups).

34. CS #1: Trial Design *If already at highest (lowest) dose, repeat current dose Assign 1 st block to middle dose Assign next block to nearest lower dose* Assign next block to nearest higher dose* Evaluate Response Response in less than 60% of patients Response in greater than 60% of patients Stopping criteria met? NO Stopping criteria met? NO Yes Yes

36. CS #1: Stopping Rule Primary Stopping Rule: At least 5 blocks of patients treated at the dose For at least four blocks of patients treated at this dose, the prior block at the next higher dose, called for a dose decrease. Secondary Stopping Rule None of the doses selected per the primary selection criteria and 15 blocks of patients (60 patients) treated at the highest dose, highest dose selected.

37. CS #1: Determining Maximum Size Boundary Rules If no dose selected after 36 blocks (200 patients treated), trial terminated without selection of dose. If the highest (or lowest) dose was selected sampling would be continued at this dose until 15 blocks of patients (60 pts) had been treated with the selected dose.

38. CS #1: Additional Sampling Additional Sampling Assuming the dose selection process was successful, sampling was continued to achieve minimal exposure to the doses above the selected dose (at least 3 blocks of patients).

39. CS #1: Changes for Practical Implementation If the results of one group were not complete and a patient was prepared for dosing in the next group, then that patient was assigned to the next lower dose. If upon receipt of results, the next dose should have been the next higher dose , then enrollment in the group at the lower dose was suspended and the trial proceeded with a group at the appropriate dose. If at some later point a group was called for at the suspended dose, then that group would be continued and completed .

41. CS #1: Simulation Used to… Evaluate operating characteristics of design Estimate probability of dose selection (> 98% select a dose, with >80% selecting an appropriate range of doses) Obtain an impression of the likely distribution of study patients across dose Assess the design-inherent bias in the response rate, biased downward for doses below selected dose upward for selected dose Rate Median Trial Size, 95% Type I Error < 5% 120 – 130, 135 – 200 Power > 80% 125 – 150 , 145 – 200

42. CS #1: Results International, multicenter, double-blind RCT 127 patients randomized, 126 treated. Placebo and 7 doses of BIBN 4096 BS included in design (0.1, 0.25, 0.5, 1, 2.5, 5, & 10 mg) 2.5 mg dose of BIBN 4096 BS selected, Oleson J et al. “Calcitonin Gene-Related Peptide Receptor Antagonist BIBN 4096 BS for the Acute Treatment of Migraine”. NEJM 2004; 350 (11): 1104-1110. 2.5 mg dose Placebo P-value 66% 27% 0.0001

43. CS #1: Ethical Most patients allocated to selected dose (38%), Slightly less (33%) at doses higher than selected dose 29% at lower doses. Lowest dose NOT investigated next to lowest dose only given to 1 patient.

44. CS #1: Efficient Sample size cut in half, even if continued to max of 200, sample size reduced by 120 patients!!!!! Fixed sample size parallel group design CS #1 Sample Size 40 pts at each dose + placebo = 320 patients 41 placebo, 85 drug (32 at optimal dose)= 126 patients

45. Other ADE Trial Types Continual Reassessment Method (CRM) randomizes patients to the next dose based on modeling maximizing information about the dose-response relationship. Prospectively define the dose assignment model Consider extending an adaptive dose-finding trial seamlessly into a confirmatory phase….

46. Seamless Phase II/III designs Learning phase + confirmatory phase combined into one trial, one protocol Most efficiency gained due to lack of delay between a typical dose response and confirmatory trial Utilizes data collected in the learning and confirmatory stage at the end of the trial in the analysis Must know primary safety and efficacy endpoints before trial begins, cannot explore which endpoint to use during learning or feasibility stage

47. Seamless Phase II/III designs TRT A TRT C TRT B Control Learning Phase Confirmatory Phase TRT A TRT B TRT C Control Period of analysis and decision making

48. Seamless Phase II/III Implementation Learn from those who go ahead Neuroprotective Agent for Stroke: (Ref: Berry DA et al. Adaptive Bayesian Designs for Dose-Ranging Drug Trials. In: Case Studies in Bayesian Statistics V). Maca J et al “Adaptive Seamless Phase II/III Designs – Background, Operational Aspects, and Examples. Drug Information Journal. 2006; 40 (4): 463 – 473.

49. Seamless Phase II/III Implementation Two separate committees recommended to monitor data External independent data monitoring committee Internal executive decision committee Strongly recommended that regulatory acceptance of design be gathered before study is started.

50. Best fit for newer adaptations Programming parameters like dosing Provide opportunity with the “drop the loser” designs to have bigger pilot trials to determine the best treatment in the confirmatory phase Select a few from many for further study Pre and Post market setting Other ideas?

51. Good fit for Medical Device Trials Sample size re-estimation to reduce risk to trial from lack of information during trial design Group sequential design to capture range of possible effect sizes and stop trial once objectives are met Utilize conditional power or predictive power to understand chance for trial success while trial is ongoing ALL ACCEPTED by FDA for market approval

52. Resources Adaptive Design Methods in Clinical Trials, Shein-Chung Chow & Mark Chang. Chapman and Hall/CRC Biostatistics Series Group Sequential Designs, Jennison and Turnbull. Bayesian Approaches to Clinical Trials and Health-Care Evaluation, Spiegelhalter et al. Gallo et al. “Adaptive Designs in Clinical Drug Development – An Executive Summary of the PhRMA Working Group”. Journal of Biopharmaceutical Statistics, 16: 275-283, 2006.

53. Thank you! Teresa Nelson, MS 218-463-5627 [email_address] www.technomicsresearch.com Ryan Wilson 612-234-8498 [email_address] www.symbiosclinical.com

54. If interested in contracting our services please contact: Kim Martinson Vice President-Business Development Email: [email_address] Ph: 218-331-2272 www.TechnomicsResearch.com