Your SlideShare is downloading. ×
Randomization: Too Important to Gamble with.
Randomization: Too Important to Gamble with.
Randomization: Too Important to Gamble with.
Randomization: Too Important to Gamble with.
Randomization: Too Important to Gamble with.
Randomization: Too Important to Gamble with.
Randomization: Too Important to Gamble with.
Randomization: Too Important to Gamble with.
Randomization: Too Important to Gamble with.
Randomization: Too Important to Gamble with.
Randomization: Too Important to Gamble with.
Randomization: Too Important to Gamble with.
Randomization: Too Important to Gamble with.
Randomization: Too Important to Gamble with.
Randomization: Too Important to Gamble with.
Randomization: Too Important to Gamble with.
Randomization: Too Important to Gamble with.
Randomization: Too Important to Gamble with.
Randomization: Too Important to Gamble with.
Randomization: Too Important to Gamble with.
Randomization: Too Important to Gamble with.
Randomization: Too Important to Gamble with.
Randomization: Too Important to Gamble with.
Randomization: Too Important to Gamble with.
Randomization: Too Important to Gamble with.
Randomization: Too Important to Gamble with.
Randomization: Too Important to Gamble with.
Randomization: Too Important to Gamble with.
Randomization: Too Important to Gamble with.
Randomization: Too Important to Gamble with.
Randomization: Too Important to Gamble with.
Randomization: Too Important to Gamble with.
Randomization: Too Important to Gamble with.
Randomization: Too Important to Gamble with.
Randomization: Too Important to Gamble with.
Randomization: Too Important to Gamble with.
Randomization: Too Important to Gamble with.
Randomization: Too Important to Gamble with.
Randomization: Too Important to Gamble with.
Randomization: Too Important to Gamble with.
Randomization: Too Important to Gamble with.
Randomization: Too Important to Gamble with.
Randomization: Too Important to Gamble with.
Randomization: Too Important to Gamble with.
Randomization: Too Important to Gamble with.
Randomization: Too Important to Gamble with.
Randomization: Too Important to Gamble with.
Randomization: Too Important to Gamble with.
Randomization: Too Important to Gamble with.
Randomization: Too Important to Gamble with.
Randomization: Too Important to Gamble with.
Randomization: Too Important to Gamble with.
Randomization: Too Important to Gamble with.
Randomization: Too Important to Gamble with.
Randomization: Too Important to Gamble with.
Randomization: Too Important to Gamble with.
Randomization: Too Important to Gamble with.
Randomization: Too Important to Gamble with.
Randomization: Too Important to Gamble with.
Randomization: Too Important to Gamble with.
Randomization: Too Important to Gamble with.
Randomization: Too Important to Gamble with.
Randomization: Too Important to Gamble with.
Randomization: Too Important to Gamble with.
Randomization: Too Important to Gamble with.
Randomization: Too Important to Gamble with.
Randomization: Too Important to Gamble with.
Randomization: Too Important to Gamble with.
Randomization: Too Important to Gamble with.
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Randomization: Too Important to Gamble with.

257

Published on

Slides from the 18 October 2012 dinner meeting of the Delaware Chapter of the ASA

Slides from the 18 October 2012 dinner meeting of the Delaware Chapter of the ASA

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
257
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
6
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Randomization: Too Important to Gamble with A Presentation for the Delaware Chapter of the ASA Oct 18, 2012 Dennis Sweitzer, Ph.D., Principal Biostatistician Medidata Randomization Center of Excellence Optimizing Clinical Trials: Concept to Conclusion™Optimizing Clinical Trials: Concept to Conclusion™ © 2012 Medidata Solutions, Inc. § 1
  • 2. Outline Randomized Controlled Trials •  Basics •  Balance Randomization methods •  Complete Randomization •  Strict Minimization •  Permuted Block •  Dynamic Allocation (Covariate-adaptive, not Response-Adaptive) Randomization Metrics •  Balance •  Predictability •  Loss of Power /Loss of Efficiency •  Secondary Imbalance: drop-outs Simulations comparing methods •  Confounding site & treatment effects (small sites) •  Overall performance •  Discontinuing patients •  Weighting stratification factors Meta-BalanceOptimizing Clinical Trials: Concept to Conclusion™ © 2012 Medidata Solutions, Inc. § 2
  • 3. Why randomize anyway? Some basic principlesOptimizing Clinical Trials: Concept to Conclusion™ © 2012 Medidata Solutions, Inc. § 3
  • 4. Why Gold Standard? Randomized Controlled Trial •  Trial:Prospective & Specific •  Controlled: •  Comparison with Control group •  (placebo or active) •  Controlled procedures ⇒ Only Test Treatment Varies •  Randomization: Minimizes biases •  Allocation bias •  Selection bias •  Permits blindingOptimizing Clinical Trials: Concept to Conclusion™ © 2012 Medidata Solutions, Inc. § 4
  • 5. Eliminating Bias¿ The Fact of bias ? •  (conscious, unconscious, or instinctive)¿ The Question of bias ? •  Always 2nd guessing •  Critics will think of unanticipated things¡ Solution ! •  Treat it as a game •  1 statistician vs N clinicians •  Statistician generates a random sequence •  Clinicians sequential guess at each assignment •  Statistician wins if clinician guesses are no better than chance (NB: 75% wrong is just as bad as 75% right)Optimizing Clinical Trials: Concept to Conclusion™ © 2012 Medidata Solutions, Inc. § 5
  • 6. Randomization Metrics What do we want in a randomization sequence or system? Randomness ó Unpredictable ⟶ Reduce Allocation Bias (All studies) ⟶ Reduce Selection Bias (All studies) ⟶ Reduce placebo effects (Blinded studies) Balance ó “Loss of Efficiency” ⟶ Maximizes statistical power ⟶ Minimize Confounding ⟶ Enhance Credibility (Face Validity)Optimizing Clinical Trials: Concept to Conclusion™ © 2012 Medidata Solutions, Inc. § 6
  • 7. BalancingOptimizing Clinical Trials: Concept to Conclusion™ © 2012 Medidata Solutions, Inc. § 7
  • 8. Balanced Study Equal allocation between treatment arms •  Maximizes Statistical Power Control TestOptimizing Clinical Trials: Concept to Conclusion™ © 2012 Medidata Solutions, Inc. § 8
  • 9. Imbalanced Statistical power limited by smallest arm •  36 subject simulation with Complete Randomization ⟶ average loss ≈ 1 subject 10% lose ≥2 subject •  Can add 2 to compensate •  BUT only large imbalances have much effect on statistical power Resulting in light weight results…. Severe Imbalances are rare in large studies Pr{worse than 60:40 split} for: •  n=25 ⟶ <42% n=100 ⟶ <4.4% n=400 ⟶ 0.006%Optimizing Clinical Trials: Concept to Conclusion™ © 2012 Medidata Solutions, Inc. § 9
  • 10. (NB: Planned Imbalance) 1:1 randomization maximizes power per patient But there are other considerations •  Utility: •  Need 100 patients on drug to monitor safety •  Study only requires 60 (30/arm) •  2:1 randomization ⟶ 100 Test & 50 Placebo •  Motivation: •  Better enrollment if 75% chance of Test drug (3:1) •  Ethics: •  85 Placebo + 255 Test vs. 125 Placebo + 125 TestOptimizing Clinical Trials: Concept to Conclusion™ © 2012 Medidata Solutions, Inc. § 10
  • 11. Imbalance•  Overall balance •  Only an issue for small studies•  Subgroup Balance •  Fixed size studies can have variable sized subgroups ⟶ Increased risk of underpowered subgroupsOptimizing Clinical Trials: Concept to Conclusion™ © 2012 Medidata Solutions, Inc. § 11
  • 12. Effective Loss of Sample Size Effective Loss = Reduction of Power Females Male as Reduction in Sample Size s Test Simulations of: Pla •  36 and 18 subjects, •  males as strata at 33% of population, Test Con •  randomized 1:1 •  (complete randomization) N=36 N=18 Overall Females Males Overall Females Males Effectively Lost Mean ± SD 1.0 ±1.4 0.9 ±1.3 1.0 ±1.4 1.0 ±1.4 1.0 ±1.4 1.0 ±1.3 ≥2 pts 12% 14% 18% 23% 16% 17% ≥4 pts 6% 4% 5% 3% 4% 5% >=100% 0.0% 0.0% 0.4% 0.0% 0.5% 7.9% Q1 0.11 0.15 0.09 0.22 0.09 0.14 Median 0.44 0.43 0.47 0.22 0.40 0.50 Q3 1.00 1.19 1.33 0.89 1.33 1.29 Imbalance (% of N) Mean ± SD 13% ±10% 16% ±12% 25% ±19% 18% ±15% 23% ±18% 35% ±28% >=50% 0.5% 1.6% 12.8% 3.1% 10.0% 27.9% Q1 6% 8% 9% 11% 9% 14% Median 11% 14% 20% 11% 20% 33% Q3 17% 22% 33% 22% 33% 50%Optimizing Clinical Trials: Concept to Conclusion™ © 2012 Medidata Solutions, Inc. § 12
  • 13. Bad Imbalance! Males Females Treatment Imbalances Pla Test within factors ⟶ spurious Test Pla findings….. Leads to conversations like: ANCOVA Higher estrogen showed no levels in patients Credibility….. differences in on Test estrogen Treatment ?? levels due to Hmm… treatment ?Optimizing Clinical Trials: Concept to Conclusion™ © 2012 Medidata Solutions, Inc. § 13
  • 14. ?" Randomization !!! ! !! ! Methods (See Animated Powerpoint Slides…)Optimizing Clinical Trials: Concept to Conclusion™ © 2012 Medidata Solutions, Inc. § 14
  • 15. Randomization4 methods•  Complete Randomization (classic approach)•  Strict Minimization•  Permuted Block (frequently used)•  Dynamic Allocation (gaining in popularity)Optimizing Clinical Trials: Concept to Conclusion™ © 2012 Medidata Solutions, Inc. § 15
  • 16. Complete Randomization Every assignment •  Same probability for each assignment •  Ignore Treatment Imbalances •  No restrictions on treatment assignments Advantages: •  Simple •  Robust against selection & accidental bias •  Maximum Unpredictability Disadvantage •  High likelihood of imbalances (smaller samples) .Optimizing Clinical Trials: Concept to Conclusion™ © 2012 Medidata Solutions, Inc. § 16
  • 17. Minimization Strict Minimization randomizes to the imbalanced armOptimizing Clinical Trials: Concept to Conclusion™ © 2012 Medidata Solutions, Inc. § 17
  • 18. Minimization Strict Minimization rebalances the Arms •  BUT at a cost in predictability •  Random only when treatments are currently balancedOptimizing Clinical Trials: Concept to Conclusion™ © 2012 Medidata Solutions, Inc. § 18
  • 19. Permuted Block Blocks of Patients (1, 2, or 3 per treatment) Here: 2:2 Allocation T P P ? T P P T T P (Unless Incomplete P * Blocks: More strata ⟶ More incomplete)Optimizing Clinical Trials: Concept to Conclusion™ © 2012 Medidata Solutions, Inc. § 19
  • 20. Dynamic Allocation Biases Randomization to the imbalanced arm •  Unpredictable •  Almost BalancedOptimizing Clinical Trials: Concept to Conclusion™ © 2012 Medidata Solutions, Inc. § 20
  • 21. Dynamic Allocation Complete Randomization •  Optimizes Unpredictability •  Ignores Balance Strict Minimization •  Optimizes Balance •  Ignores Predictability Dynamic Allocation 2nd Best Probability Parameter Controls Balance vs. Predictability TradeoffOptimizing Clinical Trials: Concept to Conclusion™ © 2012 Medidata Solutions, Inc. § 21
  • 22. Dynamic Allocation Flexibility 2nd Best Probability= 0 ⟶ Strict MinimizationOptimizing Clinical Trials: Concept to Conclusion™ © 2012 Medidata Solutions, Inc. § 22
  • 23. Dynamic Allocation Flexibility 2nd Best Probability= 0.5 ⟶ Complete Randomization (for 2 treatment arms)Optimizing Clinical Trials: Concept to Conclusion™ © 2012 Medidata Solutions, Inc. § 23
  • 24. Stratification FactorsOptimizing Clinical Trials: Concept to Conclusion™ © 2012 Medidata Solutions, Inc. § 24
  • 25. Stratification Factors Over both sexes Factors Males Females ≣ Main Effects 18-35 yo Pla Strata Pla Pla Test Test Test ≣ 1st Order Interactions 35-65 yo Test Test Pla Pla Test Pla ce Randomizing a al Balan Marg in 25 yo Male: >65 yo To PLA Test Pla Test Pla Test ⟶ Worsens Male Pla balance lanceMarg inal Ba To Test Over all ⟶ Worsens Ages: Test Pla Pla Test 18-35yo balance lance Pla Test O verall BaOptimizing Clinical Trials: Concept to Conclusion™ © 2012 Medidata Solutions, Inc. § 25
  • 26. Permuted Block Stratified Randomization Over both sexes •  Only balances Males Females within strata T P P * Pla •  Most strata will 18-35 yo P T * * Test have incomplete blocks T P P T 35-65 yo T * * * Test Pla •  Imbalances accumulate at margins >65 yo T T P * P * * * Pla Test Over all Ages: Test Pla Pla Test Pla TestOptimizing Clinical Trials: Concept to Conclusion™ © 2012 Medidata Solutions, Inc. § 26
  • 27. Minimization & Marginal Balance * Only balances on margins Over both sexes * Useful if too many Males Females strata, e.g.: 18-35 yo Pla Pla Pla Test Test Test N # Strata > blocksize 35-65 yo Test Test Pla Test Pla Pla nce inal Bala * Appropriate for a Marg main effects analysis >65 yo Test Pla (ie, no interactions) Pla Test Pla Test BalanceMarg inalOver all Ages: Test Pla Test Pla Pla alance Test Overall BOptimizing Clinical Trials: Concept to Conclusion™ © 2012 Medidata Solutions, Inc. § 27
  • 28. Stratification & Dynamic Allocation Over both sexes DA: uses weighted Males Females combination of 18-35 yo Pla Pla •  Overall balance Pla Test Test Test •  Marginal balances 35-65 yo Test Test Pla Test •  Strata balance Pla Pla lance inal Ba Marg ⇒ Flexible >65 yo Test Pla Pla Test Pla Test Balance Marg inalOver all Ages: Test Pla Pla Test Pla alance Test Overall BOptimizing Clinical Trials: Concept to Conclusion™ © 2012 Medidata Solutions, Inc. § 28
  • 29. Site as a Special Subgroup (Max 2 lines, 35 characters)Optimizing Clinical Trials: Concept to Conclusion™ © 2012 Medidata Solutions, Inc. § 29
  • 30. Imbalance•  Overall balance •  Only an issue for small studies•  Subgroup Balance •  Fixed size studies can have variable sized subgroups ⟶ Increased risk of underpowered subgroups•  Site as special case of subgroup •  Small sites ⟶ Increased risk of "monotherapy” at site ⟶ Confounding site & treatment effects ⟶ Effectively non-informative/”lost” patients •  Actual vs Assumed distribution of site sizeOptimizing Clinical Trials: Concept to Conclusion™ © 2012 Medidata Solutions, Inc. § 30
  • 31. Enrollment per Center (Densities) Data Sample •  13 Studies •  7.7 mo Average Enrollment period •  3953 Obs.Pts •  460 Listed Sites •  372 Active.SitesSize Categories:{0, 1, 2, 3, 4-7, 8-11, 12-15, 16-19, 20-29, 30-39, 40-49,50-59, 60-79, 80-99, 100-149, 150-199, ≥200 }Optimizing Clinical Trials: Concept to Conclusion™ © 2012 Medidata Solutions, Inc. § 31
  • 32. Enrollment per Site (#Sites) Data Sample •  13 Studies •  7.7 mo Average Enrollment period •  3953 Obs.Pts •  460 Listed Sites •  372 Active.Sites # Sites per Size Category {0, 1, 2, 3, 4-7, 8-11, 12-15, 16-19, 20-29, 30-39, 40-49, 50-59, 60-79, 80-99, 100-149, 150-199, ≥200 }Optimizing Clinical Trials: Concept to Conclusion™ © 2012 Medidata Solutions, Inc. § 32
  • 33. Site Enrollment SimulationSimulation based on Observations•  4 mo Enrollment Period•  Enrollment ~ Poisson distribution µ = Obs. Pts/mo (active sites) or µ ≈ 0.5 / Enrollment period (non-active sites)•  Randomize using CR, PB(2:2), or DA(0.15). •  Confounded Pts ≣ Patients at centers with only one treatment ⇒ treatment & center effects are confoundedOptimizing Clinical Trials: Concept to Conclusion™ © 2012 Medidata Solutions, Inc. § 33
  • 34. Results mean ±SD (80% C.I.) •  Affected studies had many sites with low enrollment •  Studies with fewer sites (and more pts at each) were rarely affected •  Dynamic Allocation reduced confounding slightly more effectively than permuted blockOptimizing Clinical Trials: Concept to Conclusion™ © 2012 Medidata Solutions, Inc. § 34
  • 35. Performance Comparison (for two treatments)s of Efficiency (Atkinson, 1999) E (Y ) z XTreatment difference A constant term and k 2 Randomization prognostic factors Metrics Var ( ) z T z z T X ( XT X ) 1 X T z Loss Ln zT X (X T X) 1 X T zatients and k factors;a n k design matrix) 5 Optimizing Clinical Trials: Concept to Conclusion™ © 2012 Medidata Solutions, Inc. § 35
  • 36. Randomization Metrics How do we measure “badness” of a randomization sequence or system? •  Predictability •  Goal: an observer can guess no better than chance ⟶ Score based on Blackwell-Hodges guessing rule •  Easily calculated •  Imbalance Imbalance ⟶ reduced statistical power ⟶ “Loss of Efficiency” •  Measure as effective loss in number of subjectsOptimizing Clinical Trials: Concept to Conclusion™ © 2012 Medidata Solutions, Inc. § 36
  • 37. Blackwell-HodgesUse Blackwell-Hodges guessing rule •  Directly corresponds to game interpretation •  Investigator always guesses the most probable treatment assignment, based on past assignments •  “ bias factor F” F ≣ abs(# Correct – Expected # Correct by chance alone) •  Measures potential for selection bias •  Modifications: •  Limits on knowledge of investigator (eg, can only know prior treatment allocation on own site) •  Score as percentage e.g., Score ≣ abs(% Correct – 50%)Optimizing Clinical Trials: Concept to Conclusion™ © 2012 Medidata Solutions, Inc. § 37
  • 38. Blackwell-Hodges Scoring (1) For treatment sequence “TCCC” Initial guess ⟶ Expectation = ½ “T” ⟶ Imbalance =+1 ⟶ Guess C ⟶ Correct “TC” ⟶ Imbalance=0 ⟶ Guess either ⟶ Expectation=½ “TCC” ⟶ Imbalance=-1 ⟶ Guess T ⟶ Wrong “TCCC” ⟶ # Correct= ½ + 1+ ½ +0 =2 Score = #Correct - 2 = 2-2 = 0Optimizing Clinical Trials: Concept to Conclusion™ © 2012 Medidata Solutions, Inc. § 38
  • 39. Blackwell-Hodges Scoring (2) For treatment sequence “TCCC” “TCCC” ⟶ # Correct= ½ + 1+ ½ +0 =2 Complete Randomization ⇒ Pr{“TCCC”} = 1/16 Dynamic Allocation (p=0.15) ⇒ Pr{“TCCC”}= 0.5 *0.85 * 0.5 * 0.15 = 0.031875 Permuted Block (length≤4) ⇒ PR{“TCCC”} = 0 Strict Minimization ⇒ Pr{“TCCC”}=0Optimizing Clinical Trials: Concept to Conclusion™ © 2012 Medidata Solutions, Inc. § 39
  • 40. Blackwell-Hodges Scoring (3) Sequence “TCCT” # Correct= ½ + 1 + ½ + 1 = 3 Score = 3 – 2 = 1 •  Complete Randomization ⇒ Pr{TCCT}= 1/16 •  Strict Minimization ⇒ Pr{TCCT} = ½*1*½*1 = ¼ •  Permuted Block ⇒ Pr{TCCT} = 1/6 (NB: 6 permutations of TTCC) •  Dynamic Allocation (2nd best prob.=0.15) ⇒ Pr{TCCT} = 0.5 * 0.85* 0.5 * 0.85 = 0.180625Optimizing Clinical Trials: Concept to Conclusion™ © 2012 Medidata Solutions, Inc. § 40
  • 41. Warning! Blackwell-Hodges •  Assesses potential selection bias ― Given known imbalance! ¿¿ But which imbalance(s)?? (Overall imbalance? Within strata? Within Factors?) •  Henceforth: only use imbalance within strata •  Proxy for center •  Assume observer only knows imbalance within “his center” Local •  Simple & unambiguous Predictability M Requires some caution ONLY in interpretationOptimizing Clinical Trials: Concept to Conclusion™ © 2012 Medidata Solutions, Inc. § 41
  • 42. Loss of EfficiencyComparison Performance (for two treatments) Loss of Efficiency (Atkinson, 1999) Inference in Covariate-Adaptive E (Y ) z X allocation Treatment difference Elsa Valdés Márquez & Nick Fieller A constant term and k prognostic factors EFSPI Adaptive Randomisation Meeting 2 Brussels, 7 December 2006 Var ( ) z T z z T X ( XT X ) 1 X T z Loss Ln zT X (X T X) 1 X T z (for n patients and k factors; X a n k design matrix) •  Loss can be expressed as equivalent # Patients 5 •  In a 100 patient study: Loss of Efficiency= 5 ⇒ A perfectly designed study would require only 95http://www.efspi.org/PDF/activities/international/adaptive-rando-docs/2ValdesMarquez.pdfOptimizing Clinical Trials: Concept to Conclusion™ © 2012 Medidata Solutions, Inc. § 42
  • 43. 2 Var ( )RCT vs DOE z T z z T X ( XT X ) 1 X T z Loss Ln zT X (X T X) 1 X T z (for n patients and k factors; X a n k design matrix) X ≣ design matrix: 5 ⟶n rows, 1 per pt ⟶K columns, 1 per covariate z ≣ Treatment assignmentsDesigned Experiment (DOE): ⟶ Select z and covariate values to minimize LnRCT ⟶ Select only z (No control of covariates)Optimizing Clinical Trials: Concept to Conclusion™ © 2012 Medidata Solutions, Inc. § 43
  • 44. Loss of Efficiency (Máquez & Fieller) Performance Comparison Performance Comparison (for two treatments) Loss of efficiency of various methods Loss of Efficiency (Atkinson, 1999) CR: Complete Randomization E (Y ) z X TV: Minimization (Taves,1974) Dynamic PS:Minimization Treatment difference A constant term and k Allocation (Pocock & Simon, 1975) prognostic factors Ds: Ds-Optimum Design (Begg&Iglewicz, 1980) 2 Var ( ) Biased Coin Design 1 Sequentially DA: DA-Optimum zT z zT X( XT X ) Xassign Z (Atkinson,1982) T z to minimize Loss Ln zT X (X T X) 1 X T z (for n patients and k factors; THE BEST (without random elements) Simulated data:- X a n k design matrix) 100 subjects, 5 prognostic factors 6Optimizing Clinical Trials: Concept to Conclusion™ © 2012 Medidata Solutions, Inc. § 44
  • 45. Loss of Efficiency (Máquez & Fieller) Different factors and samples Covariate adaptive methods always more efficient than complete randomisation method with random element (PS) only efficient for larger sample sizes 1,000 group of patients 7Optimizing Clinical Trials: Concept to Conclusion™ © 2012 Medidata Solutions, Inc. § 45
  • 46. 25%# Dynamic)Alloca&on:)Readjus&ng)balance)for) discon&nuing)pa&ents) Randomization 20%# PB(2:2)# PB(2:2)# αδϕυστ( PB(2:2),#25%DC# Performance DA(0.15),#Eq.Wts#Poten&al)Selec&on)Bias) 15%# δισχοντινυ( DA(0.15),#Eq.Wts,#25%DC# DA(0.15),EqWts,Adj.25%DC# 10%# Simulations DA(0.15),#Margins# DA(0.15),#Margins,#25%#DC# DA(0.15),#Margins,Adj.25%Dc# 5%# CR# CR(25%DC)# νωο Δισχ.( CR# 0%# 0%# 5%# 10%# 15%# 20%# 25%# %)Loss)of)Efficiency))) Optimizing Clinical Trials: Concept to Conclusion™ © 2012 Medidata Solutions, Inc. § 46
  • 47. Simulation Set up 3 methods: 4 Measures: •  Complete Randomization •  Loss of Efficiency •  Permuted Block •  B-H Score (“Within Strata”) •  Dynamic Allocation •  Overall Imbalance •  Relative Loss of Efficiency vs CR Each simulated patient •  % Loss of Efficiency (of #pts) randomized w/ each method 6 Strata (Factors: Sex, Age) •  48 subjects Total •  33% or 50% Males •  With random 25% Dropout •  1:1:1, 1:1:2, 1:2:3 (Young : Middle : Old)Optimizing Clinical Trials: Concept to Conclusion™ © 2012 Medidata Solutions, Inc. § 47
  • 48. Note on Figures Simula&on)results)as)80%)Confidence)Intervals) 25%# Plot B-H score vs 20%# DA(0),#Margin#Balance# Loss of Efficiency PB(1:1)# 15%# DA(0),#Margin#Balance# MedianPoten&al)Selec&on)Bias) PB(1:1)# + 80% C.I. 10%# ⇒ 10% lower & 10% higher 5%# 0%# 0# 1# 2# 3# 4# 5# 6# Loss)of)Efficiency) Optimizing Clinical Trials: Concept to Conclusion™ © 2012 Medidata Solutions, Inc. § 48
  • 49. Simulation Results(1) Predictability %Imbalance Efficiency Loss ⟵AveragesDA(0.00) 22% 0.6% 0.87 of MetricsDA(0.15) 16% 1.6% 1.45DA(0.25) 13% 2.8% 1.99 But forDA(0.33) 8% 4.3% 2.64 managingDA(0.50) 4% 11.3% 4.99 risk, needCR 4% 11.4% 5.03 Worst CasePB(8:8) 7% 7.1% 3.00PB(4:4) 13% 4.9% 1.52PB(3:3) 16% 4.2% 1.13 80% ⟶PB(2:2) 19% 3.5% 0.79 Confidence IntervalsPB(1:1) 23% 2.6% 0.47 Both DA & PB are stratified. Simulation: 48 subjects, 2 stratification factors, 6 strata, uneven sizes (DA) Dynamic Allocation (PB) Permuted Block (CR) Completely Random DA( 2nd Best Probability ), PB( Allocation Ratio ) Simulated subjects were randomized by all 3 methodsOptimizing Clinical Trials: Concept to Conclusion™ © 2012 Medidata Solutions, Inc. § 49
  • 50. Randomizations Plotted by Metrics 25%# 25%# PB(1:1)# PB(1:1), PB(2:2)# PB(2:2), 20%# DA(0.00),#Wt(3:3:3)# DA(0) 20%# DA(0.15),#Wt(3:3:3)# DA(0.15) (Essentially Strict Poten&al)Selec&on)Bias) Poten&al)Selec&on)Bias) 15%# 15%# Minimization) 10%# 10%# 5%# 25%# 5%# 25%# PB(4:4)# PB(8:8)# DA(0.33),#Wt(3:3:3)# DA(0.50),#Wt(3:3:3)# 0%# 20%# CR# 0%# CR# 20%# DA(0.5) ≣ CR 0.000# 1.000# 2.000# 3.000# 4.000# 5.000# 6.000# 7.000# 8.000# 9.000# 10.000# 0.000# 1.000# 2.000# 3.000# 4.000# 5.000# 6.000# 7.000# 8.000# 9.000# 10.000# Loss)of)Efficiency) Loss)of)Efficiency) PB(4:4) PB⟶CR Poten&al)Selec&on)Bias) Poten&al)Selec&on)Bias) 15%# 15%# PB(8:8) 10%# 10%# DA(0.33) DA(0.5) 5%# 5%# CR 0%# CR 0%# 0.000# 1.000# 2.000# 3.000# 4.000# 5.000# 6.000# 7.000# 8.000# 9.000# 10.000# 0.000# 1.000# 2.000# 3.000# 4.000# 5.000# 6.000# 7.000# 8.000# 9.000# 10.000# Loss)of)Efficiency) Loss)of)Efficiency)Optimizing Clinical Trials: Concept to Conclusion™ © 2012 Medidata Solutions, Inc. § 50
  • 51. Correlation of Metrics Correlaons*of*Predictability*and*Loss*of*Efficiency* 0.40% 0.20% 0.00% % % % % DA CR% DA 0)% DA 5)% DA 5)% DA 3)% PB )% PB )% PB )% PB )% PB )% )% CR CR CR CR 0 :1 :2 :3 :4 :8 .0 .1 .2 .3 .5 (1 (2 (3 (4 (8 (0 (0 (0 (0 (0 !0.20% !0.40% !0.60% !0.80% !1.00%Optimizing Clinical Trials: Concept to Conclusion™ © 2012 Medidata Solutions, Inc. § 51
  • 52. Backup scatterplots 25%# PB(4:4)# DA(0.33),#Wt(3:3:3)# 20%# CR# 25%# PB(8:8)# DA(0.50),#Wt(3:3:3)# Poten&al)Selec&on)Bias) 15%# 20%# CR# 25%# PB(8:8) 10%# PB(3:3)# DA(0.25),#Wt(3:3:3)# Poten&al)Selec&on)Bias) 15%# 20%# CR# 10%# DA(0.5), 5%# CR Poten&al)Selec&on)Bias) 15%# 5%# 0%# 0.000# 1.000# 2.000# 3.000# PB(3:3), 4.000# 5.000# 6.000# 7.000# 8.000# 9.000# 10.000# 10%# DA(0.25) Loss)of)Efficiency) 0%# 0.000# 1.000# 2.000# 3.000# 4.000# 5.000# 6.000# 7.000# 8.000# 9.000# 10.000# Loss)of)Efficiency) 5%# 0%# 0.000# 1.000# 2.000# 3.000# 4.000# 5.000# 6.000# 7.000# 8.000# 9.000# 10.000# Loss)of)Efficiency)Optimizing Clinical Trials: Concept to Conclusion™ © 2012 Medidata Solutions, Inc. § 52
  • 53. Simulated Comparison 25%# Predictability,vs,Loss,of,Efficiency, 20%# Permuted#Block#{1:1,#2:2,#3:3,#4:4,#8:8}# Dynamic#{0%,#15%,#25%,33%,#50%}# DA(0.25)Predictability,Score, Complete#RandomizaGon# 15%# PB(3:3) •  1,000 simulations per case * 48 subjects each 10%# * 6 Strata, 2 factor, Variety of proportions 5%# 0%# 0.0# 1.0# 2.0# 3.0# 4.0# 5.0# 6.0# 7.0# 8.0# 9.0# Optimizing Clinical Trials: Concept to Conclusion™ Loss,of,Efficiency, © 2012 Medidata Solutions, Inc. § 53
  • 54. Simulated Comparison 25%# Predictability,vs,%,Loss,of,Efficiency, 20%# Permuted#Block#{1:1,#2:2,#3:3,#4:4,#8:8}# Dynamic#{0%,#15%,#25%,33%,#50%}# DA(0.25)Predictability,Score, Complete#RandomizaDon# 15%# PB(3:3) 10%# Loss of Efficiency %Loss of Efficiency = Sample Size 5%# 0%# 0%# 2%# 4%# 6%# 8%# 10%# 12%# 14%# 16%# 18%# 20%# Optimizing Clinical Trials: Concept to Conclusion™ %Loss,of,Efficiency, © 2012 Medidata Solutions, Inc. § 54
  • 55. Relative Loss of Efficiency 25%# Predictability,vs,Rela0ve,Loss,of,Efficiency,, •  20%# Permuted#Block#{1:1,#2:2,#3:3,#4:4,#8:8}# DA(0.25)Predictability,Score, Dynamic#{0%,#15%,#25%,33%,#50%}# 15%# PB(3:3) 10%# 5%# 0%# 0.00# 0.20# 0.40# 0.60# 0.80# 1.00# 1.20# 1.40# 1.60# 1.80# 2.00#Optimizing Clinical Trials: Concept to Conclusion™ Rela0ve,Loss,of,Efficiency, © 2012 Medidata Solutions, Inc. § 55
  • 56. Local Predictability ONLY Special TopicsOptimizing Clinical Trials: Concept to Conclusion™ © 2012 Medidata Solutions, Inc. § 56
  • 57. Dynamic Allocation Weighting Dynamic)Alloca&on)Weights) Dynamic)Alloca&on)Weights) 25%# 25%# Balancing)on){Strata,)Margin,)Overall}) versus)Permuted)Block,)Complete)Randomiza&on) PB(1:1)# PB(1:1)# 20%# 20%# DA(0),#Strata#Balance# DA(0),#Strata#Balance# DA(0),#Margin#Balance# DA(0),#Margin#Balance# DA(0),#Overall#Balance# DA(0),#Overall#Balance#Poten&al)Selec&on)Bias) Poten&al)Selec&on)Bias) 15%# CR# 15%# CR# 10%# 10%# 5%# 5%# 0%# 0%# 0# 1# 2# 3# 4# 5# 6# 7# 8# 9# 10# 0.00# 1.00# 2.00# 3.00# 4.00# 5.00# 6.00# 7.00# 8.00# 9.00# 10.00# Loss)of)Efficiency) Loss)of)Efficiency) DA(0) balanced only within strata ó Approximates PB(1:1) Local Predictability DA(0) equal weighting ó Approximates PB(1:1) ONLY DA(0) balanced on margins ó Intermediate properties DA(0) balanced only overall ó Approximates CR (large N) NB: Predictability is limited to imbalance within a stratum! Optimizing Clinical Trials: Concept to Conclusion™ © 2012 Medidata Solutions, Inc. § 57
  • 58. Dynamic Allocation Weighting Dynamic)Alloca&on) Weighting: 25%# Various)Weigh&ngs) (Strata, Margins, Overall) DA(0),#Strata#Balance# DA(0) Equal Weighting (1,1,1) 20%# DA(0),#Equal#WeighCng# ó Strata Balance Dominates DA(0),#Margin&Strata# ó Approximates PB(1:1) DA(0),#Unequal#WeighCng# Poten&al)Selec&on)Bias) 15%# DA(0),#Margin#Balance# DA(0) Margin & Strata (1:9:0) DA(0),#Overall#Balance# ó Separates from PB(1:1) 10%# DA(0) Unequal Weighting (1,6,20) DA(0) Margin Balance (0,1,0) 5%# DA(0) Overall Balance (0,0,1) ó Approx. CR 0%# Local 0.00# 1.00# 2.00# 3.00# 4.00# 5.00# 6.00# 7.00# 8.00# 9.00# 10.00# Predictability Loss)of)Efficiency) ONLYOptimizing Clinical Trials: Concept to Conclusion™ © 2012 Medidata Solutions, Inc. § 58
  • 59. addition way of usingstudy and factor imbalances. Furthermore, because of the importance of main- A to the overall a random element to prevent determinism and avoid potential bias.taining site balance and the fact that the International Conference on Harmonisation (ICH) guidelinesDA Algorithmemphasizeswe introduce a new a multicenter trial should be stratified by study sites (ICH E9, 1998) is hig Here, that randomization in generalized multidimensional dynamic allocation method that[12], the method here specifically singlesrandomizationsite imbalance in the scoring formula. flexible and can be applied to most out the overall scenarios. In this generalized MDA method, when a new subject c needs to be assigned to a study arm Ai , wecalculate the weighted sum of the distance measure factor imbalances. 2.1. Marginal imbalance as study, site, strata and Distance function ≣ Weighted Sum of Imbalances IMB.c; Ai / D is a key rIMB.Study.c/; Ai // C .wSTRATUM rIMB.St ratum.c/; Ai // Distance measure.wSTUDY component in DA methods. A number of distance measures have been p posed, including range, standard deviation and variance [3, 7]. In this paper we use the marginal bala C .wSITE rIMB.Site.c/; Ai // X function as another measure of imbalance. For a actor.v; c/; Alevel, marginal balance has been descri C .wFACTOR .v/ rIMB.F given factor i // (2) as evaluating the overall balance of treatment allocation [10], and here the marginal imbalance func 16v6K is defined as:wSTUDY ; wSTRATUM ; Imbalance: •  Relative wSITE are the weights assigned to the study, stratum, and site imbalance respec- ˇ ˇ X ˇ ˇ X Av t h C ı.i; j /D 1; : : : ; K. Similarlytively. Similarly, wFACTOR .v/ is the imbalance weight assigned the j factor, v ˇ ˇS t udy.c/ is the set of all subjects randomized before c ˇinto the study, S i t e.c/ is rj ˇ set of subjects rIMB.X; Ai / D the ˇ .kX k C 1/ ˇrandomized before c at c’s site, S t rat um.c/ is the subset of those that belong to the same site and share 16j 6Nthe same factor levels as c across all factors, and F act or.v; c/ is the set of all the already randomized where X share as Union of Strata already been randomized, kX k is the cardinality of •  Factor subset of the subjects c on the v factor.subjects thatis any the same level or state as that haveth⇒ set X , N is the number of arms in the study, for i D 1; : : : ; N , Ai is the set of subjects already assig P2.3. arm Ai , ri assignment using the arm weight, (or ratio) for arm Ai1(so to Treatment is the normalized generalized method , ri 1 1/, and ı.i; j / is D X= X ⇒ X ≥ X ⇒As expected of a DAk method, arms that provide the least imbalance are collected into the Kronecker delta.first-choice set:X ∈X k X +1 ≤ 16i 6N X +1 rIMB.X; Ai / provides a measure of the imbalance that would result from randomizing a new m k k ber of X into arm AiC.c/ Dmeasure is general, it does;:::;AN g IMB.c; Aj /gnumber of arms, and can han F . This fAi W IMB.c; Ai / D minfA1 not depend on the (3) ⇒ and uneven arm ratios. This dominate Distance functionthe new class of multi- both even Strata Imbalances feature makes it particularly useful forTo keep the study balanced, it is also that unlike other distance measures, the any one of the arms adaptive clinical trials. Note preferable that the subject c will be assigned to measure here is inversely pin F C.c/. to the size of X . This ensures that an imbalance of n > 0 subjects on a small group will ‘cou portional more than the method allows for the incorporation of a random element, a ‘Second Best Probability’ However, an n subject imbalance on a larger group.parameter that sets the ConclusionOptimizing Clinical Trials: Concept to probability that even when there is just one best minimizing arm, 2012 Medidata Solutions, Inc. § 59 ™ © that arm will
  • 60. Weighting Over both sexes Males Females 18-35 yo Pla Test Pla Test Pla Test •  Stratified Randomization weights35-65 yo Pla Test Pla Test Pla Test on strata, not margins or overall Over both sexes Males Females •  Imbalances within strata tend to >65 yo Test Pla 18-35 yo Pla Pla Pla Test Pla Test Pla Test Test TestOver all Ages: Test Pla dominate in DA 35-65 yo Test Pla Test Test Pla Test Pla Pla Pla Test >65 yo •  Minimization weights on margins, not strata. Test Pla Pla Test Pla Test •  DA can weight exclusively on margins Over all Ages: Test Pla Pla Test Pla Test Over both sexes Males Females 18-35 yo Pla Test Pla Test Pla Test •  If a Strata is balanced, the next assignment 35-65 yo Test Pla Test Pla Test attempts to balance the margins. Pla >65 yo Pla Test Pla Test Pla Test •  Since small groups are more likely to have Over all Ages: Test Pla imbalances which reduce efficiency, balancing strata 1st is appropriate Pla Test Pla TestOptimizing Clinical Trials: Concept to Conclusion™ © 2012 Medidata Solutions, Inc. § 60
  • 61. Hierarchical Balancing •  While Imbalances within strata tends to dominate in DA, if a Strata is balanced, the next assignment attempts to balance the margins •  Since small group imbalances tend to dominate, balancing tends to be sequential Males Females Over both sexes ⟵ This example: 18-35 yo Pla Test Pla Test Pla Test (1)  Balance within strata 35-65 yo Test Pla Pla Test Pla Test (2)  If balanced within the strata, balance by age group >65 yo Pla Test Pla Test Pla Test (since age groups tend to be smaller than sex groups) Over all (3)  If balanced within age group, balance within sex group Ages: Test Pla Pla Test Pla Test (4)  If balanced within sex group, balance overall However: cumulative imbalances may change this orderOptimizing Clinical Trials: Concept to Conclusion™ © 2012 Medidata Solutions, Inc. § 61
  • 62. ?" Replacement Randomization !!! ! !! !Optimizing Clinical Trials: Concept to Conclusion™ © 2012 Medidata Solutions, Inc. § 62
  • 63. Dynamically Adapting to Dropouts Patients discontinue 25%# Effect)of)Drop9outs)on)Permuted)Block)and)Dynamic) ⟶ Imbalances Alloca&on) ⟶ Reduced efficiency 20%# PB(2:2)# 25% DC PB(2:2)# “Tight” randomizations (PB with small blocks, PB(2:2),#25%DC# DA with small 2nd best Prob.) ⟶ Lose morePoten&al)Selec&on)Bias) 15%# DA(0.15),#Eq.Wts# efficiency DA(0.15),#Eq.Wts,#25%DC# 10%# DA(0.15),#Margins# “Loose” randomizations DA(0.15),#Margins,#25%#DC# (CR, PB with large blocks, DA with large 2nd best Prob.) CR# 5%# ⟶ Lose less efficiency CR(25%DC)# ⟶ Little or no change CR# No DC 0%# 0%# 5%# 10%# 15%# 20%# 25%# %)Loss)of)Efficiency))) Optimizing Clinical Trials: Concept to Conclusion™ © 2012 Medidata Solutions, Inc. § 63
  • 64. Dynamically Adapting to Dropouts Dynamic Allocation: Can Effect)of)Drop9outs)&)Rerandomiza&on)) allocate new patients to 24%$ on)Permuted)Block)and)Dynamic)Alloca&on) restore balance PB(2:2)$ PB(2:2)$ 22%$ PB(2:2),$25%DC$ DA(0.15),$Eq.Wts$ DA(0.15),$Eq.Wts,$25%DC$ DA(0.15),EqWts,Adj.25%DC$ 20%$Poten&al)Selec&on)Bias) 18%$ 25% DC 16%$ 14%$ DA Adj. No DC 12%$ 0%$ 1%$ 2%$ 3%$ 4%$ 5%$ 6%$ 7%$ 8%$ %)Loss)of)Efficiency))) Optimizing Clinical Trials: Concept to Conclusion™ © 2012 Medidata Solutions, Inc. § 64
  • 65. Dynamically Adapting to Dropouts 25%# Dynamic)Alloca&on:)Readjus&ng)balance)for) discon&nuing)pa&ents) “Tight” randomizations (PB with small blocks, 20%# PB(2:2)# DA with small 2nd best Prob.) PB(2:2)# ⟶ Lose more efficiency DA Adj. PB(2:2),#25%DC# ⟶ Benefit most DA(0.15),#Eq.Wts#Poten&al)Selec&on)Bias) 15%# 25% DC DA(0.15),#Eq.Wts,#25%DC# “Loose” DA(0.15),EqWts,Adj.25%DC# randomizations (CR, PB with large blocks, 10%# DA(0.15),#Margins# DA with large 2nd best Prob.) DA(0.15),#Margins,#25%#DC# ⟶ Lose less efficiency ⟶ Little or no benefit DA(0.15),#Margins,Adj.25%Dc# 5%# CR# CR(25%DC)# No DC CR# 0%# 0%# 5%# 10%# 15%# 20%# 25%# %)Loss)of)Efficiency))) Optimizing Clinical Trials: Concept to Conclusion™ © 2012 Medidata Solutions, Inc. § 65
  • 66. Applications •  High drop-out ⇒ PB, DA ⟶ CR •  Drop-out before becoming evaluable •  Constrained resources (small sample size, limited drug supply, ….) •  Crossover studies: Requires completers •  Evaluable ó Complete Sequence of Treatments •  Provisional Randomization / Randomize to ship •  Screening visit triggers: •  Randomize at screening •  If randomized treatment not on-site, ship blinded supplies •  Randomization visit: •  If patient eligible ⇒ dispense assigned treatment •  If not eligible ⇒store for next eligible patient •  Minimizes on-site drug supplyOptimizing Clinical Trials: Concept to Conclusion™ © 2012 Medidata Solutions, Inc. § 66
  • 67. Randomization Optimization Factors•  Equipose ⇒ (less random is acceptable)•  Small Study ⇒ Efficiency important ⟶ Lower 2nd Best Probability•  Large Study ⇒ Are there small subgroups? All subgroups large ⟶ CR is acceptable•  Small subgroups ⇒ Need more efficiency ⟶ Smaller 2nd best ProbOptimizing Clinical Trials: Concept to Conclusion™ © 2012 Medidata Solutions, Inc. § 67
  • 68. Balancing Considerations •  Large Studies •  Smaller Studies •  Studies with large subgroups •  Studies with small •  Late phase studies subgroups •  Strong Treatment preferences •  Early phase studies •  Weak Blinding •  Interim Analyses •  Subjective Endpoints •  Equipoise •  Strong Blinding •  Objective Endpoints •  Many Strata / Many centers •  Limited blinded supplies Unpredictable ⟵ ⟶ ⟶ ⟶ Balanced ⟵⟵Optimizing Clinical Trials: Concept to Conclusion™ © 2012 Medidata Solutions, Inc. § 68
  • 69. BibliographyElsa Valdés Márquez & Nick Fieller. Inference inCovariate-Adaptive allocation. EFSPI AdaptiveRandomisation Meeting, Brussels, 7 December 2006Optimizing Clinical Trials: Concept to Conclusion™ © 2012 Medidata Solutions, Inc. § 69

×