Randomization: Too Important to Gamble with.

Randomization:
Too Important to Gamble with

A Presentation for the Delaware Chapter of the ASA
Oct 18, 2012

Dennis Sweitzer, Ph.D., Principal Biostatistician
Medidata Randomization Center of Excellence

Optimizing Clinical Trials:
Concept to Conclusion™

Optimizing Clinical Trials: Concept to Conclusion™ © 2012 Medidata Solutions, Inc. § 1

Outline
Randomized Controlled Trials
•  Basics
•  Balance
Randomization methods
•  Complete Randomization
•  Strict Minimization
•  Permuted Block
•  Dynamic Allocation (Covariate-adaptive, not Response-Adaptive)
Randomization Metrics
•  Balance
•  Predictability
•  Loss of Power /Loss of Efficiency
•  Secondary Imbalance: drop-outs
Simulations comparing methods
•  Confounding site & treatment effects (small sites)
•  Overall performance
•  Discontinuing patients
•  Weighting stratification factors
Meta-Balance


Why randomize
anyway?
Some basic principles


Why Gold Standard?

Randomized Controlled Trial
•  Trial:Prospective & Specific
•  Controlled:
•  Comparison with Control group
•  (placebo or active)
•  Controlled procedures ⇒ Only Test Treatment Varies
•  Randomization: Minimizes biases
•  Allocation bias
•  Selection bias
•  Permits blinding


Eliminating Bias
¿ The Fact of bias ?
•  (conscious, unconscious, or instinctive)
¿ The Question of bias ?
•  Always 2nd guessing
•  Critics will think of unanticipated things

¡ Solution !
•  Treat it as a game
•  1 statistician vs N clinicians
•  Statistician generates a random sequence
•  Clinicians sequential guess at each assignment
•  Statistician wins if clinician guesses are no better than chance
(NB: 75% wrong is just as bad as 75% right)



What do we want in a randomization sequence or
system?
Randomness ó Unpredictable
⟶ Reduce Allocation Bias (All studies)
⟶ Reduce Selection Bias (All studies)
⟶ Reduce placebo effects (Blinded studies)
Balance ó “Loss of Efficiency”
⟶ Maximizes statistical power
⟶ Minimize Confounding
⟶ Enhance Credibility (Face Validity)


Balancing


Balanced Study

Equal allocation
between
treatment arms
•  Maximizes
Statistical
Power

Control Test


Imbalanced
Statistical power limited by
smallest arm
•  36 subject simulation with
Complete Randomization
⟶
average loss ≈ 1 subject
10% lose ≥2 subject
•  Can add 2 to compensate
•  BUT only large imbalances
have much effect
on statistical power
Resulting in light weight results….

Severe Imbalances are rare in large studies
Pr{worse than 60:40 split} for:
•  n=25 ⟶ <42% n=100 ⟶ <4.4% n=400 ⟶ 0.006%


(NB: Planned Imbalance)

1:1 randomization maximizes power per patient
But there are other considerations
•  Utility:
•  Need 100 patients on drug to monitor safety
•  Study only requires 60 (30/arm)
•  2:1 randomization ⟶ 100 Test & 50 Placebo

•  Motivation:
•  Better enrollment if 75% chance of Test drug (3:1)
•  Ethics:
•  85 Placebo + 255 Test vs. 125 Placebo + 125 Test


Imbalance

•  Overall balance
•  Only an issue for small studies
•  Subgroup Balance
•  Fixed size studies can have variable sized subgroups
⟶ Increased risk of underpowered subgroups


Effective Loss of Sample Size
Effective Loss = Reduction of Power
Females Male as Reduction in Sample Size
s
Test
Simulations of:
Pla
•  36 and 18 subjects,
•  males as strata at 33% of population,
Test Con •  randomized 1:1
•  (complete randomization)

N=36 N=18
Overall Females Males Overall Females Males
Effectively Lost
Mean ± SD 1.0 ±1.4 0.9 ±1.3 1.0 ±1.4 1.0 ±1.4 1.0 ±1.4 1.0 ±1.3
≥2 pts 12% 14% 18% 23% 16% 17%
≥4 pts 6% 4% 5% 3% 4% 5%
>=100% 0.0% 0.0% 0.4% 0.0% 0.5% 7.9%
Q1 0.11 0.15 0.09 0.22 0.09 0.14
Median 0.44 0.43 0.47 0.22 0.40 0.50
Q3 1.00 1.19 1.33 0.89 1.33 1.29
Imbalance (% of N)
Mean ± SD 13% ±10% 16% ±12% 25% ±19% 18% ±15% 23% ±18% 35% ±28%
>=50% 0.5% 1.6% 12.8% 3.1% 10.0% 27.9%
Q1 6% 8% 9% 11% 9% 14%
Median 11% 14% 20% 11% 20% 33%
Q3 17% 22% 33% 22% 33% 50%


Bad Imbalance!

Males
Females Treatment
Imbalances
Pla
Test within factors
⟶ spurious
Test Pla findings…..

Leads to conversations like:
ANCOVA
Higher estrogen showed no
levels in patients Credibility…..
differences in
on Test estrogen
Treatment ??
levels due to
Hmm… treatment
?

?"
Randomization
!!!
! !!
!
Methods

(See Animated Powerpoint Slides…)


Randomization
4 methods
•  Complete Randomization (classic approach)
•  Strict Minimization
•  Permuted Block (frequently used)
•  Dynamic Allocation (gaining in popularity)



Every assignment
•  Same probability for each assignment
•  Ignore Treatment Imbalances
•  No restrictions on treatment assignments
Advantages:
•  Simple
•  Robust against selection & accidental bias
•  Maximum Unpredictability
Disadvantage
•  High likelihood of imbalances (smaller samples)

.

Minimization

Strict Minimization
randomizes to the
imbalanced arm


Minimization

Strict Minimization
rebalances the Arms
•  BUT at a cost in
predictability
•  Random only when
treatments are currently
balanced


Permuted Block

Blocks of Patients
(1, 2, or 3 per treatment)
Here: 2:2 Allocation
T P
P ?

T P
P T

T P (Unless Incomplete
P * Blocks:
More strata
⟶ More incomplete)


Dynamic Allocation

Biases Randomization to
the imbalanced arm
•  Unpredictable
•  Almost Balanced


Dynamic Allocation

•  Optimizes Unpredictability
•  Ignores Balance

Strict Minimization
•  Optimizes Balance
•  Ignores Predictability

Dynamic Allocation
2nd Best Probability Parameter
Controls
Balance vs. Predictability
Tradeoff


Dynamic Allocation Flexibility

2nd Best Probability= 0
⟶ Strict Minimization


Dynamic Allocation Flexibility

2nd Best Probability= 0.5
⟶ Complete
Randomization
(for 2 treatment arms)


Stratification Factors


Stratification Factors
Over both sexes Factors
Males Females
≣ Main Effects
18-35 yo Pla
Strata
Pla
Pla Test Test Test ≣ 1st Order
Interactions
35-65 yo Test Test
Pla
Pla Test Pla ce Randomizing a
al Balan
Marg in 25 yo Male:
>65 yo To PLA
Test Pla
Test Pla Test ⟶ Worsens Male
Pla
balance
lance
Marg inal Ba To Test
Over all ⟶ Worsens
Ages: Test Pla Pla Test 18-35yo balance
lance
Pla Test O verall Ba

Permuted Block Stratified Randomization

Over both sexes •  Only balances
Males Females within strata

T P P * Pla •  Most strata will
18-35 yo
P T * *
Test have incomplete
blocks
T P P T
35-65 yo T * * * Test
Pla •  Imbalances
accumulate at
margins
>65 yo T T P *
P * * *
Pla Test

Over all
Ages:
Test Pla
Pla Test
Pla Test


Minimization & Marginal Balance

* Only balances on
margins
Over both sexes * Useful if too many
Males Females
strata, e.g.:
18-35 yo Pla Pla
Pla Test Test Test
N
# Strata >
blocksize
35-65 yo Test Test
Pla Test
Pla Pla nce
inal Bala * Appropriate for a
Marg
main effects analysis
>65 yo
Test Pla (ie, no interactions)
Pla Test Pla Test

Balance
Marg inal
Over all
Ages: Test Pla Test
Pla
Pla alance
Test Overall B


Stratification & Dynamic Allocation

Over both sexes DA: uses weighted
Males Females combination
of
18-35 yo Pla Pla •  Overall balance
Pla Test Test Test
•  Marginal
balances
35-65 yo Test Test
Pla Test •  Strata balance
Pla
Pla lance
inal Ba
Marg ⇒ Flexible
>65 yo
Test Pla
Pla Test Pla Test

Balance
Marg inal
Over all
Ages:
Test Pla
Pla Test
Pla alance
Test Overall B


Site as a Special
Subgroup
(Max 2 lines, 35 characters)


Imbalance

•  Overall balance
•  Only an issue for small studies
•  Subgroup Balance
•  Fixed size studies can have variable sized subgroups
⟶ Increased risk of underpowered subgroups
•  Site as special case of subgroup
•  Small sites ⟶ Increased risk of "monotherapy” at site
⟶ Confounding site & treatment effects
⟶ Effectively non-informative/”lost” patients
•  Actual vs Assumed distribution of site size


Enrollment per Center (Densities)

Data Sample
•  13 Studies
•  7.7 mo Average Enrollment period
•  3953 Obs.Pts
•  460 Listed Sites
•  372 Active.Sites
Size Categories:
{0, 1, 2, 3, 4-7, 8-11, 12-15, 16-19, 20-29, 30-39, 40-49,
50-59, 60-79, 80-99, 100-149, 150-199, ≥200 }


Enrollment per Site (#Sites)

Data Sample
•  13 Studies
•  7.7 mo Average Enrollment period
•  3953 Obs.Pts
•  460 Listed Sites
•  372 Active.Sites
# Sites per Size Category {0, 1, 2, 3, 4-7, 8-11, 12-15,
16-19, 20-29, 30-39, 40-49, 50-59, 60-79, 80-99, 100-149, 150-199, ≥200 }


Site Enrollment Simulation

Simulation based on Observations
•  4 mo Enrollment Period
•  Enrollment ~ Poisson distribution
µ = Obs. Pts/mo (active sites)
or
µ ≈ 0.5 / Enrollment period (non-active sites)
•  Randomize using CR, PB(2:2), or DA(0.15).
•  Confounded Pts ≣ Patients at centers with only one
treatment
⇒ treatment & center effects are confounded


Results
mean ±SD
(80% C.I.)

•  Affected studies had many sites with low enrollment
•  Studies with fewer sites (and more pts at each) were rarely affected
•  Dynamic Allocation reduced confounding slightly more effectively than
permuted block


Performance Comparison
(for two treatments)
s of Efficiency (Atkinson, 1999)

E (Y ) z X
Treatment difference A constant term and k

2
Randomization
prognostic factors

Metrics
Var ( )
z T z z T X ( XT X ) 1 X T z

Loss Ln zT X (X T X) 1 X T z
atients and k factors;
a n k design matrix)

5



How do we measure “badness” of a randomization
sequence or system?
•  Predictability
•  Goal: an observer can guess no better than chance
⟶ Score based on Blackwell-Hodges guessing rule
•  Easily calculated
•  Imbalance
Imbalance ⟶ reduced statistical power
⟶ “Loss of Efficiency”
•  Measure as effective loss in number of subjects


Blackwell-Hodges
Use Blackwell-Hodges guessing rule
•  Directly corresponds to game interpretation
•  Investigator always guesses the most probable treatment
assignment, based on past assignments
•  “ bias factor F”
F ≣ abs(# Correct – Expected # Correct by chance alone)
•  Measures potential for selection bias
•  Modifications:
•  Limits on knowledge of investigator (eg, can only know
prior treatment allocation on own site)
•  Score as percentage
e.g., Score ≣ abs(% Correct – 50%)


Blackwell-Hodges Scoring (1)

For treatment sequence “TCCC”

Initial guess ⟶ Expectation = ½
“T” ⟶ Imbalance =+1 ⟶ Guess C ⟶ Correct
“TC” ⟶ Imbalance=0 ⟶ Guess either
⟶ Expectation=½
“TCC” ⟶ Imbalance=-1 ⟶ Guess T ⟶ Wrong
“TCCC” ⟶ # Correct= ½ + 1+ ½ +0 =2
Score = #Correct - 2 = 2-2 = 0



For treatment sequence “TCCC”
“TCCC” ⟶ # Correct= ½ + 1+ ½ +0 =2

Complete Randomization ⇒ Pr{“TCCC”} = 1/16

Dynamic Allocation (p=0.15)
⇒ Pr{“TCCC”}= 0.5 *0.85 * 0.5 * 0.15 = 0.031875

Permuted Block (length≤4) ⇒ PR{“TCCC”} = 0
Strict Minimization ⇒ Pr{“TCCC”}=0



Sequence “TCCT”
# Correct= ½ + 1 + ½ + 1 = 3
Score = 3 – 2 = 1

•  Complete Randomization ⇒ Pr{TCCT}= 1/16
•  Strict Minimization ⇒ Pr{TCCT} = ½*1*½*1 = ¼
•  Permuted Block ⇒ Pr{TCCT} = 1/6
(NB: 6 permutations of TTCC)
•  Dynamic Allocation (2nd best prob.=0.15)
⇒ Pr{TCCT} = 0.5 * 0.85* 0.5 * 0.85 = 0.180625


Warning!

Blackwell-Hodges
•  Assesses potential selection bias
― Given known imbalance!
¿¿ But which imbalance(s)??
(Overall imbalance? Within strata? Within Factors?)
•  Henceforth: only use imbalance within strata
•  Proxy for center
•  Assume observer only knows
imbalance within “his center” Local
•  Simple & unambiguous Predictability
M Requires some caution ONLY
in interpretation


Loss of EfficiencyComparison
Performance
Loss of Efficiency (Atkinson, 1999)
Inference in Covariate-Adaptive
E (Y ) z X allocation

Treatment difference Elsa Valdés Márquez & Nick Fieller
A constant term and k
prognostic factors EFSPI Adaptive Randomisation
Meeting
2 Brussels, 7 December 2006
Var ( )
z T z z T X ( XT X ) 1 X T z

(for n patients and k factors;
X a n k design matrix)
•  Loss can be expressed as equivalent # Patients
5

•  In a 100 patient study:
Loss of Efficiency= 5
⇒ A perfectly designed study would require only 95
http://www.efspi.org/PDF/activities/international/adaptive-rando-docs/2ValdesMarquez.pdf


2
Var ( )
RCT vs DOE z T z z T X ( XT X ) 1 X T z


X ≣ design matrix: 5

⟶n rows, 1 per pt
⟶K columns,
1 per covariate
z ≣ Treatment
assignments
Designed Experiment (DOE):
⟶ Select z and covariate values to minimize Ln
RCT ⟶ Select only z (No control of covariates)

Loss of Efficiency (Máquez & Fieller)
Loss of efficiency of various methods
Loss of Efficiency (Atkinson, 1999)
CR: Complete Randomization
E (Y ) z X
TV: Minimization (Taves,1974)
Dynamic
PS:Minimization
Treatment difference A constant term and k
Allocation
(Pocock & Simon, 1975)
prognostic factors
Ds: Ds-Optimum Design
(Begg&Iglewicz, 1980)
2
Var ( ) Biased Coin Design 1 Sequentially
DA: DA-Optimum
zT z zT X( XT X ) Xassign Z
(Atkinson,1982)
T
z
to minimize
THE BEST
(without random elements) Simulated data:-
100 subjects, 5 prognostic factors
6

Loss of Efficiency (Máquez & Fieller)
Different factors and samples

Covariate adaptive methods always more
efficient than complete randomisation

method with random element (PS)
only efficient for larger sample sizes
1,000 group of patients

7


25%#

Dynamic)Alloca&on:)Readjus&ng)balance)for)
discon&nuing)pa&ents)

Randomization
20%# PB(2:2)#

PB(2:2)#

αδϕυστ( PB(2:2),#25%DC#

Performance
DA(0.15),#Eq.Wts#
Poten&al)Selec&on)Bias)

15%#

δισχοντινυ( DA(0.15),#Eq.Wts,#25%DC#

DA(0.15),EqWts,Adj.25%DC#

10%#

Simulations
DA(0.15),#Margins#

DA(0.15),#Margins,#25%#DC#

DA(0.15),#Margins,Adj.25%Dc#

5%# CR#

CR(25%DC)#
νωο Δισχ.(
CR#

0%#
0%# 5%# 10%# 15%# 20%# 25%#
%)Loss)of)Eﬃciency)))


Simulation Set up

3 methods: 4 Measures:
•  Complete Randomization •  Loss of Efficiency
•  Permuted Block •  B-H Score (“Within Strata”)
•  Dynamic Allocation •  Overall Imbalance
•  Relative Loss of Efficiency vs CR
Each simulated patient •  % Loss of Efficiency (of #pts)
randomized w/ each method

6 Strata (Factors: Sex, Age) •  48 subjects Total
•  33% or 50% Males •  With random 25% Dropout
•  1:1:1, 1:1:2, 1:2:3
(Young : Middle : Old)


Note on Figures
Simula&on)results)as)80%)Conﬁdence)Intervals)
25%#

Plot B-H score
vs
20%#
DA(0),#Margin#Balance#
Loss of Efficiency
PB(1:1)#

15%#
DA(0),#Margin#Balance# Median

PB(1:1)# +
80% C.I.
10%# ⇒
10% lower
& 10% higher
5%#

0%#
0# 1# 2# 3# 4# 5# 6#
Loss)of)Eﬃciency)

Simulation Results(1)

Predictability %Imbalance Efficiency Loss ⟵Averages
DA(0.00) 22% 0.6% 0.87 of Metrics
DA(0.15) 16% 1.6% 1.45
DA(0.25) 13% 2.8% 1.99
But for
DA(0.33) 8% 4.3% 2.64 managing
DA(0.50) 4% 11.3% 4.99 risk, need
CR 4% 11.4% 5.03 Worst Case
PB(8:8) 7% 7.1% 3.00
PB(4:4) 13% 4.9% 1.52
PB(3:3) 16% 4.2% 1.13 80% ⟶
PB(2:2) 19% 3.5% 0.79 Confidence
Intervals
PB(1:1) 23% 2.6% 0.47

Both DA & PB are stratified.
Simulation: 48 subjects, 2 stratification factors, 6 strata, uneven sizes
(DA) Dynamic Allocation (PB) Permuted Block (CR) Completely Random
DA( 2nd Best Probability ), PB( Allocation Ratio )
Simulated subjects were randomized by all 3 methods

Randomizations Plotted by Metrics
25%# 25%#

PB(1:1)# PB(1:1), PB(2:2)#
PB(2:2),
20%#
DA(0.00),#Wt(3:3:3)#
DA(0) 20%#
DA(0.15),#Wt(3:3:3)#

DA(0.15)

(Essentially Strict

15%# 15%#

Minimization)
10%# 10%#

5%#
25%# 5%#
25%#

PB(4:4)# PB(8:8)#
DA(0.33),#Wt(3:3:3)# DA(0.50),#Wt(3:3:3)#

0%#
20%# CR# 0%# CR#
20%#

DA(0.5) ≣ CR
0.000# 1.000# 2.000# 3.000# 4.000# 5.000# 6.000# 7.000# 8.000# 9.000# 10.000# 0.000# 1.000# 2.000# 3.000# 4.000# 5.000# 6.000# 7.000# 8.000# 9.000# 10.000#
Loss)of)Efficiency) Loss)of)Efficiency)

PB(4:4) PB⟶CR

15%# 15%#

PB(8:8)
10%# 10%#

DA(0.33) DA(0.5)
5%# 5%#

CR
0%#
CR
0%#
0.000# 1.000# 2.000# 3.000# 4.000# 5.000# 6.000# 7.000# 8.000# 9.000# 10.000#
0.000# 1.000# 2.000# 3.000# 4.000# 5.000# 6.000# 7.000# 8.000# 9.000# 10.000#
Loss)of)Efficiency)
Loss)of)Efficiency)


Correlation of Metrics

Correla'ons*of*Predictability*and*Loss*of*Eﬃciency*
0.40%

0.20%

0.00%
%

%

%

%
DA CR%

DA 0)%

DA 5)%

DA 5)%

DA 3)%

PB )%

PB )%

PB )%

PB )%

PB )%
)%
CR

CR

CR

CR

0

:1

:2

:3

:4

:8
.0

.1

.2

.3

.5
(1

(2

(3

(4

(8
(0

(0

(0

(0

(0
!0.20%

!0.40%

!0.60%

!0.80%

!1.00%


Backup scatterplots
25%#

PB(4:4)#

DA(0.33),#Wt(3:3:3)#

20%# CR#

25%#

PB(8:8)#

DA(0.50),#Wt(3:3:3)#

15%#
20%# CR#
25%#

PB(8:8) 10%#
PB(3:3)#

DA(0.25),#Wt(3:3:3)#

15%#
20%# CR#

10%#
DA(0.5), 5%#

CR

15%#

5%#
0%#
0.000# 1.000# 2.000# 3.000#
PB(3:3),
4.000# 5.000# 6.000# 7.000# 8.000# 9.000# 10.000#
10%#
DA(0.25)
Loss)of)Efficiency)

0%#
0.000# 1.000# 2.000# 3.000# 4.000# 5.000# 6.000# 7.000# 8.000# 9.000# 10.000#
Loss)of)Efficiency) 5%#

0%#
0.000# 1.000# 2.000# 3.000# 4.000# 5.000# 6.000# 7.000# 8.000# 9.000# 10.000#
Loss)of)Efficiency)


Simulated Comparison
25%#

Predictability,vs,Loss,of,Eﬃciency,

20%#
Permuted#Block#{1:1,#2:2,#3:3,#4:4,#8:8}#

Dynamic#{0%,#15%,#25%,33%,#50%}#
DA(0.25)
Predictability,Score,

Complete#RandomizaGon#
15%# PB(3:3)

•  1,000 simulations per case
* 48 subjects each
10%#
* 6 Strata, 2 factor,
Variety of proportions
5%#

0%#
0.0# 1.0# 2.0# 3.0# 4.0# 5.0# 6.0# 7.0# 8.0# 9.0#

Optimizing Clinical Trials: Concept to Conclusion™ Loss,of,Eﬃciency, © 2012 Medidata Solutions, Inc. § 53

Simulated Comparison
25%#

Predictability,vs,%,Loss,of,Eﬃciency,

20%#
Permuted#Block#{1:1,#2:2,#3:3,#4:4,#8:8}#

Dynamic#{0%,#15%,#25%,33%,#50%}#
DA(0.25)

Complete#RandomizaDon#
15%# PB(3:3)

10%#

Loss of Efficiency
%Loss of Efficiency =
Sample Size
5%#

0%#
0%# 2%# 4%# 6%# 8%# 10%# 12%# 14%# 16%# 18%# 20%#

Optimizing Clinical Trials: Concept to Conclusion™ %Loss,of,Eﬃciency, © 2012 Medidata Solutions, Inc. § 54

Relative Loss of Efficiency
25%#

Predictability,vs,Rela0ve,Loss,of,Eﬃciency,,
• 
20%#
Permuted#Block#{1:1,#2:2,#3:3,#4:4,#8:8}#

DA(0.25)

Dynamic#{0%,#15%,#25%,33%,#50%}#
15%#
PB(3:3)

10%#

5%#

0%#
0.00# 0.20# 0.40# 0.60# 0.80# 1.00# 1.20# 1.40# 1.60# 1.80# 2.00#

Optimizing Clinical Trials: Concept to Conclusion™ Rela0ve,Loss,of,Eﬃciency, © 2012 Medidata Solutions, Inc. § 55

Local
Predictability
ONLY Special Topics


Dynamic Allocation Weighting
Dynamic)Alloca&on)Weights) Dynamic)Alloca&on)Weights)
25%# 25%#
Balancing)on){Strata,)Margin,)Overall}) versus)Permuted)Block,)Complete)Randomiza&on)

PB(1:1)#
PB(1:1)#
20%# 20%#
DA(0),#Strata#Balance# DA(0),#Strata#Balance#
DA(0),#Margin#Balance# DA(0),#Margin#Balance#
DA(0),#Overall#Balance#
DA(0),#Overall#Balance#

15%# CR# 15%#

CR#

10%# 10%#

5%# 5%#

0%#
0%#
0# 1# 2# 3# 4# 5# 6# 7# 8# 9# 10#
0.00# 1.00# 2.00# 3.00# 4.00# 5.00# 6.00# 7.00# 8.00# 9.00# 10.00#
Loss)of)Eﬃciency)
Loss)of)Eﬃciency)

DA(0) balanced only within strata ó Approximates PB(1:1) Local
Predictability
DA(0) equal weighting ó Approximates PB(1:1)
ONLY
DA(0) balanced on margins ó Intermediate properties
DA(0) balanced only overall ó Approximates CR (large N)
NB: Predictability is limited to imbalance within a stratum!

Dynamic Allocation Weighting
Dynamic)Alloca&on) Weighting:
25%#
Various)Weigh&ngs) (Strata, Margins, Overall)

DA(0),#Strata#Balance# DA(0) Equal Weighting (1,1,1)
20%# DA(0),#Equal#WeighCng# ó Strata Balance Dominates
DA(0),#Margin&Strata# ó Approximates PB(1:1)
DA(0),#Unequal#WeighCng#

15%# DA(0),#Margin#Balance# DA(0) Margin & Strata (1:9:0)
DA(0),#Overall#Balance# ó Separates from PB(1:1)

10%#
DA(0) Unequal Weighting (1,6,20)

DA(0) Margin Balance (0,1,0)

5%#
DA(0) Overall Balance (0,0,1)
ó Approx. CR

0%# Local
0.00# 1.00# 2.00# 3.00# 4.00# 5.00# 6.00# 7.00# 8.00# 9.00# 10.00# Predictability
Loss)of)Eﬃciency)
ONLY


addition way of usingstudy and factor imbalances. Furthermore, because of the importance of main-
A to the overall a random element to prevent determinism and avoid potential bias.
taining site balance and the fact that the International Conference on Harmonisation (ICH) guidelines
DA Algorithm
emphasizeswe introduce a new a multicenter trial should be stratified by study sites (ICH E9, 1998) is hig
Here, that randomization in generalized multidimensional dynamic allocation method that
[12], the method here specifically singlesrandomizationsite imbalance in the scoring formula.
flexible and can be applied to most out the overall scenarios.
In this generalized MDA method, when a new subject c needs to be assigned to a study arm Ai , we
calculate the weighted sum of the distance measure factor imbalances.
2.1. Marginal imbalance as study, site, strata and
Distance function ≣ Weighted Sum of Imbalances
IMB.c; Ai / D is a key rIMB.Study.c/; Ai // C .wSTRATUM rIMB.St ratum.c/; Ai //
Distance measure.wSTUDY component in DA methods. A number of distance measures have been p
posed, including range, standard deviation and variance [3, 7]. In this paper we use the marginal bala
C .wSITE rIMB.Site.c/; Ai //
X
function as another measure of imbalance. For a actor.v; c/; Alevel, marginal balance has been descri
C .wFACTOR .v/ rIMB.F given factor i // (2)
as evaluating the overall balance of treatment allocation [10], and here the marginal imbalance func
16v6K
is defined as:
wSTUDY ; wSTRATUM ; Imbalance:
•  Relative wSITE are the weights assigned to the study, stratum, and site imbalance respec-
ˇ ˇ
X ˇ ˇ X Av t h C ı.i; j /D 1; : : : ; K. Similarly
tively. Similarly, wFACTOR .v/ is the imbalance weight assigned the j factor, v ˇ
ˇ
S t udy.c/ is the set of all subjects randomized before c ˇinto the study, S i t e.c/ is rj ˇ set of subjects
rIMB.X; Ai / D the
ˇ .kX k C 1/ ˇ
randomized before c at c’s site, S t rat um.c/ is the subset of those that belong to the same site and share
16j 6N
the same factor levels as c across all factors, and F act or.v; c/ is the set of all the already randomized
where X share as Union of Strata already been randomized, kX k is the cardinality of
•  Factor subset of the subjects c on the v factor.
subjects thatis any the same level or state as that haveth⇒
set X , N is the number of arms in the study, for i D 1; : : : ; N , Ai is the set of subjects already assig
P
2.3. arm Ai , ri assignment using the arm weight, (or ratio) for arm Ai1(so
to Treatment is the normalized generalized method , ri 1 1/, and ı.i; j / is
D
X= X ⇒ X ≥ X ⇒
As expected of a DAk method, arms that provide the least imbalance are collected into the
Kronecker delta.
first-choice set:X ∈X
k
X +1
≤
16i 6N

X +1
rIMB.X; Ai / provides a measure of the imbalance that would result from randomizing a new m
k k
ber of X into arm AiC.c/ Dmeasure is general, it does;:::;AN g IMB.c; Aj /gnumber of arms, and can han
F . This fAi W IMB.c; Ai / D minfA1 not depend on the (3)
⇒ and uneven arm ratios. This dominate Distance functionthe new class of multi-
both even Strata Imbalances feature makes it particularly useful for
To keep the study balanced, it is also that unlike other distance measures, the any one of the arms
adaptive clinical trials. Note preferable that the subject c will be assigned to measure here is inversely p
in F C.c/. to the size of X . This ensures that an imbalance of n > 0 subjects on a small group will ‘cou
portional
more than the method allows for the incorporation of a random element, a ‘Second Best Probability’
However,
an n subject imbalance on a larger group.
parameter that sets the Conclusion
Optimizing Clinical Trials: Concept to probability that even when there is just one best minimizing arm, 2012 Medidata Solutions, Inc. § 59
™ © that arm will

Weighting
Over both sexes
Males Females

18-35 yo
Pla Test
Pla
Test
Pla
Test •  Stratified Randomization weights
35-65 yo

Pla
Test
Pla Test
Pla
Test on strata, not margins or overall Over both sexes
Males Females

•  Imbalances within strata tend to
>65 yo
Test Pla
18-35 yo Pla Pla
Pla Test Pla Test
Pla Test Test Test

Over all
Ages:
Test Pla
dominate in DA 35-65 yo Test
Pla Test Test
Pla Test
Pla Pla Pla
Test

>65 yo

•  Minimization weights on margins, not strata.
Test Pla

Pla Test Pla Test

•  DA can weight exclusively on margins
Over all
Ages:
Test Pla
Pla Test
Pla
Test

Over both sexes
Males Females

18-35 yo
Pla Test
Pla
Test
Pla
Test •  If a Strata is balanced, the next assignment
35-65 yo
Test
Pla Test
Pla
Test attempts to balance the margins.
Pla

>65 yo

Pla
Test Pla
Test Pla Test
•  Since small groups are more likely to have
Over all
Ages:
Test Pla
imbalances which reduce efficiency, balancing
strata 1st is appropriate
Pla Test
Pla
Test


Hierarchical Balancing

•  While Imbalances within strata tends to dominate in DA,
if a Strata is balanced, the next assignment attempts to balance the
margins

•  Since small group imbalances tend to dominate, balancing tends to be
sequential

Males Females Over both sexes
⟵ This example:
18-35 yo
Pla Test
Pla
Test
Pla
Test (1)  Balance within strata
35-65 yo Test
Pla Pla Test Pla
Test
(2)  If balanced within the strata, balance by age group
>65 yo
Pla
Test Pla
Test Pla Test
(since age groups tend to be smaller than sex groups)
Over all
(3)  If balanced within age group, balance within sex group
Ages: Test Pla
Pla Test
Pla Test
(4)  If balanced within sex group, balance overall

However: cumulative imbalances may change this order


?" Replacement
Randomization
!!!
! !!
!


Dynamically Adapting to Dropouts
Patients discontinue
25%#
Eﬀect)of)Drop9outs)on)Permuted)Block)and)Dynamic) ⟶ Imbalances
Alloca&on) ⟶ Reduced efficiency

20%# PB(2:2)#
25% DC PB(2:2)#
“Tight” randomizations
(PB with small blocks,
PB(2:2),#25%DC# DA with small 2nd best Prob.)
⟶ Lose more

15%#
DA(0.15),#Eq.Wts#
efficiency
DA(0.15),#Eq.Wts,#25%DC#

10%# DA(0.15),#Margins#
“Loose”
randomizations
DA(0.15),#Margins,#25%#DC# (CR, PB with large blocks,
DA with large 2nd best Prob.)
CR#
5%#
⟶ Lose less efficiency
CR(25%DC)# ⟶ Little or no change
CR#
No DC
0%#
0%# 5%# 10%# 15%# 20%# 25%#


Dynamic Allocation: Can
Eﬀect)of)Drop9outs)&)Rerandomiza&on)) allocate new patients to
24%$
on)Permuted)Block)and)Dynamic)Alloca&on)
restore balance
PB(2:2)$
PB(2:2)$
22%$
PB(2:2),$25%DC$
DA(0.15),$Eq.Wts$
DA(0.15),$Eq.Wts,$25%DC$
DA(0.15),EqWts,Adj.25%DC$
20%$

18%$

25% DC
16%$

14%$
DA Adj.
No DC

12%$
0%$ 1%$ 2%$ 3%$ 4%$ 5%$ 6%$ 7%$ 8%$


25%#

Dynamic)Alloca&on:)Readjus&ng)balance)for)
discon&nuing)pa&ents) “Tight” randomizations
(PB with small blocks,
20%# PB(2:2)# DA with small 2nd best Prob.)

PB(2:2)#
⟶ Lose more
efficiency
DA Adj. PB(2:2),#25%DC#
⟶ Benefit most
DA(0.15),#Eq.Wts#

15%#

25% DC DA(0.15),#Eq.Wts,#25%DC# “Loose”
DA(0.15),EqWts,Adj.25%DC#
randomizations
(CR, PB with large blocks,
10%# DA(0.15),#Margins# DA with large 2nd best Prob.)
DA(0.15),#Margins,#25%#DC# ⟶ Lose less efficiency
⟶ Little or no benefit
DA(0.15),#Margins,Adj.25%Dc#

5%# CR#

CR(25%DC)#
No DC
CR#

0%#
0%# 5%# 10%# 15%# 20%# 25%#


Applications

•  High drop-out ⇒ PB, DA ⟶ CR
•  Drop-out before becoming evaluable
•  Constrained resources (small sample size, limited drug
supply, ….)

•  Crossover studies: Requires completers
•  Evaluable ó Complete Sequence of Treatments

•  Provisional Randomization / Randomize to ship
•  Screening visit triggers:
•  Randomize at screening
•  If randomized treatment not on-site, ship blinded supplies
•  Randomization visit:
•  If patient eligible ⇒ dispense assigned treatment
•  If not eligible ⇒store for next eligible patient
•  Minimizes on-site drug supply


Randomization Optimization Factors
•  Equipose ⇒ (less random is acceptable)
•  Small Study ⇒ Efficiency important
⟶ Lower 2nd Best Probability
•  Large Study ⇒ Are there small subgroups?
All subgroups large ⟶ CR is acceptable
•  Small subgroups ⇒ Need more efficiency
⟶ Smaller 2nd best Prob


Balancing Considerations

•  Large Studies •  Smaller Studies
•  Studies with large subgroups •  Studies with small
•  Late phase studies subgroups
•  Strong Treatment preferences •  Early phase studies
•  Weak Blinding •  Interim Analyses
•  Subjective Endpoints •  Equipoise
•  Strong Blinding
•  Objective Endpoints
•  Many Strata / Many centers
•  Limited blinded supplies

Unpredictable ⟵ ⟶ ⟶ ⟶ Balanced
⟵⟵

Bibliography
Elsa Valdés Márquez & Nick Fieller. Inference in
Covariate-Adaptive allocation. EFSPI Adaptive
Randomisation Meeting, Brussels, 7 December 2006


Randomization: Too Important to Gamble with.

More Related Content

What's hot

Viewers also liked

Similar to Randomization: Too Important to Gamble with.

More from Dennis Sweitzer

Randomization: Too Important to Gamble with.