Modeling & Simulation in Managing Studies: Some Examples

1. SERC
M&S:
Examples
(Screening,
Enrollment,
Randomization,
Completion
Modeling
&
Simulation)
Dennis
Sweitzer,
Ph.D.
April
2016

2. Application
Scopes
A
priori
Assumptions
⟶ Simulate⟶ Expected
Outcomes,
Thresholds
(
e.g.,
planned
timeline,
resources,
and
expected
variability)
Ongoing
study⟶ Model⟶ Simulate⟶ Projections
(
e.g.,
projected
timeline,
resources,
and
expected
variability
given
real
information)
Projections
v. A
priori
Assumptions
⟶Validation
(Consistency)
(
e.g.,
are
projections
from
incoming
data
consistent
with
assumptions)
Projections
v. Observations
⟶Validation
(Reality)
(
e.g.,
do
projections
from
incoming
data
match
planning
expectations)
Model
+
Scenarios ⟶ Simulate
⟶ Alterative
Projections
✔
✔
✔
✔
✔
Using
patient
milestone
dates
(blinded)
(SERC
≣ Screening,
Enrollment,
Randomization,
Discontinuation)
And/or
Assumptions
used
in
planning
Simple
Modeling
&
Simulation
can
be
used:
Modeling:
Survival
analysis
of
time
between
events
Simulation:
Competing
Events
model
using
survival
results
Examples
⟹

3. Example:
Multi-­‐Segment
Studies
Study Flowchart
Randomized Treatment
Phase
28 to 104 weeks
Screening
&
Enrollment
Open-Label Treatment
Phase
12 to 36 weeks Active
Placebo
Inclusion/Exclusion
Criteria
Inclusion/Exclusion
Criteria
Screen
Failure
Drop
Outs
Drop
Outs
• Long
term
randomized
withdrawal
maintenance
studies
(AstraZeneca)
• Open
Label
Stabilization
(3-­‐9mo)
+
Follow
to
Relapse
(1-­‐2yr)
• Standard
design,
but
not
in
Schizophrenia,
bipolar,
&
other
mood
– ⟶ Uncertain
dropout,
relapse,
&
response
rates
• Risks
of
enrolling
– Too
few
(subjects
dropout
before
relapse)⟶ Failed
Study
– Too
many
(subjects
in
Open
Label
at
last
relapse)⟶ Costs,
Ethics

4. Competing
Events
Model
1. Best
guess
for
initial
planning
2. As
study
was
running,
every
month:
• Update
Statistical
Model
using
patient
status
data
• Simulate
remainder
of
study
from
model
3. Summarize
Simulations
to:
• Predict
milestones
(timelines,
resources)
• Test
scenarios
(of
changes
in
plans)
• Validate
study
assumptions
&
detect
deviations
Enroll OL Pts
OL
Dropouts
Relapse
Rand’d
Patients
Rand’d
Dropouts

5. M&S
ProjectionTrial B, Dates of 200th Event Predicted on 29 Oct
by Enrollment Cutoff
12-Feb-06
23-May-06
31-Aug-06
9-Dec-06
19-Mar-07
27-Jun-07
5-Oct-07
13-Jan-08
22-Apr-08
31-Jul-08
10-Sep-0524-Sep-058-O
ct-0522-O
ct-055-Nov-0519-N
ov-053-Dec-05
17-D
ec-0531-D
ec-0514-Jan-0628-Jan-0611-Feb-0625-Feb-0611-M
ar-0625-M
ar-068-Apr-0622-Apr-066-M
ay-06
20-M
ay-06
Enrollment Cutoffs
Region Based Simulation Actual
Projected
End
of
Study,
IF…
…
Enrollment
ends
on
this
date
Reduced
costs:
stop
enrollment
on
3
Dec
Reduced
Risks:
stop
by
11
March

6. Maintenance
Studies
in
2005
Trial A, Predicted Dates of 200th Event
22-Feb-06
8-Mar-06
22-Mar-06
5-Apr-06
19-Apr-06
3-May-06
17-May-06
31-May-06
14-Jun-06
28-Jun-06
12-Jul-06
26-Jul-06
9-Aug-06
23-Aug-06
6-Sep-06
20-Sep-06
4-Oct-06
9-O
ct-05
23-O
ct-05
6-N
ov-05
20-N
ov-05
4-D
ec-05
18-D
ec-05
1-Jan-06
15-Jan-06
29-Jan-06
12-Feb-06
26-Feb-06
12-M
ar-06
26-M
ar-06
9-Apr-06
23-Apr-06
Date of Prediction (Oct 1 Enrollment Cutoff)
PredictedDateof200thEvent
Region Based Model (Median) Trial Based Actual
Stop
enrolling Stop
Randomizing
Wait as
Patients
Relapse
or
Drop
out

7. Another
Case
Study
Management
feedback:
“… the simulations are very valuable and the only
way we have to plan our timelines. As it has
turned out, your simulations seems to be pretty
accurate ...”
... We would have been guessing and spinning
our wheels without them.”
Date # Randomized Relapses
/
Dropouts Prediction:
101st Relapse
3 Aug’06 73 3
/
2 1
Dec …
15
June
6
Sep’06 182 16 /
7 12
Nov
…
21
Feb
2
Oct’06
Stopped
Enrolling
Patients
(NB:
3-­‐4
month
open
label)
Dec‘06
Stopped
Randomizing
Patients
(All
eligible
or
discontinued)
1
Jan’07
101st Relapse
Event

8. Examples
Validation: Protocols A&B assumed: (50% randomized, 30% Relapse) rate
Models estimated: Trial A: (33%, 37%) Trial B: (55%, 41%)
Early
Issue
Identification
⟶ Quick
Corrections
Scenario:
¿Add Sites to compensate for low enrollment?
• Run
simulation
with
additional
sites
• Compare
between
simulations
Scenario:
EMEA
requested
secondary
endpoint
of
Late
Relapses
(>4wk
off
Tx),
Trial
A
had
stopped
enrolling.
Should
Trial
A
be
reopened?
Should
Trial
B
be
extended?
• Build
new
endpoint
into
simulations
• Report

9. More
A
presentation
I
gave
at
JSM
2006
on
the
method,
with
a
proceedings
paper.
https://sites.google.com/site/dennissweitzer/home/modeling-­‐multiphase-­‐clinical-­‐trials-­‐time-­‐to-­‐completion-­‐
study-­‐management
Simple
simulation
methods
using
Excel.
I’ve
long
used
Excel
simulations
to
aid
in
planning
clinical
trials
(for
quick
&
transparent
models),
although
methods
for
doing
so
are
not
well
publicized.
Here’s
a
presentation
of
how-­‐to:
https://sites.google.com/site/dennissweitzer/home/quick-­‐simple-­‐simulation-­‐using-­‐ms-­‐excel