CDAC 2018 Angaroni optimal control

Optimal Control of chronic myeloid leukemia treatment
F.Angaroni
May 24, 2018
F.Angaroni (U. Bicocca) Optimal Control of the chronic myeloid leukemia treatment 23 May 2018 1 / 36

1 The disease
Diﬀerential equations model
2 Optimal control
Analytic solution: the Pontryagins maximum principle
Numerical solution: the Pontryagins maximum principle
Optimal control: the CML therapy examples
Optimal control: the CML therapy
3 Future works

The disease: Mathematical hypothesis
This disease is driven by the BRC-ABL oncogene.
Since BCR-ABL mutation is present in all leukemic cells the ratio of
cancer cells respect to healthy cells is a ”simple” measurements.
About 2000 follow up could be found in literature. Their are an
example of measurements of the in vivo kinetics (F.Michor)
Exponential decay of cancer cells, and exponential blast after the stop
of the therapy, tell us that N(t) should be an exponential (solution of
ﬁrst order diﬀerential equation) or described by a Poisson process.

The disease: Mathematical hypothesis
We divide the cells of the tiusse under study in ne non intersecting
ensambles, every ensamble represents a certain stage of cell diﬀerentiation.

Compartments Model
In this case, we divide the cells in
four ensambles:
1 Stem cells (SC)
2 Precursors Cells (PC)
3 Diﬀerentiated cells (PC)
4 Terminally diﬀerentiated cells
(TD)
Every collection of these ensembles
is a branch.

Since BCR-ABL mutation is present in all leukemic cells we can distinguish
between
Healthy branch
Leukemic branch

We attach a function ci,l (t) to every ensemble, it represents the number
of cells in one ensemble.
We will study transition rate between classes.
Given the following parameters:
pi,k the division rate of the cells
di,k the death rate of the cells
ai,k ∈ [0, 1] the probability of self-renewal
λ the probability for unit of time for a SC to develop the mutation
s(t) = 1
1+k( 4
i=1
h
j=l ci,j (t))
biochemical signal that regulate the cells
proliferation, it depends only on the number of mature cells

Considering only symmetric diﬀerentiation, the transition rate for the ci,k
ensamble are:
+2pi,kai,kci (t)s(t) a in-going ﬂux caused by the replication, this
contribute is absent for the TD ensembles,

ensamble are:
+s(t)pi−1,k(1 − ai−1,k)ci−1,k(t) a in-going ﬂux due to the
diﬀerentiation in the previous ensemble of the branch, it is not present
for the SC (i = 1)

ensamble are: :
−di,kci,k out-going ﬂux due to the death of cells,

ensamble are:
−s(t)pi ci (t) out-going ﬂux due to the diﬀerentiation,

Considering only symmetric differentiation, the transition rate are give by:
λc1,k in-going flux for c1,k+1 and out-going flux for c1,k. It represents
the generation of the tumor SC from the healthy SC.

From the diagram it is easy to obtain a system of 8 ﬁrst order diﬀerential
equations:
dc1,h(t)
dt
= [(2a1,h − 1)p1,hs(t) − d1,h − λ]c1,h(t),
dc2,h(t)
dt
= 2(1 − a1,h)p1,hs(t)c1,h(t) + [−d2,h + (2a2,h − 1)p2,hs(t)]c2,h(t),
dc3,h(t)
dt
= 2(1 − a2,h)p2,hs(t)c2,h(t) + [−d3,h + (2a3,h − 1)p3,hs(t)]c3,h(t), (1)
dc4,h(t)
dt
= 2(1 − a3,h)p3,hs(t)c4,h(t) − d4,hc4,h(t),
dc1,l (t)
dt
= [(2a1,l − 1)p1,l s(t) − d1,l ]c1,l (t) + λc1,h(t),
dc2,l (t)
dt
= 2(1 − a1,l )p1,l s(t)c1,l (t) + [−d2,l + (2a2,l − 1)p2,l s(t)]c2,l (t),
dc3,l (t)
dt
= 2(1 − a2,l )p2,l s(t)c2,l (t) + [−d3,l + (2a3,l − 1)p3,l s(t)]c3,l (t),
dc4,l (t)
dt
= 2(1 − a3,l )p3,l s(t)c4,l (t) − d4,l c4,l (t),

This tumor has a molecularly targeted therapy: Imatinib an inhibitor of
BCR-ABL gene.
The therapy is simulate as a decrease of 3 order of magnitude of division
rate of tumoral cells.
Figure: From:Dynamics of chronic myeloid leukemia F.Michor et al.

The model is in a good agreement with the experimental data,
deﬁning a ﬁgure of merit (FOM)
T(t) =
4
i=1 ci,l (t)
4
i=1
h
k=l ci,k(t)
. (2)

The model presents a steady state where the tumoral stem cells are
constant: life-long disease.
c∗
1,l =
λc∗
1,h
(2a1,h − 1)p1,hs∗ − d1,l
, (3)

The steady state is present only if:
d1,l c∗
1,l = [(2a1,l − 1)p1,l s∗
)]c∗
1,l + λc∗
1,h (4)
An early estimation of parameters could help in clinical management

Optimal control: motivation
The therapy has several drawbacks:
1 Fails in eradicating disease
Figure: From:Dynamics of chronic myeloid leukemia F.Michor et al.

2 Too expensive to be eﬀective in a epidemiology contest
Figure: From:The price of drugs for chronic myeloid leukemia (CML) is a
reﬂection of the unsustainable prices of cancer drugs: from the perspective of a
large group of CML experts

3 Is a life-long therapy (?)
Figure: From:Early molecular response and female sex strongly predict stable
undetectable BCR-ABL1, the criteria for imatinib discontinuation in patients with
CML

3 Is a life-long therapy (?)
4 The therapy fails in approximately 15 − 25% of patients due to the
presence of resistant subclones, but Pharmacologic inhibitors for
imatinib-resistant CML exist and dose escalation can improve the
response in a subset of patients with resistance to standard dose
imatinib

Control Theory
Control Theory deals with systems that can be controlled, i.e. whose
evolution can be inﬂuenced by some external agent described by
u ∈ U ⊆ Rn, e.g.:
dx
dt
= f (x, u) (5)
There are two classes of control:
1 Open loop. Choose u as function of time t,
2 Closed loop or Feedback. Choose u as function of space variable x
Optimal control belongs to the ﬁrst class.

Open loop control: the problem
Given the dynamics:
dx
dt
= f (x, u) (6)
T is the ensemble of final possible state x(T) at time t = T, if
x(0) = x0
L(x(t), u)dt is the running cost
φ(x(T)) is the pay-off
the optimal control problem is to find u∗(t) such that:
min[φ(x(T)) +
t
0
dtL(x(t), u(t))] x(0) = x0 x(T) ∈ T (7)
One approach to the optimal problem is the maximum principle :

Optimal control: Pontryagins Maximum Principle
The cornerstone of optimal control is the Pontryagins maximum
principle.
If u∗(t) and x∗(t) are optimal optimal solution and H(t, x(t), u(t), λ(t)) is
an Hamiltonian deﬁned as follows:
H(t, x(t), u(t), λ(t)) = λf (t, u(t), x(t)) + L(u(t), x(t)) (8)
then there exists a piecewise diﬀerentiable adjoint variable λ(t) such that:
dλ(t)
dt
= −
∂H(t, x(t), u(t), λ(t))
dx
(9)
λ(T) = φ(x(T)) (10)
H(t, x∗
(t), u(t), λ(t)) ≤ H(t, x∗
(t), u∗
(t), λ(t)) (11)
with
∂H
∂u u=u∗
= 0 (12)

Optimal control: Numerical solution
Consider an optimal control system:
1) Cost
max φ(x(T)) +
T
t0
L(t, x(t), u(t))dt (13)
2) Dynamics
x(t0) = a
dx(t)
dt
= f (t, x(t), u(t)) (14)
3) The dynamics of the adjoint
λ(T) = φ(x(T))
dλ
dt
= −
∂H
∂x
(15)
4) Characterization of the optimal control
∂H
∂u u=u∗
= 0 (16)

Numerical solution: Forward-Backward Sweep Method
A rough outline of the algorithm to solve the optimal control problem is
the following:
1 Make an initial guess for u over the interval.
2 Solve Forward in time the Dynamics:
x(t0) = a
dx(t)
dt
= f (t, x(t), u(t)) (17)
3 Solve Backward in time the dynamics of the adjoint:
λ(T) = φ(x(T))
dλ
dt
= −
∂H
∂x
(18)
4 Update u using λ(t) and x(t) into the characterization of the
optimal control
∂H
∂u u=u∗
= 0 (19)
5 Check convergence. If values of the variables in this iteration and the
last iteration are negligibly close, output the current values as
solutions. If values are not close, return to Step 2.

The dynamics of the system under treatment is given by:
dc1,h(t)
dt
= [(2a1,h − 1)p1,hs(t) − d1,h − λ]c1,h(t),
dc2,h(t)
dt
= 2(1 − a1,h)p1,hs(t)c1,h(t) + [−d2,h + (2a2,h − 1)p2,hs(t)]c2,h(t),
dc3,h(t)
dt
= 2(1 − a2,h)p2,hs(t)c2,h(t) + [−d3,h + (2a3,h − 1)p3,hs(t)]c3,h(t), (20)
dc4,h(t)
dt
= 2(1 − a3,h)p3,hs(t)c4,h(t) − d4,hc4,h(t),
dc1,l (t)
dt
= [(2a1,l − 1)
p1,l
u(t)
s(t) − d1,l ]c1,l (t) + λc1,h(t),
dc2,l (t)
dt
= 2(1 − a1,l )
p1,l
u(t)
s(t)c1,l (t) + [−d2,l + (2a2,l − 1)
p2,l
u(t)
s(t)]c2,l (t),
dc3,l (t)
dt
= 2(1 − a2,l )
p2,l
u(t)
s(t)c2,l (t) + [−d3,l + (2a3,l − 1)p3,l s(t)]c3,l (t),
dc4,l (t)
dt
= 2(1 − a3,l )p3,l s(t)c4,l (t) − d4,l c4,l (t),

Where the control is given by:
U = {u(t)|∀t, 1 ≤ u(t)≤103}; (21)
We use the following cost:
C(t, x(t), u(t)) =
T
0
dt{Au2
(t) + B
4
i=1
c2
i,l (t)} (22)
where:
A economic cost per dose
u(t) is the control,
B 4
i=1 c2
i,l represents a running cost that depends on the size of the
tumor

We deﬁne an Hamiltonian:
H = Au(t)2
+ B
4
i=1
c2
i,l (t) +
4
i=1
h
k=l
λi,k
dc1,k(t)
dt
(23)
We have 8 adjoint equation:
dλi,k
dt
= −
∂H
∂ci,h
i = 1, 2, 3, 4 k = l, h
and the characterization of the optimal control:
0 = −2Au(t) + log(u(t))(2a1,l − 1)p1,l s(t)c1,l +
+ log(u(t))(2(1 − a1,l )p1,l c1,l (t) + (2a2,l − 1)p2,l s(t)c2,l (t)))+
+ log(u(t))2(1 − a2,l )p2,l s(t)c2,l
(24)

The Greedy controller (A = 100, B = 0 =⇒ u(t) = 1)
150 200 250 300
-4
-3
-2
-1
0
1
2
t (days)
Log[T]+2
150 200 250 300
0
2000
4000
6000
8000
10000
t (days)
cost
The Careful controller (A = 0, B = 1 =⇒ u(t) = 103)
150 200 250 300
-4
-3
-2
-1
0
1
2
t (days)
Log[T]+2
150 200 250 300
0
5.0×1014
1.0×1015
1.5×1015
t (days)
cost

Problems:
the optimal solution exists and it is unique only for certain costs
Optimal control solutions implies continuous measurements and
action on the system: Optimal control is very complicated to apply to
a real system,
Since the functional space is very rich compared to a “discrete” space
the optimal control gives the upper bond of a decision process and
could be used to evaluate the sustainability of a therapy

Therapy control: future works
Using a random optimizer:
1 Consider the real pharmacodynamics and pharmacokinetics
Figure: From:Pharmacokinetics and pharmacodynamics of dasatinib in the chronic
phase of newly diagnosed chronic myeloid leukemia

1 Consider the real pharcodynamics
2 Consider a more realistic cost, not quadratic
C(t, x(t), u(t)) =
T
0
dt{Ad(u) + Bd(u) + D
4
i=1
ci,l (t)} (25)
where:
d(u) the function that represent the dosage of the molecular therapy,
A economic cost per dose, B quality of life cost per dose (toxicity)
D 4
i=1 ci,l (t) represents a cost that depends on the number of
leukemic cells

2 Consider a more realistic cost
3 Consider a resistant branch and then manage the switch of the
therapy

3 Consider a resistant branch and then manage the switch of the
therapy
4 Consider a discrete follow up

3 Consider a discrete follow up
4 Consider resistant branch and then the switch of the therapy
5 Consider a hybrid therapy with “conventional“ radiotherapy or
chemotherapy with the aim of eradicate residual cancer stem cells

Thank you

CDAC 2018 Angaroni optimal control

Recommended

Recommended

More Related Content

What's hot

What's hot (6)

Similar to CDAC 2018 Angaroni optimal control

Similar to CDAC 2018 Angaroni optimal control (20)

More from Marco Antoniotti

More from Marco Antoniotti (14)

Recently uploaded

Recently uploaded (20)

CDAC 2018 Angaroni optimal control