This presentation proposes an algorithmic approach to synthesize stabilizing control laws for discrete-time piecewise affine probabilistic (PWAP) systems based on computations of probabilistic reachable sets. The considered class of systems contains probabilistic components (with Gaussian distribution) modeling additive disturbances and state initialization. The probabilistic reachable state sets contain all states that are reachable with a given confidence level under the effect of time-variant control laws. The control synthesis uses principles of the ellipsoidal calculus, and it considers that the system parametrization depends on the partition of the state space. The proposed algorithm uses LMI-constrained semi-definite programming (SDP) problems to compute stabilizing controllers, while polytopic input constraints and transitions between regions of the state space are considered. The formulation of the SDP is adopted from a previous work in [1] for switched systems, in which the switching of the continuous dynamics is triggered by a discrete input variable. Here, as opposed to [1], the switching occurs autonomously and an algorithmic procedure is suggested to synthesis a stabilizing controller. An example for illustration is included.

1. Control & System
Theory
Control of Discrete-Time Piecewise Affine Probabilistic
Systems using Reachability Analysis
Leonhard Asselborn Olaf Stursberg
Control and System Theory
University of Kassel (Germany)
l.asselborn@uni-kassel.de
stursberg@uni-kassel.de
CACSD 2016
www.control.eecs.uni-kassel.de 21.09.2016

2. Introduction: Motivation in Uncertain and Partitioned Environment Control & System
Theory
• piecewise linearization of nonlinear dynamic system
• different dynamics in different regions of the state space
• stochastic disturbances (wind, waves)
• stochastic initialization (GPS coordinates)
• input constraints: uk ∈ U
• steer the system into a terminal region T with confidence δ
x0
p
p
xN
Introduction Problem Definition Method Example Conclusion Appendix 2

3. Relevant Literature (Excerpt): Control & System
Theory
Piecewise Linear Systems
• Sontag [1981], Kerrigan and Mayne [2002], Rakovic et al. [2004], Koutsoukos
and Antsaklis [2003]: piecewise linearization and controller synthesis
Reachability sets for stochastic hybrid systems
• Hu et al. [2000], Blom and Lygeros [2006], Cassandras and Lygeros [2006],
Kamgarpour et al. [2013], Abate et al. [2008]: control design
Previous own work
• Controller synthesis for nonlinear systems: [NOLCOS, 2013]
• Synthesis for stochastic discrete-time linear systems: [ROCOND, 2015]
• Synthesis for stochastic discrete-time switched linear systems: [ADHS, 2015]
• Synthesis for stochastic discrete-time switched linear systems with chance
constraints: [ECC, 2016]
Introduction Problem Definition Method Example Conclusion Appendix 3

4. Contribution Control & System
Theory
Controller synthesis based on probabilistic reachability computation for
discrete-time piecewise affine probabilistic systems
Solution Approach:
• forward propagation of ellipsoidal reachable sets Xδ
k with confidence δ
Xδ
0
Θ(1)
Xδ
N
Θ(2)
T
• offline controller synthesis by semi-definite programming (SDP)
• push-and-branch procedure introduced
Introduction Problem Definition Method Example Conclusion Appendix 4

5. Sets, Distributions and Dynamic System (1) Control & System
Theory
Set representation:
• Ellipsoid: E := ε(q, Q) = x ∈ Rn
| (x − q)T
Q−1
(x − q) ≤ 1
with q ∈ Rn
, Q ∈ Rn×n
• Polytope: P := {x ∈ Rn
| Rx ≤ b} with R ∈ Rnp×n
, b ∈ Rnp
Multivariate Normal Distribution:
ξ ∼ N(µ, Ω)
The sum of two Gaussian variables ξ1 ∼ N(µ1, Ω1) and ξ2 ∼ N(µ2, Ω2) is
again a Gaussian variable:
ξ1 + ξ2 ∼ N(µ1 + µ2, Ω1 + Ω2)
Introduction Problem Definition Method Example Conclusion Appendix 5

6. Sets, Distributions and Dynamic System (2) Control & System
Theory
Piecewise Affine Probabilistic System (PWAPS):
xk+1 = Azk xk + Bzk uk + Gzk vk (1)
x0 ∼ N(qx,0, Qx,0) (2)
vk ∼ N(qv, Qv) (3)
uk ∈ U = {uk ∈ Rm
| Ruuk ≤ bu} (4)
zk ∈ Z = {1, 2, . . . , nz} (5)
¯Θ = {Θ(1)
, . . . , Θ(nz)
} (6)
Feasible system execution for k ∈ N0:
1. given the continuous and discrete state is xk ∈ Θ(i)
and zk = i,
2. sample the disturbance vk ∼ N(qv, Qv)
3. choose a suitable input uk ∈ U
4. evaluate the continuous dynamics with the tuple (Azk , Bzk , Gzk ) to
compute xk+1
5. compute zk+1 according to the current partition element, which contains
xk+1
Introduction Problem Definition Method Example Conclusion Appendix 6

7. Probabilistic Reachable Sets with Confidence δ Control & System
Theory
• Surfaces of equal density for ξ ∼ N(µ, Ω)
(Krzanowski and Marriott [1994]):
(ξ − µ)T
Ω−1
(ξ − µ) = c
with χ2
-distributed random variable and:
δ := Pr (ξ ∈ ε(µ, Ωc)) = Fχ2 (c, n)
cumulative distribution function
• Initial state confidence ellipsoid:
Xδ
0 := ε(qx,0, Qx,0c) with Pr(x0 ∈ Xδ
0 ) = δ contour of pdf
samples of ξ ∼ N (µ, Ω)
ξ2
ξ1
• Evolution of the state distribution:
qx,k+1 = Azk qk + Bzk uk, Qx,k+1 = Azk Qx,kAT
zk
+ Gzk QvGT
zk
Xδ
k+1 := ε(qx,k+1, Qx,k+1c
=:Qδ
x,k+1
)
Xδ
k+1 is the confidence ellipsoid for xk+1 with confidence δ.
Introduction Problem Definition Method Example Conclusion Appendix 7

8. Problem Definition Control & System
Theory
Problem
Given PWAPS, determine a control law κk = λk(xk) for which it holds that:
• uk = λk(xk) ∈ U and xk ∈ Xδ
k ∀ k ∈ {0, 1, . . . , N − 1}, N ∈ N
• Xδ
N ⊆ T for a finite N ≤ Nmax.
Thus, any initial state x0 ∈ Xδ
0 has to be transferred into the terminal set T
with probability δ after N steps.
Introduction Problem Definition Method Example Conclusion Appendix 8

9. Main Idea Control & System
Theory
Solution procedure:
• Solution of an SDP provides continuous control law:
uk = λ(xk) = −Kkxk + dk
with closed-loop dynamics:
xk+1 = Azk xk + Bzk uk + Gzk vk
= (Azk − Bzk Kk)
:=Acl,k,zk
xk + Bzk dk + Gzk vk
Main Challenge:
• intersection of reachable set with any boundary:
Xδ
k ∩ ∂Θ(i)
= ∅
• partial consideration of reachable set is intractable
Idea: push-and-branch procedure to retain ellipsoidal set representation
Introduction Problem Definition Method Example Conclusion Appendix 9

10. Solution based on SDP (1) Control & System
Theory
• Convergence of the covariance matrix of N(qx,k+1, Qx,k+1):
Sk+1 ≥ Qx,k+1 = Acl,k,zk
Qx,kAT
cl,k,zk
+ Gzk QvGT
zk
or with Schur complement:


Sk+1 Acl,k,zk
Qx,k Gzk Qv
Qx,kAT
cl,k,zk
Qx,k 0
QvGT
zk
0 Qv

 ≥ 0
• Convergence of the expected value qx,k
use of flexible Lyapunov functions. (Lazar et al. [2009] ) suitable for
switched dynamics
V
k
Introduction Problem Definition Method Example Conclusion Appendix 10

11. Solution based on SDP (2) Control & System
Theory
Proposition
The input constraint uk = −Kkxk + dk ∈ U holds for Kk, dk and all xk ∈ Xδ
k
if:
(bu,i − ru,idk)In −ru,iKk(Qδ
x,k)− 1
2
−(Qδ
x,k)− 1
2 KT
k rT
u,i bu,i − ru,idk
≥ 0 ∀i = {1, . . . , nu}.
• ru,i and bu,i denote the i−th row of Ru and bu, respectively.
• Xδ
k is mapped into a unit ball by a suitable coordinate transformation
h(xk)
• the Euclidean norm ||h(xk)||2 ≤ 1 can be expressed as LMI, which results
in the above formulation
• complete proof can be found in Asselborn et al. [2013]
Introduction Problem Definition Method Example Conclusion Appendix 11

12. Determination of the Continuous Controller Control & System
Theory
Semidefinite program to be solved for chosen zk ∈ Z:
min
Sk+1,Kk,dk
Jk,zk
center point convergence:



qT
x,k+1,zk
Lqx,k+1,zk
− ρqT
x,kLqx,k ≤ αk
qx,k+1,zk
= (Azk − Bzk Kk)qx,k + Bzk dk
αk ≤ maxl∈{1,...,k} ωl
αk−l
ellipsoidal shape convergence:






Sk+1 Acl,k,zk
Qx,k Gzk Qv
Qx,kAT
cl,k,zk
Qx,k 0
QvGT
zk
0 Qv


 ≥ 0
trace(Sk+1) ≤ trace(Qk)
input constraint:



(bu,i − ru,idk)In −ru,iKk(Qδ
x,k)− 1
2
−(Qδ
x,k)− 1
2 KT
k rT
u,i bu,i − ru,idk
≥ 0,
∀i = {1, . . . , nu}
Introduction Problem Definition Method Example Conclusion Appendix 12

13. Push-and-Branch Procedure (1) Control & System
Theory
Push:
Xδ
k
Xδ
k+1
Θ(1)
Θ(2)
• initial solution for Xδ
k+1 intersects with boundary
• solve SDP again for each intersecting region with additional constraints:
r
(i)
j qx,k+1 − b
(i)
j ≥ max ∆(Qδ
x,k)
• choose best solution (if available)
Introduction Problem Definition Method Example Conclusion Appendix 13

14. Push-and-Branch Procedure (2) Control & System
Theory
Branch:
Xδ
k
Xδ
k+1
Xδ
k+2,γ1
Xδ
k+2,γ2
Θ(1)
Θ(2)
• branching is needed, if it fails to push the reachable set Xδ
k+1 into any
region Θ(i)
• entire, instead of a partial, consideration of Xδ
k+1 for the controller
synthesis for k + 2 → preserve the ellipsoidal set representation
• required tree structure: Γk = {γ1, . . . , γnγ,k }, with
γi = (Preγ, Sucγ , Xδ
k, Zint,k, ǫγ )
Introduction Problem Definition Method Example Conclusion Appendix 14

15. Push-and-Branch Procedure (3) Control & System
Theory
Probabilistic branch evaluation:
Xδ
k+1
Xδ
k+2,γ1
Xδ
k+2,γ2
Θ(1)
Θ(2)
• Probability for each region:
ǫγi := Pr xk+1 ∈ Θ(i)
=
ζ∈Θ(i)
N(qx,k+1, Qx,k+1)dζ
= getProbPart(Xδ
k, Θ(i)
) · ǫP re(γi)
• Approximation of multidimensional integral adopted from Asselborn and
Stursberg [2015], Blackmore and Ono [2009]
Introduction Problem Definition Method Example Conclusion Appendix 15

16. Controller Synthesis Control & System
Theory
Probabilistic Ellipsoidal Control Algorithm (PECA)
given: PWAPS with x0 ∼ N (qx,0, Qx,0), vk ∼ N (qv, Qv), ¯Θ, and U = {uk | Ruuk ≤ bu}; T, δ, πmin, ω, ρ,
and α0
define: k := 0, Zint,0 = getIntReg(Xδ
0 , ¯Θ), π0 := πmin, γ1 = (∅, ∅, Xδ
0 , Zint,0, 1), Γ0 := {γ1}
while ∃ γ ∈ Γk with Xδ
k T and πk ≥ πmin do
Γk+1 := ∅
for γi ∈ Γk do
for p ∈ Zint,k do
solve the SDP with zk = p
⋆ compute the distribution of xk+1,p
compute Xδ
k+1,p
Zint,k+1 := getIntReg(Xδ
k+1,p , ¯Θ)
if |Zint,k+1| > 1 do
“push” Xδ
k+1,p into one region by solving the SDP with the additional distance-constraint
if a feasible solution exists do go to line ⋆
else
for j ∈ Zint,k+1 do
ǫj := getP robP art Xδ
k+1,p, Θ(j) · ǫ(γi)
γj := (γi, ∅, Xδ
k+1,p, Zint,k+1, ǫγj
), Sucγi
:= Sucγi
∪ γj
Γk+1 := Γk+1 ∪ γj
end end end end end
compute πk+1
k := k + 1
end while
return (Kk,γ , dk,γ) for all γ ∈ Γk and 0 ≤ k ≤ N − 1
Introduction Problem Definition Method Example Conclusion Appendix 16

17. Termination with success Control & System
Theory
Lemma
The control problem with a confidence δ, an initialization x0 ∼ N(qx,0, Qx,0),
vk ∼ N(0, Σ), λk(xk) ∈ U ∀ k, and Pr(xk ∈ Xk) ≥ δx is successfully solved
with selected parameters γmin, ω, ρ and α0, if PECA terminates in N steps
with Xδ
N,γi
⊆ T, ∀γi ∈ ΓN .
Proof: by construction
If no success: adjust δ, πmin, ω, ρ, α0.
Introduction Problem Definition Method Example Conclusion Appendix 17

18. Numerical Example (1) Control & System
Theory
Initial distribution and disturbance:
x0 ∼ N(qx,0, Qx,0) with qx,0 =
−10
50
, Qx,0 =
1 0
0 1
vk ∼ N(0, Σ) with Σ =
0.02 0.01
0.01 0.02
.
The continuous dynamic is specified by the following system matrices:
A1 =
9.41 0.19
−0.38 9.99
10−1
, A2 =
9.22 0.19
−0.58 10.4
10−1
, A3 =
11.2 −0.21
0.42 9.79
10−1
B1 =
1.98 0.02
3.96 2.00
10−1
, B2 =
1.96 0.02
4.02 2.04
10−1
, B3 =
2.12 −0.04
0.04 3.96
10−1
G1 = G2 = G3 =
0.1 0.05
0.08 0.2
Note that all three subsystems are chosen to have unstable state matrices.
Introduction Problem Definition Method Example Conclusion Appendix 18

19. Numerical Example (2) Control & System
Theory
Input constraints:
uk ∈ U =



u ∈ R2
|




1 0
0 1
−1 0
0 −1



 u ≤




4
4
4
8







,
Target set:
T = ε 0,
0.96 0.64
0.64 0.8
Cost function:
Jk = trace
Sk+1 0
0 0.8 qx,k+1
Discrete input set: Z = {1, 2, 3}
State space partition: ¯Θ := {Θ(1)
, Θ(2)
, Θ(3)
}
Parameters: δ = 0.95, γmin = 0.01, α0 = 10−4
, ω = 0.8 and ρ = 0.98
Introduction Problem Definition Method Example Conclusion Appendix 19

20. Numerical Example (3) Control & System
Theory
Θ(1)
Θ(2)
Θ(3)
x1
x2
Xδ
0
branching
T
• Termination with N = 30 steps
in 48s using a standard PC (Intel
Core i7 − 6700 CPU, 16GB
RAM)
• Implementation with Matlab
2016a, YALMIP 3.0, SeDuMi
1.3, and ellipsoidal toolbox ET
(Kurzhanskiy and Varaiya [2006])
• Branching occurs after 3 time
steps with ǫγ1 = 0.89 and
ǫγ2 = 0.11
• Attractiveness to each other
results from the underlying
Lyapunov condition
Introduction Problem Definition Method Example Conclusion Appendix 20

21. Conclusion and Outlook Control & System
Theory
Summary:
• Algorithm for control of PWAPS
• Offline control law synthesis based on probabilistic reachability analysis
• Explicit consideration of input constraints
• Push-and-Branch procedure to preserve the ellipsoidal set representation
Future work:
• Development of methods to reduce the computational complexity
Introduction Problem Definition Method Example Conclusion Appendix 21

22. End Control & System
Theory
Thank you for your attention!
Introduction Problem Definition Method Example Conclusion Appendix 22

23. Probailistic branch evaluation Control & System
Theory
Probability of necessity for non-existing controller
Pr

xk+1 /∈


i∈Zint,k+1
Θ(i)



 = 1 −
i∈Zint,k+1
ǫi
Introduction Problem Definition Method Example Conclusion Appendix 23

24. Attractivity and Stochastic Stability Control & System
Theory
Stability with confidence δ
The switched uncertain linear system is called attractive with confidence δ on a
bounded time domain [0, N], if for any initial condition x0 ∈ Xδ
0 and any
vk ∈ ε(0, Σc), finite parameters ¯q ∈ Rn
and ¯Q ∈ Rn×n
exist such that:
||qN || ≤ ||¯q||, ||QN || ≤ || ¯Q||.
The system is said stable with confidence δ on a bounded time domain [0, N] if
in addition
||qk+1|| ≤ ||qk||, ||Qk+1|| ≤ ||Qk||.
holds for any 0 ≤ k ≤ N − 1.
Interpretation:
• qk converges to a finite neighborhood of the origin
• covariance matrix Qk converges, such that the confidence ellipsoid is of
decreasing size over k (while rotation is still possible).
Introduction Problem Definition Method Example Conclusion Appendix 24

25. References Control & System
Theory
Liberzon, D.
Switching in Systems and Control
Birkhaeuser, 2003
Sun, Z.
Switched Linear Systems: Control and Design
Springer, 2006
Sun, Z. and Ge, S.S.
Stability theory of switched dynamical systems;
Springer. 2011
Blackmore, L. and Ono, M.
Convex chance constrained predictive control without sampling
Proceedings of the AIAA Guidance, Navigation and Control Conference, 2009
Blackmore, L., Ono, M., William, B.C.
Chance-constrained optimal path planning with obstacles
IEEE Transactions on Robotics, 2011
Vitus, M.P. and Tomlin, C.J.
Closed-loop belief space planning for linear gaussian systems
IEEE Conference on Robotics and Automation, 2011
Calafiore, G. and Campi, M.C.
Uncertain convex programs: randomized solutions and confidence levels
Mathematical Programming, 2005
Blackmore, L., Ono, M., Bektassov, A., Williams, B.C.
A probabilistic particle-control approximation of chance-constrained stochastic predictive control
IEEE Transactions on Robotics, 2010
Introduction Problem Definition Method Example Conclusion Appendix 25

26. References Control & System
Theory
Asselborn, L., Jilg, M., and Stursberg, O.
Control of uncertain hybrid nonlinear systems using particle filters
In 4th IFAC Conf. on Analysis and Design of Hybrid Systems, 2012
Prandini, M., Garatti, S., Vignali, R.
Performance assessment and design of abstracted models for stochastic hybrid systems through a randomized approach
Autmatica, vol. 50, 2014
Hu, J., Lygeros, J., and Sastry, S.
Towards a theory of stochastic hybrid systems;
In Hybrid systems: Computation and Control, volume 1790, 160-173. Springer. 2000
Blom, H.A. and Lygeros, J.
Stochastic hybrid systems: theory and safety critical applications;
volume 337. Springer. 2006
Cassandras, C.G. and Lygeros, J.
Stochastic hybrid systems;
CRC Press.2006
Kamgarpour, M., Summers, S., and Lygeros, J.
Control design for specifications on stochastic hybrid systems;
In Hybrid systems: computation and control, 303-312. ACM. 2013
Abate, A., Prandini, M., Lygeros, J., and Sastry, S.
Probabilistic reachability and safety for controlled discrete time stochastic hybrid systems;
Automatica,44(11), 2724-2734. 2008
Asselborn, L., Gross, D., and Stursberg, O.
Control of uncertain nonlinear systems using ellipsoidal reachability calculus;
In 9th IFAC Symp. on Nonlinear Control Systems, 50-55. 2013
Introduction Problem Definition Method Example Conclusion Appendix 26

27. References Control & System
Theory
Asselborn, L. and Stursberg, O.
Probabilistic control of uncertain linear systems using stochastic reachability;
In 8th IFAC Symp. on Robust Control Design. 2015
Asselborn, L. and Stursberg, O.
Robust Control of Uncertain Switched Linear Systems based on Stochastic Reachability
In 5th IFAC Conf. on Analysis and Design of Hybrid Systems, 2015
Soranzo, E.E.A.
Very simple explicit invertible approximation of normal cumulative and normal quantile function
Applied Mathematical Science, 2014
Boyd, S.P., El Ghaoui, L., Feron, E., and Balakrishnan, V.
Linear matrix inequalities in system and control theory;
volume 15. SIAM. 1994
Lazar, M.
Flexible control lyapunov functions;
In American Control Conf., 102-107.2009
Kurzhanski, A. and Varaiya, P.:
Ellipsoidal Calculus for Estimation and Control;
Birkh¨auser, 1996.
Introduction Problem Definition Method Example Conclusion Appendix 27