This presentation proposes an approach to algorithmically synthesize control strategies for set-to-set transitions of uncertain discrete-time switched linear systems based on a combination of tree search and reachable set computations in a stochastic setting. For given Gaussian distributions of the initial states and disturbances, state sets wich are reachable to a chosen confidence level under the effect of time-variant hybrid control laws are computed by using principles of the ellipsoidal calculus. The proposed algorithm iterates over sequences of the discrete states and LMI-constrained semi-definite programming (SDP) problems to compute stabilizing controllers, while polytopic input constraints are considered. An example for illustration is included.

1. Robust Control of Uncertain Switched Linear Systems
based on Stochastic Reachability
Leonhard Asselborn Olaf Stursberg
Control and System Theory
University of Kassel (Germany)
l.asselborn@uni-kassel.de
stursberg@uni-kassel.de
www.control.eecs.uni-kassel.de

2. Introduction
Problem:
• discrete-time switched uncertain linear systems:
xk+1 = Azk xk + Bzk uk + Gzk vk
• Gaussian distribution of the initial state x0 and the disturbances vk
• continuous input uk ∈ U and discrete input zk ∈ Z = {1, 2, . . . , nz}
• steer the system into a terminal region T with confidence δ
Xδ
0
Xδ
N
x2
x1
T
Solution Approach:
• forward propagation of ellipsoidal reachable sets Xδ
k with confidence δ
• offline algorithmic synthesis by semidefinite programming (SDP) and tree
search
• result: time-variant hybrid control laws for set-to-set transitions
Introduction Problem Definition Method Example Conclusion Appendix 2

3. Relevant literature (excerpt):
Switched Linear Systems
• Liberzon [2003], Sun [2006], Sun and Ge [2011]: stability conditions for arbitrary
switching, no input constraints
• Kerrigan and Mayne [2002], Rakovic et al [2004]: optimal control problem
adressed by dynamic programming
• Dehghan and Ong [2012]: computation of invariant sets under dwell-time
restriction
• Lin and Antsaklis [2006,2007]: BMI synthesis conditions for exponential
stabilization
Reachability sets for stochastic hybrid systems
• Prandini and Hu [2006], Abate [2007], Bujorianu [2012], Ding [2012]: verification
• Hu et al [2000], Blom Lygeros [2006], Cassandras and Lygeros [2006],
Kamgarpour et al. [2013], Abate et al. [2008]: control design
Previous work
• Asselborn et al. [2013]: Controller synthesis for nonlinear systems
• Asselborn and Stursberg [2015]: Control of stochastic discrete-time linear systems
Here: algorithmic control law synthesis for discrete-time uncertain switched
linear systems based on tree search with embedded SDP solution
Introduction Problem Definition Method Example Conclusion Appendix 3

4. Sets, Distributions and the Dynamic System (1)
Set representation:
Ellipsoid: E := ε(q, Q) = x ∈ Rn
| (x − q)T
Q−1
(x − q) ≤ 1
with q ∈ Rn
, Q ∈ Rn×n
Polytope: P := {x ∈ Rn
| Rx ≤ b} with R ∈ Rnp×n
, b ∈ Rnp
Multivariate Normal Distribution:
ξ ∼ N(µ, Ω)
The sum of two Gaussian variables ξ1 ∼ N(µ1, Ω1) and ξ2 ∼ N(µ2, Ω2) is
again a Gaussian variable:
ξ1 + ξ2 ∼ N(µ1 + µ2, Ω1 + Ω2)
Introduction Problem Definition Method Example Conclusion Appendix 4

5. Sets, Distributions and the Dynamic System (2)
Dynamical system:
xk+1 = Azk xk + Bzk uk + Gzk vk, k ∈ {0, 1, 2, . . .}
x0 ∼ N(q0, Q0), xk ∈ Rn
vk ∼ N(0, Σ), vk ∈ Rn
, iid
uk ∈ U = {uk | Ruuk ≤ bu} ⊆ Rm
zk ∈ Z = {1, 2, . . . , nz}
Feasible system execution:
for every k ∈ N0:
1. select zk ∈ Z to determine the tuple (Azk , Bzk , Gzk )
2. sample the disturbance vk ∼ N(0, Σ)
3. choose a suitable input uk ∈ U
4. evaluate the continuous dynamics to get the new state xk+1
Introduction Problem Definition Method Example Conclusion Appendix 5

6. Probabilistic Reachable Sets with Confidence δ
• Surfaces of equal density for ξ ∼ N(µ, Ω)
(Krzanowski and Marriott [1994]):
(ξ − µ)T
Ω−1
(ξ − µ) = c
c is a χ2
-distributed random variable.
δ := Pr (ξ ∈ ε(µ, Ωc)) = Fχ2 (c, n)
cumulative distribution function
• Initial state confidence ellipsoid:
Xδ
0 := ε(q0, Q0c) with Pr(x0 ∈ Xδ
0 ) = δ contour of pdf
samples of ξ ∼ N (µ, Ω)
ξ2
ξ1
• Evolution of the initial state distribution:
qk+1 = Azk qk + Bzk uk, Qk+1 = Azk QkAT
zk
+ Gzk ΣGT
zk
Xδ
k+1 := ε(qk+1, Qk+1c
=:Qδ
k+1
)
Xδ
k+1 is the confidence ellipsoid for xk+1 with confidence δ.
Introduction Problem Definition Method Example Conclusion Appendix 6

7. Attractivity and Stochastic Stability
Stability with confidence δ
The switched uncertain linear system is called attractive with confidence δ on a
bounded time domain [0, N], if for any initial condition x0 ∈ Xδ
0 and any
vk ∈ ε(0, Σc), finite parameters ¯q ∈ Rn
and ¯Q ∈ Rn×n
exist such that:
||qN || ≤ ||¯q||, ||QN || ≤ || ¯Q||.
The system is said stable with confidence δ on a bounded time domain [0, N] if
in addition
||qk+1|| ≤ ||qk||, ||Qk+1|| ≤ ||Qk||.
holds for any 0 ≤ k ≤ N − 1.
Interpretation:
• qk converges to a finite neighbourhood of the origin
• covariance matrix Qk converges, such that the confidence ellipsoid is of
decreasing size over k (while rotation is still possible).
Introduction Problem Definition Method Example Conclusion Appendix 7

8. Problem Definition
Problem 1
Determine a hybrid control law κk = (λk, νk) for which it holds that:
• uk = λk ∈ U, zk = νk ∈ Z and xk ∈ Xδ
k ∀ k ∈ {0, 1, . . . , N − 1}, N ∈ N
• the closed-loop system is attractive with confidence δ,
• Xδ
N ⊆ T for a finite N ≤ Nmax.
Thus, any initial state x0 ∈ Xδ
0 has to be transferred into the terminal set T
with probability δ after N steps.
• selected structure of the continuous control law for xk ∈ Xδ
k = ε(qk, Qkc):
uk = λ(xk) = −Kkxk + dk
• νk is determined by a tree search algorithm
• Closed-loop dynamics:
xk+1 = Azk xk + Bzk uk + Gzk vk
= (Aνk − Bνk Kk)
:=Acl,k,νk
xk + Bνk dk + Gνk vk
Introduction Problem Definition Method Example Conclusion Appendix 8

9. Main Idea
Solution procedure:
• Discrete control input νk ∈ Z
spans a decision tree.
• Determine (Kk, dk) by solving
an SDP in each explored node
of the tree.
• The tree is used to find an
appropriate sequence of
discrete states.
k = 0
k = 1
k = 2
k = 3
Search strategy: Depth-First Search:
A branch is explored as far as possible, and backtracking is applied if no
solution is found.
Introduction Problem Definition Method Example Conclusion Appendix 9

10. Solution based on SDP: Covariance Matrix and Expected Value
Convergence of the covariance matrix of N(qk+1, Qk+1):
Sk+1 ≥ Qk+1 = Acl,k,zk
QkAT
cl,k,zk
+ GΣGT
or with Schur complement:


Sk+1 Acl,k,zk
Qk GΣ
QkAT
cl,k,zk
Qk 0
ΣGT
0 Σ

 ≥ 0
Convergence of the expected value qk
is achieved by the use of flexible Lyapunov functions. (Lazar et al. [2009] )
V
k
Introduction Problem Definition Method Example Conclusion Appendix 10

11. Solution based on SDP: Input Constraint
Proposition
The input constraint uk = −Kkxk + dk ∈ U holds for Kk, dk and all xk ∈ Xδ
k
if:
(bu,i − ru,idk)In −ru,iKk(Qδ
k)− 1
2
−(Qδ
k)− 1
2 KT
k rT
u,i bi − ru,idk
≥ 0 ∀i = {1, . . . , nu}.
• ru,i and bu,i denote the i−th row of Ru and bu, respectively.
• Xδ
k is mapped into a unit ball by a suitable coordinate transformation
h(xk)
• the Euclidean norm ||h(xk)||2 ≤ 1 can be expressed as LMI, which results
in the above formulation
• complete proof can be found in Asselborn et al. [2013]
Introduction Problem Definition Method Example Conclusion Appendix 11

12. Determination of the Continuous Controller
Semidefinite program to be solved for every explored node in the tree, i. e.
∀zk ∈ Z, in any k:
min
Sk+1,Kk,dk
trace




Sk+1 0 0
0 w1 qk+1 0
0 0 w2 uk




center point convergence:



qT
k+1,zk
Lqk+1,zk
− ρqT
k Lqk ≤ αk
qk+1,zk
= (Azk − Bzk Kk)qk + Bzk dk
αk ≤ maxl∈{1,...,k} ωl
αk−l
ellipsoidal shape convergence:






Sk+1 Acl,k,zk
Qk G,zk Σ
QkAT
cl,k,zk
Qk 0
ΣGT
,zk
0 Σ


 ≥ 0
trace(Sk+1) ≤ trace(Qk)
input constraint:



(bu,i − ru,idk)In −ru,iKk(Qδ
k)− 1
2
−(Qδ
k)− 1
2 KT
k rT
u,i bi − ru,idk
≥ 0,
∀i = {1, . . . , nu}
Introduction Problem Definition Method Example Conclusion Appendix 12

13. Determination of the Discrete Controller
Probabilistic Ellipsoidal Control Algorithm (PECA)
Given: x0 ∼ N(q0, Q0), vk ∼ N(0, Σ), U = {uk | Ruk ≤ b}, T, δ, γmin, ω, ρ, α0
Define: k := 0, γ0 = γmin, O0 = ∅
while Xδ
k T and γk ≥ γmin do
(1) for i = 1, . . . , nz do
◮ compute Xδ
k, and solve the SDP problem for z = i and Xδ
k
◮ if solution exists do Ok := Ok ∪ {i} else Ok := Ok end
end
(2) if Ok = ∅ do choose the tuple (Kk, dk, νk) with best performance
else
if k = 0 do Termination without success
else k = k − 1, Ok := Ok {νk}, go to step (2) end
end
(3) compute (qk+1, Qk+1) with the selected controller (Kk, dk, νk)
(4) compute γk+1 = qk+1 − qk , k := k + 1
end while
Result: hybrid control law κk = (λk, νk) ∀ k ∈ {0, 1, . . . , N − 1}
Introduction Problem Definition Method Example Conclusion Appendix 13

14. Termination of PECA
Lemma
The control problem with a confidence δ, an initialization x0 ∼ N(q0, Q0),
vk ∼ N(0, Σ), and (λk, νk) ∈ U × Z ∀ k is successfully solved with selected
parameters γmin, ω, ρ and α0, if PECA terminates in N steps with Xδ
N ⊆ T.
Proof (sketch):
• successful termination of PECA implies Xδ
N ∈ T
• for a chosen νk ∈ Z, the solution of the SDP for k by construction ensures
that uk = −Kkxk + dk transforms Xδ
k into Xδ
k+1 while satisfying the
input constraint
• backwards induction for k ∈ {N − 1, . . . , 0} implies the set
{Xδ
N−1, . . . , Xδ
0 }, and thus: any initial state x0 ∈ Xδ
0 is transferred to Xδ
N
with confidence δ for vk ∼ N(0, Σ)
Introduction Problem Definition Method Example Conclusion Appendix 14

15. Example in 2-D: Problem (1)
Initial distribution and disturbance:
x0 ∼ N(q0, Q0) with q0 =
−10
50
, Q0 =
1 0
0 1
vk ∼ N(0, Σ) with Σ =
0.02 0.01
0.01 0.02
.
Discrete state set: Z = {1, 2, 3}
The continuous dynamic is specified by the following system matrices:
A1 =
1.35 −0.06
0.11 0.95
, A2 =
0.82 0.05
−0.14 1.10
, A3 =
0.86 0.05
−0.09 0.99
B1 =
0.58 −0.03
0.03 0.97
, B2 =
0.48 0.01
1.01 0.53
, B3 =
0.49 0.01
0.98 0.50
G1 = G2 = G3 =
0.1 0.05
0.08 0.2
Note that all three subsystems are chosen to have unstable state matrices.
Introduction Problem Definition Method Example Conclusion Appendix 15

16. Example in 2-D: Problem (2)
Input constraints:
uk ∈ U =



u ∈ R2
|




1 0
0 1
−1 0
0 −1



 u ≤




3
3
3
3







,
Target set:
T = ε 0,
0.96 0.64
0.64 0.8
Cost function:
J = trace
Sk+1 0
0 0.8 qk+1
Parameters: δ = 0.95, γmin = 0.01, α0 = 10−4
, ω = 0.8 and ρ = 0.98
Introduction Problem Definition Method Example Conclusion Appendix 16

17. 2-Dimensional Example: Results (1)
1
2
x2
x1
−20 −15 −10 −5 0
0
5
10
15
20
25
30
35
40
45
50
• Termination after 38 time steps
in 80s using 2.8 Ghz Quad-Core
CPU
• Implementation with Matlab
7.12.0, YALMIP 3.0, SeDuMi
1.3, and ellipsoidal toolbox ET
(Kurzhanskiy and Varaiya
[2006]).
Validation of the confidence:
• 1000 samples each for
k ∈ {0, 1, . . . , 5}
• P r: ratio of samples with
xk ∈ Xδ
k
k P r(xk ∈ Xδ
k
)
0 95.3%
1 95.1%
2 95.0%
3 94.9%
4 95.0%
5 95.0%
Introduction Problem Definition Method Example Conclusion Appendix 17

18. Conclusion and Outlook
Summary:
• Algorithm for the stabilization with confidence δ of a discrete-time
switched uncertain linear system
• Control law synthesis based on a combination of probabilistic reachability
analysis and tree search
• Explicit consideration of input constraints
• Stabilization problem reformulated as an iterative problem
• Countermeasures, if PECA terminates without success: adjust δ, γmin
• So far only applied to low dimensional exapmles
Future work:
• Extension to state chance constraints and autonomous switching
• Different confidence levels for the state and disturbance distribution
• Explore measures to reduce the computational complexity
Introduction Problem Definition Method Example Conclusion Appendix 18

19. References
[1] Abate, A.
Probabilistic reachability for stochastic hybrid systems;
Theory, computations, and applications. Ph.D. thesis, University of California, Berkeley,2007
[2] Abate, A., Prandini, M., Lygeros, J., and Sastry, S.
Probabilistic reachability and safety for controlled discrete time stochastic hybrid systems;
Automatica,44(11), 2724-2734. 2008
[3] Asselborn, L., Jilg, M., and Stursberg, O.
Control of uncertain hybrid nonlinear systems using particle filters;
In 4th IFAC Conf. on Analysis and Design of Hybrid Systems, 436-441. 2012
[4] Asselborn, L., Gross, D., and Stursberg, O.
Control of uncertain nonlinear systems using ellipsoidal reachability calculus;
In 9th IFAC Symp. on Nonlinear Control Systems, 50-55. 2013
[5] Asselborn, L. and Stursberg, O.
Probabilistic control of uncertain linear systems using stochastic reachability;
In 8th IFAC Symp. on Robust Control Design. 2015
[6] Blom, H.A. and Lygeros, J.
Stochastic hybrid systems: theory and safety critical applications;
volume 337. Springer. 2006
[7] Boyd, S.P., El Ghaoui, L., Feron, E., and Balakrishnan, V.
Linear matrix inequalities in system and control theory;
volume 15. SIAM. 1994
[8] Bujorianu, L.
Stochastic reachability analysis of hybrid systems;
Springer. 2012
[9] Cassandras, C.G. and Lygeros, J.
Stochastic hybrid systems;
CRC Press.2006
[10] Dehghan, M. and Ong, C.J.
Characterization and computation of disturbance invariant sets for constrained switched linear systems with dwell time restriction;
Automatica, 48(9), 2175-2181. 2012
Introduction Problem Definition Method Example Conclusion Appendix 19

20. References
[11] Dehghan, M. and Ong, C.J.
Discrete-time switching linear system with constraints: Characterization and computation of invariant sets under dwell-time
consideration;
Automatica, 48(5), 964-969. 2012
[12] Ding, J.
Methods for Reachability-based Hybrid Controller Design;
Ph.D. thesis, University of California, Berkeley. 2012
[13] Habets, L., Collins, P.J., and van Schuppen, J.H.
Reachability and control synthesis for piecewise-affine hybrid systems on simplices;
IEEE Trans. on Automatic Control, 51(6), 938-948. 2006
[14] Habets, L.C. and van Schuppen, J.H.
Control of piecewise-linear hybrid systems on simplices and rectangles;
In Hybrid Systems: Computation and Control, volume 2034, 261-274. Springer LNCS. 2001
[15] Hu, J., Lygeros, J., and Sastry, S.
Towards a theory of stochastic hybrid systems;
In Hybrid systems: Computation and Control, volume 1790, 160-173. Springer. 2000
[16] Kamgarpour, M., Summers, S., and Lygeros, J.
Control design for specifications on stochastic hybrid systems;
In Hybrid systems: computation and control, 303-312. ACM. 2013
[17] Kerrigan, E.C. and Mayne, D.Q.
Optimal control of constrained, piecewise affine systems with bounded disturbances;
In 41st IEEE Conf. on Decision and Control, 1552-1557. 2002
[18] Krzanowski, W.J. and Marriott, F.H.C.:
Multivariate Analysis: Distributions, Ordination and Inference;
Wiley Interscience, volume 2, 1994.
[19] Kurzhanski, A. and Varaiya, P.:
Ellipsoidal Calculus for Estimation and Control;
Birkh¨auser, 1996.
Introduction Problem Definition Method Example Conclusion Appendix 20

21. References
[20] A. A. Kurzhanskiy, P. Varaiya:
Ellipsoidal Toolbox;
Technical Report UCB/EECS-2006-46, EECS Department, University of California, Berkeley. URL
http://code.google.com/p/ellipsoids, 2006.
[21] Lazar, M.
Flexible control lyapunov functions;
In American Control Conf., 102-107.2009
[22] Liberzon, D.
Switching in systems and control;
Birkhaeuser. 2003
[23] Lin, H. and Antsaklis, P.J.
Switching stabilization and l2 gain performance controller synthesis for discrete-time switched linear systems;
In 45th IEEE Conf. on Decision Control, 2673-2678. 2006
[24] Lin, H. and Antsaklis, P.J.
Hybrid h state feedback control for discrete-time switched linear systems;
In 22nd Int. Symp. on Intelligent Control, 112-117.2007
[25] Prandini, M. and Hu, J.
A stochastic approximation method for reachability computations;
Stochastic Hybrid Systems, 337, 107-139. 2006
[26] Rakovic, S.V., Kerrigan, E.C., and Mayne, D.Q.
Optimal control of constrained piecewise affine systems with state-and input-dependent disturbances;
In 16th Int. Symp. on Mathematical Theory of Networks and Systems, MP8-01.25. 2004
[27] Sun, Z.
Switched linear systems: control and design;
Springer. 2006
[28] Sun, Z. and Ge, S.S.
Stability theory of switched dynamical systems;
Springer. 2011
[29] Zhai, G., Lin, H., and Antsaklis, P.J.
Quadratic stabilizability of switched linear systems with polytopic uncertainties;
Int. Journal of Control, 76(7), 747-753. 2003
Introduction Problem Definition Method Example Conclusion Appendix 21

22. 2-Dimensional Example: Results (2)
Partially explored search tree:
• Search strategy: depth-first
search
• Infeasible Solution: red cross
• No backtracking needed
Introduction Problem Definition Method Example Conclusion Appendix 22

23. 2-Dimensional Example: Results (3)
Partially explored search tree:
• No feasible solution at k = 30
• Backtracking needed
Introduction Problem Definition Method Example Conclusion Appendix 23