“Optimization of positive linear systems via geometric programming,” Shenzhen University, 2019.

1. Optimization of Positive Linear Systems
via Geometric Programming
Masaki Ogura
Osaka University, Japan
1
M. Ogura, M. Kishida, and J. Lam, “Geometric programming for optimal positive linear
systems,” IEEE Transactions on Automatic Control (accepted for publication), 2020.
Shenzhen University
12/26/2019

2.  Positive linear systems
 Parameter Tuning Problem
 Synthesis by geometric programming
 (Hidden convexity)
 Numerical example
Outline 2

3. Positive linear systems
3

4. Nonnegative variables
Probability Chemistry Economics
Math biology Project management
4

5. Dynamical systems with positive variables
 Lotka-Volterra equation
Rantzer, Valcher, “A tutorial on positive systems and large scale control,”
2018 IEEE Conference on Decision and Control, 2018.
Positive systems 5
 Buffer network

6. Positivity
 𝐴 Metzler, 𝑡 ≥ 0 ⇒ exp 𝐴𝑡 ≥ 0 (nonnegative matrix)
 𝑥0 ≥ 0 and 𝑤 𝑡 ≥ 0 ⇒ 𝑥 𝑡 ≥ 0
 𝑥0 ≥ 0 and 𝑤 𝑡 ≥ 0 ⇒ 𝑦 𝑡 ≥ 0
Positive linear system
𝑑𝑥
𝑑𝑡
= 𝐴𝑥 + 𝐵𝑤
𝑦 = 𝐶𝑥 非負行列
Nonnegative matrices
6
Metzler matrix
(nonnegative off-diagonals)

7. Informal statement...
Problem statement 7
𝑑𝑥
𝑑𝑡
= 𝐴𝑥 + 𝐵𝑤
𝑦 = 𝐶𝑥
𝑑𝑥
𝑑𝑡
= 𝐴𝑥 + 𝐵𝑤
𝑦 = 𝐶𝑥
Investment
Desirable control-theoretical
properties
Positive linear
system

8.  Parametrized positive linear system
 Parameter tuning problem
Problem statement 8
𝑑𝑥
𝑑𝑡
= 𝐴(𝜃)𝑥 + 𝐵(𝜃)𝑤
𝑦 = 𝐶 𝜃 𝑥,
Σ𝜃: 𝜃 ∈ Θ ⊂ ℝ𝑛𝜃
𝐿(𝜃)
minimize
𝐽 Σ𝜃 ≤ 𝛾
𝜃 ∈ Θ
subject to Σ𝜃 is internally stable
Performance functional
𝐻2 norm, 𝐻∞ norm, Hankel norm, ...
Parameter tuning cost

9. Networked epidemic processes
https://www.barabasilab.com/
Example: Epidemic Containment 9
H1N1 Flu outbreak

10. Networked Susceptible-Infected-Susceptible model
Nodes ＝ individuals
Edges ＝ relationships
Example: Epidemic Containment
Infection rate b
Infected
Susceptible
Recovery rate d
10
Infection probability ≥ 0

11. Epidemic containment problem [Preciado 2014, IEEE TCNS]
Finite, limited amount of medicine
Example: Epidemic Containment 11
Larger recovery rate di Smaller infection rate bi
𝑔 𝛿𝑖 𝑓 𝛽𝑖
Total cost
𝑖=1
𝑁
(𝑓 𝛽𝑖 + 𝑔(𝛿𝑖))
Network 𝑉, 𝐸
𝑉 = {1, … , 𝑁}

12.  Linearlized SIS model
 Cost function
𝐿 𝜃 =
𝑖=1
𝑁
(𝑓 𝛽𝑖 + 𝑔(𝛿𝑖))
Example: Epidemic Containment 12
𝑑𝑥
𝑑𝑡
= diag 𝛽1, … , 𝛽𝑁 𝐴𝐺 − diag 𝛿1, … , 𝛿𝑁 𝑥 + 𝐵𝑤
𝑦 = 𝐶𝑥,
Σ𝛽,𝛿:
Adjacency matrix
External disturbance
Performance output
Infection
probability vector

13.  Parametrized positive linear system
 Parameter tuning problem
Problem statement (p. 8) 13
𝑑𝑥
𝑑𝑡
= 𝐴(𝜃)𝑥 + 𝐵(𝜃)𝑤
𝑦 = 𝐶 𝜃 𝑥,
Σ𝜃: 𝜃 ∈ Θ ⊂ ℝ𝑛𝜃
𝐿(𝜃)
minimize
𝐽 Σ𝜃 ≤ 𝛾
𝜃 ∈ Θ
subject to Σ𝜃 is internally stable

14. Convexity results
 Assumption: Off-diagonals of 𝐴 fixed. 𝐵 and 𝐶 fixed.
 Convexity of 𝑥 𝑇 [Colaneri 2014, Automatica]
 Convexity of 𝐻2 or 𝐻∞ norm [Dhingra 2018, IEEE TCNS]
 Don’t apply to epidemic containment (and other problems)
Research question
Related works 14
𝑑𝑥
𝑑𝑡
= 𝐴(𝜃)𝑥 + 𝐵(𝜃)𝑤
𝑦 = 𝐶 𝜃 𝑥,
Σ𝜃:
Is it possible to optimally tune the off-diagonals of 𝑨
as well as 𝑩 and 𝑪 matrices?

15. Geometric programming
15

16. A framework for optimizing “posynomial functions”
Posynomial
 𝑓 𝑥1, … , 𝑥𝑛 = (sum of monomials)
Monomial
𝑔 𝑥1, … , 𝑥𝑛 = 𝑐 𝑥1
𝑎1 ⋯ 𝑥𝑛
𝑎𝑛
 Positive coefficient: 𝑐 > 0
 Positive variables: 𝑥1, … , 𝑥𝑛 > 0
 Real exponents: 𝑎1, … , 𝑎𝑛 ∈ ℝ
Boyd, Kim, Vandenberghe, Hassibi, “A tutorial on geometric programming,” Optimization and
Engineering, vol. 8, no. 1, pp. 67–127, 2007.
Geometric programming 16

17. Definition
 An example
Geometric program
minimize 𝑓(𝑥1, … , 𝑥𝑛)
subject to 𝑓𝑖 𝑥1, … , 𝑥𝑛 ≤ 1 (𝑖 = 1, … , 𝑝)
𝑔𝑗 𝑥1, … , 𝑥𝑛 = 1 ( 𝑗 = 1, … , 𝑞)
Posynomial
Monomial
17
𝑥1, … , 𝑥𝑛 > 0
𝑥, 𝑦, 𝑧 > 0

18. Reduction to a convex optimization
Posynomial
ℝ𝑛
ℝ+
𝑛
ℝ+
ℝ
exp log
Convex
Geometric program Convex optimization
Log transformation
18

19.  Geometric program
 Variable transformations: 𝑥 = 𝑒𝑡, 𝑦 = 𝑒𝑠
 Equivalent convex optimization
An example
minimize 𝑥−1/3
𝑦−1/4
subject to 𝑥1/2 + 𝑦1/2 ≤ 1
k
𝑥
𝑦
minimize 𝑒
−
𝑡
3−
𝑠
4
subject to 𝑒𝑡/2
+ 𝑒𝑠/2
≤ 1
minimize −
𝑡
3
−
𝑠
4
subject to 𝑠 ≤ 2 log(1 − 𝑒𝑡/2)
𝑡
𝑠
19

20. Circuit design
Applications 20
Other fields
Boyd, Kim, Vandenberghe, Hassibi, “A tutorial on geometric programming,” Optimization and
Engineering, vol. 8, no. 1, pp. 67–127, 2007.
Geometric programming to solve parameter tuning
problem for positive linear systems?

21. Solution by geometric
programming
21

22.  Parametrized positive linear system
 Parameter tuning problem
 Assumptions in literature: Off-diagonals of 𝐴 fixed. 𝐵 and 𝐶
fixed.
Problem statement (p. 8) 22
𝑑𝑥
𝑑𝑡
= 𝐴(𝜃)𝑥 + 𝐵(𝜃)𝑤
𝑦 = 𝐶 𝜃 𝑥,
Σ𝜃: 𝜃 ∈ Θ ⊂ ℝ𝑛𝜃
𝐿(𝜃)
minimize
𝐽 Σ𝜃 ≤ 𝛾
𝜃 ∈ Θ
subject to Σ𝜃 is internally stable

23.  Parametrized positive linear system
Assumption: system matrices 23
𝐴(𝜃) = 𝐴(𝜃) − 𝑅(𝜃)
Posynomials
𝐵(𝜃)
Posynomials
Posynomials
𝐶(𝜃)
𝑑𝑥
𝑑𝑡
= 𝐴(𝜃)𝑥 + 𝐵(𝜃)𝑤
𝑦 = 𝐶 𝜃 𝑥,
Σ𝜃: 𝜃 ∈ Θ ⊂ ℝ𝑛𝜃

24.  Parametrized positive linear system
Assumption: parameter set 24
𝑑𝑥
𝑑𝑡
= 𝐴(𝜃)𝑥 + 𝐵(𝜃)𝑤
𝑦 = 𝐶 𝜃 𝑥,
Σ𝜃: 𝜃 ∈ Θ ⊂ ℝ𝑛𝜃
Θ = {𝜃 ∈ ℝ𝑛𝜃 | 𝜃 > 0, 𝑓1 𝜃 ≤ 1, … , 𝑓𝑝 𝜃 ≤ 1}
Posynomials

25.  Parametrized positive linear system
 Parameter tuning problem
Assumption: cost function 25
𝑑𝑥
𝑑𝑡
= 𝐴(𝜃)𝑥 + 𝐵(𝜃)𝑤
𝑦 = 𝐶 𝜃 𝑥,
Σ𝜃: 𝜃 ∈ Θ ⊂ ℝ𝑛𝜃
𝐿(𝜃)
minimize
𝐽 Σ𝜃 ≤ 𝛾
𝜃 ∈ Θ
subject to Σ𝜃 is internally stable
Posynomial

26. If 𝐽 Σ𝜃 is either
 𝐻2 norm;
 𝐻∞ norm;
 Worst-case growth rate under structural uncertainty;
Main result: An overview 26
Σ𝜃
Δ

27. Main result: An overview 27
https://www.slideserve.com/helene/introduction-to-the-hankel-based-model-order-
reduction-for-linear-systems
Hankel operator
LTI SYSTEM
X (state)
t
u
t
y
Hankel operator
Past
input
(u(t>0)=0)
Future
output
2 2
: ( ,0) (0, )
H L L
  
Hankel operator:
- maps past inputs to future system outputs
- ignores any system response before time 0.
- Has finite rank (connection only by the state at t=0)
- As an operator on L2 it has an induced norm (energy
amplification)!
If 𝐽 Σ𝜃 is either
 𝐻2 norm;
 𝐻∞ norm;
 Worst-case growth rate under structural uncertainty;
 Hankel norm;
 Schatten 𝑝-norm;
then, the parameter tuning problem can be exactly solved by
geometric programming.
No need to fix the elements of 𝑨, 𝑩, 𝑪 matrices!

28. 35
𝐿(𝜃)
minimize
𝐽 Σ𝜃 ≤ 𝛾
𝜃 ∈ Θ
subject to Σ𝜃 is internally stable
Strategy
Reduction to a geometric program 28
Algebraic
How to rewrite into a
constraint in terms of
posynomials?
Posynomial
Original 𝐽 Σ𝜃
 Linear systems theory  Perron-Frobenius
 Matrix theory tools

29. Example: 𝑯∞
-norm 29
𝐺(0) ≤ 𝛾 Posynomial
Σ𝜃 ∞ ≤ 𝛾
DC-dominance property
[Tanaka 2013, SCL]
𝐺(0) = 𝐶𝜃𝐴𝜃
−1
𝐵𝜃
Matrix inversion
lemma
Max singular
value lemma

30. Maximum singular value lemma
Let 𝑀 be a nonnegative matrix and 𝛾 be a positive number. Then, the
following conditions are equivalent:
 𝑀 < 𝛾
 ∃ positive vectors 𝑢 and 𝑣 such that 𝑀𝑢 < 𝛾𝑣 and 𝑀𝑣 < 𝛾𝑢
Matrix inversion lemma
Let 𝐹 be a Metzler matrix, 𝑔 be a nonnegative vector, and 𝐻 be a
nonnegative matrix. The following conditions are equivalent:
 𝐹 is Hurwitz stable and −𝐻𝐹−1𝑔 < 𝑣
 ∃ positive vector 𝜒 such that 𝐻𝜒 < 𝑣 and 𝐹𝜒 + 𝑔 < 0
Lemmas 30

31. Example: 𝑯∞
norm 31

32. Example: Hankel norm 32
Any fundamental reason for the effectiveness of
geometric programming?

33. Hidden convexity
33

34. Positive linear system (not parametrized)
 Impulse response function
Φ 𝑡 = 𝐶 exp 𝐴𝑡 𝐵
 Claim (informal)
Φ(𝑡) is a “posynomial” in the entries of 𝐴, 𝐵, 𝐶
Hidden convexity
𝑑𝑥
𝑑𝑡
= 𝐴𝑥 + 𝐵𝑤
𝑦 = 𝐶𝑥,
Σ:
34
Hidden convexity 26
“(log𝐴, log 𝐵, log 𝐶)”
(𝐴, 𝐵, 𝐶) ℝ+
ℝ
exp log
Convex
Φ(𝑡)

35. Diagonal shift
 Φ 𝑡 = 𝑒−𝑟𝑡
𝐶 exp 𝐴𝑡 𝐵
Diagonals of 𝑨 matrix 35
𝐴 = 𝐴 − 𝑟𝐼
≥ 𝟎
≥ 𝟎
≥ 𝟎

36. Posynomial in
entries of 𝐴, 𝐵, 𝐶.
Claim
Each entry of Φ(𝑡) is the limit of a sequence of posynomials in
the entries of 𝐴, 𝐵, 𝐶.
Proof
Hidden convexity of impulse response 36
= 𝑒−𝑟𝑡
𝐶
𝑘=0
∞
𝐴𝑘
𝑡𝑘
𝑘!
𝐵
Φ 𝑡 = 𝑒−𝑟𝑡𝐶 exp 𝐴𝑡 𝐵
= lim
𝐿→∞
𝑒−𝑟𝑡
𝑘=0
𝐿
𝑡𝑘
𝑘
𝐶𝐴𝑘 𝐵

37. Corollary
The following quantities are log-convex in entries of 𝐴, 𝐵, 𝐶
 Output: 𝑦 𝑡 = 𝑒−𝑟𝑡𝐶 exp 𝐴𝑡 𝑥 0 + 0
𝑡
Φ 𝑡 − 𝜏 𝑤 𝜏 𝑑𝜏
 ℓ𝑝 norm: 𝑦(𝑡)
 𝐿𝑝 norm: 𝑦 𝑝 = 0
∞
𝑦 𝑡 𝑝 𝑑𝑡
1/𝑝
 𝐿𝑝 gain: Σ 𝑝 = sup
𝑤∈𝐿𝑝
𝑦 𝑝
𝑤 𝑝
Extensions 37

38.  Appearance of GP not necessarily surprising!
Hidden convexity 38
“(log 𝐴, log 𝐵, log 𝐶)”
(𝐴, 𝐵, 𝐶) ℝ+
ℝ
exp log
Convex
𝐿𝑝
gain

39. Example
39

40.  Privacy issue
 Information acquisition cost
 Data reliability
Network uncertainty 40
？
？
？
？
？
Perfect information Uncertain network
Optimal medical investment
taking into account
uncertainty in network
structure?

41. Additive uncertainty on adjacency matrix
Network uncertainty
38
？
？
？
？
？
37
？
？
？
？
？
= +
Adjacency
matrix
= +
𝐴𝐺 Δ
Nominal graph
41
Fixed Uncertain

42.  Linearlized SIS model w/ uncertainty
 Cost function
𝐿 𝜃 =
𝑖=1
𝑁
(𝑓 𝛽𝑖 + 𝑔(𝛿𝑖))
 Problem: maximize the worst-case decay rate of the size of
infected population geometric programming
Example: Epidemic Containment 42
𝑑𝑥
𝑑𝑡
= diag 𝛽1, … , 𝛽𝑁 (𝑨𝑮+𝚫) − diag 𝛿1, … , 𝛿𝑁 𝑥 + 𝐵𝑤
𝑦 = 𝐶𝑥,
Σ𝛽,𝛿:

43. PageRank and optimal investments
 We need to invest more to prepare for worst case scenario
 GP allows us to find cost-efficient medical resource allocation
Numerical simulation: results
Without uncertainty With uncertainty
43
Larger investment

44. Conclusions
44

45. Parameter tuning of positive linear systems
 Geometric programs w/ finitely many variables
 Hidden convexity
Future research directions
 Switched linear systems
 Discrete-time systems
 Nonlinear systems
Research questions
 Can we combine convexity results and log-convexity results
to better synthesize positive linear systems?
Conclusion 45