invited talk Shim

1. Blended dynamics theorem and its applications
for networked dynamic systems
Hyungbo Shim
Department of Electrical & Computer Engineering
Seoul National University, Korea
LABORATORY
CONTROL & DYNAMIC SYSTEMS
LABORATORY
CONTROL & DYNAMIC SYSTEMS
@ SNU

2. Collaborators and funding agencies related to today’s topic
서진헌 교수, 백주훈 교수, 김홍근 교수, 김정수 교수, 이찬화 교수, 안효성 교수,
Stephan Trenn 교수, Aneel Tanwani 교수, Bayu Jayawardhana 교수
김재용 박사, 양종욱 박사, 윤현준 박사, 이승준 박사,
이진규 박사, 김태규 박사, 김준수 박사
이동길 박사과정, 김정우 박사과정,
성정모 박사과정
한국연구재단 / 국방과학연구소
1 / 39

3. Control by multiple control agents
T-3000 from Terminator (Movie)
2 / 39

4. Features of such systems
▶ (multi-agent) There are manymanymany···
(possibly heterogeneous) agents.
▶ (distributed operation) ∃ no central unit. Agents communicate only with their
neighbors.
▶ (decentralized design) Each agent self-organizes.
▶ (plug-and-play) No reset occurs. Agents join or leave during operation.
▶ (robustness) The system works even with some malfunctioning agents.
3 / 39

5. How to “design” multi-agent system for such features?
As a toolkit, we want to propose Blended Dynamics approach in this talk.
And talk about a few examples:
1. Economic Power Dispatch for Smart Grid
2. Network Size Estimation
3. Distributed Control of Multi-channel Plant
4. Distributed p-quantile solver
5. Coupled Oscillator
Standing assumption: The network is undirected and connected.
4 / 39

6. Blended dynamics theorem (Jaeyong Kim et al., TAC 2016), (Jin Gyu Lee & S, AUT 2020)
Heterogeneous multi-agent system:1
ẋi = fi(xi, t) + k
X
j∈Ni
(xj − xi), xi ∈ Rn
, i ∈ N := {1, · · · , N}
Theorem:2 ∀ϵ > 0, ∃k∗ such that, with k > k∗,
lim sup
t→∞
∥xi(t) − s(t)∥ ≤ ϵ, ∀i ∈ N
where s is the solution to
ṡ =
1
N
N
X
i=1
fi(s, t), s(0) =
1
N
N
X
i=1
xi(0) (Blended Dynamics)
if (Blended Dynamics) is incrementally stable.
1
fi(xi, ·) is uniformly bounded
2
semi-global result; becomes global if fi are globally Lipschitz in xi
5 / 39

7. Example
ẋ1 = −x1 + sin t + k
X
j∈N1
(xj − x1)
ẋ2 = −3x2 + cos t + k
X
j∈N2
(xj − x2)
ẋ3 = −2x3 + 1 + k
X
j∈N3
(xj − x3)
x1(0) = x2(0) = x3(0) = 1
Blended dynamics:
ṡ = −2s + (sin t + cos t + 1)/3
s(0) = 1
k = 0 k = 10 k = 50
6 / 39

8. Implication of the theorem
Heterogeneous multi-agent system
ẋi = fi(xi, t) + k
X
j∈Ni
(xj − xi) with strong coupling gain k ≫ 1
has the following properties:
1. has ability to “assemble” the desired vector field
2. exhibits emergent behavior, i.e. xi(t) ≈ s(t), governed by “emergent
dynamics”
ṡ =
1
N
N
X
i=1
fi(s, t) =: femergent(s, t)
3. stability of blended dynamics makes initial conditions of agents forgotten
→ useful to design initialization-free algorithm → helpful for plug-and-play
7 / 39

9. Finding the average of {a1, a2, a3}
Average consensus3
vs. blended dynamics approach
Using ‘average consensus’ algorithm
ẋ1 =
X
j∈N1
(xj − x1), x1(0) = a1
ẋ2 =
X
j∈N2
(xj − x2), x2(0) = a2
ẋ3 =
X
j∈N3
(xj − x3), x3(0) = a3
xi(t) →
a1 + a2 + a3
3
, ∀i
Using ‘blended dynamics’ approach
ẋ1 = −x1 + a1 + k
X
j∈N1
(xj − x1)
ẋ2 = −x2 + a2 + k
X
j∈N2
(xj − x2)
ẋ3 = −x3 + a3 + k
X
j∈N3
(xj − x3)
ṡ = −s +
a1 + a2 + a3
3
∴ xi(t) →≈
a1 + a2 + a3
3
, ∀i
3
e.g., the textbook Lectures on Network Systems (F. Bullo)
8 / 39

10. Implication of the theorem
Heterogeneous multi-agent system
ẋi = fi(xi, t) + k
X
j∈Ni
(xj − xi) with strong coupling gain k ≫ 1
has the following properties:
4. does not ask each agent to be stable, as long as the blended dynamics is
stable
5. when N ≫ 1, robust against a few malfunctioning agents because stability
is traded in the sense that
ṡ =
1
N
N
X
i=1
fi(s, t) =
1
N
X
i∈Ngood
f∗
(s, t) +
X
i∈Nbad
fi(s, t)
9 / 39

11. An alternative way to control a plant
(How to educate your children?)
desired: ẋ = f∗(x)
(teach them) (make them good friends)
10 / 39

12. A few examples:
1. Economic Power Dispatch for Smart Grid
2. Network Size Estimation
3. Distributed Control of Multi-channel Plant
4. Distributed p-quantile solver
5. Coupled Oscillator

13. Economic Power Dispatch Problem for Smart Grid
An example of distributed optimization
xi: power generation, di: power demand,
Ji(xi): strict convex cost function of power generation at node i
11 / 39

14. Problem formulation
Find x1, · · · , xN that solve
minimize
N
X
i=1
Ji(xi)
subject to
N
X
i=1
xi =
N
X
i=1
di,
xi ≤ xi ≤ x̄i, ∀i = 1, · · · , N
▶ solve it at a ‘center’? → not scalable, privacy problem
▶ solve it at each node? → need distributed optimization
12 / 39

15. (Centralized) Solution
Make the dual function
g(λ) =
N
X
i=1
inf
xi≤xi≤x̄i
(Ji(xi) + λ(xi − di)) =:
N
X
i=1
gi(λ)
Solve λ∗ of the dual problem
maxλ g(λ)
The optimal x∗
i is
x∗
i = vi(λ∗
) :=
dJi
dxi
−1
sat
dJi
dxi
(xi)
dJi
dxi
(xi)
(λ∗
)
!
(consisting of private information)
The dual problem can be solved (at the center) by
λ̇ = ∇g(λ) = ∇g1(λ) + · · · + ∇gN (λ) (so that λ(t) → λ∗
)
13 / 39

16. Distributed Solution (Hyeonjun Yun, S, Hyo-Sung Ahn, AUT 2019)
Agent i:
λ̇i = ∇gi(λi) + k
X
j∈Ni
(λj − λi), xi = vi(λi)
Blended dynamics:
ṡ =
1
N
(∇g1(s) + · · · + ∇gN (s)) =
1
N
∇g(s)
Therefore,
s(t) → λ∗;
λi(t) → λ∗ approximately;
xi(t) = vi(λi(t)) → x∗
i approximately
♣ distributed operation + plug-and-play + (decentralized design)
▶ no private information is exchanged
14 / 39

17. Simulation for IEEE 118-bus system
15 / 39

18. Power generations xi(t) and the cost
PN
i=1 Ji(xi(t))
0 10 20 30 40 50
Time
0
50
100
150
200
250
300
350
400
450
Power
0 10 20 30 40 50
Time
5
5.5
6
6.5
7
Cost
×104
dg+sac
Global opt.
16 / 39

19. Blended dynamics theorem: PI coupling
(Seungjoon Lee S, arxiv 2021), (Taekyoo Kim, Donggil Lee, S, arxiv 2021)
Heterogeneous multi-agent system:
ẋi = fi(xi) + k
X
j∈Ni
(xj − xi) + k
X
j∈Ni
(ξj − ξi),
˙
ξi = −k
X
j∈Ni
(xj − xi), i ∈ N = {1, · · · , N}
⇒ ẋi = fi(xi) + uPI
i (k)
Theorem (PI): ∃k∗ s.t. ∀k k∗,
lim
t→∞
∥xi(t) − s∗
∥ = 0
if the blended dynamics
ṡ =
1
N
N
X
i=1
fi(s)
has an exponentially stable equilibrium s∗.
17 / 39

20. Network size estimation
Prob: How to compute # of agents in a network?
18 / 39

21. Network size estimation (Donggil Lee et al., CDC 2018), (Taekyoo Kim, Donggil Lee, S, arxiv 2021)
Sol: Let all participating agents run
ẋi = 1 + uPI
i (k)
and add a special agent 0 which runs
ẋ0 = −x0 + uPI
0 (k)
Reason: Blended dynamics becomes
ṡ = −
1
N + 1
s +
N
N + 1
and so s(t) → N
∴ ∃k∗ s.t., with k k∗,
lim
t→∞
xi(t) = N
♣ distributed operation
♣ decentralized design with
k∗
= 0
♣ plug-and-play:
# of agents: 3 ⇒ 2 ⇒ 3
19 / 39

22. A few examples:
1. Economic Power Dispatch for Smart Grid
2. Network Size Estimation
3. Distributed Control of Multi-channel Plant
4. Distributed p-quantile solver
5. Coupled Oscillator

23. Distributed stabilization of multi-channel linear systems4
Each control agent i
knows (A, Bi, Ci),
measures yi, and
generates ui,
only under the local
communication with its neighbors.
Prob: Design control agents for
▶ distributed operation
▶ decentralized design
▶ plug-and-play
4
a problem extended from (Wang, Fullmer, Liu, Morse, ACC 2020).
20 / 39

24. Assumption
If (A, Bi, Ci) were controllable and observable (← maybe too much), one agent
˙
x̂i = Ax̂i + Biui + Li(Cix̂i − yi), ui = Fix̂i
could stabilize the plant.
Assumption:
A, {Bi}N(t), {Ci}N(t)
is controllable and observable
where N(t) is the index set of participating agents at time t
21 / 39

25. Initial thought: design by separation principle (distributed observer + state feedback)
for the time being, let us assume
▶ N = {1, 2, . . . , N} is fixed
▶ every agent knows N
▶ each agent i knows Fi and Li, which satisfy
A +
B1, · · · , BN



F1
.
.
.
FN


 is Hurwitz and A +
L1, · · · , LN



C1
.
.
.
CN


 is Hurwitz
(proposed) control agent i:
˙
x̂i = Ax̂i + NBiFix̂i + NLi(Cix̂i − yi) + k
X
i∈N
(x̂j − x̂i), ui = Fix̂i
(BD) ṡ = (A + BF + LC)s − Ly (plant) ẋ = Ax +
PN
i=1 BiFix̂i ⇒ Ax + BFs
22 / 39

26. Idea of decentralized design of Fi and Li (Taekyoo Kim, Donggil Lee, S, arxiv 2020)
Bass’ Algorithm: The solution X 0 to
−(A + βI)X − X(A + βI)T
+ 2BBT
= 0
where (A, B) is controllable and β max Reλ(A), makes (A + BF) Hurwitz where
F = −BT
X−1
or



F1
.
.
.
FN


 = −



BT
1 X−1
.
.
.
BT
N X−1


 .
Can we solve X distributedly? Yes. Because X(t) → X where
Ẋ(t) = −(A + βI)X(t) − X(t)(A + βI)T
+ 2BBT
(♡)
let us solve (♡) distributely!
23 / 39

27. Blended dynamics:
Ẋ(t) = −(A + βI)X(t) − X(t)(A + βI)T
+ 2BBT
(♡)
Agent i runs:
Ẋi = −(A + βI)Xi − Xi(A + βI)T
+ 2NBiBT
i + uPI
i (kX), Fi = −BT
i X−1
i
We do the same for the observer gain Li.
Ẏi = −Yi(A + βI) − (A + βI)T
Yi − 2NCT
i Ci + uPI
i (kY ), Li = Y −1
i CT
i
24 / 39

28. Proposed control agent i
With added agent 0, N can be replaced by the network size estimator.
˙
x̂i = Ax̂i + +Ni(t)BiFi(t)x̂i + Ni(t)Li(t)(Cix̂i − yi) + kO
X
i∈Ni
(x̂j − x̂i), ui = Fi(t)x̂i
Ṅi = 1 + uPI
i (kN )
Ẋi = −(A + βI)Xi − Xi(A + βI)T
+ 2Ni(t)BiBT
i + uPI
i (kX), Fi(t) = −BT
i X−1
i
Ẏi = −Yi(A + βI) − (A + βI)T
Yi − 2Ni(t)CT
i Ci + uPI
i (kY ), Li(t) = Y −1
i CT
i
achieves
♣ distributed operation
♣ decentralized design
♣ plug-and-play (for not too often changes)
25 / 39

29. Collaborative mobile robots
⇒ ⇒
26 / 39

30. A few examples:
1. Economic Power Dispatch for Smart Grid
2. Network Size Estimation
3. Distributed Control of Multi-channel Plant
4. Distributed p-quantile solver
5. Coupled Oscillator

31. Distributed p-quantile solver (Jeong Mo Seong, Jeong Woo Kim, Seungjoon Lee, S, CDC 2021)
Prob: Given distributed data {z̄1, · · · , z̄N },
find p-quantile distributedly.
Sol:
ẋi = p −
1
2
+
1
2
sign(z̄i − xi) + uPI
i (k)
Reason: Blended dynamics becomes
ṡ = p −
1
2
+
1
2N
N
X
i=1
sign(z̄i − s)
which is the gradient descent algorithm for
min
s
1
N
N
X
i=1
1
2
− p
(s − z̄i) +
1
2
|s − z̄i|
(when p = 0.3 and data = {1, 3, 6, 6, 8})
1
2 − p
(s − z̄i) + 1
2 |s − z̄i|
1
N
PN
i=1
1
2 − p
(s − z̄i) + 1
2 |s − z̄i|
27 / 39

32. Application for cyber-security (Jin Gyu Lee, Junsoo Kim, S, TAC, Special Issue, 2020)
Q: How to ignore outliers by distributed computation?
A: Use a distributed median solver
28 / 39

33. Coupled oscillators
Tom Morris (@palerlotus)
29 / 39

34. Stability of Coupled Oscillators (Jin Gyu Lee S; NOLCOS 2019)
Liénard system
ẍ + f(x)ẋ + g(x) = 0
exhibits a stable oscillatory behavior if
xg(x) 0 ∀x ̸= 0
lim
x→±∞
Z x
0
f(ξ)dξ = ±∞
x
Z x
0
f(ξ)dξ 0 ∀x(̸= 0) near 0.
Networked Liénard system
ẍi + fi(xi)ẋi + gi(xi) = ui
yi = xi + ẋi
ui = k
X
j∈Ni
(yj − yi)
exhibits a stable oscillatory behavior if
x
1
N
N
X
i=1
gi(x)
!
0 ∀x ̸= 0
lim
x→±∞
Z x
0
1
N
N
X
i=1
fi(ξ)
!
dξ = ±∞
x
Z x
0
1
N
N
X
i=1
fi(ξ)
!
dξ 0 ∀x(̸= 0) near 0
30 / 39

35. Blended dynamics theorem: Output coupling (Jin Gyu Lee S, AUT 2020)
Heterogeneous multi-agent system:
żi = gi(zi, yi)
ẏi = hi(zi, yi) + k
X
j∈Ni
(yj − yi), i ∈ N = {1, · · · , N}
Theorem (output): ∀ϵ 0, ∃k∗ s.t. with k k∗,
lim sup
t→∞
zi(t)
yi(t)
−
ẑi(t)
s(t)
≤ ϵ, ∀i
where ẑi and s are the solution to
˙
ẑi = gi(ẑi, s)
ṡ =
1
N
N
X
i=1
hi(ẑi, s)
(Blended Dynamics)
if (Blended Dynamics) is incrementally stable.
31 / 39

36. Analysis of coupled oscillators
With zi = xi and yi = xi + ẋi, networked Liénard system:
żi = −zi + yi,
ẏi = (1 − fi(zi))(−zi + yi) − gi(zi) + k
X
j∈Ni
(yj − yi)
Blended dynamics:
˙
ẑi = −ẑi + s, i ∈ N,
ṡ =
1
N
N
X
i=1
[(1 − fi(zi)) (−zi + s) − gi(zi)]
Since zi(t) → z(t), ∀i ∈ N as t → ∞, blended dynamics tends to
ż = −z + s
ṡ =
1
N
N
X
i=1
[(1 − fi(z)) (−z + s) − gi(z)] = 1 −
1
N
N
X
i=1
fi(z)
!
(−z + s) −
1
N
N
X
i=1
gi(z)
32 / 39

37. Simulation
♣ distributed operation + plug-and-play + (decentralized design)
33 / 39

38. Robustness of coupled oscillators (Jaeyong Kim et al., TAC 2016)
Prob
lim sup
t→∞
zi(t)
yi(t)
−
ẑi(t)
s(t)
≤ ϵ, ∀i
→ 1 as N → ∞
34 / 39

39. Impulsive coupling
35 / 39

40. Hybrid system analysis (Tanwani S, CDC 2021, under review)
Networked Liénard system:
żi = −zi + yi,
ẏi = (1 − fi(zi))(−zi + yi) − gi(zi)
+ k
X
j∈Ni
(yj − yi)
Networked Liénard system:
żi = −zi + yi,
ẏi = (1 − fi(zi))(−zi + yi) − gi(zi)
At a random time, for any pair (j, k)
randomly chosen,
y+
j = (1 − ϵ)yj + ϵyk
y+
k = (1 − ϵ)yk + ϵyj.
If these updates are sufficiently rich, we
recover the same result.
36 / 39

41. Take-home message
multi-agent system: ẋi = fi(xi) + k
X
j∈Ni
(xj − xi)
blended dynamics: ṡ =
1
N
N
X
i=1
fi(s)
If blended dynamics is stable, then, with k ≫ 1,
xi(t) →≈ s(t).
37 / 39

42. Sketch of proof: in view of singular perturbation theory
ẋi = fi(xi) + k
X
j∈Ni
(xj − xi), i ∈ N
⇒ ẋ =



ẋ1
.
.
.
ẋN


 =



f1(x1)
.
.
.
fN (xN )


 − kLx
Let zo = (1/N)1⊤
N x and z̃ = R⊤x where
1
N 1⊤
N
R⊤
L
1N R
=
0 0
0 Λ
⇒



x1
.
.
.
xN


 = 1N zo + Rz̃ =



zo + r1z̃
.
.
.
zo + rN z̃



żo =
1
N
N
X
i=1
fi(zo + riz̃)
1
k
˙
z̃ = −Λz̃ +
1
k
R⊤



f1(zo + r1z̃)
.
.
.
fN (zo + rN z̃)



Boundary-layer subsystem:
dz̃
dτ
= −Λz̃ with τ = kt
Quasi-steady-state subsystem:
żo =
1
N
N
X
i=1
fi(zo) blended dynamics
38 / 39

43. THANK YOU!
Recommended reading for interested audience:
▶ 심형보, “Multi-agent system의 Consensus에 대해 알아보자,”
제어로봇시스템학회지 2019년 12월호 (available online at dbpia.co.kr)
▶ J. Lee and H. Shim (2021). Design of heterogeneous multi-agent system for
distributed computation. In Z.- P. Jiang et al. (Eds.), Trends in Nonlinear and
Adaptive Control, Lecture Notes in Control and Information Sciences 488,
Springer. (available online at www.arxiv.org)
For more information, visit http://cdsl.kr.
39 / 39