ADI for Tensor Structured Equations

Workshop on Matrix Equations and
Tensor Techniques 2011 Aachen,
21 November 2011

ADI for Tensor Structured Equations
Thomas Mach and Jens Saak

Max Planck Institute for Dynamics of Complex Technical Systems
Computational Methods in Systems and Control Theory

MAX PLANCK INSTITUTE
FOR DYNAMICS OF COMPLEX
TECHNICAL SYSTEMS
MAGDEBURG

Max Planck Institute Magdeburg Thomas Mach, Jens Saak, Tensor-ADI 1/37

ADI ADI for Tensors Numerical Results and Shifts Conditioning of the Problem Conclusions

Classic ADI [Peaceman/Rachford ’55]

Developed to solve linear systems related to Poisson problems

−∆u = f in Ω ⊂ Rd , d = 2
u=0 on ∂Ω.

uniform grid size h, centered diﬀerences, d = 1,

⇒ ∆1,h u = h2 f
 
2 −1
−1 2 −1 
 
∆1,h =
 .. .. .. .

 . . . 
 −1 2 −1
−1 2




Developed to solve linear systems related to Poisson problems

−∆u = f in Ω ⊂ Rd , d = 2
u=0 on ∂Ω.

uniform grid size h, 5-point diﬀerence star, d = 2,

⇒ ∆2,h u = h2 f
   
K −I 4 −1
−I K −I  −1 4 −1 
   
∆2,h =
 .. .. ..  and K = 
  .. .. .. .

 . . .   . . . 
 −I K −I   −1 4 −1
−I K −1 4




Observation
∆2,h = (∆1,h ⊗ I ) + (I ⊗ ∆1,h ).
=:H =:V




Observation
∆2,h = (∆1,h ⊗ I ) + (I ⊗ ∆1,h ).
=:H =:V

˜
Solve ∆2,h u = h2 f =: f exploiting structure in H and V .




Observation
∆2,h = (∆1,h ⊗ I ) + (I ⊗ ∆1,h ).
=:H =:V

˜
Solve ∆2,h u = h2 f =: f exploiting structure in H and V .

For certain shift parameters perform
˜
(H + pi I ) ui+ 1 = (pi I − V ) ui + f ,
2
˜
(V + pi I ) ui+1 = (pi I − H) ui+ 1 + f ,
2

until ui is good enough.



ADI and Lyapunov Equations [Wachspress ’88]

Lyapunov Equation
FX + XF T = −GG T




Lyapunov Equation
FX + XF T = −GG T

Vectorized Lyapunov Equation
(I ⊗ F ) + (F ⊗ I ) vec(X ) = −vec(GG T )
=:HF =:VF

Same structure ⇒ apply ADI




Lyapunov Equation
FX + XF T = −GG T

Vectorized Lyapunov Equation
(I ⊗ F ) + (F ⊗ I ) vec(X ) = −vec(GG T )
=:HF =:VF

Same structure ⇒ apply ADI

(F + pi I ) Xi+ 1 = −GG T − Xi F T − pi I
2

(F + pi I ) Xi+1 = −GG T − Xi+ 1 F T − pi I
T
2



LR-ADI for Lyapunov Equations

Lyapunov Equation
FX + XF T = −GG T




Lyapunov Equation
FX + XF T = −GG T

Often singular values of X decay rapidly when G is “thin”.
X = ZZ T with Z “thin”.




Lyapunov Equation
FX + XF T = −GG T

Often singular values of X decay rapidly when G is “thin”.
X = ZZ T with Z “thin”.

LR-ADI [Penzl ’99, Li/White ’02]

Z0 = [] V1 = −2 Re (p1 )(F + p1 I )−1 G
Re (pi )
Vi = I − (pi + pi−1 )(F + pi I )−1 Vi−1
Re (pi−1 )
Zi = [Zi−1 , Vi ]



Generalizing Matrix Equations

∆2,h vec(X ) = vec(B)
I ⊗ ∆1,h + ∆1,h ⊗ I vec(X ) = vec(B)

=H =V =u =f

∆µa a
Xa c

+ = Ba c
c ∆µc
Xa c



Generalizing Matrix Equations

∆4,h vec(X ) = vec(B)
I ⊗ I ⊗ I ⊗ ∆1,h + I ⊗ I ⊗ ∆1,h ⊗ I + I ⊗ ∆1,h ⊗ I ⊗ I + ∆1,h ⊗ I ⊗ I ⊗ I vec(X ) = vec(B)

=H =V =R =Q =u =f

∆µa a
Xabcd + Xabcd
∆µb b
+ = Babcd
c ∆µc
Xabcd + Xabcd
d ∆µd



Generalizing ADI

I ⊗ ∆1,h + ∆1,h ⊗ I vec(X ) = vec(B)

=H =V =u =f

(H + I ⊗ pi,1 I )Xi+ 1 = (pi,1 I − V )Xi + B
2
(V + pi,2 I ⊗ I )Xi+ 1 = (pi,2 I − H)Xi+ 1 + B
2 2



Generalizing ADI

I ⊗ ∆1,h + ∆1,h ⊗ I vec(X ) = vec(B)

=H =V =u =f

(H + I ⊗ pi,1 I )Xi+ 1 = (pi,1 I − V )Xi + B
2
(V + pi,2 I ⊗ I )Xi+ 1 = (pi,2 I − H)Xi+ 1 + B
2 2

I ⊗ I ⊗ I ⊗ ∆1,h + I ⊗ I ⊗ ∆1,h ⊗ I + I ⊗ ∆1,h ⊗ I ⊗ I + ∆1,h ⊗ I ⊗ I ⊗ I vec(X ) = vec(B)

=H =V =R =Q =u =f

(H + I ⊗ I ⊗ I ⊗ pi,1 I )Xi+ 1 = (pi,1 I − V − R − Q)Xi +B
4
(V + I ⊗ I ⊗ pi,2 I ⊗ I )Xi+ 1 = (pi,2 I − H − R − Q)Xi+ 1 +B
2 4
(R + I ⊗ pi,3 I ⊗ I ⊗ I )Xi+ 3 = (pi,3 I − H − V − Q)Xi+ 1 +B
4 2
(Q + pi,4 I ⊗ I ⊗ I ⊗ I )Xi+1 = (pi,4 I − H − V − R)Xi+ 3 +B
4



Goal

Solve AX = B

A = I ⊗ I ⊗ · · · ⊗ I ⊗ I ⊗ A1 +
I ⊗ I ⊗ · · · ⊗ I ⊗ A2 ⊗ I +
... +
Ad ⊗ I ⊗ · · · ⊗ I ⊗ I ⊗ I

B is given in tensor train decomposition
⇒ X is sought in tensor train decomposition.



Tensor Trains [Oseledets, Tyrtyshnikov ’09]

r1 ,...,rd−1
T (i1 , i2 , . . . , id ) = G1 (i1 , α1 )G2 (α1 , i2 , α2 )
α1 ,...,αd−1 =1

· · · Gj (αj−1 , ij , αj ) · · ·
Gd−1 (αd−2 , id−1 , αd−1 )Gd (αd−1 , id ).




r1 ,...,rd−1
T (i1 , i2 , . . . , id ) = G1 (i1 , α1 )G2 (α1 , i2 , α2 )
α1 ,...,αd−1 =1

· · · Gj (αj−1 , ij , αj ) · · ·
Gd−1 (αd−2 , id−1 , αd−1 )Gd (αd−1 , id ).

G1 (i1 , α1 ) α1 G2 (α1 , i2 , α2 ) α2 ··· Gd (αd−1 , id )




Tensor trains are
computable, and
d
require only O(dnr 2 ) storage, with TT-rank r and T ∈ Rn .

Canonical representation

T (i1 , i2 , . . . , id ) = G1 (i1 , α) · · · Gd (id , α)
α

Tucker decomposition

T (i1 , i2 , . . . , id ) = C (α1 , . . . , αd )G1 (i1 , α1 ) · · · Gd (id , αd )
α1 ,...,αd




(I ⊗ · · · ⊗ I ⊗ A1 ) T




(I ⊗ · · · ⊗ I ⊗ A1 ) T

A1 (β, i1 )

i1





(I ⊗ · · · ⊗ I ⊗ A1 ) T

A1 (β, i1 )

i1


T (i1 , i2 , . . . , id ) ×1 A1 = A1 β,i1 G1 (i1 , α1 )G2 (α1 , i2 , α2 )
α1 ,...,αd−1

· · · Gd−1 (αd−2 , id−1 , αd−1 )Gd (αd−1 , id )



(I ⊗ · · · ⊗ I ⊗ A1 ) T

˜
= G (β, α1 ) = A1 G1
A1 (β, i1 )

i1


T (i1 , i2 , . . . , id ) ×1 A1 = A1 β,i1 G1 (i1 , α1 )G2 (α1 , i2 , α2 )
α1 ,...,αd−1

· · · Gd−1 (αd−2 , id−1 , αd−1 )Gd (αd−1 , id )



(I ⊗ · · · ⊗ I ⊗ A1 ) T

˜
= G (β, α1 ) = A1 G1
A1 (β, i1 )

i1


T (i1 , i2 , . . . , id ) ×1 A1 −1 = A1 −1 β,i1 G1 (i1 , α1 )G2 (α1 , i2 , α2 )
α1 ,...,αd−1

· · · Gd−1 (αd−2 , id−1 , αd−1 )Gd (αd−1 , id )


Eigenvalues

A = I ⊗ · · · ⊗ I ⊗ A1 + I ⊗ · · · ⊗ I ⊗ A2 ⊗ I + . . . + Ad ⊗ I ⊗ · · · ⊗ I
St´phanos’ theorem:
e

⇒ λi (A) = λi1 (A1 ) + λi2 (A2 ) + · · · + λid (Ad ),
d−1
with i = i1 + i2 n1 + · · · + id nj .
j=1



Eigenvalues

A = I ⊗ · · · ⊗ I ⊗ A1 + I ⊗ · · · ⊗ I ⊗ A2 ⊗ I + . . . + Ad ⊗ I ⊗ · · · ⊗ I
e

⇒ λi (A) = λi1 (A1 ) + λi2 (A2 ) + · · · + λid (Ad ),
d−1
with i = i1 + i2 n1 + · · · + id nj .
j=1

d
AX = B ⇔ X ×j Aj = B
j=1

A is regular ⇔ λi (A) = 0 ∀i ⇐ Ai Hurwitz ∀i



Algorithm

Input: {A1 , . . . , Ad }, tensor train B, accuracy
Output: tensor train X , with AX = B
forall j ∈ {1, . . . , d} do
(0)
Xj := zeros(n, 1, 1)
end
while r (i) > do
Choose shift pi
forall k ∈ {1, . . . , d} do
d
×j Aj ×k (Ak + pi I )−1
k k−1 k−1
X (i+ d ) := B +pi X (i+ d ) − X (i+ d )

j=1
j=k
end
end



Algorithm
r (i) := B
forall j ∈ {1, . . . , d} do
Output: tensor train X , with AX(i) B =
r (i) := r − Xi ×j Aj
forall j ∈ {1, . . . , d} do
(0) end
end
while r (i) > do
Choose shift pi
forall k ∈ {1, . . . , d} do
d
×j Aj ×k (Ak + pi I )−1
k k−1 k−1
X (i+ d ) := B +pi X (i+ d ) − X (i+ d )

j=1
j=k
end
end



Algorithm

Output: tensor train X , with AX = B
forall j ∈ {1, . . . , d} do
(0)
end (I ⊗ I ⊗ · · · ⊗ I ⊗ Aj ⊗ I ⊗ · · · ⊗ I ) Xi+ k−1
d
while r (i) > do
Choose shift pi
forall k ∈ {1, . . . , d} do
d
×j Aj ×k (Ak + pi I )−1
k k−1 k−1
X (i+ d ) := B +pi X (i+ d ) − X (i+ d )

j=1
j=k
end
end



Improvements
better shifts, e.g. Wachspress/Penzl
test residual in innermost loop
replace inner loop by random k
use tensor train truncation and start with low accuarcy



Improvements

[...]
while r (i) > do
Choose shifts pi,j
forall k ∈ {1, . . . , d} do

k −1
d
k k −1
X (i+ d ) := B + pi,k X (i+ d ) − X (i+ d ) ×j Aj ×k (Ak − pi,k I )
j=1
j=k
end
end



Improvements

[...]
while r (i) > do
Choose shifts pi,j
forall = 1, . . . , 5 do
k := random(1, . . . , d)
−1
d
−1
X (i+ 5 ) := B + pi, X (i+ 5 ) − X (i+ 5 ) ×j Aj ×k (Ak − pi, I )
j=1
j=k
end
end



Improvements
Additional remark:
[...] The improvement with ran-
while r (i) > do dom chosen directions works
for the Lyapunov case with
Choose shifts pi,j
the special right hand side as
forall = 1, . . . , 5 do
k := random(1, . . . , d) in the examples. The investi-
gation of the convergence be-
−1
d
−1
X (i+ 5 ) := B + pi, X (i+ 5havior for(i+ 5 ) ×j Arhs×k (Ak − pi, I )
)
− X random j is work
j=1
in progress.
j=k
end
end



Improvements

1 · 105
constant truncation error 10−2

i
Storage in Double

80,000 tightened truncation error

Truncation Error
60,000 10−8
40,000
10−14
20,000

0 10−20
0 5 10 15 20 25 30
Iteration


Lemma

Lemma [Grasedyck ’04]
The tensor equation
d
j=1 X ×j Aj = B

with λi (A) = 0 ∀i and Ak Hurwitz has the solution
∞
X =− 0 B ×1 exp(A1 t) ×2 · · · ×d exp(Ad t)dt

Z (t) = B ×1 exp(A1 t) ×2 · · · ×d exp(Ad t)
d ∞
˙
Z (t) = Z (t) ×j Aj Z (∞) − Z (0) = ˙
Z (t)dt,
j=1 0

d ∞
0−B = Z (t)dt ×j Aj
j=1 0



Theorem
Theorem
{A1 , . . . , Ad } ⇒ A, Λ(A) ⊂ [−λmax , −λmin ] ⊕ ı [−µ, µ] ⊂ C− .
Let k ∈ N and use the quadrature points and weights:
√
hst := √k , tj := log e jhst + 1 + e 2jhst , wj := √ hst−2jh .
π
1+e st

Then the solution X can be approximated by
r1 ,...,rd−1
˜
X (i1 , i2 , . . . , id ) = − H1 (i1 , α1 ) · · · Hd (αd−1 , id ),
α1 ,...,αd−1 =1

2tj
2wj Ap
with Hp (αp−1 , ip , αp ) := k
j=−k λmin βp e
λmin
ip ,βp
Gp (αp−1 , βp , αp )
with the approximation error
2µλ−1 +1 √
(λI − 2A/λmin )−1
min
˜
X −X ≤ Cst −π k
πλmin e dΓ λ B 2.
2 π
Γ 2
extending [Grasedyck ’04] (X and B of low Kronecker rank) to low TT-rank


Theorem
Theorem
{A1 , . . . , Ad } ⇒ A, Λ(A) ⊂ [−λmax , −λmin ] ⊕ ı [−µ, µ] ⊂ C− .
Let k ∈ N and use the quadrature points and weights:
√
hst := √k , tj := log e jhst + 1 + e 2jhst , wj := √ hst−2jh .
π
1+e st
B (i1 , i2 , . . . , id ) = G1 (i1 , α1 )G2 (α1 , i2 , α2 )
α1 ,...,αd−1
Then the solution X can be approximated by
· · · Gj (αj−1 , ij , αj ) · · ·
r1 ,...,rd−1
˜
X (i1 , i2 , . . . , id ) = − Gd−1 (αd−2 ,(i , α d−1·)GH (α , id ). i
H id−1 , α ) · · d (αd−1 ,
1 1 1 d d−1 d ),
α1 ,...,αd−1 =1

2tj
2wj Ap
with Hp (αp−1 , ip , αp ) := k
j=−k λmin βp e
λmin
ip ,βp
Gp (αp−1 , βp , αp )
with the approximation error
2µλ−1 +1 √
(λI − 2A/λmin )−1
min
˜
X −X ≤ Cst −π k
πλmin e dΓ λ B 2.
2 π
Γ 2
extending [Grasedyck ’04] (X and B of low Kronecker rank) to low TT-rank


Proof – Part 1
The quadrature formula (tj , wj ) can be found in [Stenger ’93,
Example 4.2.11], with d = π/2, α = β = = 1, n = N = M = k.
[Hackbusch ’09, D.4.3] shows, that d = π/2 is optimal. The
quadrature formula is used to approximate 1 resp. the inverse of
r
matrices by
∞ k
1 tr
= e dt ≈ wj e tj r .
−r 0 j=−k

The quadrature error is bounded by [Stenger ’93, (4.2.60)]
∞ k √
tr
e dt − wj e tj r ≤ C3 e −π k
,
0 j=−k

with [Grasedyck, Hackbusch, Khoromskij ’03],
C3 ≤ Cst e | (z)|/π
.


Proof – Part 2
The lemma
∞
X =− B ×1 exp(A1 t) ×2 · · · ×d exp(Ad t)dt
0
2
together with scaling λmin we get the formula

r1 ,...,rd−1
˜
X (i1 , i2 , . . . , id ) = − H1 (i1 , α1 ) · · · Hd (αd−1 , id ),
α1 ,...,αd−1 =1

k 2tj
2wj Ap
where Hp (αp−1 , ip , αp ) := λmin e λmin ip ,βp
Gp (αp−1 , βp , αp ).
j=−k βp

∞ k
1
−r = e tr dt ≈ wj e tj r
0 j=−k



Proof – Part 3
For the error holds
r1 ,...,rd−1 d
˜
X −X = Gp (αp−1 , βp , αp )
2
α1 ,...,αd−1 =1 p=1 βp
 
∞ k 2tj
2t
A
λmin p
A
λmin p
− e + wj e  .
0 j=−k
ip ,βp 2

The Dunford-Cauchy formula and the quadrature error give
∞ k 2tj
2t Ap Ap
− e λmin + wj e λmin
0 j=−k

∞ −1 k −1
1 2Ap 2Ap
≤ − e tλ λI − dΓ λ + wj e tj λ λI − dΓ λ
2π Γ 0 λmin Γ λmin
j=−k



Proof – Part 3

Dunford-Cauchy representation of the matrix exponential:

e tAp = 1
2πı Γe
tλ (λI − Ap )−1 dΓ λ

∞ k 2tj
2t Ap Ap
0 j=−k

∞ −1 k −1
1 2Ap 2Ap
j=−k



Proof – Part 3
∞ k 2tj
2t Ap Ap
0 j=−k

∞ −1 k −1
1 2Ap 2Ap
j=−k



Proof – Part 3
∞ k 2tj
2t Ap Ap
0 j=−k

∞ −1 k −1
1 2Ap 2Ap
j=−k
 
∞ k −1
1  2Ap
≤ − e tλ + wj e tj λ  λI − dΓ λ
2π 0 Γ λmin
j=−k



Proof – Part 3
∞ k 2tj
2t Ap Ap
0 j=−k

∞ −1 k −1
1 2Ap 2Ap

The quadrature error is bounded by [Stenger ’93, (4.2.60)]

j=−k

∞ k −1
1  2Ap
≤ − e tλ + w∞tj λ 
je λI −
k dΓ λ √
2π 0 tr Γ λmin t r −π k
j=−k
e dt − wj e j
≤ C3 e ,
0 j=−k

with [Grasedyck, Hackbusch, Khoromskij ’03],

C3 ≤ Cst e | (z)|/π
.



Proof – Part 3
∞ k 2tj
2t Ap Ap
0 j=−k

∞ −1 k −1
1 2Ap 2Ap
j=−k
 
∞ k −1
1  2Ap
≤ − e tλ + wj e tj λ  λI − dΓ λ
2π 0 Γ λmin
j=−k

−1
1 | (z)| −π
√ 2Ap
≤ Cst e π e k λI − dΓ λ.
2π Γ λmin
2

Summing over p completes the proof.



Numerical Results



Example: Laplace – Ai = ∆1, 11
1

Ai = ∆1, 1
11

B = 0 0 ... 0 1

Shifts:
pi := e1 (∗1 ) + . . . + ed (∗d ) — random chosen eigenvalue



Numerical Results – Ai = ∆1, 11
1

d t in s residual mean(#it)
2 0.3887 e+00 7.015 e−10 112.8
5 5.3975 e+00 7.467 e−10 45.8
8 6.0073 e+00 6.936 e−10 12.8
10 3.6624 e+00 7.685 e−10 6.8
25 3.1421 e+01 2.437 e−10 5.0
50 2.2682 e+02 2.049 e−10 5.0
75 7.1918 e+02 4.036 e−10 5.0
100 1.6997 e+03 1.864 e−10 5.0
150 5.5375 e+03 1.801 e−10 5.0
200 1.2795 e+04 1.472 e−10 5.0
250 2.4991 e+04 1.816 e−10 5.0
300 4.2979 e+04 2.535 e−10 5.0
500 1.9515 e+05 2.039 e−10 5.0



1

sparse dense
d TADI MESS Penzl’s sh. lyap
2 0.310 0.0006 0.024 0.003 0.0003 0.0005
4 3.130 0.1695 0.011 0.049 6.331 0.012
6 8.147 — 0.076 0.094 — 7.165
8 5.458 — 5.863 1.097 — 13 698.212
10 5.306 — 3 445.523 249.464 — —



1

105

104
Computation Time in s

103

102
Tensor ADI
101 sparse
MESS
100
Penzl’s shifts
10−1 dense
lyap
10−2
10 100 300
Dimension d


Single Shift and Convergence

A = I ⊗ · · · ⊗ I ⊗ A1 + I ⊗ · · · ⊗ I ⊗ A2 ⊗ I + . . . + Ad ⊗ I ⊗ · · · ⊗ I
We assume Λ(Ak ) ⊂ R− .

Error Propagation, Single Shift
p− λk + λl λk
 
d d
k k
G1 2 ≤ max = 1 −  .
λk ∈Λ(Ak ), p + λl p + λl
k=1,...,d l=0 l=0

If G1 2 < 1, then the ADI iteration converges.
p < 0 and p > −∞




A = I ⊗ · · · ⊗ I ⊗ A1 + I ⊗ · · · ⊗ I ⊗ A2 ⊗ I + . . . + Ad ⊗ I ⊗ · · · ⊗ I

p− λk + λl λk
 
d d
k k
G1 2 ≤ max = 1 −  .
k=1,...,d l=0 l=0

p < 0 and p > −∞
d
p < λi (A) = k=1 λk (Ak ) ∀i




A = I ⊗ · · · ⊗ I ⊗ A1 + I ⊗ · · · ⊗ I ⊗ A2 ⊗ I + . . . + Ad ⊗ I ⊗ · · · ⊗ I

p− λk + λl λk
 
d d
k k
G1 2 ≤ max = 1 −  .
k=1,...,d l=0 l=0

p < 0 and p > −∞
d
p < λi (A) = k=1 λk (Ak ) ∀i
d−2
Lyapunov case (Ak = A0 ∀k): p < 2 λmin (A0 )




A = I ⊗ · · · ⊗ I ⊗ A1 + I ⊗ · · · ⊗ I ⊗ A2 ⊗ I + . . . + Ad ⊗ I ⊗ · · · ⊗ I

p− λk + λl λk
 
d d
k k
G1 2 ≤ max = 1 −  .
k=1,...,d l=0 l=0

p < 0 and p > −∞
d
p < λi (A) = k=1 λk (Ak ) ∀i
2−2
Lyapunov case (Ak = A0 ∀k): p < 2 λmin (A0 ) =0



Shifts

Min-Max-Problem
d
pi,k − j=k λj
min max
{p1,1 ,...,p ,d }⊂C λk ∈Λ(Ak ) ∀k pi,k + λk
i=0 k=0



Shifts

Min-Max-Problem
d
pi,k − j=k λj
min max
i=0 k=0

Min-Max-Problem, Lyapunov case (Ak = A0 ∀k, A0 Hurwitz)
d
pi,k − j=k λj
min max
{p1,1 ,...,p ,d }⊂C λk ∈Λ(A0 ) ∀k pi,k + λk
i=0 k=0



Shifts

Min-Max-Problem
d
pi,k − j=k λj
min max
i=0 k=0

d
pi − j=k λj
min max
{p1 ,...,p }⊂C λk ∈Λ(A0 ) ∀k pi + λk
i=0 k=0



Shifts

Min-Max-Problem
d
pi,k − j=k λj
min max
i=0 k=0

d
pi − j=k λj
min max
{p1 ,...,p }⊂C λk ∈Λ(A0 ) ∀k pi + λk
i=0 k=0

λk = λ0 ∀k
Penzl’s idea: {p1 , . . . , p } ⊂ (d − 1)Λ(A0 )



Random Example

seed := 1;
R := rand(10);
R := R + R ;
R := R − λmin + 0.1;
A0 = −R;
Λ(A0 ) = {−0.1000, −0.2250, −1.1024, −1.7496, −2.0355,
−2.4402, −3.1330, −3.3961, −3.9347, −11.9713}

⇒ The random shifts do not lead to convergence.

p0 = λ10 (A0 )(d − 1)
p1 = λ9 (A0 )(d − 1)



Numerical Results – Ai = −R
random k, test residual every 5 inner iterations, max. 250 iterations

d t in s residual #it
2 0.3627 2.6327 e−06 250
5 17.6850 1.4517 e−07 250
8 62.4336 9.3164 e−09 200
10 44.1547 8.5963 e−09 125
15 12.2231 5.0356 e−09 60
20 15.7506 3.3142 e−09 50
25 25.2221 3.6501 e−09 45
50 49.2004 5.4141 e−09 35
75 118.1297 6.8682 e−09 30
100 614.4017 2.4598 e−09 30



Conditioning of the Problem



Eigenvalues

A = I ⊗ · · · ⊗ I ⊗ A1 + I ⊗ · · · ⊗ I ⊗ A2 ⊗ I + . . . + Ad ⊗ I ⊗ · · · ⊗ I
e

⇒ λi (A) = λi1 (A1 ) + λi2 (A2 ) + · · · + λid (Ad ),
d−1
with i = i1 + i2 n1 + · · · + id nj .
j=1



Eigenvalues

A = I ⊗ · · · ⊗ I ⊗ A1 + I ⊗ · · · ⊗ I ⊗ A2 ⊗ I + . . . + Ad ⊗ I ⊗ · · · ⊗ I
e

⇒ λi (A) = λi1 (A1 ) + λi2 (A2 ) + · · · + λid (Ad ),
d−1
with i = i1 + i2 n1 + · · · + id nj .
j=1

d d
max |λl (A)| ≥ λmk (Ak ) ≥ Re (λmk (Ak )) , if ∀k : Ak Hurwitz
l
k=1 k=1
d
min |λl (A)| ≤ min |λlk (Ak )| .
l lk
k=1



Eigenvalues

A = I ⊗ · · · ⊗ I ⊗ A1 + I ⊗ · · · ⊗ I ⊗ A2 ⊗ I + . . . + Ad ⊗ I ⊗ · · · ⊗ I
e

⇒ λimk = λi1 (A1 argmax 2 ) + ·|λ·j (Ak )| (Ad ),
(A) = ) + λi2 (A · + λid
j∈{1,...,nk : Im (λi (Ak ))≥0}
d−1
with i = i1 + i2 n1 + · · · + id nj .
j=1

d d
max |λl (A)| ≥ λmk (Ak ) ≥ Re (λmk (Ak )) , if ∀k : Ak Hurwitz
l
k=1 k=1
d
min |λl (A)| ≤ min |λlk (Ak )| .
l lk
k=1



Normal Matrices Aj

Lemma
If all Ai , i = 1, . . . , d are normal, then also A is normal.

Proof.
Here d = 3, the extension to larger d is obvious.
AAT = A3 ⊗ I ⊗ I + I ⊗ A2 ⊗ I + I ⊗ I ⊗ A1 A3 ⊗ I ⊗ I + I ⊗ A2 ⊗ I + I ⊗ I ⊗ A1



Normal Matrices Aj

Lemma

Proof.
AAT = A3 ⊗ I ⊗ I + I ⊗ A2 ⊗ I + I ⊗ I ⊗ A1 AT ⊗ I ⊗ I + I ⊗ AT ⊗ I + I ⊗ I ⊗ AT
3 2 1



Normal Matrices Aj

Lemma

Proof.
3 2 1

= A3 AT ⊗ II ⊗ II
3 + A3 I ⊗ IAT ⊗ II
2 + A3 I ⊗ II ⊗ IAT
1

+ IAT ⊗ A2 I ⊗ II
3 + II ⊗ A2 AT ⊗ II
2 + II ⊗ A2 I ⊗ IAT
1

+ IAT ⊗ II ⊗ A1 I
3 + II ⊗ IAT ⊗ A1 I
2 + II ⊗ II ⊗ A1 AT
1



Normal Matrices Aj

Lemma

Proof.
3 2 1

= AT A3 ⊗ II ⊗ II
3 + IA3 ⊗ AT I ⊗ II
2 + IA3 ⊗ II ⊗ AT I
1

+ AT I ⊗ IA2 ⊗ II
3 + II ⊗ AT A2 ⊗ II
2 + II ⊗ IA2 ⊗ AT I
1

+ AT I ⊗ II ⊗ IA1 + II ⊗ AT I ⊗ IA1 + II ⊗ II ⊗ AT A1
3 2 1



Normal Matrices Aj

Lemma

Proof.
3 2 1

= AT A3 ⊗ II ⊗ II
3 + IA3 ⊗ AT I ⊗ II
2 + IA3 ⊗ II ⊗ AT I
1

+ AT I ⊗ IA2 ⊗ II
3 + II ⊗ AT A2 ⊗ II
2 + II ⊗ IA2 ⊗ AT I
1

3 2 1
T
= A3 ⊗ I ⊗ I + I ⊗ A2 ⊗ I + I ⊗ I ⊗ A1 A3 ⊗ I ⊗ I + I ⊗ A2 ⊗ I + I ⊗ I ⊗ A1
T
=A A



Normal Matrices Aj

Lemma

Proof. max |λl (A)|
σmax (A)
κ3, the= A 2 A−1 larger d is obvious.l
Here d = 2 (A) extension to 2 = =
σmin (A) min |λl (A)|
l
3
max |λlu (A0 )|
2 1
∀i lu
= AT A3 ⊗ II ⊗ II + IA3 ⊗ AT I ⊗ II + IA3 ⊗ II ⊗ AT I
3 2 = 1
Ai =A0 min |λll (A0 )|
+ A3 I ⊗ IA2 ⊗ II + II ⊗ A2 A2 ⊗ II + II ⊗ IAl2 ⊗ AT I
T T
l 1

3 2 1
T
= A3 ⊗ I ⊗ I + I ⊗ A2 ⊗ I + I ⊗ I ⊗ A1 A3 ⊗ I ⊗ I + I ⊗ A2 ⊗ I + I ⊗ I ⊗ A1
T
=A A



Lower Bounds for Non-Normal Matrices Aj

FX + XF T = −G (i.e., d = 2, A1 = A2 = F Hurwitz) [Zhou ’02]

max |λl (F )|
l
κ2 (A) ≥
min |λl (F )|
l





max |λl (F )|
l
κ2 (A) ≥
min |λl (F )|
l

by observing
σmax (A) = A 2 = sup Ay 2
y 2 =1

≥ sup Ay 2
y 2 =1,y EV

= max |λl (A)|
l
= 2 max |λl (F )|
l





max |λl (F )|
l
κ2 (A) ≥
min |λl (F )|
l

and
−1 A−1 y 2 −1
σmin (A) = A−1 2
= sup
y 2
y 2 Ay 2
= inf −1 y
= inf
A 2 y 2
Ay 2
≤ inf
y EV y 2
= min |λl (A)| = 2 min |λl (F )|
l l




Tensor Lyapunov Equations
(i.e., ∀i = 1, . . . , d, Ai = A0 Hurwitz) [M./S. ’11]

max |λl (A0 )|
l
κ2 (A) ≥
min |λl (A0 )|
l

σmax (A) ≥ max |λl (A)| = d max |λl (A0 )|
l l

σmin (A) ≤ min |λl (A)| = d min |λl0 (A0 )|
l l0


ADI for Tensor Structured Equations

ADI for Tensor Structured Equations

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (8)

Similar to ADI for Tensor Structured Equations

Similar to ADI for Tensor Structured Equations (20)

More from Thomas Mach

More from Thomas Mach (8)

Recently uploaded

Recently uploaded (20)

ADI for Tensor Structured Equations