Preconditioned Inverse Iteration for Hierarchical Matrices

Hierarchical (H-)Matrices PINVIT Numerical Results

Preconditioned Inverse Iteration for H-Matrices

Peter Benner and Thomas Mach

Mathematics in Industry and Technology
Department of Mathematics
TU Chemnitz

23rd Chemnitz FEM Symposium 2010

1/31 Thomas Mach Preconditioned Inverse Iteration for H-Matrices


Preconditioned Inverse Iteration for H-Matrices

Peter Benner and Thomas Mach

Computational Methods in Systems and Control Theory
Max Planck Institute for Dynamics of Complex Technical Systems
Magdeburg

23rd Chemnitz FEM Symposium 2010

MAX−PLANCK−INSTITUT
FÜR DYNAMIK KOMPLEXER
TECHNISCHER SYSTEME
MAGDEBURG



Outline

1 Hierarchical (H-)Matrices

2 PINVIT

3 Numerical Results



H-Matrices [Hackbusch 1998]

Some dense matrices, e.g. BEM or FEM, can be approximated by
H-matrices in a data-sparse manner.

hierarchical tree TI block H-tree TI × I
I = {1, 2, 3, 4, 5, 6, 7, 8}
12345678 12345678 12345678 12345678
1 1 1 1
2 2 2 2
{1, 2, 3, 4} {5, 6, 7, 8} 3 3 3 3
4 4 4 4
5 5 5 5
6 6 6 6
{1, 2} {3, 4} {5, 6} {7, 8} 7 7 7 7
8 8 8 8

{1}{2}{3}{4}{5}{6}{7}{8} dense matrices, rank-k-matrices

rank-k-matrix: Ms×t = AB T , A ∈ Rn×k , B ∈ Rm×k (k n, m)



H-Matrices [Hackbusch 1998]

Hierarchical matrices
H(TI × I , k) = M ∈ RI × I rank (Ma×b ) ≤ k ∀a × b admissible
3 3

22 7 10 14 8 11 9 8 5
3 7 19 10 11 8

3 10 10 31 11 11 9 16 12 8 15 8 12 11 8
19 10 11 8

14

8
11
11
11
8
10

11
31 11
31 11
9
3
16 11
7
3
8
6

15
5
6
7
13
5
8

8
15 9

15 12 13
adaptive rank k(ε)
19
13
11 61 11 10 7
8

9 16 11 11 8 11 8

13 13
9 3 25 10 13 8

7 11 10 19 11 6 5 8 11 9

11 16
3

storage NSt,H (T , k) = O(n log n k(ε))
8 11 11 31 11 15 6 9 15 11
11 8

8 8 15 10 10 15 9

5 8 11 61 3

6 14 9 11 10 10 7

13
6 15 10 3 6 25 10 6 13 8

12 5 6 10 14 6
10 19

10
10

31 10 16 9
11

8
8

15 11 12

20
13 8

complexity of approximate arithmetic
7 8 11 15 9 10 10
5 8 11 8 11 10 51 7 9
3
7
3
9 7
6 15 10 25 11 8

13 13
7 13

11 5 6 10 16 10
9

7
3

3
11
25
10
10

19 9
13
8
8

11
11
11 8 3 7

8 15 8 39 10 10 10 6 5
10
3

9 15 3
10 3
25 7 10
3
6
3
15 10 13 6 11

13
7 10 7 22 7 10 11 9 8 5

12
O(n log n k(ε))
10 7 19 10 11 8

MH v
3

7 12 10 6 3 10 10 31 11 9 16 12 8 15 8

20
13 8
8 11 8 34 10 13 10 6 5
10 9 15 15 11 10 25 7 11 13 6 11

13 13
11 8 13 7

O(n log n k(ε)2 )
7 11 10 16 61 11 23

+H , −H
9 10 11
13 9 6 13 20 9 9 7
11 8 3 7

9 8 8 15 5 6 12 9 39 10 3 10 15 10

12 13
11 8

8 8 15 9 10 15 9

7 11 11 3 3

61 10

∗H , HLU(·), (·)−1 O(n (log n)2 k(ε)2 )
5 8 7 7 10 9 11

19
13 9 6 13 20 9 9 7

9 8
13
8
8
11 5 6 10 15 10 9 34 10 13

12 12 15 10

H
9 9
7 11 11 23 10 8 11 7 13 51



Goal: Eigenvalues of symmetric H-Matrices

M = M T ∈ H(T , k),
Λ(M) = {λ1, λ2, . . . , λn },
λ1 ≥ λ2 ≥ · · · ≥ λn > 0
⇓
How to ﬁnd λi in O(n (log n)α k β )?




M = M T ∈ H(T , k),
Λ(M) = {λ1, λ2, . . . , λn },
λ1 ≥ λ2 ≥ · · · ≥ λn > 0
⇓
How to ﬁnd λn in O(n (log n)α k β )?



Preconditioned Inverse Iteration
[Knyazev, Neymeyr, et al.]

Deﬁnition
The function
x T Mx
µ(x) = µ(x, M) =
xT x
is called the Rayleigh quotient.




Deﬁnition
The function
x T Mx
µ(x) = µ(x, M) =
xT x

Minimize the Rayleigh quotient by a gradient method:
2
xi+1 := xi − α µ(xi ), µ(x) = (Mx − xµ(x)) ,
xT x




Residual r (x) = Mx − xµ(x).

2
xT x




Deﬁnition
The function
x T Mx
µ(x) = µ(x, M) =
xT x

2
xT x
+ preconditioning ⇒ update equation:
xi+1 := xi − B −1 (Mxi − xi µ(xi )) .




Preconditioned residual B −1 r (x) = B −1 (Mx − xµ(x)).

+ preconditioning ⇒ update equation:
xi+1 := xi − B −1 (Mxi − xi µ(xi )) .



[Knyazev, Neymeyr 2009]

xi+1 := xi − B −1 (Mxi − xi µ(xi ))

If
M symmetric positive deﬁnite and
B −1 approximates the inverse of M, so that

I − B −1 M M
≤ c < 1,



[Knyazev, Neymeyr 2009]

xi+1 := xi − B −1 (Mxi − xi µ(xi ))

If
M symmetric positive deﬁnite and
B −1 approximates the inverse of M, so that

I − B −1 M M
≤ c < 1,

then Preconditioned INVerse ITeration (PINVIT) converges.




The residual

ri = Mxi − xi µ(xi )

converges to 0, so that

ri 2 <

is a useful termination criterion.



Why is it called Preconditioned Inverse Iteration?

Inverse Iteration:

xi+1 = µ(xi )A−1 xi ,
xi+1 = xi − A−1 Axi +µ(xi )A−1 xi ,
=0
−1
xi+1 = xi − A (Axi + µ(xi )xi ) .

Replace A−1 with the inexact solution by the preconditioner B:

xi+1 = xi − B −1 (Axi + µ(xi )xi ) .



Variants of PINVIT
[Neymeyr 2001: A Hierarchy of Preconditioned Eigensolvers for Elliptic Diﬀ. Operators]

Classiﬁcation by Neymeyr:

PINVIT(1): xi+1 := xi − B −1 ri .
PINVIT(2): xi+1 := arg minv ∈span{xi ,B −1 ri } µ(v ).
PINVIT(3): xi+1 := arg minv ∈span{xi−1 ,xi ,B −1 ri } µ(v ).
PINVIT(n): Analogously.
PINVIT(·,d): Replacing x by a rectangular full rank matrix
X ∈ Rn×d one gets the subspace version of PINVIT(·).



Algorithm

H-PINVIT(1,d)
Input: M ∈ Rn×n , X0 ∈ Rn×d (e.g. randomly chosen)
Output: Xp ∈ Rn×d , µ ∈ Rd×d , with MXp − Xp µ ≤
B −1 = (M)−1H
Orthogonalize X0
T
R := MX0 − X0 µ, µ = X0 MX0
for (i := 1; R F > ; i + +) do
Xi := Xi−1 − B −1 R
Orthogonalize Xi
R := MXi − Xi µ, µ = XiT MXi
end



Complexity

The number of iterations is independent of matrix size n.

Once:
adaptive H-matrix inversion: O(n (log n)2 k (c)2 ).
Per iteration:
H-matrix-vector products with M and B −1 : O(n (log n) k (c))
and O(n (log n) k (c)2 ), and
some dense arithmetic handling vectors of length n.

The complexity of the algorithm is determined by the H-matrix
inversion: ⇒ O(n (log n)2 k (c)2 ).




M = M T ∈ H(T , k),
Λ(M) = {λ1, λ2, . . . , λn },
λ1 ≥ λ2 ≥ · · · ≥ λn > 0
⇓
How to ﬁnd λn in O(n (log n)α k β )?




M = M T ∈ H(T , k),
Λ(M) = {λ1, λ2, . . . , λn },
λ1 ≥ λ2 ≥ · · · ≥ λn > 0
⇓



Folded Spectrum Method

If i = n − d with d < O(log n),
use subspace version PINVIT(·,d ).

...
0λn λn−1 λn−2 λ1




If i = n − d with d log n?

... ...
0λn λi+1 λi λi−1 λ1




If i = n − d with d log n,
shift with σ near λi .

... ...
σ




If i = n − d with d log n,
shift with σ near λi .

... ...
σ

But (M − σI) is not positive deﬁnite!



Folded Spectrum Method [Wang, Zunger 1994]

Mσ = (M − σI)2

Mσ is s.p.d., if M is s.p.d. and σ = λi .
Assume all eigenvalues of Mσ are simple.

Mv = λv ⇔ Mσ v = (M − σI)2 v
= M 2 v − 2σMv + σ 2 v
= λ2 v − 2σλv + σ 2 v
= (λ − σ)2 v




1 Choose σ.
2 Compute
2
a) Mσ = (M − σI) and
−1
b) Mσ .
3 Use PINVIT to ﬁnd the smallest eigenpair (µσ , v ) of Mσ .
4 Compute µ = v T Mv /v T v .
(µ, v ) is the nearest eigenpair to σ.




1 Choose σ.
2 Compute
2
a) Mσ = (M − σI) and
−1
b) Mσ .
3 Use PINVIT to ﬁnd the smallest eigenpair (µσ , v ) of Mσ .
4 Compute µ = v T Mv /v T v .
(µ, v ) is the nearest eigenpair to σ.

H-arithmetic enables us to shift, square and invert M resp.
Mσ in linear-polylogarithmic complexity.




M = M T ∈ H(T , k),
Λ(M) = {λ1, λ2, . . . , λn },
λ1 ≥ λ2 ≥ · · · ≥ λn > 0
⇓



Numerical Results

Numerical Results



Hlib

Hlib [B¨rm, Grasedyck, et al.]
o
We use the Hlib1.3 (www.hlib.org) for the
H-arithmetic operations and some examples out of
the library for testing the eigenvalue algorithm.



2D Laplace, smallest eigenvalues

2D Laplace over [−1, 1] × [−1, 1]

ti N(ni )
Name ni error ti ti−1 N(ni−1 )
FEM8 64 4.3466E-08 0.006
FEM16 256 1.2389E-07 0.014 2.33 80.00
FEM32 1 024 4.4308E-07 0.036 2.57 21.67
FEM64 4 096 1.8003E-06 0.228 6.33 6.72
FEM128 16 384 6.7069E-06 0.992 4.35 4.67
FEM256 65 536 1.6631E-05 2.397 2.42 4.57
FEM512 262 144 4.4015E-05 8.120 3.39 5.79

d = 4, c = 0.2, eig = 10−4 , N(ni ) = ni (log2 ni ) Csp Cid
Time t only PINVIT (without H-inversion)




103
PINVIT(1,4)
102 PINVIT(3,4)
O(N(ni ))
CPU time in s

101 H-inversion
O(N(ni ) log ni )
100

10−1

10−2

10−3
8

16

32

64

8

6

2
M

12

25

51
M

M

M
FE

M

M

M
FE

FE

FE

FE

FE

FE



103
PINVIT(1,4)
102 PINVIT(3,4)
O(N(ni ))
CPU time in s

101 H-inversion
O(N(ni ) log ni )
100 eigs
10−1

10−2

10−3
8

16

32

64

8

6

2
M

12

25

51
M

M

M
FE

M

M

M
FE

FE

FE

FE

FE

FE


Complexity

The number of iterations is independent of matrix size n.

Once:
Per iteration:

The complexity of the algorithm is determined by the H-matrix
inversion: ⇒ O(n (log n)2 k (c)2 ).



Complexity

Competitive to MATLAB eigs.
Once:
Per iteration:



Complexity

Once:
Per iteration:

Expensive.



2D Laplace, vn , FEM64

·10−2
6

4
·10−2

5 2

0
0
−2
1
−5 −4
−1 0
−0.5 0
0.5 −6
1 −1



2D Laplace, vn−1 , FEM64

·10−2
6

4
·10−2

5 2

0
0
−2
1
−5 −4
−1 0
−0.5 0
0.5 −6
1 −1




·10−2
6

4
·10−2

5 2

0
0
−2
1
−5 −4
−1 0
−0.5 0
0.5 −6
1 −1



2D Laplace, inner eigenvalues

2D Laplace over [−1, 1] × [−1, 1]

ti N(ni )
FEM8 64 2.2560E-08 <0.01
FEM16 256 3.4681E-06 0.01 — 80.00
FEM32 1 024 1.1875E-04 0.04 4.00 21.67
FEM64 4 096 7.2115E-04 0.20 5.00 6.72
FEM128 16 384 2.0747E-05 0.93 4.65 4.67
FEM256 65 536 9.2370E-05 3.59 3.86 4.57




2D Laplace, L-shape, smallest eigenvalues

2D Laplace over [−1, 1] × [−1, 1] [0, 1] × [0, 1]

ti N(ni )
Name n error t ti−1 N(ni−1 )
LFEM8 48 6.0999E-07 0.023
LFEM16 192 5.7426E-07 0.037 1.57 10.86
LFEM32 768 1.4644E-06 0.180 4.91 10.11
LFEM64 3 072 4.4753E-06 0.497 2.76 108.78
LFEM128 12 288 1.8770E-04 0.770 1.55 9.49
LFEM256 49 152 — 2.133 2.77 4.59





103
PINVIT(1,4)
102 PINVIT(3,4)
O(N(ni ))
CPU time in s

101 H-inversion
O(N(ni ) log ni )
100

10−1

10−2

10−3
8

16

32

64

8

6
M

12

25
EM

EM

EM
E

EM

EM
LF

LF

LF

LF

LF

LF



103
PINVIT(1,4)
102 PINVIT(3,4)
O(N(ni ))
CPU time in s

101 H-inversion
O(N(ni ) log ni )
100 eigs
10−1

10−2

10−3
8

16

32

64

8

6
M

12

25
EM

EM

EM
E

EM

EM
LF

LF

LF

LF

LF

LF


2D Laplace, L-shape, vn , LFEM64

·10−2
6

4
·10−2

5 2

0
0
−2
1
−5 −4
−1 0
−0.5 0
0.5 −6
1 −1



2D Laplace, L-shape, vn−1 , LFEM64

·10−2
6

4
·10−2

5 2

0
0
−2
1
−5 −4
−1 0
−0.5 0
0.5 −6
1 −1




·10−2
6

4
·10−2

5 2

0
0
−2
1
−5 −4
−1 0
−0.5 0
0.5 −6
1 −1



BEM example, smallest eigenvalues

dense matrix, but approximable by an H-matrix ⇒ MATLAB eigs
is expensive
ti N(ni )
BEM8 258 2.0454E-05 0.01
BEM16 1 026 3.5696E-05 0.03 3.00 26.81
BEM32 4 098 7.1833E-05 0.14 4.67 14.42
BEM64 16 386 — 0.46 3.29 16.51
BEM128 65 538 — 2.00 4.36 26.35





PINVIT(1,4)
104 PINVIT(3,4)
O(N(ni ))
CPU time in s

102 H-inversion
O(N(ni ) log ni )

100

10−2
8

16

32

64

8
M

12
M

M

M
BE

M
BE

BE

BE

BE



PINVIT(1,4)
104 PINVIT(3,4)
O(N(ni ))
CPU time in s

102 H-inversion
O(N(ni ) log ni )
eigs
100

10−2
8

16

32

64

8
M

12
M

M

M
BE

M
BE

BE

BE

BE


Conclusions

Once we have inverted M ∈ H(T , k) ﬁnding the eigenvalues
by PINVIT is cheap and storage eﬃcient.
For dense, data-sparse matrices, H-PINVIT can solve
problems where eigs (implicitly, restarted Arnoldi/Lanczos
iteration) is not applicable.
The folded spectrum method enables us to compute inner
eigenvalues, too.
We will investigate the use of H-Cholesky factorizations
instead of the H-inversion.



Conclusions

Once we have inverted M ∈ H(T , k) ﬁnding the eigenvalues
by PINVIT is cheap and storage eﬃcient.
For dense, data-sparse matrices, H-PINVIT can solve
problems where eigs (implicitly, restarted Arnoldi/Lanczos
iteration) is not applicable.
The folded spectrum method enables us to compute inner
eigenvalues, too.
We will investigate the use of H-Cholesky factorizations
instead of the H-inversion.

Thank you for your attention.


Appendix

Adaptive H-Inversion [Grasedyck 2001]
Adaptive H-Inversion
c
Input: M ∈ H (TI ×I ), c = √
˜ ∈R
M 2
−1 −1
Output: MH , with I − MH M 2 < c
˜
−1 H
Compute MH with local = c /(Csp (M) M
˜ 2 )
0 −1 H
δM := I − MH M 2
/˜c
i > 1 do
while δM
−1 i
Compute MH with local = local /δM
i −1 H
δM := I − MH M 2
/˜c
end

−1 c −1
I − MH M 2
<c =
˜ ⇔ I − MH M M
≤c
M 2


Preconditioned Inverse Iteration for Hierarchical Matrices

Recommended

Recommended

More Related Content

More from Thomas Mach

More from Thomas Mach (7)

Recently uploaded

Recently uploaded (20)

Preconditioned Inverse Iteration for Hierarchical Matrices