1. November 5, 2010
Hierarchical Matrices:
Concept, Applications and Eigenvalues
Thomas Mach
Max Planck Institute for Dynamics of Complex Technical Systems
Computational Methods in Systems and Control Theory
MAX PLANCK INSTITUTE
FOR DYNAMICS OF COMPLEX
TECHNICAL SYSTEMS
MAGDEBURG
HIERARCHICAL STRUCTURES
Max Planck Institute Magdeburg Thomas Mach, Hierarchical Matrices 1/28

2. November 5, 2010
Hierarchical Matrices:
Concept, Applications and Eigenvalues
Thomas Mach
Max Planck Institute for Dynamics of Complex Technical Systems
Computational Methods in Systems and Control Theory
MAX PLANCK INSTITUTE
FOR DYNAMICS OF COMPLEX
TECHNICAL SYSTEMS
MAGDEBURG
HIERARCHICAL STRUCTURES
Max Planck Institute Magdeburg Thomas Mach, Hierarchical Matrices 1/28

3. November 5, 2010
Hierarchical Matrices:
Concept, Applications and Eigenvalues
Thomas Mach
Max Planck Institute for Dynamics of Complex Technical Systems
Computational Methods in Systems and Control Theory
MAX PLANCK INSTITUTE
FOR DYNAMICS OF COMPLEX
TECHNICAL SYSTEMS
MAGDEBURG
HIERARCHICAL STRUCTURES
Max Planck Institute Magdeburg Thomas Mach, Hierarchical Matrices 1/28

4. November 5, 2010
Hierarchical Matrices:
Concept, Applications and Eigenvalues
Thomas Mach
Max Planck Institute for Dynamics of Complex Technical Systems
Computational Methods in Systems and Control Theory
MAX PLANCK INSTITUTE
FOR DYNAMICS OF COMPLEX
TECHNICAL SYSTEMS
MAGDEBURG
HIERARCHICAL STRUCTURES
Max Planck Institute Magdeburg Thomas Mach, Hierarchical Matrices 1/28

5. Concept Application Eigenvalues
Householder Notation
Scalar
λ∈R
Max Planck Institute Magdeburg Thomas Mach, Hierarchical Matrices 2/28

6. Concept Application Eigenvalues
Householder Notation
Vector
 
x1
 x2 
 
x =  x 3  ∈ Rn
 
.
..
xn
Max Planck Institute Magdeburg Thomas Mach, Hierarchical Matrices 3/28

7. Concept Application Eigenvalues
Householder Notation
Matrix
 
a11 a12 a13 . . . a1n
a21 a22 a23 . . . a2n 
 
A = a31 a32 a33 . . . a3n  ∈ Rn×n

 . . . .. . 

 .. .
. .
. . . 
.
an1 an2 an3 . . . ann
Max Planck Institute Magdeburg Thomas Mach, Hierarchical Matrices 4/28

8. Concept Application Eigenvalues
Dense Matrix Format
Store n vectors (Fortran):
       
a11 a12 a13 a1n
a21  a22  a23  a2n 
       
a31  a32  a33 
      · · · a3n 
 
 .   .   .   . 
 .   .   . 
. . .  . 
.
an1 an2 an3 ann
n2 entries in the storage
Ax costs O(n2 ) flops
AB costs O(nδ ) flops, δ ≥ 2.376 usually δ = 3
A−1 costs O(n3 ) flops
Max Planck Institute Magdeburg Thomas Mach, Hierarchical Matrices 5/28

9. Concept Application Eigenvalues
Dense Matrix Format
Landau Symbol
Store n vectors (Fortran):
   that    more than cn2 flops
There is a constant c, so  Ax costs not
a11 a12 a13 a1n
a21  a22  a23  a2n 
       
      · · · a3n 
a31  a32  a33    Flops
 .   .   . 
.  .  .   . 
. 
 . . .  . 1 flop = (αβ + γ → γ)
an1 an2 an3 ann
n2 entries in the storage
Ax costs O(n2 ) flops
AB costs O(nδ ) flops, δ ≥ 2.376 usually δ = 3
A−1 costs O(n3 ) flops
Max Planck Institute Magdeburg Thomas Mach, Hierarchical Matrices 5/28

10. Concept Application Eigenvalues
Dense Matrix Format
Store n vectors (Fortran):
       
a11 a12 a13 a1n
a21  a22  a23  a2n 
       
a31  a32  a33 
      · · · a3n 
 
 .   .   .   . 
 .   .   . 
. . .  . 
.
an1 an2 an3 ann
n2 entries in the storage expensive!
Ax costs O(n2 ) flops
AB costs O(nδ ) flops, δ ≥ 2.376 usually δ = 3
A−1 costs O(n3 ) flops
Max Planck Institute Magdeburg Thomas Mach, Hierarchical Matrices 5/28

11. Concept Application Eigenvalues
Sparse Matrix Format
4 −1 0 −1 0 0 0 0 0
 
−1 4 −1 0 −1 0 0 0 0
 
 0 −1 4 0 0 −1 0 0 0
 
−1 0 0 4 −1 0 −1 0 0
 
 0 −1 0 −1 4 −1 0 −1 0 
 
0
 0 −1 0 −1 4 0 0 −1
0
 0 0 −1 0 0 4 −1 0 

0 0 0 0 −1 0 −1 4 −1
0 0 0 0 0 −1 0 −1 4
Max Planck Institute Magdeburg Thomas Mach, Hierarchical Matrices 6/28

12. Concept Application Eigenvalues
Sparse Matrix Format
4 −1 0 −1 0 0 0 0 0
 
−1 4 −1 0 −1 0 0 0 0
 
 0 −1 4 0 0 −1 0 0 0
 
−1 0 0 4 −1 0 −1 0 0
 
 0 −1 0 −1 4 −1 0 −1 0 
 
0
 0 −1 0 −1 4 0 0 −1
0
 0 0 −1 0 0 4 −1 0 

0 0 0 0 −1 0 −1 4 −1
0 0 0 0 0 −1 0 −1 4
non-zero entries per row/column = c ≤ 5
Max Planck Institute Magdeburg Thomas Mach, Hierarchical Matrices 6/28

13. Concept Application Eigenvalues
Sparse Matrix Format
4 −1 −1 −1
 
−1 −1 −1 4
 
−1 4 −1 −1
 
−1 −1 4 −1
 
4
 −1 −1 −1

−1 −1 −1 4
 
−1 −1 4 
 
−1 4 −1 
4 −1 −1
non-zero entries per row/column = c ≤ 5
Max Planck Institute Magdeburg Thomas Mach, Hierarchical Matrices 6/28

14. Concept Application Eigenvalues
Sparse Matrix Format (Matlab)
Store vector of tripel:
 
(1, 1, 4)
(2, 1, −1)
 
(4, 1, −1)
 
(1, 2, −1)
.
 
.
.
O(n) entries in the storage (if #non-zeros/column < c n)
Ax costs O(n) flops
AB may be much denser ⇒ better use A(Bx)
A−1 not possible, only solve Ax = b
Max Planck Institute Magdeburg Thomas Mach, Hierarchical Matrices 7/28

15. Concept Application Eigenvalues
Is there something in between?
Yes, e.g.
Low rank matrices and Tucker tensor format
Toeplitz/Hankel matrices
Semiseparable matrices [Gantmacher, Krein 1937]
Cauchy matrices
Fast Multipole Method (FMM) [Greengard, Rokhlin ’87]
Mosaic-skeleton matrices [Tyrtyshnikov ’96]
...
Max Planck Institute Magdeburg Thomas Mach, Hierarchical Matrices 8/28

16. Concept Application Eigenvalues
Is there something in between?
Yes, e.g.
Low rank matrices and Tucker tensor format
Toeplitz/Hankel matrices
Semiseparable matrices [Gantmacher, Krein 1937]
Cauchy matrices
Fast Multipole Method (FMM) [Greengard, Rokhlin ’87]
Mosaic-skeleton matrices [Tyrtyshnikov ’96]
...
Max Planck Institute Magdeburg Thomas Mach, Hierarchical Matrices 8/28

17. Concept Application Eigenvalues
Is there something in between?
Yes, e.g.
Low rank matrices and Tucker tensor format
Toeplitz/Hankel matrices
Semiseparable matrices [Gantmacher, Krein 1937]
Cauchy matrices
Fast Multipole Method (FMM) [Greengard, Rokhlin ’87]
Mosaic-skeleton matrices [Tyrtyshnikov ’96]
Hierarchical (H-) Matrices [Hackbusch ’98]
Max Planck Institute Magdeburg Thomas Mach, Hierarchical Matrices 8/28

18. Concept Application Eigenvalues
Integral Equation
Fredholm equation of first kind:
Ω g (x, y )u(y )dy = f (x)
Max Planck Institute Magdeburg Thomas Mach, Hierarchical Matrices 9/28

19. Concept Application Eigenvalues
Integral Equation
unknown
Fredholm equation of first kind:
Ω g (x, y )u(y )dy = f (x)
Max Planck Institute Magdeburg Thomas Mach, Hierarchical Matrices 9/28

20. Concept Application Eigenvalues
Integral Equation
Fredholm equation of first kind:
Ω g (x, y )u(y )dy = f (x)
electrostatic problem:
g (x, y ) = 4π 1x−y
u(y ) charge density
f (x) electrostatic potential
inverse of elliptic differential operators
population dynamics [Koch, Hackbusch, Sundmacher]
Max Planck Institute Magdeburg Thomas Mach, Hierarchical Matrices 9/28

21. Concept Application Eigenvalues
H-Matrices [Hackbusch ’98]
Ω g (x, y )u(y )dy = f (x)
Ritz-Galerkin method:
u(y ) = i ui ψi (y )
⇒ g (x, y )ψi (y )dy ψj (x)dx ui = f (x)ψj (x)dx
i Ω Ω Ω
:=Mji =fj
Max Planck Institute Magdeburg Thomas Mach, Hierarchical Matrices 10/28

22. Concept Application Eigenvalues
H-Matrices [Hackbusch ’98]
discretization error ∼ Ω1g,(x, <)u(y )dy = f (x)
nκ 0
y κ<1
Ritz-Galerkin method:
u(y ) = i ui ψi (y )
⇒ g (x, y )ψi (y )dy ψj (x)dx ui = f (x)ψj (x)dx
i Ω Ω Ω
:=Mji =fj
Max Planck Institute Magdeburg Thomas Mach, Hierarchical Matrices 10/28

23. Concept Application Eigenvalues
H-Matrices [Hackbusch ’98]
Ω g (x, y )u(y )dy = f (x)
Ritz-Galerkin method:
u(y ) = i ui ψi (y )
⇒ g (x, y )ψi (y )dy ψj (x)dx ui = f (x)ψj (x)dx
i Ω Ω Ω
:=Mji =fj
Mu = f
Max Planck Institute Magdeburg Thomas Mach, Hierarchical Matrices 10/28

24. Concept Application Eigenvalues
H-Matrices [Hackbusch ’98]
Ω g (x, y )u(y )dy = f (x)
Ritz-Galerkin method:
u(y ) = i ui ψi (y )
⇒ g (x, y )ψi (y )dy ψj (x)dx ui = f (x)ψj (x)dx
i Ω Ω Ω
:=Mji =fj
k x,s y ,t
x ∈ Ωs , y ∈ Ωt : g (x, y ) ≈ p=1 gp (x)gp (y )
Max Planck Institute Magdeburg Thomas Mach, Hierarchical Matrices 10/28

25. Concept Application Eigenvalues
H-Matrices [Hackbusch ’98]
Ω g (x, y )u(y )dy = f (x)
Ritz-Galerkin method:
u(y ) = i ui ψi (y )
⇒ g (x, y )ψi (y )dy ψj (x)dx ui = f (x)ψj (x)dx
i Ω Ω Ω
:=Mji =fj
k x,s y ,t
x ∈ Ωs , y ∈ Ωt : g (x, y ) ≈ p=1 gp (x)gp (y )
Mji = g (x, y )ψj (x)ψi (y )dydx
Ω Ω
k
x y T
≈ gp (x)ψj (x)dx gp (y )ψi (y )dy = Aj· Bi·
p=1 Ω Ω
Max Planck Institute Magdeburg Thomas Mach, Hierarchical Matrices 10/28

26. Concept Application Eigenvalues
H-Matrices [Hackbusch ’98]
3 3
22 7 10 14 8 11 9 8 5
3 7 19 10 11 8
3 10 10 31 11 11 9 16 12 8 15 8 12 11 8
19 10 11 8
14
8
11
11
10
11
31 11
31 11
9
16 11
7
6
15
5
6
13 8
8
15 9
15 12 13
11 8 3 3
8 7 5
19
13
11 61 11 10 7
8
9 16 11 11 8 11 8
13 13
25
9
7
3
11 10
10
19 11 6 5
13
8
8
11 9 k
11 16
3
11 11 x,s y ,t
11 8
8 11 11 31 15 6 9 15
x ∈ Ωs , y ∈ Ωt : g (x, y ) ≈ gp (x)gp (y )
8 8 15 10 10 15 9
5 8 11 61 3
6 14 9 11 10 10 7 p=1
13
6 15 10 3 6 25 10 6 13 8
12 5 6 10 14 6
10 19
10
10
31 10 16 9
11
8
8
15 11 12
20
13 8
7 8 11 15 9 10 10
5 8 11 8 11 10 51 7 9
3
7
3
9 7
6 15 10 25 11 8
13 13
7 13
11 5 6 10 16 10
9
7
3
3
11
25
10
10
19 9
13
8
8
11
11
Mji = g (x, y )ψj (x)ψi (y )dydx
11 8 3 7
8 15 8 39 10 10 10 6 5
10
3
15 10 11
9 15 3
10 3
25 7 10
3
6
3
13 6
Ω Ω
13
7 10 7 22 7 10 11 9 8 5
7 12 10
10
6
3
3
7
10 10
19 10
31 11 9 16 12
11
8
8
15 8
12 k
x y T
20
13
gp (x)ψj (x)dx gp (y )ψi (y )dy = Aj· Bi·
8
10
8
9
11 8
15 15 11
34
10
10
25
13
7
10
11
6
13
5
6 11 ≈
Ω Ω
13 13 p=1
11 8 13 7
7 11 10 9 16 10 11 61 11 23
13 9 6 13 20 9 9 7
11 8 3 7
9 8 8 15 5 6 12 9 39 10 3 10 15 10
12 13
11 8
8 8 15 9 10 15 9
7 11 11 5 8 7
3
7
3
10 61 10 9 11
19
13 9 6 13 20 9 9 7
9 8
13
8
8
11 5 6 10 15 10 9 34 10 13
12 7 11 12 11 23 10
15
8
9
11
9
7
10
13 51
Max Planck Institute Magdeburg Thomas Mach, Hierarchical Matrices 11/28

27. Concept Application Eigenvalues
H-Matrices [Hackbusch ’98]
3 3
22 7 10 14 8 11 9 8 5
3 7 19 10 11 8
3 10 10 31 11 11 9 16 12 8 15 8 12 11 8
19 10 11 8
14
8
11
11
10
11
31 11
31 11
9
16 11
7
6
15
5
6
13 8
8
15 9
15 12 13
11 8 3 3
8 7 5
19
13
11 61 11 10 7
8
9 16 11 11 8 11 8
13 13
25
9
7
3
11 10
10
19 11 6 5
13
8
8
11 9 k
11 16
3
11 11 x,s y ,t
11 8
8 11 11 31 15 6 9 15
x ∈ Ωs , y ∈ Ωt : g (x, y ) ≈ gp (x)gp (y )
8 8 15 10 10 15 9
5 8 11 61 3
6 14 9 11 10 10 7 p=1
13
6 15 10 3 6 25 10 6 13 8
12 5 6 10 14 6
10 19
10
10
31 10 16 9
11
8
8
15 11 12
20
13 8
7 8 11 15 9 10 10
5 8 11 8 11 10 51 7 9
3
7
3
9 7
6 15 10 25 11 8
13 13
7 13
11 5 6 10 16 10
9
7
3
3
11
25
10
10
19 9
13
8
8
11
11
Mji = g (x, y )ψj (x)ψi (y )dydx
11 8 3 7
8 15 8 39 10 10 10 6 5
10
3
15 10 11
9 15 3
10 3
25 7 10
3
6
3
13 6
Ω Ω
13
7 10 7 22 7 10 11 9 8 5
7 12 10
10
6
3
3
7
10 10
19 10
31 11 9 16 12
11
8
8
15 8
12 k
x y T
20
13
gp (x)ψj (x)dx gp (y )ψi (y )dy = Aj· Bi·
8
10
8
9
11 8
15 15 11
34
10
10
25
13
7
10
11
6
13
5
6 11 ≈
Ω Ω
13 13 p=1
11 8 13 7
7 11 10 9 16 10 11 61 11 23
13 9 6 13 20 9 9 7
11 8 3 7
9 8 8 15 5 6 12 9 39 10 3 10 15 10
12 13
11 8
s 7 11 11
8
5
8
8
15 9
7
3
7
10 3
10 61 10
15
9
9
11
19
13 9 6 13 20 9 9 7
9 8
13
8
8
11 5 6 10 15 10 9 34 10 13
12 7 11 12 11 23 10
15
8
9
11
9
7
10
13 51
t
Max Planck Institute Magdeburg Thomas Mach, Hierarchical Matrices 11/28