Computing Inner Eigenvalues of Matrices in Tensor Train Matrix Format

GAMM Workshop Applied and Numerical Linear Algebra 2011 September 22, 2011, Bremen Computing Inner Eigenvalues of Matrices in Tensor Train Matrix Format Thomas Mach joint work with Peter Benner Max Planck Institute for Dynamics of Complex Technical Systems Computational Methods in Systems and Control Theory Magdeburg MAX PLANCK INSTITUTE FOR DYNAMICS OF COMPLEX TECHNICAL SYSTEMS MAGDEBURGMax Planck Institute Magdeburg Thomas Mach, Computing Eigenvalues of Matrices in TTM Format 1/21

Tensor Trains PINVIT and Folded Spectrum Method Numerical Results Outline 1 Tensor Trains 2 PINVIT and Folded Spectrum Method 3 Numerical ResultsMax Planck Institute Magdeburg Thomas Mach, Computing Eigenvalues of Matrices in TTM Format 2/21

Tensor Trains PINVIT and Folded Spectrum Method Numerical Results Tensor Trains [Oseledets, Tyrtyshnikov ’09] d T ∈ RmMax Planck Institute Magdeburg Thomas Mach, Computing Eigenvalues of Matrices in TTM Format 3/21

Tensor Trains PINVIT and Folded Spectrum Method Numerical Results Tensor Trains [Oseledets, Tyrtyshnikov ’09] d T ∈ Rm r T = α=1 U1 (i1 , α)U2 (i2 , α) · · · Ud (id , α), with Uj (·, α) ∈ Rm U1 (i1 , α) U4 (i4 , α) α U2 (i2 , α) U3 (i3 , α)Max Planck Institute Magdeburg Thomas Mach, Computing Eigenvalues of Matrices in TTM Format 3/21

Tensor Trains PINVIT and Folded Spectrum Method Numerical Results Tensor Trains [Oseledets, Tyrtyshnikov ’09] d T ∈ Rm r T = α=1 U1 (i1 , α)U2 (i2 , α) · · · Ud (id , α), with Uj (·, α) ∈ Rm r r d T (i1 , i2 , . . . , id ) = α1 =1 · · · αd =1 Gα1 ,...,αd 1 d j=1 Uj (ij , αj ), with G ∈ Rr1 ×···×rd and Uj (·, αj ) ∈ Rm U1 (i1 , α1 ) U3 (i3 , α3 ) α1 α3 U4 (i4 , α4 ) α4 G (α1 , . . . , α4 ) α2 U2 (i2 , α2 )Max Planck Institute Magdeburg Thomas Mach, Computing Eigenvalues of Matrices in TTM Format 3/21

Tensor Trains PINVIT and Folded Spectrum Method Numerical Results Tensor Trains [Oseledets, Tyrtyshnikov ’09] d T ∈ Rm r T = α=1 U1 (i1 , α)U2 (i2 , α) · · · Ud (id , α), with Uj (·, α) ∈ Rm r r d T (i1 , i2 , . . . , id ) = α1 =1 · · · αd =1 Gα1 ,...,αd 1 d j=1 Uj (ij , αj ), with G ∈ Rr1 ×···×rd and Uj (·, αj ) ∈ Rm T (i1 , i2 , . . . , id ) = G1 (i1 , α1 )G2 (α1 , i2 , α2 ) · · · α1 ,...,αd−1 Gd−1 (αd−2 , id−1 , αd−1 )Gd (αd−1 , id )Max Planck Institute Magdeburg Thomas Mach, Computing Eigenvalues of Matrices in TTM Format 3/21

Tensor Trains PINVIT and Folded Spectrum Method Numerical Results Tensor Trains [Oseledets, Tyrtyshnikov ’09] T (i1 , i2 , . . . , id ) = G1 (i1 , α1 )G2 (α1 , i2 , α2 ) · · · α1 ,...,αd−1 Gd−1 (αd−2 , id−1 , αd−1 )Gd (αd−1 , id ) G1 (i1 , α1 ) α1 G2 (α1 , i2 , α2 ) α2 G3 (α2 , i3 , α3 ) α3 ··· αd−1 Gd (αd−1 , id )Max Planck Institute Magdeburg Thomas Mach, Computing Eigenvalues of Matrices in TTM Format 4/21

Tensor Trains PINVIT and Folded Spectrum Method Numerical Results Tensor Train Matrix Format (TTM) [Oseledets ’10] G1 (i1 , j1 , α1 ) α1 G2 (α1 , i2 , j2 , α2 ) α2 ··· αd−1 Gd (αd−1 , id , jd ) d ×md M ∈ RmMax Planck Institute Magdeburg Thomas Mach, Computing Eigenvalues of Matrices in TTM Format 5/21

Tensor Trains PINVIT and Folded Spectrum Method Numerical Results Tensor Train Matrix Format (TTM) [Oseledets ’10] G1 (i1 , j1 , α1 ) α1 G2 (α1 , i2 , j2 , α2 ) α2 ··· αd−1 Gd (αd−1 , id , jd ) d ×md M ∈ Rm M (i1 , i2 , . . . , id ; j1 , j2 , . . . , jd ) = α1 ,...,αd−1 G1 (i1 , j1 , α1 )G2 (α1 , i2 , j2 , α2 ) · · · · · · Gd (αd−1 , id , jd )Max Planck Institute Magdeburg Thomas Mach, Computing Eigenvalues of Matrices in TTM Format 5/21

Tensor Trains PINVIT and Folded Spectrum Method Numerical Results Tensor Train Matrix Format (TTM) [Oseledets ’10] G1 (i1 , j1 , α1 ) α1 G2 (α1 , i2 , j2 , α2 ) α2 ··· αd−1 Gd (αd−1 , id , jd ) d ×md M ∈ Rm M (i1 , i2 , . . . , id ; j1 , j2 , . . . , jd ) = α1 ,...,αd−1 G1 (i1 , j1 , α1 )G2 (α1 , i2 , j2 , α2 ) · · · · · · Gd (αd−1 , id , jd ) TTM is a data-sparse matrix format.Max Planck Institute Magdeburg Thomas Mach, Computing Eigenvalues of Matrices in TTM Format 5/21

Tensor Trains PINVIT and Folded Spectrum Method Numerical Results Tensor Train Matrix Format (TTM) [Oseledets ’10] G1 (i1 , j1 , α1 ) α1 G2 (α1 , i2 , j2 , α2 ) α2 ··· αd−1 Gd (αd−1 , id , jd ) d ×md M ∈ Rm M (i1 , i2 , . . . , id ; j1 , j2 , . . . , jd ) = α1 ,...,αd−1 G1 (i1 , j1 , α1 )G2 (α1 , i2 , j2 , α2 ) · · · · · · Gd (αd−1 , id , jd ) TTM is a data-sparse matrix format. m = 2 ⇒ QTT matrix formatMax Planck Institute Magdeburg Thomas Mach, Computing Eigenvalues of Matrices in TTM Format 5/21

Tensor Trains PINVIT and Folded Spectrum Method Numerical Results Matrix Vector Product in TT and TTM Format [Oseledets ’10] T ∈ (Q)TT , M ∈ (Q)TTM ⇒ MT = W ∈ (Q)TTMax Planck Institute Magdeburg Thomas Mach, Computing Eigenvalues of Matrices in TTM Format 6/21

Tensor Trains PINVIT and Folded Spectrum Method Numerical Results Matrix Vector Product in TT and TTM Format [Oseledets ’10] T ∈ (Q)TT , M ∈ (Q)TTM ⇒ MT = W ∈ (Q)TT W (i1 , i2 , . . . , id ) = M(i1 , j1 , i2 , j2 , . . . , id , jd )T (j1 , j2 , . . . , jd ) j1 ,j2 ,...,jdMax Planck Institute Magdeburg Thomas Mach, Computing Eigenvalues of Matrices in TTM Format 6/21

Tensor Trains PINVIT and Folded Spectrum Method Numerical Results Matrix Vector Product in TT and TTM Format [Oseledets ’10] T ∈ (Q)TT , M ∈ (Q)TTM ⇒ MT = W ∈ (Q)TT W (i1 , i2 , . . . , id ) = M(i1 , j1 , i2 , j2 , . . . , id , jd )T (j1 , j2 , . . . , jd ) j1 ,j2 ,...,jd G1 (i1 , j1 , α1 ) j1 H1 (j1 , β1 ) α1 β1 G2 (α1 , i2 , j2 , α2 ) j2 H2 (β1 , j2 , β2 ) α2 β2 . . . . . .Max Planck Institute Magdeburg Thomas Mach, Computing Eigenvalues of Matrices in TTM Format 6/21

Tensor Trains PINVIT and Folded Spectrum Method Numerical Results Matrix Vector Product in TT and TTM Format [Oseledets ’10] T ∈ (Q)TT , M ∈ (Q)TTM ⇒ MT = W ∈ (Q)TT W (i1 , i2 , . . . , id ) = M(i1 , j1 , i2 , j2 , . . . , id , jd )T (j1 , j2 , . . . , jd ) j1 ,j2 ,...,jd j1 G1 (i1 , j1 , αK1 (i1 , (α1 , β1 )) H1 (j1 , β1 ) 1) α1 β1 G2 (α1 , i2 , j2 , α2 ) j2 H2 (β1 , j2 , β2 ) α2 β2 . . . . . .Max Planck Institute Magdeburg Thomas Mach, Computing Eigenvalues of Matrices in TTM Format 6/21

Tensor Trains PINVIT and Folded Spectrum Method Numerical Results Matrix Vector Product in TT and TTM Format [Oseledets ’10] T ∈ (Q)TT , M ∈ (Q)TTM ⇒ MT = W ∈ (Q)TT W (i1 , i2 , . . . , id ) = M(i1 , j1 , i2 , j2 , . . . , id , jd )T (j1 , j2 , . . . , jd ) j1 ,j2 ,...,jd j1 G1 (i1 , j1 , αK1 (i1 , (α1 , β1 )) H1 (j1 , β1 ) 1) α1 β1 G2 (α1 , iK2j((α1 ,)β1 ),j22 , (α2H22 ))1 , j2 , β2 ) 2 , 2 , α2 i , β (β α2 β2 . . . . . .Max Planck Institute Magdeburg Thomas Mach, Computing Eigenvalues of Matrices in TTM Format 6/21

Tensor Trains PINVIT and Folded Spectrum Method Numerical Results Matrix Vector Product in TT and TTM Format [Oseledets ’10] T ∈ (Q)TT , M ∈ (Q)TTM ⇒ MT = W ∈ (Q)TT W (i1 , i2 , . . . , id ) = M(i1 , j1 , i2 , j2 , . . . , id , jd )T (j1 , j2 , . . . , jd ) j1 ,j2 ,...,jd j1 G1 (i1 , j1 , αK1 (i1 , (α1 , β1 )) H1 (j1 , β1 ) 1) (α1 , β1 ) G2 (α1 , iK2j((α1 ,)β1 ),j22 , (α2H22 ))1 , j2 , β2 ) 2 , 2 , α2 i , β (β α2 β2 . . . . . .Max Planck Institute Magdeburg Thomas Mach, Computing Eigenvalues of Matrices in TTM Format 6/21

Tensor Trains PINVIT and Folded Spectrum Method Numerical Results Matrix Vector Product in TT and TTM Format [Oseledets ’10] T ∈ (Q)TT , M ∈ (Q)TTM ⇒ MT = W ∈ (Q)TT W (i1 , i2 , . . . , id ) = M(i1 , j1 , i2 , j2 , . . . , id , jd )T (j1 , j2 , . . . , jd ) j1 ,j2 ,...,jd j1 G1 (i1 , j1 , αK1 (i1 , (α1 , β1 )) H1 (j1 , β1 ) 1) o n ati (α1 , β1 ) c t run G2 (α1 , iK2j((α1 ,)β1 ),j22 , (α2H22 ))1 , j2 , β2 ) 2 , 2 , α2 i , β (β + α2 β2 . . . . . .Max Planck Institute Magdeburg Thomas Mach, Computing Eigenvalues of Matrices in TTM Format 6/21

Tensor Trains PINVIT and Folded Spectrum Method Numerical Results Inversion of a Matrix in TTM Format [Schulz 1933, Oseledets ’10] Newton-Schulz Iteration Xk+1 = 2Xk − Xk MXk X0 initial approximation to M −1 with, ρ(MX0 − I ) < 1. 2 If M is symmetric, positive deﬁnite, then X0 = M I is an 2 admissible initial approximation. Hk+1 = I − Yk Yk+1 = Yk Hk+1 Xk+1 = Hk+1 XkMax Planck Institute Magdeburg Thomas Mach, Computing Eigenvalues of Matrices in TTM Format 7/21

Tensor Trains PINVIT and Folded Spectrum Method Numerical Results Inversion of a Matrix in TTM Format [Schulz 1933, Oseledets ’10] Newton-Schulz Iteration Xk+1 = 2Xk − Xk MXk X0 initial approximation to M −1 with, ρ(MX0 − I ) < 1. 2 If M is symmetric, positive deﬁnite, then X0 = M I is an 2 admissible initial approximation. Hk+1 = T (I − Yk , ) Yk+1 = T (Yk Hk+1 , ) Xk+1 = T (Hk+1 Xk , )Max Planck Institute Magdeburg Thomas Mach, Computing Eigenvalues of Matrices in TTM Format 7/21

Tensor Trains PINVIT and Folded Spectrum Method Numerical Results Eigenvalue Problem Problem Setting d d Assume M ∈ R2 ×2 is given in TTM. M is sym. pos. deﬁnite. d Compute eigenvalue λ and eigenvector v ∈ R2 of M. d R2 Mv = λv quantum molecular dynamics [Lebedeva ’11]Max Planck Institute Magdeburg Thomas Mach, Computing Eigenvalues of Matrices in TTM Format 8/21

Tensor Trains PINVIT and Folded Spectrum Method Numerical Results Preconditioned Inverse Iteration [Knyazev, Neymeyr, et al.] Deﬁnition The function x T Mx µ(x) = µ(x, M) = xT x is called the Rayleigh quotient.Max Planck Institute Magdeburg Thomas Mach, Computing Eigenvalues of Matrices in TTM Format 9/21

Tensor Trains PINVIT and Folded Spectrum Method Numerical Results Preconditioned Inverse Iteration [Knyazev, Neymeyr, et al.] Deﬁnition The function x T Mx µ(x) = µ(x, M) = xT x is called the Rayleigh quotient. Minimize the Rayleigh quotient by a gradient method: 2 xi+1 := xi − α µ(xi ), µ(x) = T (Mx − xµ(x)) , x xMax Planck Institute Magdeburg Thomas Mach, Computing Eigenvalues of Matrices in TTM Format 9/21

Tensor Trains PINVIT and Folded Spectrum Method Numerical Results Preconditioned Inverse Iteration [Knyazev, Neymeyr, et al.] Residual r (x) = Mx − xµ(x). Minimize the Rayleigh quotient by a gradient method: 2 xi+1 := xi − α µ(xi ), µ(x) = T (Mx − xµ(x)) , x xMax Planck Institute Magdeburg Thomas Mach, Computing Eigenvalues of Matrices in TTM Format 9/21

Tensor Trains PINVIT and Folded Spectrum Method Numerical Results Preconditioned Inverse Iteration [Knyazev, Neymeyr, et al.] Deﬁnition The function x T Mx µ(x) = µ(x, M) = xT x is called the Rayleigh quotient. Minimize the Rayleigh quotient by a gradient method: 2 xi+1 := xi − α µ(xi ), µ(x) = T (Mx − xµ(x)) , x x + preconditioning ⇒ update equation: xi+1 := xi − B −1 (Mxi − xi µ(xi )) .Max Planck Institute Magdeburg Thomas Mach, Computing Eigenvalues of Matrices in TTM Format 9/21

Tensor Trains PINVIT and Folded Spectrum Method Numerical Results Preconditioned Inverse Iteration [Knyazev, Neymeyr, et al.] Preconditioned residual B −1 r (x) = B −1 (Mx − xµ(x)). ⇒ inexact Newton-method + preconditioning ⇒ update equation: xi+1 := xi − B −1 (Mxi − xi µ(xi )) .Max Planck Institute Magdeburg Thomas Mach, Computing Eigenvalues of Matrices in TTM Format 9/21

Tensor Trains PINVIT and Folded Spectrum Method Numerical Results Preconditioned Inverse Iteration [Knyazev, Neymeyr 2009] xi+1 := xi − B −1 (Mxi − xi µ(xi )) If M ∈ Rn×n symmetric positive deﬁnite and B −1 approximates the inverse of M, so that I − B −1 M M ≤ c < 1, then Preconditioned INVerse ITeration (PINVIT) converges and the number of iterations is independent of n.Max Planck Institute Magdeburg Thomas Mach, Computing Eigenvalues of Matrices in TTM Format 10/21

Tensor Trains PINVIT and Folded Spectrum Method Numerical Results Preconditioned Inverse Iteration [Knyazev, Neymeyr, et al.] The residual ri = Mxi − xi µ(xi ) converges to 0, so that ri 2 < is a useful termination criterion.Max Planck Institute Magdeburg Thomas Mach, Computing Eigenvalues of Matrices in TTM Format 11/21

Tensor Trains PINVIT and Folded Spectrum Method Numerical Results Algorithm The number of iterations is independent of matrix size n = 2d . PINVIT(1,s) Input: M ∈ Rn×n , X0 ∈ Rn×s (X0 X0 = I , e.g. randomly chosen) T Output: Xp ∈ R n×s , µ ∈ Rs×s , with MX − X µ ≤ p p Approximative inversion B −1 ≈ (M)−1 R := MX0 − X0 µ, µ = X0 MX0T for (i := 1; R F > ; i + +) do Xi := Orthogonalize Xi−1 − B −1 R R := MXi − Xi µ, µ = XiT MXi endMax Planck Institute Magdeburg Thomas Mach, Computing Eigenvalues of Matrices in TTM Format 12/21

Tensor Trains PINVIT and Folded Spectrum Method Numerical Results Algorithm The number of iterations is independent of matrix size n = 2d . PINVIT(1,s) Input: M ∈ Rn×n , X0 ∈ Rn×s (X0 X0 = I , e.g. randomly chosen) T Output: Xp ∈ R n×s , µ ∈ Rs×s , with MX − X µ ≤ p p Approximative inversion B −1 ≈ (M)−1 R := MX0 − X0 µ, µ = X0 MX0T for (i := 1; R F > ; i + +) do Xi := Orthogonalize Xi−1 − B −1 R R := MXi − Xi µ, µ = XiT MXi end Newton-Schulz iteration: Bk+1 = 2Bk − Bk MBk [Oseledets ’10] for TTMMax Planck Institute Magdeburg Thomas Mach, Computing Eigenvalues of Matrices in TTM Format 12/21

Tensor Trains PINVIT and Folded Spectrum Method Numerical Results Algorithm The number of iterations is independent of matrix size n = 2d . PINVIT(1,s) Input: M ∈ Rn×n , X0 ∈ Rn×s (X0 X0 = I , e.g. randomly chosen) T Output: Xp ∈ R n×s , µ ∈ Rs×s , with MX − X µ ≤ p p Approximative inversion B −1 ≈ (M)−1 R := MX0 − X0 µ, µ = X0 MX0T for (i := 1; R F > ; i + +) do Xi := Orthogonalize Xi−1 − B −1 R R := MXi − Xi µ, µ = XiT MXi end TTM-TT productsMax Planck Institute Magdeburg Thomas Mach, Computing Eigenvalues of Matrices in TTM Format 12/21

Tensor Trains PINVIT and Folded Spectrum Method Numerical Results Folded Spectrum Method How to ﬁnd λi ? If i = n − s with s < O(log n), use subspace version PINVIT(·,s). ... 0λn λn−1 λn−2 λ1Max Planck Institute Magdeburg Thomas Mach, Computing Eigenvalues of Matrices in TTM Format 13/21

Tensor Trains PINVIT and Folded Spectrum Method Numerical Results Folded Spectrum Method How to ﬁnd λi ? If i = n − s with s log n? ... ... 0λn λi+1 λi λi−1 λ1Max Planck Institute Magdeburg Thomas Mach, Computing Eigenvalues of Matrices in TTM Format 13/21

Tensor Trains PINVIT and Folded Spectrum Method Numerical Results Folded Spectrum Method How to ﬁnd λi ? If i = n − s with s log n, shift with σ near λi . ... ... 0λn λi+1 λi λi−1 λ1 σMax Planck Institute Magdeburg Thomas Mach, Computing Eigenvalues of Matrices in TTM Format 13/21

Tensor Trains PINVIT and Folded Spectrum Method Numerical Results Folded Spectrum Method How to ﬁnd λi ? If i = n − s with s log n, shift with σ near λi . ... ... 0λn λi+1 λi λi−1 λ1 σ But (M − σI) is not positive deﬁnite.Max Planck Institute Magdeburg Thomas Mach, Computing Eigenvalues of Matrices in TTM Format 13/21

Tensor Trains PINVIT and Folded Spectrum Method Numerical Results Folded Spectrum Method Folded Spectrum Method [Wang, Zunger 1994, Morgan 1991] Mσ = (M − σI)2 Mσ is s.p.d., if M is s.p.d. and σ = λi . Assume all eigenvalues of Mσ are simple. Mv = λv ⇔ Mσ v = (M − σI)2 v = M 2 v − 2σMv + σ 2 v = λ2 v − 2σλv + σ 2 v = (λ − σ)2 vMax Planck Institute Magdeburg Thomas Mach, Computing Eigenvalues of Matrices in TTM Format 14/21

Tensor Trains PINVIT and Folded Spectrum Method Numerical Results Folded Spectrum Method Folded Spectrum Method [Wang, Zunger 1994, Morgan 1991] Mσ = (M − σI)2 Mσ is s.p.d., if M is s.p.d. and σ = λi . Assume all eigenvalues of Mσ are simple. (2, v2 ), (3, v3 ), σ = 2.5 ⇒ Mσ has eigenvalue 0.25 of multiplicity 2.Max Planck Institute Magdeburg Thomas Mach, Computing Eigenvalues of Matrices in TTM Format 14/21

Tensor Trains PINVIT and Folded Spectrum Method Numerical Results Folded Spectrum Method Folded Spectrum Method [Wang, Zunger 1994, Morgan 1991] Mσ = (M − σI)2 Mσ is s.p.d., if M is s.p.d. and σ = λi . Assume all eigenvalues of Mσ are simple. (2, v2 ), (3, v3 ), σ = 2.5 ⇒ Mσ has eigenvalue 0.25 of multiplicity 2. PINVIT computes v ∈ span(v2 , v3 ). ⇒ v T Mv /v T v ∈ [2, 3]Max Planck Institute Magdeburg Thomas Mach, Computing Eigenvalues of Matrices in TTM Format 14/21

Tensor Trains PINVIT and Folded Spectrum Method Numerical Results Folded Spectrum Method Folded Spectrum Method [Wang, Zunger 1994, Morgan 1991] Mσ = (M − σI)2 Mσ is s.p.d., if M is s.p.d. and σ = λi . Assume all eigenvalues of Mσ are simple. (2, v2 ), (3, v3 ), σ = 2.5 ⇒ Mσ has eigenvalue 0.25 of multiplicity 2. PINVIT computes v ∈ span(v2 , v3 ). ⇒ v T Mv /v T v ∈ [2, 3] Use PINVIT to compute V ∈ Rn×2 ⇒ Λ(V T MV /V T V ) = {2, 3}Max Planck Institute Magdeburg Thomas Mach, Computing Eigenvalues of Matrices in TTM Format 14/21

Tensor Trains PINVIT and Folded Spectrum Method Numerical Results Folded Spectrum Method 1 Choose σ. 2 Compute 2 a) Mσ := (M − σI) and b) B −1 :≈ Mσ . −1 3 Use PINVIT to ﬁnd the smallest eigenpair (µσ , v ) of Mσ . 4 Compute µ := v T Mv /v T v . (µ, v ) is the nearest eigenpair to σ.Max Planck Institute Magdeburg Thomas Mach, Computing Eigenvalues of Matrices in TTM Format 15/21

Tensor Trains PINVIT and Folded Spectrum Method Numerical Results Folded Spectrum Method 1 Choose σ. 2 Compute 2 a) Mσ := (M − σI) and b) B −1 :≈ Mσ . −1 3 Use PINVIT to ﬁnd the smallest eigenpair (µσ , v ) of Mσ . 4 Compute µ := v T Mv /v T v . (µ, v ) is the nearest eigenpair to σ. In TTM M can be shifted, squared and inverted with reasonable costs.Max Planck Institute Magdeburg Thomas Mach, Computing Eigenvalues of Matrices in TTM Format 15/21

Tensor Trains PINVIT and Folded Spectrum Method Numerical Results Folded Spectrum Method 1 Choose σ. 2 Compute 2 a) Mσ := (M − σI) and b) B −1 :≈ Mσ . −1 3 Use PINVIT to ﬁnd the smallest eigenpair (µσ , v ) of Mσ . 4 Compute µ := v T Mv /v T v . (µ, v ) is the nearest eigenpair to σ. In TTM M can be shifted, squared and inverted with reasonable costs. If M is sparse, then squaring and inverting is prohibitive.Max Planck Institute Magdeburg Thomas Mach, Computing Eigenvalues of Matrices in TTM Format 15/21

Tensor Trains PINVIT and Folded Spectrum Method Numerical Results Drawbacks The condition number of (M − σI )2 is larger. −1 ⇒ The computation of Mσ is more expensive. −1 ⇒ Mσ has larger local ranks. −1 ⇒ Mσ v is more expensive.Max Planck Institute Magdeburg Thomas Mach, Computing Eigenvalues of Matrices in TTM Format 16/21

Tensor Trains PINVIT and Folded Spectrum Method Numerical Results Drawbacks The condition number of (M − σI )2 is larger. −1 ⇒ The computation of Mσ is more expensive. −1 ⇒ Mσ has larger local ranks. −1 ⇒ Mσ v is more expensive. Multiple eigenvalues of Mσ may lead to incomplete subspace information. ⇒ v T Mv /v T v does not approximate λ.Max Planck Institute Magdeburg Thomas Mach, Computing Eigenvalues of Matrices in TTM Format 16/21

Tensor Trains PINVIT and Folded Spectrum Method Numerical Results Numerical Results Numerical ResultsMax Planck Institute Magdeburg Thomas Mach, Computing Eigenvalues of Matrices in TTM Format 17/21

Tensor Trains PINVIT and Folded Spectrum Method Numerical Results TT-Toolbox 2.1 [Oseledets et al. ’09–’11] We use TT-Toolbox 2.1 for MATLAB from I.V. Oseledets at al.Max Planck Institute Magdeburg Thomas Mach, Computing Eigenvalues of Matrices in TTM Format 18/21

Tensor Trains PINVIT and Folded Spectrum Method Numerical Results 2D Laplace d )2 ×(2d )2 M = α∆2 = α (∆1 ⊗ I + I ⊗ ∆1 ) ∈ R(2 , with ∆1 = tridiag([−1, 2, −1]). without shift, 3 smallest eigenvalues n d tinv in s tPINVIT in s # it. error 64 3 0.449 1.577 17 1.2230 e−07 256 4 0.345 1.376 18 1.1450 e−07 1 024 5 0.997 3.236 27 2.7432 e−07 4 096 6 2.347 8.172 19 1.8911 e−07 16 384 7 4.789 19.885 16 6.4020 e−08 65 536 8 11.895 43.717 20 1.0688 e−07 262 144 9 19.076 63.727 20 4.1304 e−09 1 048 576 10 29.865 99.808 20 3.7722 e−09 4 194 304 11 110.712* 331.059 27 5.8789 e−10 16 777 216 12 165.560* 439.341 23 4.5062 e−09 67 108 864 13 240.226* 587.796 26 8.9795 e−09Max Planck Institute Magdeburg Thomas Mach, Computing Eigenvalues of Matrices in TTM Format 19/21

Tensor Trains PINVIT and Folded Spectrum Method Numerical Results 2D Laplace d )2 ×(2d )2 M = α∆2 = α (∆1 ⊗ I + I ⊗ ∆1 ) ∈ R(2 , with ∆1 = tridiag([−1, 2, −1]). with shift σ = 203.3139, folded spectrum method, 4 eigenvalues n d tinv in s tPINVIT in s # it. error 64 3 0.796 0.843 8 5.5601 e−10 256 4 3.360 3.053 45 2.9268 e−09 1 024 5 22.659 4.581 20 1.1411 e−10 4 096 6 60.755 32.264 31 2.5295 e−12 16 384 7 411.165* 80.827 28 6.3665 e−12 65 536 8 1 808.302* 344.696 27 2.5835 e−11Max Planck Institute Magdeburg Thomas Mach, Computing Eigenvalues of Matrices in TTM Format 19/21

Tensor Trains PINVIT and Folded Spectrum Method Numerical Results 3D Laplace M = α∆3 = α (∆1 ⊗ I ⊗ I + I ⊗ ∆1 ⊗ I + I ⊗ I ⊗ ∆1 ) , d 3 d 3 M ∈ R(2 ) ×(2 ) . without shift, 4 smallest eigenvalues n d tinv in s tPINVIT in s # it. error 64 2 0.219 4.518 30 2.4762 e−07 512 3 0.561 6.742 27 2.9333 e−07 4 096 4 1.109 23.715 24 1.5441 e−07 32 768 5 7.197 99.543 28 9.1209 e−09 262 144 6 11.052 249.084 24 1.9956 e−08 2 097 152 7 20.893 923.874 26 1.6577 e−07 16 777 216 8 34.937 5 131.664 34 1.1977 e−08Max Planck Institute Magdeburg Thomas Mach, Computing Eigenvalues of Matrices in TTM Format 19/21

Tensor Trains PINVIT and Folded Spectrum Method Numerical Results 3D Laplace M = α∆3 = α (∆1 ⊗ I ⊗ I + I ⊗ ∆1 ⊗ I + I ⊗ I ⊗ ∆1 ) , d 3 d 3 M ∈ R(2 ) ×(2 ) . with shift σ = 230.6195, folded spectrum method, 6 eigenvalues n d tinv in s tPINVIT in s # it. error 512 3 14.300 45.979 29 2.8479 e−07 4 096 4 137.502 293.154 54 1.8060 e−08 32 768 5 1 952.980* 1 223.395 19 1.0971 e−11 262 144 6 37 149.998* out of mem1 1 canceled after 60 hours while using > 300 GB RAMMax Planck Institute Magdeburg Thomas Mach, Computing Eigenvalues of Matrices in TTM Format 19/21

Tensor Trains PINVIT and Folded Spectrum Method Numerical Results 4D Laplace M = α∆4 = α (∆2 ⊗ I + I ⊗ ∆2 ) , d 4 d 4 M ∈ R(2 ) ×(2 ) without shift, 5 smallest eigenvalues n d tinv in s tPINVIT in s # it. error 256 2 0.240 9.265 36 2.5997 e−07 4 096 3 0.873 23.993 28 1.7104 e−07 65 536 4 1.436 119.348 28 5.9876 e−08 1 048 576 5 5.975 497.812 32 2.2100 e−08 16 777 216 6 12.655 1 710.326 31 7.9299 e−09 268 435 456 7 23.628 7 898.374 41 5.1963 e−10Max Planck Institute Magdeburg Thomas Mach, Computing Eigenvalues of Matrices in TTM Format 20/21

Tensor Trains PINVIT and Folded Spectrum Method Numerical Results Conclusions Finding the eigenvalues by PINVIT is cheap and storage eﬃcient.Max Planck Institute Magdeburg Thomas Mach, Computing Eigenvalues of Matrices in TTM Format 21/21

Tensor Trains PINVIT and Folded Spectrum Method Numerical Results Conclusions Finding the eigenvalues by PINVIT is cheap and storage eﬃcient. The folded spectrum method enables us to compute inner eigenvalues, too.Max Planck Institute Magdeburg Thomas Mach, Computing Eigenvalues of Matrices in TTM Format 21/21

Tensor Trains PINVIT and Folded Spectrum Method Numerical Results Conclusions Finding the eigenvalues by PINVIT is cheap and storage eﬃcient. The folded spectrum method enables us to compute inner eigenvalues, too. The use of the folded spectrum method leads to not well conditioned problems.Max Planck Institute Magdeburg Thomas Mach, Computing Eigenvalues of Matrices in TTM Format 21/21

Tensor Trains PINVIT and Folded Spectrum Method Numerical Results Conclusions Finding the eigenvalues by PINVIT is cheap and storage eﬃcient. The folded spectrum method enables us to compute inner eigenvalues, too. The use of the folded spectrum method leads to not well conditioned problems. Choose the shift and the subspace dimension carefully.Max Planck Institute Magdeburg Thomas Mach, Computing Eigenvalues of Matrices in TTM Format 21/21

Tensor Trains PINVIT and Folded Spectrum Method Numerical Results Conclusions Finding the eigenvalues by PINVIT is cheap and storage eﬃcient. The folded spectrum method enables us to compute inner eigenvalues, too. The use of the folded spectrum method leads to not well conditioned problems. Choose the shift and the subspace dimension carefully. Thank you for your attention.Max Planck Institute Magdeburg Thomas Mach, Computing Eigenvalues of Matrices in TTM Format 21/21

http://www.mpi-magdeburg.mpg.de	99
http://www.de.mpi-magdeburg.mpg.de	2
http://www.en.mpi-magdeburg.mpg.de	1

Computing Inner Eigenvalues of Matrices in Tensor Train Matrix Format

by Thomas Mach on Dec 06, 2011

Accessibility

Categories

Upload Details

Usage Rights

3 Embeds 102

Statistics

Computing Inner Eigenvalues of Matrices in Tensor Train Matrix Format Presentation Transcript