Successfully reported this slideshow.
Upcoming SlideShare
×

# Taylor's Theorem for Matrix Functions and Pseudospectral Bounds on the Condition Number

429 views

Published on

Discusses a generalization of Taylor's theorem to matrix functions followed by new upper bounds on their condition numbers.

The resulting algorithm is shown to approximate the condition number of the function A^t much faster than current alternatives. We would recommend using this algorithm first, reverting to other (slower) algorithms if a tighter bound is required.

Published in: Science
• Full Name
Comment goes here.

Are you sure you want to Yes No
• Be the first to comment

• Be the first to like this

### Taylor's Theorem for Matrix Functions and Pseudospectral Bounds on the Condition Number

1. 1. Taylor’s Theorem for Matrix Functions and Pseudospectral Bounds on the Condition Number Samuel Relton samuel.relton@maths.man.ac.uk @sdrelton samrelton.com blog.samrelton.com Joint work with Edvin Deadman edvin.deadman@nag.co.uk University of Strathclyde June 23rd, 2015 Sam Relton (UoM) Taylor’s Theorem for f (A) June 23rd, 2015 1 / 21
2. 2. Outline • Taylor’s Theorem for Scalar Functions • Matrix Functions, their Derivatives, and the Condition Number • Taylor’s Theorem for Matrix Functions • Pseudospectral Bounds on the Condition Number • Numerical Experiments Sam Relton (UoM) Taylor’s Theorem for f (A) June 23rd, 2015 2 / 21
3. 3. Taylor’s Theorem - 1 Theorem (Taylor’s Theorem) When f : R → R is k times continuously diﬀerentiable at a ∈ R there exists Rk : R → R such that f (x) = k j=0 f (j)(a) j! (x − a)j + Rk(x). Diﬀerent expressions for the remainder term Rk(x) include • the Lagrange form. • the Cauchy form. • the contour integral form. Sam Relton (UoM) Taylor’s Theorem for f (A) June 23rd, 2015 3 / 21
4. 4. Taylor’s Theorem - 2 We can extend this to complex analytic functions. If f (z) is complex analytic in an open set D ⊂ C then for any a ∈ D f (z) = k j=0 f (k)(a) j! (z − a)j + Rk(z), where Rk(z) = (z − a)k+1 2πi Γ f (w)dw (w − a)k+1(w − z) , and Γ is a closed curve in D containing a. Sam Relton (UoM) Taylor’s Theorem for f (A) June 23rd, 2015 4 / 21
5. 5. Matrix Functions We are interested in extending this to matrix functions f : Cn×n → Cn×n. For example: • the matrix exponential eA = ∞ j=0 Aj j! . • the matrix cosine cos(A) = ∞ j=0 (−1)j A2j (2j)! . Applications include: • Diﬀerential equations: du dt = Au(t), u(t) = etAu(0). • Second order ODEs with sine and cosine. • Ranking importance of nodes in a graph etc. . . Sam Relton (UoM) Taylor’s Theorem for f (A) June 23rd, 2015 5 / 21
6. 6. Fr´echet derivatives Let f : Cn×n → Cn×n be a matrix function. Deﬁnition (Fr´echet derivative) The Fr´echet derivative of f at A is the unique linear function Lf (A, ·) : Cn×n → Cn×n such that for all E f (A + E) − f (A) − Lf (A, E) = o( E ). • Lf (A, E) is just a linear approximation to f (A + E) − f (A). • Higher order derivatives are deﬁned recursively (Higham & R., 2014). Sam Relton (UoM) Taylor’s Theorem for f (A) June 23rd, 2015 6 / 21
7. 7. Condition Numbers A condition number describes the sensitivity of f at A to small perturbations arising from rounding error etc. The absolute condition number is given by condabs(f , A) := lim →0 sup E ≤ f (A + E) − f (A) = max E =1 Lf (A, E) , whilst the relative condition number is condrel(f , A) := condabs(f , A) A f (A) . Sam Relton (UoM) Taylor’s Theorem for f (A) June 23rd, 2015 7 / 21
8. 8. Matrix Functions and Taylor’s Theorem - 1 Previous results combining these two ideas include: • an expansion around αI f (A) = ∞ j=0 f (j)(α) j! (A − αI)j . • an expansion in terms of derivatives f (A + E) = ∞ j=0 1 j! dj dtj t=0 f (A + tE). Note that: • neither expansion has an explicit remainder term. • dj dtj t=0 f (A + tE) = Lf (A, E) when j = 1. Sam Relton (UoM) Taylor’s Theorem for f (A) June 23rd, 2015 8 / 21
9. 9. Matrix Functions and Taylor’s Theorem - 2 Let us take D [j] f (A, E) := dj dtj t=0 f (A + tE) then we have the following. Theorem (Taylor’s Theorem for Matrix Functions) Let f : Cn×n → Cn×n we analytic in an open set D ⊂ C with A, E satisfying Λ(A), Λ(A + E) ⊂ D. Then f (A + E) = Tk(A, E) + Rk(A, E), where Tk(A, E) = k j=0 1 j! D[j] (A, E), and Rk(A, E) = 1 2πi Γ f (z)(zI − A − E)−1 [E(zI − A)−1 ]k+1 dz, where Γ is a closed contour enclosing Λ(A) and Λ(A + E). Sam Relton (UoM) Taylor’s Theorem for f (A) June 23rd, 2015 9 / 21
10. 10. Matrix Functions and Taylor’s Theorem - 3 As an example take f (z) = z−1. D [1] z−1 (A, E) = −A−1 EA−1 , D [2] z−1 (A, E) = 2A−1 EA−1 EA−1 . Therefore we have (A + E)−1 = 1 0! A−1 − 1 1! A−1 EA−1 + 2 2! A−1 EA−1 EA−1 + Γ 1 z (zI − A − E)−1 [E(zI − A)−1 ]3 dz. Sam Relton (UoM) Taylor’s Theorem for f (A) June 23rd, 2015 10 / 21
11. 11. Applying Pseudospectral Theory - 1 Recall that the -pseudospectrum of X is the set Λ (X) = {z ∈ C : (zI − X)−1 ≥ −1 }. The -psuedospectral radius is ρ = max |z| for z ∈ Λ (X). -1 0 1 2 3 -3 -2 -1 0 1 2 3 -2.5 -2 -1.5 -1 Sam Relton (UoM) Taylor’s Theorem for f (A) June 23rd, 2015 11 / 21
12. 12. Applying Pseudospectral Theory - 1 Recall that the -pseudospectrum of X is the set Λ (X) = {z ∈ C : (zI − X)−1 ≥ −1 }. The -psuedospectral radius is ρ = max |z| for z ∈ Λ (X). Using this we can bound the remainder term by Rk(A, E) ≤ E k+1˜L 2π k+1 max z∈ ˜Γ |f (z)|, where • ˜Γ is a contour enclosing Λ (A) and Λ (A + E). • ˜L is the length of the contour ˜Γ . • is a parameter to be chosen. Sam Relton (UoM) Taylor’s Theorem for f (A) June 23rd, 2015 11 / 21
13. 13. Applying Pseudospectral Theory - 2 Applying this to R0(A, E) gives a bound on the condition number. condabs(f , A) ≤ L 2π 2 max z∈Γ |f (z)|, where Γ encloses Λ (A) and has length L . Interesting because: • Usually only lower bounds on condition number are known. • Computing (or estimating) this eﬃciently could be of considerable interest in practice or for algorithm design. Sam Relton (UoM) Taylor’s Theorem for f (A) June 23rd, 2015 12 / 21
14. 14. The Condition Number of At - 1 This upper bound is extremely eﬃcient to compute for the matrix function given by f (x) = xt for t ∈ (0, 1). Our experiments will • determine how tight the upper bound is as changes. • see how fast evaluating the upper bound is in comparison to computing it exactly. Other methods for this problem are: • “CN Exact” – computes condition number exactly. • “CN Normest” – lower bound using norm estimator. Sam Relton (UoM) Taylor’s Theorem for f (A) June 23rd, 2015 13 / 21
15. 15. The Condition Number of At - 2 This function has a branch cut along the negative real line, meaning we need to choose a keyhole contour. Overall: condabs(xt , A) ≤ 2(π + 1)ρ1+t√ n 2π 2 , where ρ is the -pseudospectral radius, computed using code by Gugliemi and Overton. Note: There is an upper limit for where the pseudospectrum intersects the branch cut. We need to take smaller than this value. Sam Relton (UoM) Taylor’s Theorem for f (A) June 23rd, 2015 14 / 21
16. 16. Test matrix - Grcar matrix -1 0 1 2 3 -3 -2 -1 0 1 2 3 -2.5 -2 -1.5 -1 Sam Relton (UoM) Taylor’s Theorem for f (A) June 23rd, 2015 15 / 21
17. 17. CN Bound as varies 0 10 -8 10 -7 10 -6 10 -5 10 -4 10 -3 10 -2 10 -1 10 0 10 1 10 -15 10 -10 10 -5 10 0 CN Pseudo CN Exact CN Normest Sam Relton (UoM) Taylor’s Theorem for f (A) June 23rd, 2015 16 / 21
18. 18. Test matrix - Almost neg. eigenvalues -1 -0.5 0 0.5 -0.3 -0.2 -0.1 0 0.1 0.2 -5 -4.5 -4 -3.5 Sam Relton (UoM) Taylor’s Theorem for f (A) June 23rd, 2015 17 / 21
19. 19. CN Bound varies 0 10 -8 10 -7 10 -6 10 -5 10 -4 10 -3 10 -2 10 -1 10 0 10 1 10 -15 10 -10 10 -5 10 0 CN Pseudo CN Exact CN Normest Sam Relton (UoM) Taylor’s Theorem for f (A) June 23rd, 2015 18 / 21
20. 20. Runtime Comparison - Timings n 0 50 100 150 200 runtime(s) 10 -4 10 -2 10 0 10 2 10 4 CN Normest t=1/5 CN Pseudo t=1/5 CN Normest t=1/10 CN Pseudo t=1/10 CN Normest t=1/15 CN Pseudo t=1/15 Sam Relton (UoM) Taylor’s Theorem for f (A) June 23rd, 2015 19 / 21
21. 21. Runtime Comparison - Speedup n 0 50 100 150 200 speedup 0 200 400 600 800 1000 t=1/5 t=1/10 t=1/15 Sam Relton (UoM) Taylor’s Theorem for f (A) June 23rd, 2015 20 / 21
22. 22. Conclusions • Extended Taylor’s theorem to matrix functions. • Applied pseudospectral theory to bound remainder term. • Bounds are very eﬃcient to compute for At. • If bound is unsatisfactorily large can revert to a more precise method. Future work: • Apply to algorithm design. • Find other classes of functions for which this is eﬃcient. Sam Relton (UoM) Taylor’s Theorem for f (A) June 23rd, 2015 21 / 21