Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Taylor’s Theorem for Matrix Functions and
Pseudospectral Bounds on the Condition
Number
Samuel Relton
samuel.relton@maths....
Outline
• Taylor’s Theorem for Scalar Functions
• Matrix Functions, their Derivatives, and the Condition Number
• Taylor’s...
Taylor’s Theorem - 1
Theorem (Taylor’s Theorem)
When f : R → R is k times continuously differentiable at a ∈ R there
exists...
Taylor’s Theorem - 2
We can extend this to complex analytic functions.
If f (z) is complex analytic in an open set D ⊂ C t...
Matrix Functions
We are interested in extending this to matrix functions f : Cn×n → Cn×n.
For example:
• the matrix expone...
Fr´echet derivatives
Let f : Cn×n → Cn×n be a matrix function.
Definition (Fr´echet derivative)
The Fr´echet derivative of ...
Condition Numbers
A condition number describes the sensitivity of f at A to small
perturbations arising from rounding erro...
Matrix Functions and Taylor’s Theorem - 1
Previous results combining these two ideas include:
• an expansion around αI
f (...
Matrix Functions and Taylor’s Theorem - 2
Let us take D
[j]
f (A, E) := dj
dtj
t=0
f (A + tE) then we have the following.
...
Matrix Functions and Taylor’s Theorem - 3
As an example take f (z) = z−1.
D
[1]
z−1 (A, E) = −A−1
EA−1
,
D
[2]
z−1 (A, E) ...
Applying Pseudospectral Theory - 1
Recall that the -pseudospectrum of X is the set
Λ (X) = {z ∈ C : (zI − X)−1
≥ −1
}.
The...
Applying Pseudospectral Theory - 1
Recall that the -pseudospectrum of X is the set
Λ (X) = {z ∈ C : (zI − X)−1
≥ −1
}.
The...
Applying Pseudospectral Theory - 2
Applying this to R0(A, E) gives a bound on the condition number.
condabs(f , A) ≤
L
2π ...
The Condition Number of At
- 1
This upper bound is extremely efficient to compute for the matrix function
given by f (x) = x...
The Condition Number of At
- 2
This function has a branch cut along the negative real line, meaning we
need to choose a ke...
Test matrix - Grcar matrix
-1 0 1 2 3
-3
-2
-1
0
1
2
3
-2.5
-2
-1.5
-1
Sam Relton (UoM) Taylor’s Theorem for f (A) June 23...
CN Bound as varies
0
10 -8
10 -7
10 -6
10 -5
10 -4
10 -3
10 -2
10 -1
10 0
10 1
10 -15
10 -10
10 -5
10 0 CN Pseudo
CN Exact...
Test matrix - Almost neg. eigenvalues
-1 -0.5 0 0.5
-0.3
-0.2
-0.1
0
0.1
0.2
-5
-4.5
-4
-3.5
Sam Relton (UoM) Taylor’s The...
CN Bound varies
0
10 -8
10 -7
10 -6
10 -5
10 -4
10 -3
10 -2
10 -1
10 0
10 1
10 -15
10 -10
10 -5
10 0 CN Pseudo
CN Exact
CN...
Runtime Comparison - Timings
n
0 50 100 150 200
runtime(s)
10 -4
10 -2
10 0
10 2
10 4
CN Normest t=1/5
CN Pseudo t=1/5
CN ...
Runtime Comparison - Speedup
n
0 50 100 150 200
speedup
0
200
400
600
800
1000
t=1/5
t=1/10
t=1/15
Sam Relton (UoM) Taylor...
Conclusions
• Extended Taylor’s theorem to matrix functions.
• Applied pseudospectral theory to bound remainder term.
• Bo...
Upcoming SlideShare
Loading in …5
×

Taylor's Theorem for Matrix Functions and Pseudospectral Bounds on the Condition Number

429 views

Published on

Discusses a generalization of Taylor's theorem to matrix functions followed by new upper bounds on their condition numbers.

The resulting algorithm is shown to approximate the condition number of the function A^t much faster than current alternatives. We would recommend using this algorithm first, reverting to other (slower) algorithms if a tighter bound is required.

Published in: Science
  • Be the first to comment

  • Be the first to like this

Taylor's Theorem for Matrix Functions and Pseudospectral Bounds on the Condition Number

  1. 1. Taylor’s Theorem for Matrix Functions and Pseudospectral Bounds on the Condition Number Samuel Relton samuel.relton@maths.man.ac.uk @sdrelton samrelton.com blog.samrelton.com Joint work with Edvin Deadman edvin.deadman@nag.co.uk University of Strathclyde June 23rd, 2015 Sam Relton (UoM) Taylor’s Theorem for f (A) June 23rd, 2015 1 / 21
  2. 2. Outline • Taylor’s Theorem for Scalar Functions • Matrix Functions, their Derivatives, and the Condition Number • Taylor’s Theorem for Matrix Functions • Pseudospectral Bounds on the Condition Number • Numerical Experiments Sam Relton (UoM) Taylor’s Theorem for f (A) June 23rd, 2015 2 / 21
  3. 3. Taylor’s Theorem - 1 Theorem (Taylor’s Theorem) When f : R → R is k times continuously differentiable at a ∈ R there exists Rk : R → R such that f (x) = k j=0 f (j)(a) j! (x − a)j + Rk(x). Different expressions for the remainder term Rk(x) include • the Lagrange form. • the Cauchy form. • the contour integral form. Sam Relton (UoM) Taylor’s Theorem for f (A) June 23rd, 2015 3 / 21
  4. 4. Taylor’s Theorem - 2 We can extend this to complex analytic functions. If f (z) is complex analytic in an open set D ⊂ C then for any a ∈ D f (z) = k j=0 f (k)(a) j! (z − a)j + Rk(z), where Rk(z) = (z − a)k+1 2πi Γ f (w)dw (w − a)k+1(w − z) , and Γ is a closed curve in D containing a. Sam Relton (UoM) Taylor’s Theorem for f (A) June 23rd, 2015 4 / 21
  5. 5. Matrix Functions We are interested in extending this to matrix functions f : Cn×n → Cn×n. For example: • the matrix exponential eA = ∞ j=0 Aj j! . • the matrix cosine cos(A) = ∞ j=0 (−1)j A2j (2j)! . Applications include: • Differential equations: du dt = Au(t), u(t) = etAu(0). • Second order ODEs with sine and cosine. • Ranking importance of nodes in a graph etc. . . Sam Relton (UoM) Taylor’s Theorem for f (A) June 23rd, 2015 5 / 21
  6. 6. Fr´echet derivatives Let f : Cn×n → Cn×n be a matrix function. Definition (Fr´echet derivative) The Fr´echet derivative of f at A is the unique linear function Lf (A, ·) : Cn×n → Cn×n such that for all E f (A + E) − f (A) − Lf (A, E) = o( E ). • Lf (A, E) is just a linear approximation to f (A + E) − f (A). • Higher order derivatives are defined recursively (Higham & R., 2014). Sam Relton (UoM) Taylor’s Theorem for f (A) June 23rd, 2015 6 / 21
  7. 7. Condition Numbers A condition number describes the sensitivity of f at A to small perturbations arising from rounding error etc. The absolute condition number is given by condabs(f , A) := lim →0 sup E ≤ f (A + E) − f (A) = max E =1 Lf (A, E) , whilst the relative condition number is condrel(f , A) := condabs(f , A) A f (A) . Sam Relton (UoM) Taylor’s Theorem for f (A) June 23rd, 2015 7 / 21
  8. 8. Matrix Functions and Taylor’s Theorem - 1 Previous results combining these two ideas include: • an expansion around αI f (A) = ∞ j=0 f (j)(α) j! (A − αI)j . • an expansion in terms of derivatives f (A + E) = ∞ j=0 1 j! dj dtj t=0 f (A + tE). Note that: • neither expansion has an explicit remainder term. • dj dtj t=0 f (A + tE) = Lf (A, E) when j = 1. Sam Relton (UoM) Taylor’s Theorem for f (A) June 23rd, 2015 8 / 21
  9. 9. Matrix Functions and Taylor’s Theorem - 2 Let us take D [j] f (A, E) := dj dtj t=0 f (A + tE) then we have the following. Theorem (Taylor’s Theorem for Matrix Functions) Let f : Cn×n → Cn×n we analytic in an open set D ⊂ C with A, E satisfying Λ(A), Λ(A + E) ⊂ D. Then f (A + E) = Tk(A, E) + Rk(A, E), where Tk(A, E) = k j=0 1 j! D[j] (A, E), and Rk(A, E) = 1 2πi Γ f (z)(zI − A − E)−1 [E(zI − A)−1 ]k+1 dz, where Γ is a closed contour enclosing Λ(A) and Λ(A + E). Sam Relton (UoM) Taylor’s Theorem for f (A) June 23rd, 2015 9 / 21
  10. 10. Matrix Functions and Taylor’s Theorem - 3 As an example take f (z) = z−1. D [1] z−1 (A, E) = −A−1 EA−1 , D [2] z−1 (A, E) = 2A−1 EA−1 EA−1 . Therefore we have (A + E)−1 = 1 0! A−1 − 1 1! A−1 EA−1 + 2 2! A−1 EA−1 EA−1 + Γ 1 z (zI − A − E)−1 [E(zI − A)−1 ]3 dz. Sam Relton (UoM) Taylor’s Theorem for f (A) June 23rd, 2015 10 / 21
  11. 11. Applying Pseudospectral Theory - 1 Recall that the -pseudospectrum of X is the set Λ (X) = {z ∈ C : (zI − X)−1 ≥ −1 }. The -psuedospectral radius is ρ = max |z| for z ∈ Λ (X). -1 0 1 2 3 -3 -2 -1 0 1 2 3 -2.5 -2 -1.5 -1 Sam Relton (UoM) Taylor’s Theorem for f (A) June 23rd, 2015 11 / 21
  12. 12. Applying Pseudospectral Theory - 1 Recall that the -pseudospectrum of X is the set Λ (X) = {z ∈ C : (zI − X)−1 ≥ −1 }. The -psuedospectral radius is ρ = max |z| for z ∈ Λ (X). Using this we can bound the remainder term by Rk(A, E) ≤ E k+1˜L 2π k+1 max z∈ ˜Γ |f (z)|, where • ˜Γ is a contour enclosing Λ (A) and Λ (A + E). • ˜L is the length of the contour ˜Γ . • is a parameter to be chosen. Sam Relton (UoM) Taylor’s Theorem for f (A) June 23rd, 2015 11 / 21
  13. 13. Applying Pseudospectral Theory - 2 Applying this to R0(A, E) gives a bound on the condition number. condabs(f , A) ≤ L 2π 2 max z∈Γ |f (z)|, where Γ encloses Λ (A) and has length L . Interesting because: • Usually only lower bounds on condition number are known. • Computing (or estimating) this efficiently could be of considerable interest in practice or for algorithm design. Sam Relton (UoM) Taylor’s Theorem for f (A) June 23rd, 2015 12 / 21
  14. 14. The Condition Number of At - 1 This upper bound is extremely efficient to compute for the matrix function given by f (x) = xt for t ∈ (0, 1). Our experiments will • determine how tight the upper bound is as changes. • see how fast evaluating the upper bound is in comparison to computing it exactly. Other methods for this problem are: • “CN Exact” – computes condition number exactly. • “CN Normest” – lower bound using norm estimator. Sam Relton (UoM) Taylor’s Theorem for f (A) June 23rd, 2015 13 / 21
  15. 15. The Condition Number of At - 2 This function has a branch cut along the negative real line, meaning we need to choose a keyhole contour. Overall: condabs(xt , A) ≤ 2(π + 1)ρ1+t√ n 2π 2 , where ρ is the -pseudospectral radius, computed using code by Gugliemi and Overton. Note: There is an upper limit for where the pseudospectrum intersects the branch cut. We need to take smaller than this value. Sam Relton (UoM) Taylor’s Theorem for f (A) June 23rd, 2015 14 / 21
  16. 16. Test matrix - Grcar matrix -1 0 1 2 3 -3 -2 -1 0 1 2 3 -2.5 -2 -1.5 -1 Sam Relton (UoM) Taylor’s Theorem for f (A) June 23rd, 2015 15 / 21
  17. 17. CN Bound as varies 0 10 -8 10 -7 10 -6 10 -5 10 -4 10 -3 10 -2 10 -1 10 0 10 1 10 -15 10 -10 10 -5 10 0 CN Pseudo CN Exact CN Normest Sam Relton (UoM) Taylor’s Theorem for f (A) June 23rd, 2015 16 / 21
  18. 18. Test matrix - Almost neg. eigenvalues -1 -0.5 0 0.5 -0.3 -0.2 -0.1 0 0.1 0.2 -5 -4.5 -4 -3.5 Sam Relton (UoM) Taylor’s Theorem for f (A) June 23rd, 2015 17 / 21
  19. 19. CN Bound varies 0 10 -8 10 -7 10 -6 10 -5 10 -4 10 -3 10 -2 10 -1 10 0 10 1 10 -15 10 -10 10 -5 10 0 CN Pseudo CN Exact CN Normest Sam Relton (UoM) Taylor’s Theorem for f (A) June 23rd, 2015 18 / 21
  20. 20. Runtime Comparison - Timings n 0 50 100 150 200 runtime(s) 10 -4 10 -2 10 0 10 2 10 4 CN Normest t=1/5 CN Pseudo t=1/5 CN Normest t=1/10 CN Pseudo t=1/10 CN Normest t=1/15 CN Pseudo t=1/15 Sam Relton (UoM) Taylor’s Theorem for f (A) June 23rd, 2015 19 / 21
  21. 21. Runtime Comparison - Speedup n 0 50 100 150 200 speedup 0 200 400 600 800 1000 t=1/5 t=1/10 t=1/15 Sam Relton (UoM) Taylor’s Theorem for f (A) June 23rd, 2015 20 / 21
  22. 22. Conclusions • Extended Taylor’s theorem to matrix functions. • Applied pseudospectral theory to bound remainder term. • Bounds are very efficient to compute for At. • If bound is unsatisfactorily large can revert to a more precise method. Future work: • Apply to algorithm design. • Find other classes of functions for which this is efficient. Sam Relton (UoM) Taylor’s Theorem for f (A) June 23rd, 2015 21 / 21

×