Discusses a generalization of Taylor's theorem to matrix functions followed by new upper bounds on their condition numbers.
The resulting algorithm is shown to approximate the condition number of the function A^t much faster than current alternatives. We would recommend using this algorithm first, reverting to other (slower) algorithms if a tighter bound is required.
Locating and isolating a gene, FISH, GISH, Chromosome walking and jumping, te...
Taylor's Theorem for Matrix Functions and Pseudospectral Bounds on the Condition Number
1. Taylor’s Theorem for Matrix Functions and
Pseudospectral Bounds on the Condition
Number
Samuel Relton
samuel.relton@maths.man.ac.uk @sdrelton
samrelton.com blog.samrelton.com
Joint work with Edvin Deadman
edvin.deadman@nag.co.uk
University of Strathclyde
June 23rd, 2015
Sam Relton (UoM) Taylor’s Theorem for f (A) June 23rd, 2015 1 / 21
2. Outline
• Taylor’s Theorem for Scalar Functions
• Matrix Functions, their Derivatives, and the Condition Number
• Taylor’s Theorem for Matrix Functions
• Pseudospectral Bounds on the Condition Number
• Numerical Experiments
Sam Relton (UoM) Taylor’s Theorem for f (A) June 23rd, 2015 2 / 21
3. Taylor’s Theorem - 1
Theorem (Taylor’s Theorem)
When f : R → R is k times continuously differentiable at a ∈ R there
exists Rk : R → R such that
f (x) =
k
j=0
f (j)(a)
j!
(x − a)j
+ Rk(x).
Different expressions for the remainder term Rk(x) include
• the Lagrange form.
• the Cauchy form.
• the contour integral form.
Sam Relton (UoM) Taylor’s Theorem for f (A) June 23rd, 2015 3 / 21
4. Taylor’s Theorem - 2
We can extend this to complex analytic functions.
If f (z) is complex analytic in an open set D ⊂ C then for any a ∈ D
f (z) =
k
j=0
f (k)(a)
j!
(z − a)j
+ Rk(z),
where
Rk(z) =
(z − a)k+1
2πi Γ
f (w)dw
(w − a)k+1(w − z)
,
and Γ is a closed curve in D containing a.
Sam Relton (UoM) Taylor’s Theorem for f (A) June 23rd, 2015 4 / 21
5. Matrix Functions
We are interested in extending this to matrix functions f : Cn×n → Cn×n.
For example:
• the matrix exponential
eA
=
∞
j=0
Aj
j!
.
• the matrix cosine
cos(A) =
∞
j=0
(−1)j A2j
(2j)!
.
Applications include:
• Differential equations: du
dt = Au(t), u(t) = etAu(0).
• Second order ODEs with sine and cosine.
• Ranking importance of nodes in a graph etc. . .
Sam Relton (UoM) Taylor’s Theorem for f (A) June 23rd, 2015 5 / 21
6. Fr´echet derivatives
Let f : Cn×n → Cn×n be a matrix function.
Definition (Fr´echet derivative)
The Fr´echet derivative of f at A is the unique linear function
Lf (A, ·) : Cn×n → Cn×n such that for all E
f (A + E) − f (A) − Lf (A, E) = o( E ).
• Lf (A, E) is just a linear approximation to f (A + E) − f (A).
• Higher order derivatives are defined recursively (Higham & R., 2014).
Sam Relton (UoM) Taylor’s Theorem for f (A) June 23rd, 2015 6 / 21
7. Condition Numbers
A condition number describes the sensitivity of f at A to small
perturbations arising from rounding error etc.
The absolute condition number is given by
condabs(f , A) := lim
→0
sup
E ≤
f (A + E) − f (A)
= max
E =1
Lf (A, E) ,
whilst the relative condition number is
condrel(f , A) := condabs(f , A)
A
f (A)
.
Sam Relton (UoM) Taylor’s Theorem for f (A) June 23rd, 2015 7 / 21
8. Matrix Functions and Taylor’s Theorem - 1
Previous results combining these two ideas include:
• an expansion around αI
f (A) =
∞
j=0
f (j)(α)
j!
(A − αI)j
.
• an expansion in terms of derivatives
f (A + E) =
∞
j=0
1
j!
dj
dtj
t=0
f (A + tE).
Note that:
• neither expansion has an explicit remainder term.
• dj
dtj
t=0
f (A + tE) = Lf (A, E) when j = 1.
Sam Relton (UoM) Taylor’s Theorem for f (A) June 23rd, 2015 8 / 21
9. Matrix Functions and Taylor’s Theorem - 2
Let us take D
[j]
f (A, E) := dj
dtj
t=0
f (A + tE) then we have the following.
Theorem (Taylor’s Theorem for Matrix Functions)
Let f : Cn×n → Cn×n we analytic in an open set D ⊂ C with A, E
satisfying Λ(A), Λ(A + E) ⊂ D. Then
f (A + E) = Tk(A, E) + Rk(A, E),
where
Tk(A, E) =
k
j=0
1
j!
D[j]
(A, E),
and
Rk(A, E) =
1
2πi Γ
f (z)(zI − A − E)−1
[E(zI − A)−1
]k+1
dz,
where Γ is a closed contour enclosing Λ(A) and Λ(A + E).
Sam Relton (UoM) Taylor’s Theorem for f (A) June 23rd, 2015 9 / 21
10. Matrix Functions and Taylor’s Theorem - 3
As an example take f (z) = z−1.
D
[1]
z−1 (A, E) = −A−1
EA−1
,
D
[2]
z−1 (A, E) = 2A−1
EA−1
EA−1
.
Therefore we have
(A + E)−1
=
1
0!
A−1
−
1
1!
A−1
EA−1
+
2
2!
A−1
EA−1
EA−1
+
Γ
1
z
(zI − A − E)−1
[E(zI − A)−1
]3
dz.
Sam Relton (UoM) Taylor’s Theorem for f (A) June 23rd, 2015 10 / 21
11. Applying Pseudospectral Theory - 1
Recall that the -pseudospectrum of X is the set
Λ (X) = {z ∈ C : (zI − X)−1
≥ −1
}.
The -psuedospectral radius is ρ = max |z| for z ∈ Λ (X).
-1 0 1 2 3
-3
-2
-1
0
1
2
3
-2.5
-2
-1.5
-1
Sam Relton (UoM) Taylor’s Theorem for f (A) June 23rd, 2015 11 / 21
12. Applying Pseudospectral Theory - 1
Recall that the -pseudospectrum of X is the set
Λ (X) = {z ∈ C : (zI − X)−1
≥ −1
}.
The -psuedospectral radius is ρ = max |z| for z ∈ Λ (X).
Using this we can bound the remainder term by
Rk(A, E) ≤
E k+1˜L
2π k+1
max
z∈ ˜Γ
|f (z)|,
where
• ˜Γ is a contour enclosing Λ (A) and Λ (A + E).
• ˜L is the length of the contour ˜Γ .
• is a parameter to be chosen.
Sam Relton (UoM) Taylor’s Theorem for f (A) June 23rd, 2015 11 / 21
13. Applying Pseudospectral Theory - 2
Applying this to R0(A, E) gives a bound on the condition number.
condabs(f , A) ≤
L
2π 2
max
z∈Γ
|f (z)|,
where Γ encloses Λ (A) and has length L .
Interesting because:
• Usually only lower bounds on condition number are known.
• Computing (or estimating) this efficiently could be of considerable
interest in practice or for algorithm design.
Sam Relton (UoM) Taylor’s Theorem for f (A) June 23rd, 2015 12 / 21
14. The Condition Number of At
- 1
This upper bound is extremely efficient to compute for the matrix function
given by f (x) = xt for t ∈ (0, 1).
Our experiments will
• determine how tight the upper bound is as changes.
• see how fast evaluating the upper bound is in comparison to
computing it exactly.
Other methods for this problem are:
• “CN Exact” – computes condition number exactly.
• “CN Normest” – lower bound using norm estimator.
Sam Relton (UoM) Taylor’s Theorem for f (A) June 23rd, 2015 13 / 21
15. The Condition Number of At
- 2
This function has a branch cut along the negative real line, meaning we
need to choose a keyhole contour. Overall:
condabs(xt
, A) ≤
2(π + 1)ρ1+t√
n
2π 2
,
where ρ is the -pseudospectral radius, computed using code by Gugliemi
and Overton.
Note: There is an upper limit for where the pseudospectrum intersects
the branch cut. We need to take smaller than this value.
Sam Relton (UoM) Taylor’s Theorem for f (A) June 23rd, 2015 14 / 21
16. Test matrix - Grcar matrix
-1 0 1 2 3
-3
-2
-1
0
1
2
3
-2.5
-2
-1.5
-1
Sam Relton (UoM) Taylor’s Theorem for f (A) June 23rd, 2015 15 / 21
21. Runtime Comparison - Speedup
n
0 50 100 150 200
speedup
0
200
400
600
800
1000
t=1/5
t=1/10
t=1/15
Sam Relton (UoM) Taylor’s Theorem for f (A) June 23rd, 2015 20 / 21
22. Conclusions
• Extended Taylor’s theorem to matrix functions.
• Applied pseudospectral theory to bound remainder term.
• Bounds are very efficient to compute for At.
• If bound is unsatisfactorily large can revert to a more precise method.
Future work:
• Apply to algorithm design.
• Find other classes of functions for which this is efficient.
Sam Relton (UoM) Taylor’s Theorem for f (A) June 23rd, 2015 21 / 21