Discusses a generalization of Taylor's theorem to matrix functions followed by new upper bounds on their condition numbers. The resulting algorithm is shown to approximate the condition number of the function A^t much faster than current alternatives. We would recommend using this algorithm first, reverting to other (slower) algorithms if a tighter bound is required.

1. Taylor’s Theorem for Matrix Functions and
Pseudospectral Bounds on the Condition
Number
Samuel Relton
samuel.relton@maths.man.ac.uk @sdrelton
samrelton.com blog.samrelton.com
Joint work with Edvin Deadman
edvin.deadman@nag.co.uk
University of Strathclyde
June 23rd, 2015
Sam Relton (UoM) Taylor’s Theorem for f (A) June 23rd, 2015 1 / 21

2. Outline
• Taylor’s Theorem for Scalar Functions
• Matrix Functions, their Derivatives, and the Condition Number
• Taylor’s Theorem for Matrix Functions
• Pseudospectral Bounds on the Condition Number
• Numerical Experiments
Sam Relton (UoM) Taylor’s Theorem for f (A) June 23rd, 2015 2 / 21

3. Taylor’s Theorem - 1
Theorem (Taylor’s Theorem)
When f : R → R is k times continuously differentiable at a ∈ R there
exists Rk : R → R such that
f (x) =
k
j=0
f (j)(a)
j!
(x − a)j
+ Rk(x).
Different expressions for the remainder term Rk(x) include
• the Lagrange form.
• the Cauchy form.
• the contour integral form.
Sam Relton (UoM) Taylor’s Theorem for f (A) June 23rd, 2015 3 / 21

4. Taylor’s Theorem - 2
We can extend this to complex analytic functions.
If f (z) is complex analytic in an open set D ⊂ C then for any a ∈ D
f (z) =
k
j=0
f (k)(a)
j!
(z − a)j
+ Rk(z),
where
Rk(z) =
(z − a)k+1
2πi Γ
f (w)dw
(w − a)k+1(w − z)
,
and Γ is a closed curve in D containing a.
Sam Relton (UoM) Taylor’s Theorem for f (A) June 23rd, 2015 4 / 21

5. Matrix Functions
We are interested in extending this to matrix functions f : Cn×n → Cn×n.
For example:
• the matrix exponential
eA
=
∞
j=0
Aj
j!
.
• the matrix cosine
cos(A) =
∞
j=0
(−1)j A2j
(2j)!
.
Applications include:
• Differential equations: du
dt = Au(t), u(t) = etAu(0).
• Second order ODEs with sine and cosine.
• Ranking importance of nodes in a graph etc. . .
Sam Relton (UoM) Taylor’s Theorem for f (A) June 23rd, 2015 5 / 21

6. Fr´echet derivatives
Let f : Cn×n → Cn×n be a matrix function.
Definition (Fr´echet derivative)
The Fr´echet derivative of f at A is the unique linear function
Lf (A, ·) : Cn×n → Cn×n such that for all E
f (A + E) − f (A) − Lf (A, E) = o( E ).
• Lf (A, E) is just a linear approximation to f (A + E) − f (A).
• Higher order derivatives are defined recursively (Higham & R., 2014).
Sam Relton (UoM) Taylor’s Theorem for f (A) June 23rd, 2015 6 / 21

7. Condition Numbers
A condition number describes the sensitivity of f at A to small
perturbations arising from rounding error etc.
The absolute condition number is given by
condabs(f , A) := lim
→0
sup
E ≤
f (A + E) − f (A)
= max
E =1
Lf (A, E) ,
whilst the relative condition number is
condrel(f , A) := condabs(f , A)
A
f (A)
.
Sam Relton (UoM) Taylor’s Theorem for f (A) June 23rd, 2015 7 / 21

8. Matrix Functions and Taylor’s Theorem - 1
Previous results combining these two ideas include:
• an expansion around αI
f (A) =
∞
j=0
f (j)(α)
j!
(A − αI)j
.
• an expansion in terms of derivatives
f (A + E) =
∞
j=0
1
j!
dj
dtj
t=0
f (A + tE).
Note that:
• neither expansion has an explicit remainder term.
• dj
dtj
t=0
f (A + tE) = Lf (A, E) when j = 1.
Sam Relton (UoM) Taylor’s Theorem for f (A) June 23rd, 2015 8 / 21

9. Matrix Functions and Taylor’s Theorem - 2
Let us take D
[j]
f (A, E) := dj
dtj
t=0
f (A + tE) then we have the following.
Theorem (Taylor’s Theorem for Matrix Functions)
Let f : Cn×n → Cn×n we analytic in an open set D ⊂ C with A, E
satisfying Λ(A), Λ(A + E) ⊂ D. Then
f (A + E) = Tk(A, E) + Rk(A, E),
where
Tk(A, E) =
k
j=0
1
j!
D[j]
(A, E),
and
Rk(A, E) =
1
2πi Γ
f (z)(zI − A − E)−1
[E(zI − A)−1
]k+1
dz,
where Γ is a closed contour enclosing Λ(A) and Λ(A + E).
Sam Relton (UoM) Taylor’s Theorem for f (A) June 23rd, 2015 9 / 21

10. Matrix Functions and Taylor’s Theorem - 3
As an example take f (z) = z−1.
D
[1]
z−1 (A, E) = −A−1
EA−1
,
D
[2]
z−1 (A, E) = 2A−1
EA−1
EA−1
.
Therefore we have
(A + E)−1
=
1
0!
A−1
−
1
1!
A−1
EA−1
+
2
2!
A−1
EA−1
EA−1
+
Γ
1
z
(zI − A − E)−1
[E(zI − A)−1
]3
dz.
Sam Relton (UoM) Taylor’s Theorem for f (A) June 23rd, 2015 10 / 21

11. Applying Pseudospectral Theory - 1
Recall that the -pseudospectrum of X is the set
Λ (X) = {z ∈ C : (zI − X)−1
≥ −1
}.
The -psuedospectral radius is ρ = max |z| for z ∈ Λ (X).
-1 0 1 2 3
-3
-2
-1
0
1
2
3
-2.5
-2
-1.5
-1
Sam Relton (UoM) Taylor’s Theorem for f (A) June 23rd, 2015 11 / 21

12. Applying Pseudospectral Theory - 1
Recall that the -pseudospectrum of X is the set
Λ (X) = {z ∈ C : (zI − X)−1
≥ −1
}.
The -psuedospectral radius is ρ = max |z| for z ∈ Λ (X).
Using this we can bound the remainder term by
Rk(A, E) ≤
E k+1˜L
2π k+1
max
z∈ ˜Γ
|f (z)|,
where
• ˜Γ is a contour enclosing Λ (A) and Λ (A + E).
• ˜L is the length of the contour ˜Γ .
• is a parameter to be chosen.
Sam Relton (UoM) Taylor’s Theorem for f (A) June 23rd, 2015 11 / 21

13. Applying Pseudospectral Theory - 2
Applying this to R0(A, E) gives a bound on the condition number.
condabs(f , A) ≤
L
2π 2
max
z∈Γ
|f (z)|,
where Γ encloses Λ (A) and has length L .
Interesting because:
• Usually only lower bounds on condition number are known.
• Computing (or estimating) this efficiently could be of considerable
interest in practice or for algorithm design.
Sam Relton (UoM) Taylor’s Theorem for f (A) June 23rd, 2015 12 / 21

14. The Condition Number of At
- 1
This upper bound is extremely efficient to compute for the matrix function
given by f (x) = xt for t ∈ (0, 1).
Our experiments will
• determine how tight the upper bound is as changes.
• see how fast evaluating the upper bound is in comparison to
computing it exactly.
Other methods for this problem are:
• “CN Exact” – computes condition number exactly.
• “CN Normest” – lower bound using norm estimator.
Sam Relton (UoM) Taylor’s Theorem for f (A) June 23rd, 2015 13 / 21

15. The Condition Number of At
- 2
This function has a branch cut along the negative real line, meaning we
need to choose a keyhole contour. Overall:
condabs(xt
, A) ≤
2(π + 1)ρ1+t√
n
2π 2
,
where ρ is the -pseudospectral radius, computed using code by Gugliemi
and Overton.
Note: There is an upper limit for where the pseudospectrum intersects
the branch cut. We need to take smaller than this value.
Sam Relton (UoM) Taylor’s Theorem for f (A) June 23rd, 2015 14 / 21

16. Test matrix - Grcar matrix
-1 0 1 2 3
-3
-2
-1
0
1
2
3
-2.5
-2
-1.5
-1
Sam Relton (UoM) Taylor’s Theorem for f (A) June 23rd, 2015 15 / 21

17. CN Bound as varies
0
10 -8
10 -7
10 -6
10 -5
10 -4
10 -3
10 -2
10 -1
10 0
10 1
10 -15
10 -10
10 -5
10 0 CN Pseudo
CN Exact
CN Normest
Sam Relton (UoM) Taylor’s Theorem for f (A) June 23rd, 2015 16 / 21

18. Test matrix - Almost neg. eigenvalues
-1 -0.5 0 0.5
-0.3
-0.2
-0.1
0
0.1
0.2
-5
-4.5
-4
-3.5
Sam Relton (UoM) Taylor’s Theorem for f (A) June 23rd, 2015 17 / 21

19. CN Bound varies
0
10 -8
10 -7
10 -6
10 -5
10 -4
10 -3
10 -2
10 -1
10 0
10 1
10 -15
10 -10
10 -5
10 0 CN Pseudo
CN Exact
CN Normest
Sam Relton (UoM) Taylor’s Theorem for f (A) June 23rd, 2015 18 / 21

20. Runtime Comparison - Timings
n
0 50 100 150 200
runtime(s)
10 -4
10 -2
10 0
10 2
10 4
CN Normest t=1/5
CN Pseudo t=1/5
CN Normest t=1/10
CN Pseudo t=1/10
CN Normest t=1/15
CN Pseudo t=1/15
Sam Relton (UoM) Taylor’s Theorem for f (A) June 23rd, 2015 19 / 21

21. Runtime Comparison - Speedup
n
0 50 100 150 200
speedup
0
200
400
600
800
1000
t=1/5
t=1/10
t=1/15
Sam Relton (UoM) Taylor’s Theorem for f (A) June 23rd, 2015 20 / 21

22. Conclusions
• Extended Taylor’s theorem to matrix functions.
• Applied pseudospectral theory to bound remainder term.
• Bounds are very efficient to compute for At.
• If bound is unsatisfactorily large can revert to a more precise method.
Future work:
• Apply to algorithm design.
• Find other classes of functions for which this is efficient.
Sam Relton (UoM) Taylor’s Theorem for f (A) June 23rd, 2015 21 / 21