SlideShare a Scribd company logo
1 of 22
Download to read offline
Taylor’s Theorem for Matrix Functions and
Pseudospectral Bounds on the Condition
Number
Samuel Relton
samuel.relton@maths.man.ac.uk @sdrelton
samrelton.com blog.samrelton.com
Joint work with Edvin Deadman
edvin.deadman@nag.co.uk
University of Strathclyde
June 23rd, 2015
Sam Relton (UoM) Taylor’s Theorem for f (A) June 23rd, 2015 1 / 21
Outline
• Taylor’s Theorem for Scalar Functions
• Matrix Functions, their Derivatives, and the Condition Number
• Taylor’s Theorem for Matrix Functions
• Pseudospectral Bounds on the Condition Number
• Numerical Experiments
Sam Relton (UoM) Taylor’s Theorem for f (A) June 23rd, 2015 2 / 21
Taylor’s Theorem - 1
Theorem (Taylor’s Theorem)
When f : R → R is k times continuously differentiable at a ∈ R there
exists Rk : R → R such that
f (x) =
k
j=0
f (j)(a)
j!
(x − a)j
+ Rk(x).
Different expressions for the remainder term Rk(x) include
• the Lagrange form.
• the Cauchy form.
• the contour integral form.
Sam Relton (UoM) Taylor’s Theorem for f (A) June 23rd, 2015 3 / 21
Taylor’s Theorem - 2
We can extend this to complex analytic functions.
If f (z) is complex analytic in an open set D ⊂ C then for any a ∈ D
f (z) =
k
j=0
f (k)(a)
j!
(z − a)j
+ Rk(z),
where
Rk(z) =
(z − a)k+1
2πi Γ
f (w)dw
(w − a)k+1(w − z)
,
and Γ is a closed curve in D containing a.
Sam Relton (UoM) Taylor’s Theorem for f (A) June 23rd, 2015 4 / 21
Matrix Functions
We are interested in extending this to matrix functions f : Cn×n → Cn×n.
For example:
• the matrix exponential
eA
=
∞
j=0
Aj
j!
.
• the matrix cosine
cos(A) =
∞
j=0
(−1)j A2j
(2j)!
.
Applications include:
• Differential equations: du
dt = Au(t), u(t) = etAu(0).
• Second order ODEs with sine and cosine.
• Ranking importance of nodes in a graph etc. . .
Sam Relton (UoM) Taylor’s Theorem for f (A) June 23rd, 2015 5 / 21
Fr´echet derivatives
Let f : Cn×n → Cn×n be a matrix function.
Definition (Fr´echet derivative)
The Fr´echet derivative of f at A is the unique linear function
Lf (A, ·) : Cn×n → Cn×n such that for all E
f (A + E) − f (A) − Lf (A, E) = o( E ).
• Lf (A, E) is just a linear approximation to f (A + E) − f (A).
• Higher order derivatives are defined recursively (Higham & R., 2014).
Sam Relton (UoM) Taylor’s Theorem for f (A) June 23rd, 2015 6 / 21
Condition Numbers
A condition number describes the sensitivity of f at A to small
perturbations arising from rounding error etc.
The absolute condition number is given by
condabs(f , A) := lim
→0
sup
E ≤
f (A + E) − f (A)
= max
E =1
Lf (A, E) ,
whilst the relative condition number is
condrel(f , A) := condabs(f , A)
A
f (A)
.
Sam Relton (UoM) Taylor’s Theorem for f (A) June 23rd, 2015 7 / 21
Matrix Functions and Taylor’s Theorem - 1
Previous results combining these two ideas include:
• an expansion around αI
f (A) =
∞
j=0
f (j)(α)
j!
(A − αI)j
.
• an expansion in terms of derivatives
f (A + E) =
∞
j=0
1
j!
dj
dtj
t=0
f (A + tE).
Note that:
• neither expansion has an explicit remainder term.
• dj
dtj
t=0
f (A + tE) = Lf (A, E) when j = 1.
Sam Relton (UoM) Taylor’s Theorem for f (A) June 23rd, 2015 8 / 21
Matrix Functions and Taylor’s Theorem - 2
Let us take D
[j]
f (A, E) := dj
dtj
t=0
f (A + tE) then we have the following.
Theorem (Taylor’s Theorem for Matrix Functions)
Let f : Cn×n → Cn×n we analytic in an open set D ⊂ C with A, E
satisfying Λ(A), Λ(A + E) ⊂ D. Then
f (A + E) = Tk(A, E) + Rk(A, E),
where
Tk(A, E) =
k
j=0
1
j!
D[j]
(A, E),
and
Rk(A, E) =
1
2πi Γ
f (z)(zI − A − E)−1
[E(zI − A)−1
]k+1
dz,
where Γ is a closed contour enclosing Λ(A) and Λ(A + E).
Sam Relton (UoM) Taylor’s Theorem for f (A) June 23rd, 2015 9 / 21
Matrix Functions and Taylor’s Theorem - 3
As an example take f (z) = z−1.
D
[1]
z−1 (A, E) = −A−1
EA−1
,
D
[2]
z−1 (A, E) = 2A−1
EA−1
EA−1
.
Therefore we have
(A + E)−1
=
1
0!
A−1
−
1
1!
A−1
EA−1
+
2
2!
A−1
EA−1
EA−1
+
Γ
1
z
(zI − A − E)−1
[E(zI − A)−1
]3
dz.
Sam Relton (UoM) Taylor’s Theorem for f (A) June 23rd, 2015 10 / 21
Applying Pseudospectral Theory - 1
Recall that the -pseudospectrum of X is the set
Λ (X) = {z ∈ C : (zI − X)−1
≥ −1
}.
The -psuedospectral radius is ρ = max |z| for z ∈ Λ (X).
-1 0 1 2 3
-3
-2
-1
0
1
2
3
-2.5
-2
-1.5
-1
Sam Relton (UoM) Taylor’s Theorem for f (A) June 23rd, 2015 11 / 21
Applying Pseudospectral Theory - 1
Recall that the -pseudospectrum of X is the set
Λ (X) = {z ∈ C : (zI − X)−1
≥ −1
}.
The -psuedospectral radius is ρ = max |z| for z ∈ Λ (X).
Using this we can bound the remainder term by
Rk(A, E) ≤
E k+1˜L
2π k+1
max
z∈ ˜Γ
|f (z)|,
where
• ˜Γ is a contour enclosing Λ (A) and Λ (A + E).
• ˜L is the length of the contour ˜Γ .
• is a parameter to be chosen.
Sam Relton (UoM) Taylor’s Theorem for f (A) June 23rd, 2015 11 / 21
Applying Pseudospectral Theory - 2
Applying this to R0(A, E) gives a bound on the condition number.
condabs(f , A) ≤
L
2π 2
max
z∈Γ
|f (z)|,
where Γ encloses Λ (A) and has length L .
Interesting because:
• Usually only lower bounds on condition number are known.
• Computing (or estimating) this efficiently could be of considerable
interest in practice or for algorithm design.
Sam Relton (UoM) Taylor’s Theorem for f (A) June 23rd, 2015 12 / 21
The Condition Number of At
- 1
This upper bound is extremely efficient to compute for the matrix function
given by f (x) = xt for t ∈ (0, 1).
Our experiments will
• determine how tight the upper bound is as changes.
• see how fast evaluating the upper bound is in comparison to
computing it exactly.
Other methods for this problem are:
• “CN Exact” – computes condition number exactly.
• “CN Normest” – lower bound using norm estimator.
Sam Relton (UoM) Taylor’s Theorem for f (A) June 23rd, 2015 13 / 21
The Condition Number of At
- 2
This function has a branch cut along the negative real line, meaning we
need to choose a keyhole contour. Overall:
condabs(xt
, A) ≤
2(π + 1)ρ1+t√
n
2π 2
,
where ρ is the -pseudospectral radius, computed using code by Gugliemi
and Overton.
Note: There is an upper limit for where the pseudospectrum intersects
the branch cut. We need to take smaller than this value.
Sam Relton (UoM) Taylor’s Theorem for f (A) June 23rd, 2015 14 / 21
Test matrix - Grcar matrix
-1 0 1 2 3
-3
-2
-1
0
1
2
3
-2.5
-2
-1.5
-1
Sam Relton (UoM) Taylor’s Theorem for f (A) June 23rd, 2015 15 / 21
CN Bound as varies
0
10 -8
10 -7
10 -6
10 -5
10 -4
10 -3
10 -2
10 -1
10 0
10 1
10 -15
10 -10
10 -5
10 0 CN Pseudo
CN Exact
CN Normest
Sam Relton (UoM) Taylor’s Theorem for f (A) June 23rd, 2015 16 / 21
Test matrix - Almost neg. eigenvalues
-1 -0.5 0 0.5
-0.3
-0.2
-0.1
0
0.1
0.2
-5
-4.5
-4
-3.5
Sam Relton (UoM) Taylor’s Theorem for f (A) June 23rd, 2015 17 / 21
CN Bound varies
0
10 -8
10 -7
10 -6
10 -5
10 -4
10 -3
10 -2
10 -1
10 0
10 1
10 -15
10 -10
10 -5
10 0 CN Pseudo
CN Exact
CN Normest
Sam Relton (UoM) Taylor’s Theorem for f (A) June 23rd, 2015 18 / 21
Runtime Comparison - Timings
n
0 50 100 150 200
runtime(s)
10 -4
10 -2
10 0
10 2
10 4
CN Normest t=1/5
CN Pseudo t=1/5
CN Normest t=1/10
CN Pseudo t=1/10
CN Normest t=1/15
CN Pseudo t=1/15
Sam Relton (UoM) Taylor’s Theorem for f (A) June 23rd, 2015 19 / 21
Runtime Comparison - Speedup
n
0 50 100 150 200
speedup
0
200
400
600
800
1000
t=1/5
t=1/10
t=1/15
Sam Relton (UoM) Taylor’s Theorem for f (A) June 23rd, 2015 20 / 21
Conclusions
• Extended Taylor’s theorem to matrix functions.
• Applied pseudospectral theory to bound remainder term.
• Bounds are very efficient to compute for At.
• If bound is unsatisfactorily large can revert to a more precise method.
Future work:
• Apply to algorithm design.
• Find other classes of functions for which this is efficient.
Sam Relton (UoM) Taylor’s Theorem for f (A) June 23rd, 2015 21 / 21

More Related Content

What's hot

Using Laplace Transforms to Solve Differential Equations
Using Laplace Transforms to Solve Differential EquationsUsing Laplace Transforms to Solve Differential Equations
Using Laplace Transforms to Solve Differential Equations
George Stevens
 
The Laplace Transform of Modeling of a Spring-Mass-Damper System
The Laplace Transform of Modeling of a Spring-Mass-Damper System The Laplace Transform of Modeling of a Spring-Mass-Damper System
The Laplace Transform of Modeling of a Spring-Mass-Damper System
Mahmoud Farg
 

What's hot (19)

Laplace transform and its application
Laplace transform and its applicationLaplace transform and its application
Laplace transform and its application
 
Laplace transform
Laplace transformLaplace transform
Laplace transform
 
Laplace transform
Laplace transformLaplace transform
Laplace transform
 
Laplace Transformation & Its Application
Laplace Transformation & Its ApplicationLaplace Transformation & Its Application
Laplace Transformation & Its Application
 
Laplace transforms
Laplace transformsLaplace transforms
Laplace transforms
 
Using Laplace Transforms to Solve Differential Equations
Using Laplace Transforms to Solve Differential EquationsUsing Laplace Transforms to Solve Differential Equations
Using Laplace Transforms to Solve Differential Equations
 
Ece4510 notes05
Ece4510 notes05Ece4510 notes05
Ece4510 notes05
 
EM3 mini project Laplace Transform
EM3 mini project Laplace TransformEM3 mini project Laplace Transform
EM3 mini project Laplace Transform
 
The Laplace Transform of Modeling of a Spring-Mass-Damper System
The Laplace Transform of Modeling of a Spring-Mass-Damper System The Laplace Transform of Modeling of a Spring-Mass-Damper System
The Laplace Transform of Modeling of a Spring-Mass-Damper System
 
Production Engineering - Laplace Transformation
Production Engineering - Laplace TransformationProduction Engineering - Laplace Transformation
Production Engineering - Laplace Transformation
 
Laplace transformation
Laplace transformationLaplace transformation
Laplace transformation
 
Laplace transform
Laplace transformLaplace transform
Laplace transform
 
Lie Group Formulation for Robot Mechanics
Lie Group Formulation for Robot MechanicsLie Group Formulation for Robot Mechanics
Lie Group Formulation for Robot Mechanics
 
14210111030
1421011103014210111030
14210111030
 
Laplace transform
Laplace transformLaplace transform
Laplace transform
 
Integral Transform
Integral  TransformIntegral  Transform
Integral Transform
 
Laplace transform and its application
Laplace transform and its applicationLaplace transform and its application
Laplace transform and its application
 
160280102011 c1 aem
160280102011 c1 aem160280102011 c1 aem
160280102011 c1 aem
 
Laplace transform
Laplace transformLaplace transform
Laplace transform
 

Similar to Taylor's Theorem for Matrix Functions and Pseudospectral Bounds on the Condition Number

Quasi-Stochastic Approximation: Algorithm Design Principles with Applications...
Quasi-Stochastic Approximation: Algorithm Design Principles with Applications...Quasi-Stochastic Approximation: Algorithm Design Principles with Applications...
Quasi-Stochastic Approximation: Algorithm Design Principles with Applications...
Sean Meyn
 
Laplace transform
Laplace transformLaplace transform
Laplace transform
joni joy
 
nagoor kani-763-790.pdf book formula for transform
nagoor kani-763-790.pdf book formula for transformnagoor kani-763-790.pdf book formula for transform
nagoor kani-763-790.pdf book formula for transform
NIETMsSaranyaRAsstPr
 
transformada de lapalace universidaqd ppt para find eaño
transformada de lapalace universidaqd ppt para find eañotransformada de lapalace universidaqd ppt para find eaño
transformada de lapalace universidaqd ppt para find eaño
luis506251
 

Similar to Taylor's Theorem for Matrix Functions and Pseudospectral Bounds on the Condition Number (20)

Frechet Derivatives of Matrix Functions and Applications
Frechet Derivatives of Matrix Functions and ApplicationsFrechet Derivatives of Matrix Functions and Applications
Frechet Derivatives of Matrix Functions and Applications
 
Circuit Network Analysis - [Chapter5] Transfer function, frequency response, ...
Circuit Network Analysis - [Chapter5] Transfer function, frequency response, ...Circuit Network Analysis - [Chapter5] Transfer function, frequency response, ...
Circuit Network Analysis - [Chapter5] Transfer function, frequency response, ...
 
NAS-Ch4-Application of Laplace Transform
NAS-Ch4-Application of Laplace TransformNAS-Ch4-Application of Laplace Transform
NAS-Ch4-Application of Laplace Transform
 
time response
time responsetime response
time response
 
EC8352-Signals and Systems - Laplace transform
EC8352-Signals and Systems - Laplace transformEC8352-Signals and Systems - Laplace transform
EC8352-Signals and Systems - Laplace transform
 
Quasi-Stochastic Approximation: Algorithm Design Principles with Applications...
Quasi-Stochastic Approximation: Algorithm Design Principles with Applications...Quasi-Stochastic Approximation: Algorithm Design Principles with Applications...
Quasi-Stochastic Approximation: Algorithm Design Principles with Applications...
 
Maths ppt partial diffrentian eqn
Maths ppt partial diffrentian eqnMaths ppt partial diffrentian eqn
Maths ppt partial diffrentian eqn
 
A Fibonacci-like universe expansion on time-scale
A Fibonacci-like universe expansion on time-scaleA Fibonacci-like universe expansion on time-scale
A Fibonacci-like universe expansion on time-scale
 
publish paper
publish paperpublish paper
publish paper
 
Laplace transform
Laplace transformLaplace transform
Laplace transform
 
Top ranking colleges in india
Top ranking colleges in indiaTop ranking colleges in india
Top ranking colleges in india
 
MM framework for RL
MM framework for RLMM framework for RL
MM framework for RL
 
Clase 02-modelado-de-sistemas-de-control (1)
Clase 02-modelado-de-sistemas-de-control (1)Clase 02-modelado-de-sistemas-de-control (1)
Clase 02-modelado-de-sistemas-de-control (1)
 
nagoor kani-763-790.pdf book formula for transform
nagoor kani-763-790.pdf book formula for transformnagoor kani-763-790.pdf book formula for transform
nagoor kani-763-790.pdf book formula for transform
 
Cost indexes
Cost indexesCost indexes
Cost indexes
 
DeepLearn2022 2. Variance Matters
DeepLearn2022  2. Variance MattersDeepLearn2022  2. Variance Matters
DeepLearn2022 2. Variance Matters
 
Laplace.pdf
Laplace.pdfLaplace.pdf
Laplace.pdf
 
from_data_to_differential_equations.ppt
from_data_to_differential_equations.pptfrom_data_to_differential_equations.ppt
from_data_to_differential_equations.ppt
 
transformada de lapalace universidaqd ppt para find eaño
transformada de lapalace universidaqd ppt para find eañotransformada de lapalace universidaqd ppt para find eaño
transformada de lapalace universidaqd ppt para find eaño
 
lcs_manual_1[1].pdf
lcs_manual_1[1].pdflcs_manual_1[1].pdf
lcs_manual_1[1].pdf
 

Recently uploaded

Human genetics..........................pptx
Human genetics..........................pptxHuman genetics..........................pptx
Human genetics..........................pptx
Silpa
 
biology HL practice questions IB BIOLOGY
biology HL practice questions IB BIOLOGYbiology HL practice questions IB BIOLOGY
biology HL practice questions IB BIOLOGY
1301aanya
 
Digital Dentistry.Digital Dentistryvv.pptx
Digital Dentistry.Digital Dentistryvv.pptxDigital Dentistry.Digital Dentistryvv.pptx
Digital Dentistry.Digital Dentistryvv.pptx
MohamedFarag457087
 
Conjugation, transduction and transformation
Conjugation, transduction and transformationConjugation, transduction and transformation
Conjugation, transduction and transformation
Areesha Ahmad
 
Porella : features, morphology, anatomy, reproduction etc.
Porella : features, morphology, anatomy, reproduction etc.Porella : features, morphology, anatomy, reproduction etc.
Porella : features, morphology, anatomy, reproduction etc.
Silpa
 

Recently uploaded (20)

Velocity and Acceleration PowerPoint.ppt
Velocity and Acceleration PowerPoint.pptVelocity and Acceleration PowerPoint.ppt
Velocity and Acceleration PowerPoint.ppt
 
Human genetics..........................pptx
Human genetics..........................pptxHuman genetics..........................pptx
Human genetics..........................pptx
 
GBSN - Microbiology (Unit 2)
GBSN - Microbiology (Unit 2)GBSN - Microbiology (Unit 2)
GBSN - Microbiology (Unit 2)
 
Site Acceptance Test .
Site Acceptance Test                    .Site Acceptance Test                    .
Site Acceptance Test .
 
Proteomics: types, protein profiling steps etc.
Proteomics: types, protein profiling steps etc.Proteomics: types, protein profiling steps etc.
Proteomics: types, protein profiling steps etc.
 
Thyroid Physiology_Dr.E. Muralinath_ Associate Professor
Thyroid Physiology_Dr.E. Muralinath_ Associate ProfessorThyroid Physiology_Dr.E. Muralinath_ Associate Professor
Thyroid Physiology_Dr.E. Muralinath_ Associate Professor
 
Dr. E. Muralinath_ Blood indices_clinical aspects
Dr. E. Muralinath_ Blood indices_clinical  aspectsDr. E. Muralinath_ Blood indices_clinical  aspects
Dr. E. Muralinath_ Blood indices_clinical aspects
 
GBSN - Biochemistry (Unit 1)
GBSN - Biochemistry (Unit 1)GBSN - Biochemistry (Unit 1)
GBSN - Biochemistry (Unit 1)
 
biology HL practice questions IB BIOLOGY
biology HL practice questions IB BIOLOGYbiology HL practice questions IB BIOLOGY
biology HL practice questions IB BIOLOGY
 
GBSN - Microbiology (Unit 3)
GBSN - Microbiology (Unit 3)GBSN - Microbiology (Unit 3)
GBSN - Microbiology (Unit 3)
 
FAIRSpectra - Enabling the FAIRification of Analytical Science
FAIRSpectra - Enabling the FAIRification of Analytical ScienceFAIRSpectra - Enabling the FAIRification of Analytical Science
FAIRSpectra - Enabling the FAIRification of Analytical Science
 
Zoology 5th semester notes( Sumit_yadav).pdf
Zoology 5th semester notes( Sumit_yadav).pdfZoology 5th semester notes( Sumit_yadav).pdf
Zoology 5th semester notes( Sumit_yadav).pdf
 
CURRENT SCENARIO OF POULTRY PRODUCTION IN INDIA
CURRENT SCENARIO OF POULTRY PRODUCTION IN INDIACURRENT SCENARIO OF POULTRY PRODUCTION IN INDIA
CURRENT SCENARIO OF POULTRY PRODUCTION IN INDIA
 
Selaginella: features, morphology ,anatomy and reproduction.
Selaginella: features, morphology ,anatomy and reproduction.Selaginella: features, morphology ,anatomy and reproduction.
Selaginella: features, morphology ,anatomy and reproduction.
 
Digital Dentistry.Digital Dentistryvv.pptx
Digital Dentistry.Digital Dentistryvv.pptxDigital Dentistry.Digital Dentistryvv.pptx
Digital Dentistry.Digital Dentistryvv.pptx
 
Climate Change Impacts on Terrestrial and Aquatic Ecosystems.pptx
Climate Change Impacts on Terrestrial and Aquatic Ecosystems.pptxClimate Change Impacts on Terrestrial and Aquatic Ecosystems.pptx
Climate Change Impacts on Terrestrial and Aquatic Ecosystems.pptx
 
An introduction on sequence tagged site mapping
An introduction on sequence tagged site mappingAn introduction on sequence tagged site mapping
An introduction on sequence tagged site mapping
 
Conjugation, transduction and transformation
Conjugation, transduction and transformationConjugation, transduction and transformation
Conjugation, transduction and transformation
 
Porella : features, morphology, anatomy, reproduction etc.
Porella : features, morphology, anatomy, reproduction etc.Porella : features, morphology, anatomy, reproduction etc.
Porella : features, morphology, anatomy, reproduction etc.
 
Locating and isolating a gene, FISH, GISH, Chromosome walking and jumping, te...
Locating and isolating a gene, FISH, GISH, Chromosome walking and jumping, te...Locating and isolating a gene, FISH, GISH, Chromosome walking and jumping, te...
Locating and isolating a gene, FISH, GISH, Chromosome walking and jumping, te...
 

Taylor's Theorem for Matrix Functions and Pseudospectral Bounds on the Condition Number

  • 1. Taylor’s Theorem for Matrix Functions and Pseudospectral Bounds on the Condition Number Samuel Relton samuel.relton@maths.man.ac.uk @sdrelton samrelton.com blog.samrelton.com Joint work with Edvin Deadman edvin.deadman@nag.co.uk University of Strathclyde June 23rd, 2015 Sam Relton (UoM) Taylor’s Theorem for f (A) June 23rd, 2015 1 / 21
  • 2. Outline • Taylor’s Theorem for Scalar Functions • Matrix Functions, their Derivatives, and the Condition Number • Taylor’s Theorem for Matrix Functions • Pseudospectral Bounds on the Condition Number • Numerical Experiments Sam Relton (UoM) Taylor’s Theorem for f (A) June 23rd, 2015 2 / 21
  • 3. Taylor’s Theorem - 1 Theorem (Taylor’s Theorem) When f : R → R is k times continuously differentiable at a ∈ R there exists Rk : R → R such that f (x) = k j=0 f (j)(a) j! (x − a)j + Rk(x). Different expressions for the remainder term Rk(x) include • the Lagrange form. • the Cauchy form. • the contour integral form. Sam Relton (UoM) Taylor’s Theorem for f (A) June 23rd, 2015 3 / 21
  • 4. Taylor’s Theorem - 2 We can extend this to complex analytic functions. If f (z) is complex analytic in an open set D ⊂ C then for any a ∈ D f (z) = k j=0 f (k)(a) j! (z − a)j + Rk(z), where Rk(z) = (z − a)k+1 2πi Γ f (w)dw (w − a)k+1(w − z) , and Γ is a closed curve in D containing a. Sam Relton (UoM) Taylor’s Theorem for f (A) June 23rd, 2015 4 / 21
  • 5. Matrix Functions We are interested in extending this to matrix functions f : Cn×n → Cn×n. For example: • the matrix exponential eA = ∞ j=0 Aj j! . • the matrix cosine cos(A) = ∞ j=0 (−1)j A2j (2j)! . Applications include: • Differential equations: du dt = Au(t), u(t) = etAu(0). • Second order ODEs with sine and cosine. • Ranking importance of nodes in a graph etc. . . Sam Relton (UoM) Taylor’s Theorem for f (A) June 23rd, 2015 5 / 21
  • 6. Fr´echet derivatives Let f : Cn×n → Cn×n be a matrix function. Definition (Fr´echet derivative) The Fr´echet derivative of f at A is the unique linear function Lf (A, ·) : Cn×n → Cn×n such that for all E f (A + E) − f (A) − Lf (A, E) = o( E ). • Lf (A, E) is just a linear approximation to f (A + E) − f (A). • Higher order derivatives are defined recursively (Higham & R., 2014). Sam Relton (UoM) Taylor’s Theorem for f (A) June 23rd, 2015 6 / 21
  • 7. Condition Numbers A condition number describes the sensitivity of f at A to small perturbations arising from rounding error etc. The absolute condition number is given by condabs(f , A) := lim →0 sup E ≤ f (A + E) − f (A) = max E =1 Lf (A, E) , whilst the relative condition number is condrel(f , A) := condabs(f , A) A f (A) . Sam Relton (UoM) Taylor’s Theorem for f (A) June 23rd, 2015 7 / 21
  • 8. Matrix Functions and Taylor’s Theorem - 1 Previous results combining these two ideas include: • an expansion around αI f (A) = ∞ j=0 f (j)(α) j! (A − αI)j . • an expansion in terms of derivatives f (A + E) = ∞ j=0 1 j! dj dtj t=0 f (A + tE). Note that: • neither expansion has an explicit remainder term. • dj dtj t=0 f (A + tE) = Lf (A, E) when j = 1. Sam Relton (UoM) Taylor’s Theorem for f (A) June 23rd, 2015 8 / 21
  • 9. Matrix Functions and Taylor’s Theorem - 2 Let us take D [j] f (A, E) := dj dtj t=0 f (A + tE) then we have the following. Theorem (Taylor’s Theorem for Matrix Functions) Let f : Cn×n → Cn×n we analytic in an open set D ⊂ C with A, E satisfying Λ(A), Λ(A + E) ⊂ D. Then f (A + E) = Tk(A, E) + Rk(A, E), where Tk(A, E) = k j=0 1 j! D[j] (A, E), and Rk(A, E) = 1 2πi Γ f (z)(zI − A − E)−1 [E(zI − A)−1 ]k+1 dz, where Γ is a closed contour enclosing Λ(A) and Λ(A + E). Sam Relton (UoM) Taylor’s Theorem for f (A) June 23rd, 2015 9 / 21
  • 10. Matrix Functions and Taylor’s Theorem - 3 As an example take f (z) = z−1. D [1] z−1 (A, E) = −A−1 EA−1 , D [2] z−1 (A, E) = 2A−1 EA−1 EA−1 . Therefore we have (A + E)−1 = 1 0! A−1 − 1 1! A−1 EA−1 + 2 2! A−1 EA−1 EA−1 + Γ 1 z (zI − A − E)−1 [E(zI − A)−1 ]3 dz. Sam Relton (UoM) Taylor’s Theorem for f (A) June 23rd, 2015 10 / 21
  • 11. Applying Pseudospectral Theory - 1 Recall that the -pseudospectrum of X is the set Λ (X) = {z ∈ C : (zI − X)−1 ≥ −1 }. The -psuedospectral radius is ρ = max |z| for z ∈ Λ (X). -1 0 1 2 3 -3 -2 -1 0 1 2 3 -2.5 -2 -1.5 -1 Sam Relton (UoM) Taylor’s Theorem for f (A) June 23rd, 2015 11 / 21
  • 12. Applying Pseudospectral Theory - 1 Recall that the -pseudospectrum of X is the set Λ (X) = {z ∈ C : (zI − X)−1 ≥ −1 }. The -psuedospectral radius is ρ = max |z| for z ∈ Λ (X). Using this we can bound the remainder term by Rk(A, E) ≤ E k+1˜L 2π k+1 max z∈ ˜Γ |f (z)|, where • ˜Γ is a contour enclosing Λ (A) and Λ (A + E). • ˜L is the length of the contour ˜Γ . • is a parameter to be chosen. Sam Relton (UoM) Taylor’s Theorem for f (A) June 23rd, 2015 11 / 21
  • 13. Applying Pseudospectral Theory - 2 Applying this to R0(A, E) gives a bound on the condition number. condabs(f , A) ≤ L 2π 2 max z∈Γ |f (z)|, where Γ encloses Λ (A) and has length L . Interesting because: • Usually only lower bounds on condition number are known. • Computing (or estimating) this efficiently could be of considerable interest in practice or for algorithm design. Sam Relton (UoM) Taylor’s Theorem for f (A) June 23rd, 2015 12 / 21
  • 14. The Condition Number of At - 1 This upper bound is extremely efficient to compute for the matrix function given by f (x) = xt for t ∈ (0, 1). Our experiments will • determine how tight the upper bound is as changes. • see how fast evaluating the upper bound is in comparison to computing it exactly. Other methods for this problem are: • “CN Exact” – computes condition number exactly. • “CN Normest” – lower bound using norm estimator. Sam Relton (UoM) Taylor’s Theorem for f (A) June 23rd, 2015 13 / 21
  • 15. The Condition Number of At - 2 This function has a branch cut along the negative real line, meaning we need to choose a keyhole contour. Overall: condabs(xt , A) ≤ 2(π + 1)ρ1+t√ n 2π 2 , where ρ is the -pseudospectral radius, computed using code by Gugliemi and Overton. Note: There is an upper limit for where the pseudospectrum intersects the branch cut. We need to take smaller than this value. Sam Relton (UoM) Taylor’s Theorem for f (A) June 23rd, 2015 14 / 21
  • 16. Test matrix - Grcar matrix -1 0 1 2 3 -3 -2 -1 0 1 2 3 -2.5 -2 -1.5 -1 Sam Relton (UoM) Taylor’s Theorem for f (A) June 23rd, 2015 15 / 21
  • 17. CN Bound as varies 0 10 -8 10 -7 10 -6 10 -5 10 -4 10 -3 10 -2 10 -1 10 0 10 1 10 -15 10 -10 10 -5 10 0 CN Pseudo CN Exact CN Normest Sam Relton (UoM) Taylor’s Theorem for f (A) June 23rd, 2015 16 / 21
  • 18. Test matrix - Almost neg. eigenvalues -1 -0.5 0 0.5 -0.3 -0.2 -0.1 0 0.1 0.2 -5 -4.5 -4 -3.5 Sam Relton (UoM) Taylor’s Theorem for f (A) June 23rd, 2015 17 / 21
  • 19. CN Bound varies 0 10 -8 10 -7 10 -6 10 -5 10 -4 10 -3 10 -2 10 -1 10 0 10 1 10 -15 10 -10 10 -5 10 0 CN Pseudo CN Exact CN Normest Sam Relton (UoM) Taylor’s Theorem for f (A) June 23rd, 2015 18 / 21
  • 20. Runtime Comparison - Timings n 0 50 100 150 200 runtime(s) 10 -4 10 -2 10 0 10 2 10 4 CN Normest t=1/5 CN Pseudo t=1/5 CN Normest t=1/10 CN Pseudo t=1/10 CN Normest t=1/15 CN Pseudo t=1/15 Sam Relton (UoM) Taylor’s Theorem for f (A) June 23rd, 2015 19 / 21
  • 21. Runtime Comparison - Speedup n 0 50 100 150 200 speedup 0 200 400 600 800 1000 t=1/5 t=1/10 t=1/15 Sam Relton (UoM) Taylor’s Theorem for f (A) June 23rd, 2015 20 / 21
  • 22. Conclusions • Extended Taylor’s theorem to matrix functions. • Applied pseudospectral theory to bound remainder term. • Bounds are very efficient to compute for At. • If bound is unsatisfactorily large can revert to a more precise method. Future work: • Apply to algorithm design. • Find other classes of functions for which this is efficient. Sam Relton (UoM) Taylor’s Theorem for f (A) June 23rd, 2015 21 / 21