slides for the talk in the SLC 2015 improved version

1. Numerical Methods for SPDEs driven by L´evy Jump
Processes: Probabilistic and Deterministic Approaches
Mengdi Zheng, George Em
Karniadakis (Brown University)
2015 SIAM Conference on
Computational Science and Engineering
March 17, 2015

2. Contents
Motivation
Introduction
L´evy process
Dependence structure of multi-dim pure jump process
Generalized Fokker-Planck (FP) equation
Overdamped Langevin equation driven by 1D TαS process
by MC and PCM (probabilistic methods)
by FP equation (deterministic method, tempered fractional PDE)
Diffusion equation driven by multi-dimensional jump processes
SPDE w/ 2D jump process in LePage’s rep
SPDE w/ 2D jump process by L´evy copula
SPDE w/ 10D jump process in LePage’s rep (ANOVA decomposition)
Future work
2 of 25

3. Section 1: motivation
D. Xiu, J.S. Hesthaven, High order collocation methods for differential
equations with random inputs, SIAM J. Sci. Comput., 27(3) (2005),
pp. 1118–1139.
R. Cont, P. Tankov, Financial Modelling with Jump Processes,
Chapman & Hall/CRC Press, 2004.
3 of 25

4. Section 2.1: introduction of L´evy processes
Definition of a L´evy process Xt (a continuous random walk):
Independent increments: for t0 < t1 < ... < tn, random variables
(RVs) Xt0
, Xt1
− Xt0
,..., Xtn−1
− Xtn−1
are independent;
Stationary increments: the distribution of Xt+h − Xt does not depend
on t;
RCLL: right continuous with left limits;
Stochastic continuity: ∀ > 0, limh→0 P(|Xt+h − Xt| ≥ ) = 0;
X0 = 0 P-a.s..
Decomposition of a L´evy process Xt = Gt + Jt + vt: a Gaussian
process (Gt), a pure jump process (Jt), and a drift (vt).
Definition of the jump: Jt = Jt − Jt− .
Definition of the Poisson random measure (an RV):
N(t, U) = 0≤s≤t I Js ∈U, U ∈ B(Rd
0 ), ¯U ⊂ Rd
0 .1
1
S. Ken-iti, L´evy Processes and Infinitely Divisible Distributions, Cambridge
University Press, Cambridge, 1999.4 of 25

5. Section 2.2: Pure jump process Jt
L´evy measure ν: ν(U) = E[N(1, U)], U ∈ B(Rd
0 ), ¯U ⊂ Rd
0 .
3 ways to describe dependence structure between components of
a multi-dimensional L´evy process:
5 of 25

6. Section 2.2: LePage’s multi-d jump processes (1)
Example 1: d-dim tempered α-stable processes (TαS) in
spherical coordinates (”size” and ”direction” of jumps):
L´evy measure (dependence structure):
νrθ(dr, dθ) = σ(dr, θ)p(dθ) = ce−λr
dr
r1+α p(dθ) = ce−λr
dr
r1+α
2πd/2
dθ
Γ(d/2) ,
r ∈ [0, +∞], θ ∈ Sd
.
Series representation by Rosinksi (simulation)2
:
L(t) =
+∞
j=1 j [(
αΓj
2cT )−1/α
∧ ηj ξ
1/α
j ] (θj1, θj2, ..., θjd )I{Uj ≤t},
for t ∈ [0, T].
P( j = 0, 1) = 1/2, ηj ∼ Exp(λ), Uj ∼ U(0, T), ξj ∼U(0, 1).
{Γj } are the arrival times in a Poisson process with unit rate.
(θj1, θj2, ..., θjd ) is uniformly distributed on the sphereSd−1
.
2
J. Ros´ınski, Series representations of infinitely divisible random vectors and a
generalized shot noise in Banach spaces, Technical Report No. 195, (1987).
J. Ros´ınski, On series representations of infinitely divisible random vectors, Ann.
Probab., 18 (1990), pp. 405–430.6 of 25

7. Section 2.2: dependence structure by L´evy copula (2)
Example 2: 2-dim jump process (L1, L2) w/ TαS components3
(L++
1 , L++
2 ), (L+−
1 , L+−
2 ), (L−+
1 , L−+
2 ), and (L−−
1 , L−−
2 )
Figure : Construction of L´evy measure for (L++
1 , L++
2 ) as an example
3
J. Kallsen, P. Tankov, Characterization of dependence of
multidimensional L´evy processes using L´evy copulas, Journal of Multivariate
Analysis, 97 (2006), pp. 1551–1572.7 of 25

8. Section 2.2: dependence structure by (L´evy copula)
Example 2 (continued):
Simulation of (L1, L2) ((L++
1 , L++
2 ) as an example) by series
representation 4
L++
1 (t) =
+∞
j=1 1j (
αΓj
2(c/2)T )−1/α
∧ ηj ξ
1/α
j I[0,t](Vj ),
L++
2 (t) =
+∞
j=1 2j U
++(−1)
2 F−1
(Wi U++
1 (
αΓj
2(c/2)T )−1/α
∧ ηj ξ
1/α
j ) I[0,t](Vj )
F−1
(v2|v1) = v1 v
− τ
1+τ
2 − 1
−1/τ
.
{Vi } ∼Uniform(0, 1) and {Wi } ∼Uniform(0, 1). {Γi } is the i-th
arrival time for a Poisson process with unit rate. {Vi }, {Wi } and {Γi }
are independent.
Concept: we ’can’ represent L´evy processes correlated by L´evy
copula by RVs
4
R. Cont, P. Tankov, Financial Modelling with Jump Processes, Chapman
& Hall/CRC Press, 2004.8 of 25

9. Section 2.3: generalized Fokker-Planck (FP) equations
For an SODE system du = C(u, t) + dL(t), where C(u, t) is a
linear operator on u.
Let us assume that the L´evy measure of the pure jump process
L(t) ∈ Rd has the symmetry ν(x) = ν(−x).
The generalized FP equation for the joint PDF satisfies5:
∂P(u, t)
∂t
= − ·(C(u, t)P(u, t))+
Rd −{0}
ν(dz) P(u+z, t)−P(u, t) .
(1)
Available in literature: if C(u, t) is non-linear, if the noise is
multiplicative in Ito’s or Marcus’s integral form, the FP eqn is
derived.
5
X. Sun, J. Duan, Fokker-Planck equations for nonlinear dynamical systems
driven by non-Gaussian L´evy processes. J. Math. Phys., 53 (2012), 072701.9 of 25

10. Section 3: 1D SODE driven by 1D TαS process
We solve6:
dx(t; ω) = −σx(t; ω)dt + dLt(ω), x(0) = x0.
L´evy measure of Lt is: ν(x) = ce−λ|x|
|x|α+1 , for x ∈ R, 0 < α < 2
FP equation as a tempered fractional PDE (TFPDE)
When 0 < α < 1, D(α) = c
α Γ(1 − α)
∂
∂t P(x, t) = ∂
∂x σxP(x, t) −D(α) −∞Dα,λ
x P(x, t)+x Dα,λ
+∞P(x, t)
−∞Dα,λ
x and x Dα,λ
+∞ are left and right Riemann-Liouville tempered
fractional derivatives7
.
We solve this by 3 methods: MC, PCM, TFPDE.
6
M. Zheng, G.E. Karniadakis, Numerical methods for SPDEs with
tempered stable processes, SIAM J. Sci. Comput., accepted in 2015.
7
M.M. Meerschaert, A. Sikorskii, Stochastic Models for Fractional
Calculus, De Gruyter Studies in Mathematics Vol. 43, 2012.10 of 25

11. Section 3: PCM vs. TFPDE in E[x2
(t; ω)]
0 0.2 0.4 0.6 0.8 1
10
4
10
3
10
2
10
1
10
0
t
err
2nd
fractional density equation
PCM/CP
0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45
10
3
10
2
10
1
10
0
t
err
2nd
fractional density equation
PCM/CP
Figure : err2nd versus time by: 1) TFPDEs; 2) PCM. α = 0.5, c = 2,
λ = 10, σ = 0.1, x0 = 1 (left); α = 1.5, c = 0.01, λ = 0.01, σ = 0.1, x0 = 1
(right). However, TFPDE costs less CPU time than PCM.
11 of 25

12. Section 3: MC vs. TFPDE in density
4 2 0 2 4 6
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
x(T = 0.5)
densityP(x,t)
histogram by MC/CP
density by fractional PDEs
4 2 0 2 4 6
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
x(T=1)
densityP(x,t)
histogram by MC/CP
density by fractional PDEs
Figure : Zoomed in plots of P(x, T) by TFPDEs and MC at T = 0.5 (left)
and T = 1 (right): α = 0.5, c = 1, λ = 1, x0 = 1 and σ = 0.01 (left and
right). The agreement can be quantified by the Kolmogorov–Smirnov test.
12 of 25

13. Section 4: SPDE w/ multi-dim jump process
We solve :


du(t, x; ω) = µ∂2u
∂x2 dt + d
i=1 fi (x)dLi (t; ω), x ∈ [0, 1]
u(t, 0) = u(t, 1) = 0
u(0, x) = u0(x) = d
i=1 fi (x)
L(t; ω): {Li (t; ω), i = 1, ..., d} are mutually dependent.
fk(x) =
√
2sin(πkx), k = 1, 2, 3, ... are orthonormal on [0, 1].
By u(x, t; ω) = +∞
i=1 ui (t; ω)fi (x) and Galerkin projection onto
{fi (x)}, we obtain an SODE system, where Dmm = −(πm)2:


du1(t) = µD11u1(t)dt + dL1, u1(0) = 1
du2(t) = µD22u2(t)dt + dL2, u2(0) = 1
...
dud (t) = µDdd ud (t)dt + dLd , ud (0) = 1
13 of 25

14. Section 4.1: SPDEs driven by multi-d jump processes
Figure : probabilistic and deterministic methods: M. Zheng, G.E.
Karniadakis, Numerical methods for SPDEs with additive
multi-dimensional L´evy jump processes, in preparation.
14 of 25

15. Section 4.2: FP eqn when Lt (2D) is in LePage’s rep
When the L´evy measure of Lt is given by (d = 2)
νrθ(dr, dθ) = ce−λr dr
r1+α
2πd/2dθ
Γ(d/2) , for r ∈ [0, +∞], θ ∈ Sd
The generalized FP equation for the joint PDF P(u, t) of solutions
in the SODE system is:
∂P(u,t)
∂t = − d
i=1 µDii (P + ui
∂P
∂ui
)
− c
α Γ(1 − α) Sd−1
Γ(d/2)dσ(θ)
2πd/2 r Dα,λ
+∞P(u + rθ, t) , where θ is a
unit vector on the unit sphere Sd−1.
x Dα,λ
+∞ is the right Riemann-Liouville TF derivative.
Solved by a multi-grid solver; I.C. introduces error
Later, for d = 10, we will use ANOVA decomposition to obtain
equations for marginal distributions from this FP equation.
15 of 25

16. Section 4.2: simulation if Lt (2D) is in LePage’s rep
Figure : TFPDE (3D contour) vs. MC (2D contour): P(u1, u2, t) of SODE
system, slices at the peak. t = 1 , c = 1, α = 0.5, λ = 5, µ = 0.01,
NSR = 16.0% at t = 1. NSR = Var[u] L∞([0,1])/ E[u] L∞([0,1]).16 of 25

17. Section 4.2: simulation if Lt (2D) is in LePage’s rep
0.2 0.4 0.6 0.8 1
10
−10
10
−8
10
−6
10
−4
10
−2
l2u2(t)
t
PCM/S Q=5, q=2
PCM/S Q=10, q=2
TFPDE
NSR 4.8%
0.2 0.4 0.6 0.8 1
10
−7
10
−6
10
−5
10
−4
10
−3
10
−2
l2u2(t)
t
PCM/S Q=10, q=2
PCM/S Q=20, q=2
TFPDE
NSR 6.4%
Figure : TFPDE vs. PCM: L2 error norm E[u] by PCM and TFPDE.
α = 0.5, λ = 5, µ = 0.001 (left and right). c = 0.1 (left); c = 1 (right).
(Talk about the restriction of 2 methods here.)
17 of 25

18. Section 4.3: FP eqn if Lt (2D) is from L´evy copula
The dependence structure btw components of Lt is given by L´evy
copula on each corners (++, +−, −+, −−)
dependence structure is described by the Clayton family of copulas
with correlation length τ on each corner
The generalized FP eqn is :
∂P(u,t)
∂t = − · (C(u, t)P(u, t))
+
+∞
0 dz1
+∞
0 dz2ν++(z1, z2)[P(u + z, t) − P(u, t)]
+
+∞
0 dz1
0
−∞ dz2ν+−(z1, z2)[P(u + z, t) − P(u, t)]
+
0
−∞ dz1
+∞
0 dz2ν−+(z1, z2)[P(u + z, t) − P(u, t)]
+
0
−∞ dz1
0
−∞ dz2ν−−(z1, z2)[P(u + z, t) − P(u, t)]
We solve this by a multi-grid solver.
M. Zheng, G.E. Karniadakis, Numerical methods for SPDEs
with additive multi-dimensional L´evy jump processes, in
preparation.
18 of 25

19. Section 4.3: FP eqn if Lt (2D) is from L´evy copula
Figure : FP (3D contour) vs. MC (2D contour): P(u1, u2, t) of SODE
system. t = 1 , c = 1, α = 0.5, λ = 5, µ = 0.005, τ = 1, NSR = 30.1%.
19 of 25

20. Section 4.3: if Lt (2D) is from L´evy copula
0.2 0.4 0.6 0.8 1
10
−5
10
−4
10
−3
10
−2
t
l2u2(t)
TFPDE
PCM/S Q=1, q=2
PCM/2 Q=2, q=2
NSR 6.4%
0.2 0.4 0.6 0.8 1
10
−3
10
−2
10
−1
10
0
t
l2u2(t)
TFPDE
PCM/S Q=2, q=2
PCM/S Q=1, q=2
NSR 30.1%
Figure : FP vs. PCM: L2 error of E[u] in heat equation α = 0.5, λ = 5,
τ = 1 (left and right). c = 0.05, µ = 0.001 (left). c = 1, µ = 0.005 (right).
20 of 25

21. Section 4.3: FP eqn if Lt is in LePage’s rep by ANOVA
The unanchored analysis of variance (ANOVA) decomposition is 8:
P(u, t) ≈ P0(t) + 1≤j1≤d Pj1 (uj1 , t) + 1≤j1<j2≤d Pj1,j2 (uj1 , uj2 , t)
+... + 1≤j1<j2...<jκ≤d Pj1,j2,...,jκ (uj1 , uj2 , ..., uκ, t)
κ is the effective dimension
ANOVA modes of P(u, t) are related to marginal distributions
P0(t) = Rd P(u, t)du
Pi (ui , t) = Rd−1 du1...dui−1dui+1...dud P(u, t) − P0(t) =
pi (ui , t) − P0(t)
Pij (xi , xj , t) = Rd−1 du1...dui−1dui+1...duj−1duj+1...dud P(u, t)
−Pi (ui , t) − Pj (uj , t) − P0(t) =
pij (x1, x2, t) − pi (x1, t) − pj (x2, t) + P0(t)
8
M. Bieri, C. Schwab, Sparse high order FEM for elliptic sPDEs, Tech.
Report 22, ETH, Switzerland, (2008).
X. Yang, M. Choi, G. Lin, G.E. Karniadakis,Adaptive ANOVA
decomposition of stochastic incompressible and compressible flows, Journal of
Computational Physics, 231 (2012), pp. 1587–1614.21 of 25

22. Section 4.3: FP eqn if Lt is in LePage’s rep by ANOVA
When the L´evy measure of Lt is given by
νrθ(dr, dθ) = ce−λr dr
r1+α
2πd/2dθ
Γ(d/2) , for r ∈ [0, +∞], θ ∈ Sd (for
0 < α < 1)
∂pi (ui ,t)
∂t = − d
k=1 µDkk pi (xi , t) − µDii xi
∂pi (xi ,t)
∂xi
−cΓ(1−α)
α
Γ( d
2
)
2π
d
2
2π
d−1
2
Γ( d−1
2
)
π
0 dφsin(d−2)(φ) r Dα,λ
+∞pi (ui +rcos(φ), t)
∂pij (ui ,uj ,t)
∂t =
− d
k=1 µDkk pij −µDii ui
∂pij
∂ui
−µDjj uj
∂pij
∂uj
−cΓ(1−α)
α
Γ( d
2
)
2π
d
2
2π
d−2
2
Γ(d−2
2
)
π
0 dφ1
π
0 dφ2sin8(φ1)sin7(φ2) r Dα,λ
+∞pij (ui + rcosφ1, uj +
rsinφ1cosφ2, t)
22 of 25

23. Section 4.3: 1D-ANOVA-FP is enough for E[u] in 10D
0 0.2 0.4 0.6 0.8 1
−2
0
2
4
6
8
10
12
x
E[u(x,T=1)]
E[uPCM
]
E[u1D−ANOVA−FP
]
E[u2D−ANOVA−FP
]
0.6 0.65 0.7 0.75 0.8 0.85 0.9 0.95 1
3.4
3.6
3.8
4
4.2
4.4
4.6
4.8
5
5.2
x 10
−4
T
L2
normofdifferenceinE[u]
||E[u1D−ANOVA−FP
−E[uPCM
]||L
2
([0,1])
/||E[uPCM
]||L
2
([0,1])
||E[u2D−ANOVA−FP
−E[uPCM
]||L
2
([0,1])
/||E[uPCM
]||L
2
([0,1])
Figure : 1D-ANOVA-FP vs. 2D-ANOVA-FP vs. PCM in 10D: the mean
(left) for heat eqn at T = 1. The L2 norms of difference in E[u] (right).
c = 1, α = 0.5, λ = 10, µ = 10−4
.NSR ≈ 18.24% at T = 1.23 of 25

24. Section 4.3: 2D-ANOVA-FP is enough for E[u] in 10D
0 0.2 0.4 0.6 0.8 1
0
20
40
60
80
100
120
x
E[u2
(x,T=1)]
E[u
2
PCM
]
E[u2
1D−ANOVA−FP
]
E[u
2
2D−ANOVA−FP
]
0.6 0.65 0.7 0.75 0.8 0.85 0.9 0.95 1
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
T
L
2
normofdifferenceinE[u
2
]
||E[u2
1D−ANOVA−FP
−E[u2
PCM
]||
L
2
([0,1])
/||E[u2
PCM
]||
L
2
([0,1])
||E[u
2
2D−ANOVA−FP
−E[u
2
PCM
]||L
2
([0,1])
/||E[u
2
PCM
]||L
2
([0,1])
Figure : 1D-ANOVA-FP vs. 2D-ANOVA-FP vs. PCM in 10D: E[u2
] (left)
for heat eqn. The L2 norms of difference in E[u2
] (right). c = 1, α = 0.5,
λ = 10, µ = 10−4
. NSR ≈ 18.24% at T = 1.
24 of 25

25. Future work
multiplicative noise (now we have additive noise)
nonlinear SPDE (now we have linear SPDE)
higher dimensions (we computed up to < 20 dimensions)
This work is partially supported by OSD-MURI (grant
FA9550-09-1-0613), NSF/DMS (grant DMS-1216437) and the new
DOE Center on Mathematics for Mesoscopic Modeling of Materials
(CM4).
Thanks!
25 of 25