Lecture presented at the QMC Transition Workshop

1. How to efficiently implement the Multivariate Decomposition
Method for ∞-dimensional integration
Alec Gilbert
Joint work with: F. Y. Kuo, D. Nuyens and G. W. Wasilkowski
Wednesday May 9, 2018
QMC transition workshop, SAMSI, Research Triangle Park, NC, USA
1 / 23

2. WG VII: The Multivariate Decomposition Method
Submitted
A. D. Gilbert, F. Y. Kuo, D. Nuyens and G. W. Wasilkowski. Efficient implementations
of the Multivariate Decomposition Method for approximating infinite-variate integrals.
arXiv: 1712.06782.
Ongoing work
A. D. Gilbert, F. Y. Kuo, D. Nguyen and D. Nuyens. Application of the Multivariate
Decomposition Method to PDEs with random coefficients.
A. D. Gilbert, F. Y. Kuo and D. Nuyens. The Anchored Decomposition Method — A
new formulation of the MDM specific to the anchored decomposition.
2 / 23

3. Outline
1. The ∞-dimensional integration problem
2. The Multivariate Decomposition Method (MDM)
3. How to efficiently implement the MDM
3.1 Constructing the active set
3.2 Applying the quadrature rules
4. Numerical results
3 / 23

4. ∞-dimensional integration
f(y1, y2, . . .)
with
y = (y1, y2, . . .) ∈ [−1
2, 1
2] × [−1
2, 1
2] × · · ·
:=[− 1
2
, 1
2
]N
.
The ∞-dimensional integral is defined by
I(f) := lim
s→∞ [− 1
2
, 1
2
]s
f(y1, . . . , ys, 0, . . .) dy1 . . . dys .
4 / 23

5. ∞-dimensional integration
f(y1, y2, . . .)
with
y = (y1, y2, . . .) ∈ [−1
2, 1
2] × [−1
2, 1
2] × · · ·
:=[− 1
2
, 1
2
]N
.
The ∞-dimensional integral is defined by
I(f) := lim
s→∞ [− 1
2
, 1
2
]s
f(y1, . . . , ys, 0, . . .) dy1 . . . dys .
Example for this talk:
f(y) =
1
1 + ∞
j=1 yj/jβ
for β ≥ 2.
4 / 23

6. MDM general idea
How to numerically integrate f over [−1
2 , 1
2]N?
I(f) ≈
[− 1
2
, 1
2
]s
f(y1, . . . , ys, 0, . . .) dy1 . . . dys
5 / 23

7. MDM general idea
How to numerically integrate f over [−1
2 , 1
2]N?
I(f) ≈
✭✭✭✭✭✭✭✭✭✭✭✭✭✭✭✭✭✭❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤[− 1
2
, 1
2
]s
f(y1, . . . , ys, 0, . . .) dy1 . . . dys
Alternative?
f(y1, y2, . . .) =
u⊂N,|u|<∞
fu((yj : j ∈ u)
yu
)
then
I(f) =
u⊂N,|u|<∞ [− 1
2
, 1
2
]|u|
fu(yu) dyu
5 / 23

8. MDM general idea
How to numerically integrate f over [−1
2 , 1
2]N?
I(f) ≈
✭✭✭✭✭✭✭✭✭✭✭✭✭✭✭✭✭✭❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤[− 1
2
, 1
2
]s
f(y1, . . . , ys, 0, . . .) dy1 . . . dys
Alternative?
f(y1, y2, . . .) =
u⊂N,|u|<∞
fu((yj : j ∈ u)
yu
)
then
I(f) ≈
u∈U [− 1
2
, 1
2
]|u|
fu(yu) dyu ≈
u∈U
Aufu
MDM
5 / 23

9. MDM general idea
How to numerically integrate f over [−1
2 , 1
2]N?
I(f) ≈
✭✭✭✭✭✭✭✭✭✭✭✭✭✭✭✭✭✭❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤[− 1
2
, 1
2
]s
f(y1, . . . , ys, 0, . . .) dy1 . . . dys
Alternative?
f(y1, y2, . . .) =
u⊂N,|u|<∞
fu((yj : j ∈ u)
yu
)
then
I(f) ≈
u∈U [− 1
2
, 1
2
]|u|
fu(yu) dyu ≈
u∈U
Aufu
MDM
Key ingredients:
- Active set: U is a finite set of subsets u ⊂ N
- Au are |u|-dimensional quadrature rules using nu points
5 / 23

10. Anchored decomposition
fu(yu) =
v⊆u
(−1)|u|−|v|
f(yv; 0)
the jth coordinate of (yv; 0) is =
yj if j ∈ v
0 if j /∈ v.
e.g.
f(y) =
1
1 + ∞
j=1 yj/jβ
the decomposition terms are
f∅ = 1
f{i}(yi) =
1
1 + yi/iβ
− 1
f{i,j}(yi, yj) =
1
1 + yi/iβ + yj/jβ
−
1
1 + yi/iβ
−
1
1 + yj/jβ
+ 1
. . . 6 / 23

11. Multivariate decomposition method
[− 1
2
, 1
2
]N
f(y) dy ≈ Aεf =
u∈Uε
Aufu
with inputs
1. Active set: Uε = u ⊂ N : γu > T(ε, {γu}u⊂N)
2. Quadrature rules Au with number of points nu = nu ε, u, {γu}u⊂N, Uε
Key points:
1. γu represents the importance of yu (or fu)
2. Inputs for the setup of the MDM are ε, {γu}u⊂N
3. Need to construct active set before calculating nu
4. Use anchored decomposition to evaluate fu
7 / 23

12. Implementation I: The active set
8 / 23

13. Constructing the active set
Uε = {u ⊂ N : γu > T(ε)}
General idea:
1. For our example γu are of Product and Order Dependent (POD) form:
γu = C|u|!
j∈u
1
jβ
2. Store each u, |u| = ℓ, as (u1, u2, . . . , uℓ) ∈ Nℓ with u1 < u2 < · · · < uℓ.
3. Systematically search through sets in order of increasing cardinality.
4. POD structure of γu adds a notion ordering and allows us to determine when to
stop.
9 / 23

14. Active set results
β = 4 β = 3 β = 2.5
ε 1e-1 1e-2 1e-3 1e-1 1e-2 1e-3 1e-1 1e-2
T 1.4e-4 2.8e-6 6.4e-8 4.0e-6 3.6e-8 3.8e-10 1.5e-8 4.9e-11
max{|u|} 3 4 5 5 6 7 8 10
max{uj} 10 28 72 86 418 1907 2528 24724
size 1 9 26 68 76 370 1686 2019 19750
2 12 48 159 195 1285 7327 10077 126882
3 5 28 132 202 1828 13117 21996 354377
4 0 4 36 80 1234 11907 26258 559155
5 0 0 1 10 361 5578 17874 536133
6 0 0 0 0 32 1145 6513 313623
7 0 0 0 0 0 69 1088 106877
8 0 0 0 0 0 0 47 18582
9 0 0 0 0 0 0 0 1210
10 0 0 0 0 0 0 0 8
Table: Results from the active set construction for various β and ε.
10 / 23

15. Implementation II: Applying the quadrature rules Au
11 / 23

16. A naive implementation
Aε(f) =
u∈Uε
Au(fu)
12 / 23

17. A naive implementation
Aε(f) =
u∈Uε v⊆u
(−1)|u|−|v|
Au(f(·v; 0))
12 / 23

18. A naive implementation
Aε(f) =
u∈Uε v⊆u
(−1)|u|−|v|
Au(f(·v; 0))
e.g.
Uε = {1}, {1, 2}, {1, 2, 3}
then
v = {1} occurs 3 times
v = {1, 2} occurs 2 times
v = {1, 3} occurs 1 time
v = {2, 3} occurs 1 time
v = {1, 2, 3} occurs 1 time
12 / 23

19. A naive implementation
Aε(f) =
u∈Uε v⊆u
(−1)|u|−|v|
Au(f(·v; 0))
Goal: Minimise the cost of evaluating Aε(f)
Strategy:
1. Use a quadrature rule that is extensible in points and dimension.
2. Rewrite Aε(f) as a single sum over sets v ⊆ u for u ∈ U:
Aε(f) =
v⊆u∈U
cvAv(f(·v; 0))
12 / 23

20. Quasi-Monte Carlo: base-2 embedded lattice rules
[− 1
2
, 1
2
]|u|
g(yu)dyu ≈ Qu,m(z)f =
1
2m
2m−1
k=0
g (tk) .
Quadrature points:
tk =
ikz
2m′ − 1
2
,
where
2m′−1
≤ k < 2m′
ik = 1, 3, 5 . . . , 2m′
− 1
m′
= 1, 2, . . . , m
z ∈ N|u| the generating vector, which is
constructed to work well for a range of m
1
2
1
2
Figure: Embedded lattice point set up to
n = 32 points (m = 5) for z = (1, 13).
13 / 23

21. Quasi-Monte Carlo: base-2 embedded lattice rules
[− 1
2
, 1
2
]|u|
g(yu)dyu ≈ Qu,m(z)f =
1
2m
2m−1
k=0
g (tk) .
Quadrature points:
tk =
ikz
2m′ − 1
2
,
where
2m′−1
≤ k < 2m′
ik = 1, 3, 5 . . . , 2m′
− 1
m′
= 1, 2, . . . , m
z ∈ N|u| the generating vector, which is
constructed to work well for a range of m
1
2
1
2
Figure: Embedded lattice point set up to
n = 32 points (m = 5) for z = (1, 13).
13 / 23

22. Quasi-Monte Carlo: base-2 embedded lattice rules
[− 1
2
, 1
2
]|u|
g(yu)dyu ≈ Qu,m(z)f =
1
2m
2m−1
k=0
g (tk) .
Quadrature points:
tk =
ikz
2m′ − 1
2
,
where
2m′−1
≤ k < 2m′
ik = 1, 3, 5 . . . , 2m′
− 1
m′
= 1, 2, . . . , m
z ∈ N|u| the generating vector, which is
constructed to work well for a range of m
1
2
1
2
Figure: Embedded lattice point set up to
n = 32 points (m = 5) for z = (1, 13).
13 / 23

23. Quasi-Monte Carlo: base-2 embedded lattice rules
[− 1
2
, 1
2
]|u|
g(yu)dyu ≈ Qu,m(z)f =
1
2m
2m−1
k=0
g (tk) .
Quadrature points:
tk =
ikz
2m′ − 1
2
,
where
2m′−1
≤ k < 2m′
ik = 1, 3, 5 . . . , 2m′
− 1
m′
= 1, 2, . . . , m
z ∈ N|u| the generating vector, which is
constructed to work well for a range of m
1
2
1
2
Figure: Embedded lattice point set up to
n = 32 points (m = 5) for z = (1, 13).
13 / 23

24. Quasi-Monte Carlo: base-2 embedded lattice rules
[− 1
2
, 1
2
]|u|
g(yu)dyu ≈ Qu,m(z)f =
1
2m
2m−1
k=0
g (tk) .
Quadrature points:
tk =
ikz
2m′ − 1
2
,
where
2m′−1
≤ k < 2m′
ik = 1, 3, 5 . . . , 2m′
− 1
m′
= 1, 2, . . . , m
z ∈ N|u| the generating vector, which is
constructed to work well for a range of m
1
2
1
2
Figure: Embedded lattice point set up to
n = 32 points (m = 5) for z = (1, 13).
13 / 23

25. Quasi-Monte Carlo: base-2 embedded lattice rules
[− 1
2
, 1
2
]|u|
g(yu)dyu ≈ Qu,m(z)f =
1
2m
2m−1
k=0
g (tk) .
Quadrature points:
tk =
ikz
2m′ − 1
2
,
where
2m′−1
≤ k < 2m′
ik = 1, 3, 5 . . . , 2m′
− 1
m′
= 1, 2, . . . , m
z ∈ N|u| the generating vector, which is
constructed to work well for a range of m
1
2
1
2
Figure: Embedded lattice point set up to
n = 32 points (m = 5) for z = (1, 13).
13 / 23

26. Quasi-Monte Carlo: base-2 embedded lattice rules
[− 1
2
, 1
2
]|u|
g(yu)dyu ≈ Qu,m(z)f =
1
2m
2m−1
k=0
g (tk) .
Quadrature points:
tk =
ikz
2m′ − 1
2
,
where
2m′−1
≤ k < 2m′
ik = 1, 3, 5 . . . , 2m′
− 1
m′
= 1, 2, . . . , m
z ∈ N|u| the generating vector, which is
constructed to work well for a range of m
1
2
1
2
Figure: Embedded lattice point set up to
n = 32 points (m = 5) for z = (1, 13).
13 / 23

27. MDM with embedded lattice rules
Properties of embedded lattice rules:
1. Extensible in dimension and number of points.
This allows us to construct a single generating vector z for every u ∈ Uε. Then:
i. mu = (max ⌈log2(nu)⌉, 0) so that 2mu−1
< nu ≤ 2mu
.
ii. Qu,mu (zu) uses the generating vector
zu = (z1, z2, . . . , z|u|)
2. Random shifting provides an unbiased estimate and an estimate of the quadrature
error.
3. But they are not isotropic, i.e. each dimension is treated differently.
14 / 23

28. Reformulating the QMC MDM
Aε(f) =
u∈Uε v⊆u
(−1)|u|−|v|
Qu,mu (zu)f(·v; 0)
where we define extended active set
Uext
ε := {v ⊂ N : v ⊆ u for u ∈ Uε} ,
15 / 23

29. Reformulating the QMC MDM
Aε(f) =
v∈Uext
ε u⊇v
u∈Uε
(−1)|u|−|v|
Qu,mu (zu)f(·v; 0)
where we define extended active set
Uext
ε := {v ⊂ N : v ⊆ u for u ∈ Uε} ,
15 / 23

30. Reformulating the QMC MDM
Aε(f) =
v∈Uext
ε
mmax
m=1 u∈Uε, u⊇v
mu=m
(−1)|u|−|v|
Qu,m(zu)f(·v; 0)
where we define extended active set
Uext
ε := {v ⊂ N : v ⊆ u for u ∈ Uε} ,
mmax := max{mu : u ∈ Uε}
15 / 23

31. How to handle Qu,m(zu)f(·v; 0)?
Qv,m(zu)f(·v; 0) = Qv,m(zv)f(·v; 0) for v u.
16 / 23

32. How to handle Qu,m(zu)f(·v; 0)?
Qv,m(zu)f(·v; 0) = Qv,m(zv)f(·v; 0) for v u.
Why?
e.g. 1. Suppose
Uε = { 2, 5 }, { 2, 3, 5 }
zu = (z1, z2), (z1, z2, z3)
and consider v = {2, 5} .
16 / 23

33. How to handle Qu,m(zu)f(·v; 0)?
Qv,m(zu)f(·v; 0) = Qv,m(zv)f(·v; 0) for v u.
Why?
e.g. 1. Suppose
Uε = { 2, 5 }, { 2, 3, 5 }
zu = (z1, z2), (z1, z2, z3)
and consider v = {2, 5} .
16 / 23

34. How to handle Qu,m(zu)f(·v; 0)?
Qv,m(zu)f(·v; 0) = Qv,m(zv)f(·v; 0) for v u.
Why?
e.g. 2. Suppose
Uε = { 2, 5 }, { 2, 3, 5 }, { 2, 4, 5 }
zu = (z1, z2), (z1, z2, z3), (z1, z2, z3)
and consider v = {2, 5} .
16 / 23

35. How to handle Qu,m(zu)f(·v; 0)?
Qv,m(zu)f(·v; 0) = Qv,m(zv)f(·v; 0) for v u.
Why?
e.g. 2. Suppose
Uε = { 2, 5 }, { 2, 3, 5 }, { 2, 4, 5 }
zu = (z1, z2), (z1, z2, z3), (z1, z2, z3)
and consider v = {2, 5} .
16 / 23

36. Reformulating the QMC MDM (again)
Aε(f) =
v∈Uext
ε w∈P(v)
mmax
m=0
cv,w,mQv,m(zw)f(·v; 0)
↑
sum over all “positions” that v comes from as a subset
where we define extended active set
Uext
ε := {v ⊂ N : v ⊆ u for u ∈ Uε} ,
mmax := max{mu : u ∈ Uε}
P(v) := collection of all possible positions that v can occur
c(v, w, m) := count of all v and w occurrences
17 / 23

37. Method for reformulated QMC MDM
Given ε, {Bu}u⊂N, {Cu}u⊂N:
1. Construct the active set Uε.
2. Construct the extended active set Uext
ε .
Loop through the active set again and for each u ∈ Uε:
2.1 Choose mu such that the number of points is 2mu−1
< nu ≤ 2mu
.
2.2 Generate all subsets v and add v to Uext
ε .
Also store the position w that v came from as a subset of u.
Update cv,w,mu .
3. Then to perform an approximation:
Aε(f) =
v∈Uext
ε w∈P(v)
mmax
m=0
cv,w,m=0
cv,w,mQv,m(zw)f(·v; 0)
Each rule is extensible in points, so for each w we actually only perform the highest
level QMC rule.
18 / 23

38. Numerical results
Timing and error results for β = 3:
f(y) =
1
1 + ∞
j=1 yj/j3
Smolyak: Trapezoidal rules were used for the 1D quadrature.
QMC: CBC used to construct a single 20-dimensional generating vector that is suitable
for nm points with m = 0, 1, . . . , 25.
Error estimate: Compare with reference value
I(f) ≈ 1.1011984577041
computed in 600 dimensions using a 222 point randomly shifted lattice with 16 random
shifts. The standard error was 8 × 1013.
19 / 23

39. Timing comparision for β = 3
efficient MDM naive MDM
ε method total error time (s) total error time (s) speedup
1e-01 QMC 2.24e-04 0.0022 2.24e-04 0.0043 2.0
Smolyak 3.21e-05 0.0035 3.21e-05 0.0096 2.7
1e-02 QMC 1.24e-05 0.035 1.24e-05 0.088 2.5
Smolyak 9.32e-06 0.046 9.32e-06 0.17 3.6
1e-03 QMC 1.47e-06 0.47 1.47e-06 1.45 3.1
Smolyak 4.94e-07 0.56 4.94e-07 2.61 4.6
1e-04 QMC 7.39e-08 5.14 7.39e-08 19.99 3.9
Smolyak 1.22e-08 6.14 1.22e-08 36.69 6.0
1e-05 QMC 2.77e-09 51.02 2.76e-09 244.06 4.8
Smolyak 1.55e-09 61.53 1.54e-09 463.77 7.5
1e-06 QMC 7.56e-10 467.60 3.38e-10 2758.02 5.9
Smolyak 4.80e-10 570.26 5.40e-10 5244.56 9.2
Table: Timing comparisons between efficient and naive MDM implementations.
20 / 23

40. Time vs total error
10−2 10−1 100 101 102 103 104
Time ( )
10−9
10−8
10−7
10−6
10−5
10−4
Totalerror
Expected rate
QMC efficient
QMC naive
Smolyak efficient
Smolyak naive
Figure: Estimated total error against time.
21 / 23

41. Conclusion
1. ∞-dimensional integration problems can be tackled using the MDM.
2. By introducing the extended active set we can exploit structure in the problem to
write the MDM without the extra sum over all subsets .
3. Numerical results indicate this significantly reduces the computational effort .
4. QMC MDM and Smolyak MDM have similar runtimes, but QMC MDM does not
see the same speedup because of the need to also store different positions of a set.
Future work?
1. How does the MDM compare with other methods?
2. Is there a better way to construct the active set? Adaptively?
3. Can we circumvent storing the positions for QMC?
22 / 23

42. Thanks for listening!
23 / 23