We are developing a surrogate for Bayesian Update formula. We perform update of prior polynomial chaos coefficients (PCE) and obtain posterior PCE.

1. Center for Uncertainty
Quantification
Solving inverse problem via
non-linear update of PCE coefficients
A. Litvinenko1, H.G. Matthies2, B. Rosic2, E. Zander2
1
CEMSE Division, KAUST, 2
TU Braunschweig, Germany
alexander.litvinenko@kaust.edu.sa
Center for Uncertainty
Quantification
Center for Uncertainty
Quantification
Motivation
Notation: Given a physical system modeled
by a PDE or ODE with uncertain coefficient
q(x, ω):
A(u, q(x, ω)) = f.
A solution operator S: u = u(x, ω) = S(q, f).
A measurement operator: y = M(u). Noisy
observations z(ω) = ˆy + ε(ω) with random
measurement error ε and ‘truth’ ˆy.
Very often, after applying stochastic Galerkin
or building a surrogate, we have all ingredi-
ents in gPCE basis. It would be nice to do
data assimilation/compute ’Bayesian’ update
with gPCE coefficients (to avoid sampling).
Aim: given noisy observations z(ω), to iden-
tify q(ω).
How?: To identify q(ω) we derived non-linear
approximation of the Bayesian update from
the variational problem associated with con-
ditional expectation. To reduce cost of the
’Bayesian’ update we offer a functional ap-
proximation, e.g. gPCE.
New: We apply ’Bayesian’ update to gPCE
coefficients of q(ω) (not to the probability den-
sity function of q).
1. Minimum Mean Square Error
Estimation (MMSE)
Let q(ξ) : Ω → RNq
be the (a priori) stochastic model
of some unknown QoI (e.g., uncertain parameter q),
Y : Ω → RNy
be the stochastic model (e.g. of mea-
surement forecast Y = M(q(ξ)) + ε(ξM)). We search
for a function ϕ : RNy
→ RNq
. The best estimator ˆϕ for
Y given q is
ˆϕ = argminϕ E[ q(ξ) − ϕ(Y + ε(ξM)) 2
2], (1)
where the expectation needs to be taken over Ω =
Ω × ΩM. Writing ξ = (ξ, ξM) ∈ Ω the best estimator
(or predictor) of q given the measurement model is
qM(ξ ) = ˆϕ(Y + ε(ξM)). (2)
Suppose the actual measurements are: yM(ξ ) =
¯yM + εM(ξ ), where ¯yM are the measured values and
εM(ξ ) is some assumed error model, then
q(ξ ) = ˆϕ(¯yM + εM(ξ )). (3)
2. Generalized PCE of mapping ϕ
Let us represent ϕ as a gPCE
ϕ ≈ ˆϕ = y →
γ∈J
ϕγΨγ(y(ξ))
γ - multi-index and J a multi-index set.
Compute unknown coefficients ϕγ by minimizing MSE,
by taking derivative w.r.t. ϕγ:
∂
∂ϕγ
E[(q −
γ∈J
ϕγΨγ(Y ))2
] = 0, (4)
2


γ∈J
E [Ψγ(y)Ψδ(y)] ϕγ − E [qΨδ(y)]

 = 0, ∀δ ∈ J ,
γ∈J
ϕγE[Ψγ(Y )Ψδ(Y )] = E[qΨδ(Y )], (5)
Aγδ := E[Ψγ(Y )Ψδ(Y )] ≈
NA
k=1
wA
k Ψγ(Y (ξk))Ψδ(Y (ξk)),
E[qΨδ(Y )] ≈
Nb
k=1
wb
kq(ξk)Ψδ(Y (ξk)), or in a matrix form,
(6)
V[diag(...wA
k ...)]VT


...
ϕβ
...

 = W


wb
1q(ξ1)
...
wb
Nb
q(ξNb
)

 , (7)
V := [..., Ψγ, ...]T
∈ R|Jγ|×NA
, [diag(...wA
k ...)] ∈ RNA×NA
,
W ∈ R|Jα|×Nb
, [wb
0q(ξ0)...wb
Nb
q(ξNb
)] ∈ RNb
.
Solving Eq. 7, obtain vector of coefficients (...ϕβ...) for
all β and then compute the update:
qnew(ξ ) = ˆϕ(¯yM + εM(ξ )). (8)
Example 1. The mapping ϕ does not exist in the
Hermite basis. y(ξ) = ξ2
, q(ξ) = ξ3
. PCE coefficients
are (1, 0, 1, 0): ξ2
= 1·H0(ξ)+0·H1(ξ)+1·H2(ξ)+0·H3(ξ)
and (0, 3, 0, 1): ξ3
= 0·H0(ξ)+3·H1(ξ)+0·H2(ξ)+1·H3(ξ).
Mapping ϕ does not exist. The matrix A is close to
singular. Support of Hermite polynomials (used for
Gaussian RVs) is (−∞, ∞).
Example 2. The mapping ϕ does exist in the La-
guerre basis.
y(ξ) = ξ2
, q(ξ) = ξ3
. gPCE coefficients are (2, −4, 2, 0)
and (6, −18, 18, −6). Mapping ϕ of order 8 and higher
produces a very accurate result. Support of Laguerre
polynomials (used for Gamma RVs) is [0, ∞).
In [1-5] we demonstrated that linear ϕ corresponds to
the well-known Kalman Filter.
Implementation: implementation in Matlab using the
Stochastic Galerkin library sglib by E. Zander [7].
3. Numerics
10 0 10
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
x
20 0 20
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
0.45
y
0 10 20
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
z
xf
x
a
yf
y
a
zf
z
a
Figure 1: Lorenz-84. Linear measurement
(x(t), y(t), z(t)) at t = 10: prior and posterior af-
ter one update.
10 5 0
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
x
x1
x2
15 10 5
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
0.45
0.5
y
y1
y2
5 10 15
0
0.1
0.2
0.3
0.4
0.5
0.6
z
z1
z2
Figure 2: Lorenz-84. Quadratic measurement
(x(t)2
, y(t)2
, z(t)2
) at t = 10: Comparison posterior for
LBU and NLBU after one update.
Example 3. Diffusion with uncertain coefficients:
− · (κ(x, ξ) u(x, ξ)) = f(x, ξ), D = [0, 1]. (9)
Measurements are in x1 = 0.2 and x2 = 0.8 with values
of y1 = 10 and y2 = 5 and noise with st. deviations of
σ1 = 0.5 and σ2 = 1.5.
0 0.2 0.4 0.6 0.8 1
−20
−10
0
10
20
30
0 0.2 0.4 0.6 0.8 1
−20
−10
0
10
20
30
Figure 3: Updating of the solution u. Left - original so-
lution u(ξ) and right the updated solution u (ξ ). Shown
are the mean +/− one to three standard deviations,
plus additionally 20 sample realisations. Uncertainty
decreases in the measurement points (0.2, 0.8).
0 0.2 0.4 0.6 0.8 1
−1
−0.5
0
0.5
1
1.5
2
0 0.2 0.4 0.6 0.8 1
−1
−0.5
0
0.5
1
1.5
2
Figure 4: Updating of the parameter κ. Left is the pa-
rameter model κ(ξ) and right the updated parameter
model κ (ξ ). Uncertainty decreases in the measure-
ment points (0.2, 0.8).
0.6 0.7 0.8 0.9 1
0.2
0.3
0.4
0.5
0.6 0.7 0.8 0.9 1
0.2
0.3
0.4
0.5
Figure 5: MMSE estimation with increasing polyno-
mial degrees pϕ = 1 and 3 from left to right. True
values X are marked by o, and estimated values
ˆX = ˆϕ(Y ) are marked by x.
Conclusion
1.We take into account the available mea-
surements and compute an update/a pos-
teriori gPCE coefficients of QoI (e.g. un-
certain coefficients).
2.We minimize MMSE and compute condi-
tional expectation
3.We developed a cheap gPCE based ap-
proximation of the Bayesian Update (see
[8]).
4.Introduced a way to derive MMSE ϕ (as a
linear, quadratic, cubic etc approximation,
i. e. compute conditional expectation of q,
given measurements .
5.Linear ϕ is equivalent to the Kalman filter.
Acknowledgements: A. Litvinenko is a member of the KAUST
ECRC and SRI UQ centers in Computational Science and Engi-
neering.
References
1. H.G. Matthies, E. Zander, B.V. Rosi´c, A. Litvinenko, O. Pajonk,
Inverse Problems in a Bayesian Setting, arXiv:1511.00524,
(2015).
2. A. Litvinenko, H. G. Matthies, Inverse problems and uncer-
tainty quantification, arXiv:1312.5048, (2013).
3. B. V. Rosi´c, A. Kucerova, J. Sykora, O. Pajonk, A. Litvinenko,
H. G. Matthies, Parameter Identification in a Probabilistic Set-
ting, Engineering Structures (2013).
4. O. Pajonk, B. V. Rosic, A. Litvinenko, H. G. Matthies, A Deter-
ministic Filter for Non-Gaussian Bayesian Estimation, - appli-
cations to dynamical system estimation with noisy measure-
ments. Physica D: Nonlinear Phenomena, Vol. 241(7), pp.
775-788, (2012).
5. B.V. Rosi´c, A. Litvinenko, O. Pajonk, H.G. Matthies, Sampling-
free linear Bayesian update of polynomial chaos representa-
tions, Journal of Computational Physics 231 (17), 5761-5787,
(2012).
6. H.G. Matthies, A. Litvinenko, O. Pajonk, B.V. Rosi´c, E. Zander,
Parametric and uncertainty computations with tensor product
representations, Uncertainty Quantification in Scientific Com-
puting, 139-150, (2012).
7. E. Zander, Stochastic Galerkin library
https://github.com/ezander/sglib
8. O. G. Ernst, B. Sprungk, H.-J. Starkloff, Analysis of the Ensem-
ble and Polynomial Chaos Kalman Filters in Bayesian Inverse
Problems, SIAM/ASA JUQ 3(1), (2015)