Model calibration or data inversion involves using experimental or field data to estimate the unknown parameters in a mathematical model. In the first part of the talk, I will present a review of a few approaches for model calibration or data inversion with the focus on model discrepancy and measurement bias. A few state-of-art methods, such as modeling the discrepancy by the Gaussian stochastic process (GaSP) or scaled Gaussian stochastic processes (S-GaSP), L2 calibration, Least squares (LS) calibration and orthogonal Gaussian process calibration, will be introduced. The connection and difference between these methods will be discussed. In the second part of talk, I will discuss our ongoing works on calibrating a geophysical model by integrating the different types of the field data, such as the interferometric synthetic aperture radar satellite (InSAR) interferograms, GPS data, velocities of tilt and lava lake from the Kilauea Volcano during the eruption in 2018. This task is complicated by the discrepancy between the model and reality different sample sizes and possible bias in field data. We introduce the scaled Gaussian stochastic process (S-GaSP), a new stochastic process to model the discrepancy function in calibration for the identifiability issue between the calibrated mathematical model and the discrepancy function. We also compare a few approaches to model the measurement bias in the data. A feasible way to fuse the field data from multiple sources will then be discussed. The calibration models are implemented in the "RobustCalibration" R Package on CRAN. The scientific goal of this work is to use data in May 2018 during the earthquake and the eruption of the Kilauea Volcano to resolve the location, volume, and pressure change in the Halema'uma'u Reservoir, as well as relating the results to the inferences from the past caldera collapses.

1. A review of model calibration methods with an
application by fusing multiple sources of data from the
eruption of the K¯ılauea Volcano in 2018
Mengyang Gu and Kyle Anderson
Department of Applied Mathematics and Statistics
Johns Hopkins University
US Geological Survey
Volcano Science Center
MUMS Transition Workshop
Mengyang Gu (Johns Hopkins University) Calibration MUMS Transition Workshop 1 / 31

2. Outline
1 Review of model calibration methods
GaSP calibration
L2 calibration and LS calibration
Reconciling the L2 calibration and GaSP calibration: S-GaSP
Orthogonal construction
Multiple sources of data and measurement bias
2 Data fusion for model calibration
Mengyang Gu (Johns Hopkins University) Calibration MUMS Transition Workshop 2 / 31

3. Review of model calibration methods
Outline
1 Review of model calibration methods
GaSP calibration
L2 calibration and LS calibration
Reconciling the L2 calibration and GaSP calibration: S-GaSP
Orthogonal construction
Multiple sources of data and measurement bias
2 Data fusion for model calibration
Mengyang Gu (Johns Hopkins University) Calibration MUMS Transition Workshop 3 / 31

4. Review of model calibration methods
Model calibration and inverse problem
A simple model can be written as
yF
(x) = fM
(x, θ) + ,
where x is a vector of the observed input and θ is a vector of calibration
parameters. is the measurement error. Assume
i.i.d.
∼ N(0, σ2
0) for this
talk. The calibration is then to estimate θ based on some field/experimental
data yF
= (yF
(x1), ..., yF
(xn))T
.
Mengyang Gu (Johns Hopkins University) Calibration MUMS Transition Workshop 4 / 31

5. Review of model calibration methods
Model calibration and inverse problem
A simple model can be written as
yF
(x) = fM
(x, θ) + ,
where x is a vector of the observed input and θ is a vector of calibration
parameters. is the measurement error. Assume
i.i.d.
∼ N(0, σ2
0) for this
talk. The calibration is then to estimate θ based on some field/experimental
data yF
= (yF
(x1), ..., yF
(xn))T
.
No one trusts a model except the man who wrote it; everyone trusts an
observation, except the man who made it. – Harlow Shapley.
Mengyang Gu (Johns Hopkins University) Calibration MUMS Transition Workshop 4 / 31

6. Review of model calibration methods GaSP calibration
Outline
1 Review of model calibration methods
GaSP calibration
L2 calibration and LS calibration
Reconciling the L2 calibration and GaSP calibration: S-GaSP
Orthogonal construction
Multiple sources of data and measurement bias
2 Data fusion for model calibration
Mengyang Gu (Johns Hopkins University) Calibration MUMS Transition Workshop 5 / 31

7. Review of model calibration methods GaSP calibration
Calibration of an imperfect model
When the mathematical model is imperfect, it is usual to model
yF
(x) = fM
(x, θ) + δ(x)
yR(x)
+ ,
where δ(x) models the discrepancy and yR
(x) denotes the reality at x ∈ X.
The additive structure between the mathematical model fM
(·, ·) and
discrepancy function δ(·) is natural to a statistician.
If there is no assumption on δ(·), for any θ, one can choose δ(·), such that
fM
(x, θ) + δ(x) fits the reality perfectly well.
Mengyang Gu (Johns Hopkins University) Calibration MUMS Transition Workshop 6 / 31

8. Review of model calibration methods GaSP calibration
Calibration of an imperfect model
When the mathematical model is imperfect, it is usual to model
yF
(x) = fM
(x, θ) + δ(x)
yR(x)
+ ,
where δ(x) models the discrepancy and yR
(x) denotes the reality at x ∈ X.
The additive structure between the mathematical model fM
(·, ·) and
discrepancy function δ(·) is natural to a statistician.
If there is no assumption on δ(·), for any θ, one can choose δ(·), such that
fM
(x, θ) + δ(x) fits the reality perfectly well.
Assume the trend and intercept are properly defined in fM
(·, ·). [Kennedy
and O’Hagan, 2001] modeled δ(·) via a Gaussian stochastic process (GaSP),
such that any marginal distribution follows a multivariate normal distribution
(δ(x1), ..., δ(xn))T
∼ MN(0, σ2
R),
where Ri,j = K(xi, xj), with K(·, ·) being a kernel function.
Advantage: The predictive accuracy for yF
(·) is improved as (θ, δ(·)) is
estimated jointly, which borrows the expert knowledge in fM
(·, ·) and a
flexible nonparametric model δ(·).
Mengyang Gu (Johns Hopkins University) Calibration MUMS Transition Workshop 6 / 31

9. Review of model calibration methods GaSP calibration
Two types of inconsistency (identifiability issue)
(First type of inconsistency.) When the random discrepancy function is
generated from a GaSP, some usual estimators of θ are inconsistent when
X is a fixed domain and the sample size goes to infinity [Gu and Anderson,
2018]. This is also observed in various studies in spatial statistics [Reich
et al., 2006, Hodges and Reich, 2010, Hughes and Haran, 2013].
(Second type of inconsistency.) When the discrepancy is a deterministic
function, the true θ is not defined. According to [Kennedy and O’Hagan,
2001], “...calibration is the activity of adjusting the unknown rate parameters
until the outputs of the model fit the observed data.”
Mengyang Gu (Johns Hopkins University) Calibration MUMS Transition Workshop 7 / 31

10. Review of model calibration methods GaSP calibration
Two types of inconsistency (identifiability issue)
(First type of inconsistency.) When the random discrepancy function is
generated from a GaSP, some usual estimators of θ are inconsistent when
X is a fixed domain and the sample size goes to infinity [Gu and Anderson,
2018]. This is also observed in various studies in spatial statistics [Reich
et al., 2006, Hodges and Reich, 2010, Hughes and Haran, 2013].
(Second type of inconsistency.) When the discrepancy is a deterministic
function, the true θ is not defined. According to [Kennedy and O’Hagan,
2001], “...calibration is the activity of adjusting the unknown rate parameters
until the outputs of the model fit the observed data.” The routinely used
estimator of (θ, δ(·)) in the GaSP calibration minimizes [Gu et al., 2018b]
(θ, δ) =
1
n
n
i=1
(yF
(xi) − fM
(xi, θ) − δ(xi))2
+ λ||δ||2
H,
with the regularization parameter λ := σ2
0/(nσ2
). The native norm || · ||H
penalizes large frequency, and it is different than the L2 norm || · ||L2(X).
The mathematical model could be far from the reality in terms of the L2
loss: ||δ||2
L2(X) := x∈X
(yR
(x) − fM
(x, θ))2
dx.
Mengyang Gu (Johns Hopkins University) Calibration MUMS Transition Workshop 7 / 31

11. Review of model calibration methods GaSP calibration
Example of the first type of inconsistency
Example 1
Assume fM
(x, θ) = θ and the experimental data is noise-free, i.e.
yF
(x) = fM
(x, θ) + δ(x) and δ(·) ∼ GaSP(0, σ2
K(·, ·)), with
K(xi, xj) = exp(−|xi − xj|/γ), the exponential correlation function. The
observation yF
(xi) is equally spaced at xi ∈ [0, 1], for i = 1, ..., n. Assume both
σ2
and γ are known.
Mengyang Gu (Johns Hopkins University) Calibration MUMS Transition Workshop 8 / 31

12. Review of model calibration methods GaSP calibration
Example of the first type of inconsistency
Example 1
Assume fM
(x, θ) = θ and the experimental data is noise-free, i.e.
yF
(x) = fM
(x, θ) + δ(x) and δ(·) ∼ GaSP(0, σ2
K(·, ·)), with
K(xi, xj) = exp(−|xi − xj|/γ), the exponential correlation function. The
observation yF
(xi) is equally spaced at xi ∈ [0, 1], for i = 1, ..., n. Assume both
σ2
and γ are known.
Lemma 1 ([Gu and Anderson, 2018])
Assume σ2
> 0 and γ > 0 are both finite. When n → ∞, the maximum
likelihood estimator ˆθMLE = (1T
n R−1
1n)−1
1T
n R−1
yF
in Example 1 has the
following limiting distribution
ˆθMLE ∼ N θ,
2σ2
γ
2γ + 1
.
The variance of the estimator does not go to zero when the sample size increases
to infinity.
Mengyang Gu (Johns Hopkins University) Calibration MUMS Transition Workshop 8 / 31

13. Review of model calibration methods GaSP calibration
Example of the second type of the inconsistency
−4000 −2000 0 2000 4000
−20000200040006000
First interferogram
−0.02
0.00
0.02
0.04
−4000 −2000 0 2000 4000
−20000200040006000
Second interferogram
−0.02
0.00
0.02
0.04
−4000 −2000 0 2000 4000
−20000200040006000
Calibrated computer model by GaSP
−0.02
0.00
0.02
0.04
−4000 −2000 0 2000 4000
−20000200040006000
Calibrated computer model by S−GaSP
−0.02
0.00
0.02
0.04
−4000 −2000 0 2000 4000
−20000200040006000
Calibrated computer model by GaSP
−0.02
0.00
0.02
0.04
−4000 −2000 0 2000 4000
−20000200040006000
Calibrated computer model by S−GaSP
−0.02
0.00
0.02
0.04
Figure 1: Two interferometric synthetic aperture radar (InSAR) interferograms
measuring the ground displacement at K¯ılauea Volcano are graphed in the 1st row [Gu
and Wang, 2018]. The left panels in the 2nd and 3rd rows give M
t=1 fM
(x, θ(t)
)/M in
the GaSP calibration at each pixel x and θ(t)
is the tth posterior samples, whereas the
right panels show the S-GaSP calibration (introduced later). Having more interferograms
and a measurement bias term improve the result [Gu and Anderson, 2018].
Mengyang Gu (Johns Hopkins University) Calibration MUMS Transition Workshop 9 / 31

14. Review of model calibration methods L2 calibration and LS calibration
Outline
1 Review of model calibration methods
GaSP calibration
L2 calibration and LS calibration
Reconciling the L2 calibration and GaSP calibration: S-GaSP
Orthogonal construction
Multiple sources of data and measurement bias
2 Data fusion for model calibration
Mengyang Gu (Johns Hopkins University) Calibration MUMS Transition Workshop 10 / 31

15. Review of model calibration methods L2 calibration and LS calibration
The L2 calibration and LS calibration
Two very sensible two-step approaches are the GaSP+L2 calibration and LS+
GaSP calibration. (Here GaSP can be replaced by any nonparametric method.)
GaSP+L2 approach (L2 calibration) in [Tuo and Wu, 2015, 2016]. First,
GaSP is used to estimate yR
(·) based on yF
and they let
ˆθL2 = argmin
θ x∈X
(ˆyR
(x) − fM
(x, θ))2
dx.
The LS+GaSP approach (LS calibration) in [Wong et al., 2017]. The
calibration parameters are first estimated by the least squares (LS),
ˆθLS = argmin
θ
n
i=1
yF
(xi) − fM
(xi, θ)
2
.
Then they plug-in ˆθLS and rely on the nonparametric regression (e.g.
GaSP) to estimate the discrepancy function.
Mengyang Gu (Johns Hopkins University) Calibration MUMS Transition Workshop 11 / 31

16. Review of model calibration methods L2 calibration and LS calibration
The L2 calibration and LS calibration
Two very sensible two-step approaches are the GaSP+L2 calibration and LS+
GaSP calibration. (Here GaSP can be replaced by any nonparametric method.)
GaSP+L2 approach (L2 calibration) in [Tuo and Wu, 2015, 2016]. First,
GaSP is used to estimate yR
(·) based on yF
and they let
ˆθL2 = argmin
θ x∈X
(ˆyR
(x) − fM
(x, θ))2
dx.
The LS+GaSP approach (LS calibration) in [Wong et al., 2017]. The
calibration parameters are first estimated by the least squares (LS),
ˆθLS = argmin
θ
n
i=1
yF
(xi) − fM
(xi, θ)
2
.
Then they plug-in ˆθLS and rely on the nonparametric regression (e.g.
GaSP) to estimate the discrepancy function.
The GaSP+L2 approach does not use the mathematical model to predict
the reality. The LS+GaSP approach does not penalize complexity of
discrepancy for estimating θ. These two-step methods do not jointly
estimate (δ(·), θ).
Mengyang Gu (Johns Hopkins University) Calibration MUMS Transition Workshop 11 / 31

17. Review of model calibration methods L2 calibration and LS calibration
The following example is from [Gu and Wang, 2018].
Example 2
Assume yF
(x) = yR
(x) + , where yR
(x) = sin(10πx) + sin(πx),
fM
(x, θ) = sin(θx) and ∼ N(0, 0.32
). xi is equally spaced from [0, 1] for
i = 1, ..., n. Two scenarios are tested with n = 20 and n = 30, each implemented
N = 200 times.
Table 1: Predictive mean squared errors for Example 2 averaged over N = 200
experiments. AvgMSEfM denotes the mean squared error using only the calibrated
mathematical model and AvgMSEfM +δ denotes the mean squared error where both the
calibrated mathematical model and the discrepancy function are allowed to be used for
prediction. The results are averaged over N = 200 repeated experiments.
n = 20 GaSP +L2 LS+ GaSP GaSP calibration S-GaSP calibration
AvgMSEfM +δ 0.40 0.12 0.024 0.024
AvgMSEfM 0.60 0.50 0.50 0.50
n = 30 GaSP +L2 LS+ GaSP GaSP calibration S-GaSP calibration
AvgMSEfM +δ 0.071 0.059 0.016 0.016
AvgMSEfM 0.50 0.50 0.50 0.50
Mengyang Gu (Johns Hopkins University) Calibration MUMS Transition Workshop 12 / 31

18. Review of model calibration methods L2 calibration and LS calibration
Prediction in one experiment
0.0 0.2 0.4 0.6 0.8 1.0
−2−10123
GaSP + L2, n = 20
x
y
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
target function
prediction by RobustGaSP
prediction by DiceKriging
0.0 0.2 0.4 0.6 0.8 1.0
−2−10123
LS + GaSP, n = 20
x
y
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
target function
prediction by RobustGaSP
prediction by DiceKriging
0.0 0.2 0.4 0.6 0.8 1.0
−2−10123
Calibration, n = 20
x
y
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
target function
S−GaSP calibration
GaSP calibration
0.0 0.2 0.4 0.6 0.8 1.0
−2−10123
GaSP + L2, n = 30
x
y
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
target function
prediction by RobustGaSP
prediction by DiceKriging
0.0 0.2 0.4 0.6 0.8 1.0
−2−10123
LS + GaSP, n = 30
x
y
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
target function
prediction by RobustGaSP
prediction by DiceKriging
0.0 0.2 0.4 0.6 0.8 1.0
−2−10123
Calibration, n = 30
x
y
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
target function
S−GaSP calibration
GaSP calibration
Figure 2: Predictions for one experiment in Example 2 based on the calibrated
mathematical model and discrepancy function.
Mengyang Gu (Johns Hopkins University) Calibration MUMS Transition Workshop 13 / 31

19. Review of model calibration methods L2 calibration and LS calibration
Parameter estimation
0
50
100
150
200
0 10 20 30 40 50
θ^
count
GaSP + L2 LS + GaSP
0
50
100
150
200
0 10 20 30 40 50
θ^
count
GaSP calibration S−GaSP calibration
0
50
100
150
200
0 10 20 30 40 50
θ^
count
GaSP + L2 LS + GaSP
0
50
100
150
200
0 10 20 30 40 50
θ^
count
GaSP calibration S−GaSP calibration
Figure 3: Histograms of the calibrated parameter of 200 experiments, where n = 20 and
n = 30 are assumed in each experiment for the upper and lower panels, respectively.
Reason: mathematical models contain information about reality. Modeling the
mathematical model and discrepancy function jointly is often more precise in prediction.
Mengyang Gu (Johns Hopkins University) Calibration MUMS Transition Workshop 14 / 31

20. Review of model calibration methods Reconciling the L2 calibration and GaSP calibration: S-GaSP
Outline
1 Review of model calibration methods
GaSP calibration
L2 calibration and LS calibration
Reconciling the L2 calibration and GaSP calibration: S-GaSP
Orthogonal construction
Multiple sources of data and measurement bias
2 Data fusion for model calibration
Mengyang Gu (Johns Hopkins University) Calibration MUMS Transition Workshop 15 / 31

21. Review of model calibration methods Reconciling the L2 calibration and GaSP calibration: S-GaSP
Scaled Gaussian stochastic process (S-GaSP)
W.l.o.g., assume X = [0, 1]p
. The idea is to give more prior probability on the
smaller ||δ||2
L2(X) = ||yR
(·) − fM
(·, ˆθ)||2
L2(X). Define the scaled Gaussian
stochastic process (S-GaSP) to model the discrepancy δz(·):
yF
(x) = fM
(x, θ) + δz(x) + ,
δz(x) = δ(x) | ξ∈X
δ2
(ξ)dξ = Z ,
δ(·) ∼ GaSP(0, σ2
K(·, ·)),
Z ∼ pZ(·), ∼ N(0, σ2
0).
(1)
Given Z = z, the S-GaSP becomes a GaSP constrained at the space
x∈X
δ2
(x)dx = z.
Mengyang Gu (Johns Hopkins University) Calibration MUMS Transition Workshop 16 / 31

22. Review of model calibration methods Reconciling the L2 calibration and GaSP calibration: S-GaSP
Scaled Gaussian stochastic process (S-GaSP)
W.l.o.g., assume X = [0, 1]p
. The idea is to give more prior probability on the
smaller ||δ||2
L2(X) = ||yR
(·) − fM
(·, ˆθ)||2
L2(X). Define the scaled Gaussian
stochastic process (S-GaSP) to model the discrepancy δz(·):
yF
(x) = fM
(x, θ) + δz(x) + ,
δz(x) = δ(x) | ξ∈X
δ2
(ξ)dξ = Z ,
δ(·) ∼ GaSP(0, σ2
K(·, ·)),
Z ∼ pZ(·), ∼ N(0, σ2
0).
(1)
Given Z = z, the S-GaSP becomes a GaSP constrained at the space
x∈X
δ2
(x)dx = z. Let pδ(Z = z) be the density of Z induced by GaSP.
Conditional on all the parameters, the default choice of pZ(·) is
pZ(z) =
gZ (z) pδ (Z = z)
∞
0
gZ (t) pδ (Z = t) dt
, (2)
with the scaling function gZ(z) = λz
2σ2 exp −λzz
2σ2 .
Mengyang Gu (Johns Hopkins University) Calibration MUMS Transition Workshop 16 / 31

23. Review of model calibration methods Reconciling the L2 calibration and GaSP calibration: S-GaSP
Some results about the S-GaSP
The first three points below are shown in [Gu and Wang, 2018]; the latter four
points are proved in [Gu et al., 2018b].
Any GaSP is a special case of S-GaSP with a constant scaling function gZ(·).
We do not put strong prior information on σ2
, because we hope to estimate
the size of the discrepancy by the data.
The marginal likelihood of δz(·) has a simple form (a transformed kernel).
Mengyang Gu (Johns Hopkins University) Calibration MUMS Transition Workshop 17 / 31

24. Review of model calibration methods Reconciling the L2 calibration and GaSP calibration: S-GaSP
Some results about the S-GaSP
The first three points below are shown in [Gu and Wang, 2018]; the latter four
points are proved in [Gu et al., 2018b].
Any GaSP is a special case of S-GaSP with a constant scaling function gZ(·).
We do not put strong prior information on σ2
, because we hope to estimate
the size of the discrepancy by the data.
The marginal likelihood of δz(·) has a simple form (a transformed kernel).
We study the “geometry” of the reproducing kernel Hilbert space of
S-GaSP, which induces a certain way to shrink the eigenvalues of the kernel.
The S-GaSP replaces the norm ||δ||2
H in GaSP by ||δz||2
H + λz||δz||2
L2(X),
which penalizes both the complexity of the discrepancy and L2 distance
between the reality and mathematical model in a joint statistical model.
We show in the S-GaSP calibration, fM
(·, ˆθz) + ˆδz(·) converges to the
reality yR
(·) with the same rate as the GaSP under the certain choice of λz.
With the same choice of λz, we also show the estimated ˆθz converge to the
L2 minimizer with a rate slightly slower than the L2 calibration (the
estimated ˆθ in the GaSP calibration does not converge to the L2 minimizer).
Mengyang Gu (Johns Hopkins University) Calibration MUMS Transition Workshop 17 / 31

25. Review of model calibration methods Reconciling the L2 calibration and GaSP calibration: S-GaSP
An example between S-GaSP and GaSP
The following example is from [Gu and Anderson, 2018].
Example 3
Assume yF
(x) = yR
(x) + where x = (x1, x2) ∈ [0, 1]2
, ∼ N(0, 0.052
) is an
independent Gaussian noise and reality is a function in [Lim et al., 2002]:
yR
(x) =
1
6
{(30 + 5x1 sin(5x1))(4 + exp(−5x2)) − 100} .
Assume fM
(x, θ) = θ1 + θ2 sin(5x1), where θ = (θ1, θ2) are two unknown
calibration parameters. The field data yF
(xi) is observed at xi, i = 1, ..., 30,
drawn from the maximin Latin hypercube design. The goal is to estimate θ and
predict the reality at all x ∈ [0, 1]2
.
Table 2: Predictive mean squared errors and the MLE of the parameters in GaSP
and S-GaSP calibration models in Example 3.
MSEfM MSEfM +δ
ˆθ ˆσ2 ˆγ ˆσ2
0
GaSP 91.5 6.12 × 10−3
(13.2, 1.90) 1.32 × 102
(1.44, 1.8) 1.76 × 10−3
S-GaSP 1.58 5.47 × 10−3
(3.76, 2.01) 3.54 × 102
(1.72, 2.13) 1.74 × 10−3
Mengyang Gu (Johns Hopkins University) Calibration MUMS Transition Workshop 18 / 31

26. Review of model calibration methods Reconciling the L2 calibration and GaSP calibration: S-GaSP
When σ2
is fixed
0 50 100 150 200
02060100
σ2
MSEfM
0.33 0.88 0.91 0.92 0.93
ρ^
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
q
GaSP
S−GaSP
0 50 100 150 200
0.0000.0100.020
σ2
MSEfM+δ
0.33 0.88 0.91 0.92 0.93
ρ^
q
q
q
q
q
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
q
GaSP
S−GaSP
Figure 4: Predictive mean squared errors of Example 3 when σ2
is fixed in the
GaSP and S-GaSP calibrations. The MSEfM and MSEfM +δ are shown in the left
and right panel, respectively. The upper x coordinate is the estimated median
value of the correlation matrix R for each σ2
in the GaSP calibration model, after
plugging in the estimated range parameters.
Mengyang Gu (Johns Hopkins University) Calibration MUMS Transition Workshop 19 / 31

27. Review of model calibration methods Orthogonal construction
Outline
1 Review of model calibration methods
GaSP calibration
L2 calibration and LS calibration
Reconciling the L2 calibration and GaSP calibration: S-GaSP
Orthogonal construction
Multiple sources of data and measurement bias
2 Data fusion for model calibration
Mengyang Gu (Johns Hopkins University) Calibration MUMS Transition Workshop 20 / 31

28. Review of model calibration methods Orthogonal construction
Orthogonal Gaussian stochastic process
In the linear regression, one has Cov( ˆYi, ˆi) = 0, where ˆYi is the fitted value
and ˆi is the residual, for the ith observation. Then it seems very sensible to
make fM
(·, ˆθ) “orthogonal” or “independent” of the ˆδ(·).
In [Plumlee, 2016], δ(·) is assumed to be a GaSP with the following
constraint
∂ x∈X
(yR
(x) − fM
(x, θ))2
dx
∂θ
= − 2
x∈X
{D(0,1)
fM
(x, θ)}δ(x)dx = 0 (3)
where D(0,1)
fM
(x, θ) is the derivative of fM
with regard to θ.
This is an innovative idea.
However, Not only the minimizer of the L2 loss satisfies the first order
conditions in (3). In fact, all the local minimizers, maximizers and saddle
points of the L2 loss satisfy these conditions. Then the likelihood is large
when θ is equal to any stationary points (illustrated later).
Second order conditions are hard to enforce.
Mengyang Gu (Johns Hopkins University) Calibration MUMS Transition Workshop 21 / 31

29. Review of model calibration methods Orthogonal construction
Example 4 ([Gu and Wang, 2018])
Suppose yF (x) = x cos(3x/2) + x + with x ∈ [0, 5] and ∼ N(0, 0.22).
The computer model is fM (x, θ) = sin(θx) + x for θ ∈ [0, 3]. We have
yF (xi), where xi is equally spaced in [0, 5] for i = 1, ..., 15.
Mengyang Gu (Johns Hopkins University) Calibration MUMS Transition Workshop 22 / 31

30. Review of model calibration methods Orthogonal construction
Example 4 ([Gu and Wang, 2018])
Suppose yF (x) = x cos(3x/2) + x + with x ∈ [0, 5] and ∼ N(0, 0.22).
The computer model is fM (x, θ) = sin(θx) + x for θ ∈ [0, 3]. We have
yF (xi), where xi is equally spaced in [0, 5] for i = 1, ..., 15.
0 1 2 3 4 5
02468
x
y
target function
computer model θ = 1
computer model θ = 2
0.0 0.5 1.0 1.5 2.0 2.5 3.0
1520253035
θ
L2Loss
0.0 0.5 1.0 1.5 2.0 2.5 3.0
−30−26−22−18
θ
loglik
0.0 0.5 1.0 1.5 2.0 2.5 3.0
−50−40−30
θ
loglik
0.0 0.5 1.0 1.5 2.0 2.5 3.0
−2000−10000
θ
loglik
Figure 5: The target function yR
(x) = x cos(3x/2) + x is graphed in the upper left
panel. The L2 loss function in 4 is graphed in the upper right panel. The log-likelihoods
of the GaSP, S-GaSP, and O-GaSP with the same kernel function are graphed in the
left, middle, and right panels in the second row, respectively.
Mengyang Gu (Johns Hopkins University) Calibration MUMS Transition Workshop 22 / 31

31. Review of model calibration methods Multiple sources of data and measurement bias
Outline
1 Review of model calibration methods
GaSP calibration
L2 calibration and LS calibration
Reconciling the L2 calibration and GaSP calibration: S-GaSP
Orthogonal construction
Multiple sources of data and measurement bias
2 Data fusion for model calibration
Mengyang Gu (Johns Hopkins University) Calibration MUMS Transition Workshop 23 / 31

32. Review of model calibration methods Multiple sources of data and measurement bias
Multiple sources of data and measurement bias
−5000 0 5000
−6000−2000020006000 interferogram 1
x1
x2
−0.05
0.00
0.05
m/yr
−5000 0 5000
−6000−2000020006000
interferogram 2
x1
x2
−0.05
0.00
0.05
m/yr
−5000 0 5000
−6000−2000020006000
interferogram 3
x1
x2
−0.05
0.00
0.05
m/yr
−5000 0 5000
−6000−2000020006000
interferogram 4
x1
x2
−0.05
0.00
0.05
m/yr
−5000 0 5000
−6000−2000020006000
interferogram 5
x1
x2
−0.05
0.00
0.05
m/yr
Figure 6: Five InSAR interferograms measuring the ground deformation at K¯ılauea. The
black curves show cliffs and other important topographic features at K¯ılauea; the large
elliptical feature is K¯ılauea Caldera.
The measurement bias here is the spatially correlated pattern caused by the
cloud and atmospheric conditions.
We hope to calibrate and improve the geophysical model to better explain
the volcanic activities (reality), not the air conditions.
Mengyang Gu (Johns Hopkins University) Calibration MUMS Transition Workshop 24 / 31

33. Review of model calibration methods Multiple sources of data and measurement bias
Modeling multiple sources of data
Calibration by repeated experiments [Bayarri et al., 2007]:
yF
l (x) = fM
(x, θ) + δ(x) + l, (4)
for each source l, l = 1, ..., k, and any x ∈ X, where δ(·) models the
discrepancy and l ∼ N(0, σ2
0l) independently.
Mengyang Gu (Johns Hopkins University) Calibration MUMS Transition Workshop 25 / 31

34. Review of model calibration methods Multiple sources of data and measurement bias
Modeling multiple sources of data
Calibration by repeated experiments [Bayarri et al., 2007]:
yF
l (x) = fM
(x, θ) + δ(x) + l, (4)
for each source l, l = 1, ..., k, and any x ∈ X, where δ(·) models the
discrepancy and l ∼ N(0, σ2
0l) independently.
In [Gu and Anderson, 2018], we consider calibration by multiple sources of
data with measurement bias:
yF
l (x) = fM
(x, θ) + µl + δdisc(x) + δbias,l(x) + l, (5)
for each source l, l = 1, ..., k, and any x ∈ X, where µl is an unknown mean
parameter (to account for the uncertainty in unwrapping the interferogram),
δbias,l(·) models the measurement bias in the source l, l = 1, ..., k,
independent of the discrepancy function. We propose to model δdisc(·) and
δbias,l(·) by the S-GaSP and GaSP respectively.
(R package.) The calibration model with no discrepancy, with the GaSP or
S-GaSP discrepancy, with or without the measurement bias is developed in
the “RobustCalibration” package [Gu, 2018]. For slow mathematical models,
the “RobustGaSP” package [Gu et al., 2018a] is called for emulation.
Mengyang Gu (Johns Hopkins University) Calibration MUMS Transition Workshop 25 / 31

35. Data fusion for model calibration
Outline
1 Review of model calibration methods
2 Data fusion for model calibration
Mengyang Gu (Johns Hopkins University) Calibration MUMS Transition Workshop 26 / 31

36. Data fusion for model calibration
Data from the eruption of the K¯ılauea Volcano in 2018
4 InSAR interferograms measuring the total ground changes during a few
days. We have thousands of pixels in each interferogram.
Measurements from 13 GPS stations, where each station gives a
three-dimensional change (east, north and up) for certain amount of time.
In total we have 39 measurements.
Measurement from 4 tilt stations, each gives us a two-dimensional velocity
of the change (derivative of the vertical measurement change for the east
and north directions). In total we have 8 measurements.
We also have the change of the surface height of the lava lake during that
period of time.
Mengyang Gu (Johns Hopkins University) Calibration MUMS Transition Workshop 27 / 31

37. Data fusion for model calibration
The calibration model
yInSAR,k(x) = µk +
3
j=1
fM
j (x, θ)ωj +
3
j=1
δj(x)ωj + δbias,k(x) + InSAR,k, k = 1, ..., 4
yGP S,j(x) = fM
j (x, θ) + δj(x) + GP S,j, j = 1, 2, 3
ytilt,i(x) =
∂fM
3 (x, θ)
∂xi
+
∂δ3(x)
∂xi
+ tilt,i, , i = 1, 2
ylake = fM
lake(θ) + lake.
δj(x) is the discrepancy of the model at the jth direction at location x.
δbias,k(x) is the measurement bias or the spatial correlated pattern. In
geophysics, the covariance of this term is typically estimated by other
interferograms and held fixed in MCMC.
The noise InSAR,k, GP S,j, tilt,i and lake are assumed to follow
independent zero-mean Gaussian distribution with known variances
(provided by the device).
We test two geophysical model (sphere and spheroid). In the simple one, we
have 7 parameters: θ = (θ1, ..., θ7)T
, which are the location of the chamber,
depth of chamber, pressure change, volume of the chamber, shear modular (
host rock properties), and average magma density of the lava lake.Mengyang Gu (Johns Hopkins University) Calibration MUMS Transition Workshop 28 / 31

38. Data fusion for model calibration
Joint density
After marginalizing out δj(·), ∂δi(·)
∂xi
and δbias,k(·), the marginal distribution of the
data Y = (ylake, yT
GP S,1, yT
GP S,2, yT
GP S,3, yT
tilt,1, yT
tilt,2, yT
InSAR,1, ..., yT
InSAR,4)T
is still a multivariate normal distribution with the covariance:









σ2
lake
˜ΣG1 ω1ΣG1I1 ω1ΣG1I2 ω1ΣG1I3 ω1ΣG1I4
˜ΣG2 ω2ΣG2I1 ω2ΣG2I2 ω2ΣG2I3 ω2ΣG2I4
˜ΣG3 ΣG3T 1 ΣG3T 2 ω3ΣG3I1 ω3ΣG3I2 ω3ΣG3I3 ω3ΣG3I4
ΣT 1G3
˜ΣT 1 ΣT 1T 2 ω3ΣT 1I1 ω3ΣT 2I1 ω3ΣT 1I3 ω3ΣT 1I4
ΣT 2G3 ΣT 2T 1
˜ΣT 2 ω3ΣT 2I1 ω3ΣT 2I2 ω3ΣT 2I3 ω3ΣT 2I4
ω1ΣI1G1 ω2ΣI1G2 ω3ΣI1G3 ω3ΣI1T 1 ω3ΣI1T 2
˜ΣI1
ω1ΣI2G1 ω2ΣI2G2 ω3ΣI2G3 ω3ΣI2T 1 ω3ΣI2T 2
˜ΣI2
ω1ΣI3G1 ω2ΣI3G2 ω3ΣI3G3 ω3ΣI3T 1 ω3ΣI3T 2
˜ΣI3
ω1ΣI4G1 ω2ΣI4G3 ω3ΣI4G3 ω3ΣI4T 1 ω3ΣI4T 2
˜ΣI4









.
We can then write down the likelihood in a full conditional way.
Mengyang Gu (Johns Hopkins University) Calibration MUMS Transition Workshop 29 / 31

39. Data fusion for model calibration
Some further issues and scientific discoveries
We develop some methods to track how each source of data influences the
model (in terms of the “effective sample size”).
Here we assume the bias term and different sources of data are independent.
The correlation can be potentially incorporated by a separable model or
linear model of coregionalization (LMC) (see e.g. [Gu and Shen, 2018]).
In the 2018 eruption at K¯ılauea, we observed around 0.8km3
basaltic caldera
collapsed. However the conditions which trigger reservoir failure and the
onset of collapse are very poorly known.
We find that the early stages of collapse occurred around the pre-existing
eruptive vent when < 2% of magma had been evacuated from the magma
stored at shallow depth (around 2 km) beneath the volcano’s summit.
Broad-scale episodic reservoir failure began around 10 days later after
withdrawal of an additional around 1 − 2% of stored magma reduced
reservoir pressure by around 20–25 MPa.
These results suggest that, under some conditions, calderas may begin to
fail more quickly than previously believed on the basis of theoretical and
experimental models. The results are preliminary.
Mengyang Gu (Johns Hopkins University) Calibration MUMS Transition Workshop 30 / 31

40. Data fusion for model calibration
Thanks!
Mengyang Gu (Johns Hopkins University) Calibration MUMS Transition Workshop 31 / 31

41. References
Maria J Bayarri, James O Berger, Rui Paulo, Jerry Sacks, John A Cafeo,
James Cavendish, Chin-Hsu Lin, and Jian Tu. A framework for
validation of computer models. Technometrics, 49(2):138–154, 2007.
Mengyang Gu. RobustCalibration: Robust Calibration of Imperfect
Mathematical Models, 2018. URL
https://CRAN.R-project.org/package=RobustCalibration. R
package version 0.5.2.
Mengyang Gu and Kyle Anderson. Calibration of imperfect mathematical
models by multiple sources of data with measurement bias. arXiv
preprint arXiv:1810.11664, 2018.
Mengyang Gu and Weining Shen. Generalized probabilistic principal
component analysis of correlated data. arXiv preprint arXiv:1808.10868,
2018.
Mengyang Gu and Long Wang. Scaled gaussian stochastic process for
computer model calibration and prediction. arXiv preprint
arXiv:1707.08215, 2018.
Mengyang Gu, Jes´us Palomo, and James O Berger. RobustGaSP: Robust
Mengyang Gu (Johns Hopkins University) Calibration MUMS Transition Workshop 31 / 31

42. References
Gaussian stochastic process emulation in R. arXiv preprint
arXiv:1801.01874, 2018a.
Mengyang Gu, Fangzheng Xie, and Long Wang. A theoretical framework
of the scaled Gaussian stochastic process in prediction and calibration.
arXiv preprint arXiv:1807.03829, 2018b.
James S Hodges and Brian J Reich. Adding spatially-correlated errors can
mess up the fixed effect you love. The American Statistician, 64(4):
325–334, 2010.
John Hughes and Murali Haran. Dimension reduction and alleviation of
confounding for spatial generalized linear mixed models. Journal of the
Royal Statistical Society: Series B (Statistical Methodology), 75(1):
139–159, 2013.
Marc C Kennedy and Anthony O’Hagan. Bayesian calibration of computer
models. Journal of the Royal Statistical Society: Series B (Statistical
Methodology), 63(3):425–464, 2001.
Yong B Lim, Jerome Sacks, WJ Studden, and William J Welch. Design
and analysis of computer experiments when the output is highly
Mengyang Gu (Johns Hopkins University) Calibration MUMS Transition Workshop 31 / 31

43. Data fusion for model calibration
correlated over the input space. Canadian Journal of Statistics, 30(1):
109–126, 2002.
Matthew Plumlee. Bayesian calibration of inexact computer models.
Journal of the American Statistical Association, (just-accepted), 2016.
Brian J Reich, James S Hodges, and Vesna Zadnik. Effects of residual
smoothing on the posterior of the fixed effects in disease-mapping
models. Biometrics, 62(4):1197–1206, 2006.
Rui Tuo and CF Jeff Wu. Efficient calibration for imperfect computer
models. The Annals of Statistics, 43(6):2331–2352, 2015.
Rui Tuo and CF Jeff Wu. A theoretical framework for calibration in
computer models: parametrization, estimation and convergence
properties. SIAM/ASA Journal on Uncertainty Quantification, 4(1):
767–795, 2016.
Raymond KW Wong, Curtis B Storlie, and Thomas Lee. A frequentist
approach to computer model calibration. Journal of the Royal Statistical
Society: Series B (Statistical Methodology), 79:635–648, 2017.
Mengyang Gu (Johns Hopkins University) Calibration MUMS Transition Workshop 31 / 31