MCMC and likelihood-free methods

MCMC and Likelihood-free Methods
Christian P. Robert
Universit´e Paris-Dauphine & CREST
http://www.ceremade.dauphine.fr/~xian
November 2, 2010

Outline
Computational issues in Bayesian statistics
The Metropolis-Hastings Algorithm
The Gibbs Sampler
Population Monte Carlo
Approximate Bayesian computation
ABC for model choice

Motivation and leading example
The Gibbs Sampler

Latent variables
Latent structures make life harder!
Even simple models may lead to computational complications, as
in latent variable models
f(x|θ) = f (x, x |θ) dx

Latent variables
f(x|θ) = f (x, x |θ) dx
If (x, x ) observed, ﬁne!

Latent variables
f(x|θ) = f (x, x |θ) dx
If (x, x ) observed, ﬁne!
If only x observed, trouble!

Latent variables
Example (Mixture models)
Models of mixtures of distributions:
X ∼ fj with probability pj,
for j = 1, 2, . . . , k, with overall density
X ∼ p1f1(x) + · · · + pkfk(x) .

Latent variables
X ∼ p1f1(x) + · · · + pkfk(x) .
For a sample of independent random variables (X1, · · · , Xn),
sample density
n
i=1
{p1f1(xi) + · · · + pkfk(xi)} .

Latent variables
X ∼ p1f1(x) + · · · + pkfk(x) .
For a sample of independent random variables (X1, · · · , Xn),
sample density
n
i=1
{p1f1(xi) + · · · + pkfk(xi)} .
Expanding this product of sums into a sum of products involves kn
elementary terms: too prohibitive to compute in large samples.

Latent variables
Simple mixture (1)
−1 0 1 2 3
−10123
µ1
µ2
Case of the 0.3N (µ1, 1) + 0.7N (µ2, 1) likelihood

Latent variables
Simple mixture (2)
For the mixture of two normal distributions,
0.3N(µ1, 1) + 0.7N(µ2, 1) ,
likelihood proportional to
n
i=1
[0.3ϕ (xi − µ1) + 0.7 ϕ (xi − µ2)]
containing 2n terms.

Latent variables
Complex maximisation
Standard maximization techniques often fail to ﬁnd the global
maximum because of multimodality or undesirable behavior
(usually at the frontier of the domain) of the likelihood function.
Example
In the special case
f(x|µ, σ) = (1 − ) exp{(−1/2)x2
} +
σ
exp{(−1/2σ2
)(x − µ)2
}
with > 0 known,

Latent variables
Complex maximisation
Standard maximization techniques often fail to ﬁnd the global
maximum because of multimodality or undesirable behavior
(usually at the frontier of the domain) of the likelihood function.
Example
In the special case
f(x|µ, σ) = (1 − ) exp{(−1/2)x2
} +
σ
exp{(−1/2σ2
)(x − µ)2
}
with > 0 known, whatever n, the likelihood is unbounded:
lim
σ→0
L(x1, . . . , xn|µ = x1, σ) = ∞

Latent variables
Unbounded likelihood
−2 0 2 4 6
1234
µ
n= 3
−2 0 2 4 6
1234
µ
σ
n= 6
−2 0 2 4 6
1234
µ
n= 12
−2 0 2 4 6
1234 µ
σ
n= 24
−2 0 2 4 6
1234
µ
n= 48
−2 0 2 4 6
1234
µ
σ
n= 96
Case of the 0.3N (0, 1) + 0.7N (µ, σ) likelihood

Latent variables
Example (Mixture once again)
press for MA Observations from
x1, . . . , xn ∼ f(x|θ) = pϕ(x; µ1, σ1) + (1 − p)ϕ(x; µ2, σ2)

Latent variables
Prior
µi|σi ∼ N (ξi, σ2
i /ni), σ2
i ∼ I G (νi/2, s2
i /2), p ∼ Be(α, β)

Latent variables
Prior
µi|σi ∼ N (ξi, σ2
i /ni), σ2
i ∼ I G (νi/2, s2
i /2), p ∼ Be(α, β)
Posterior
π(θ|x1, . . . , xn) ∝
n
j=1
{pϕ(xj; µ1, σ1) + (1 − p)ϕ(xj; µ2, σ2)} π(θ)
=
n
=0 (kt)
ω(kt)π(θ|(kt))
n

Latent variables
Example (Mixture once again (cont’d))
For a given permutation (kt), conditional posterior distribution
π(θ|(kt)) = N ξ1(kt),
σ2
1
n1 +
× I G ((ν1 + )/2, s1(kt)/2)
×N ξ2(kt),
σ2
2
n2 + n −
× I G ((ν2 + n − )/2, s2(kt)/2)
×Be(α + , β + n − )

Latent variables
Example (Mixture once again (cont’d))
where
¯x1(kt) = 1
t=1 xkt , ˆs1(kt) = t=1(xkt − ¯x1(kt))2,
¯x2(kt) = 1
n−
n
t= +1 xkt , ˆs2(kt) = n
t= +1(xkt − ¯x2(kt))2
and
ξ1(kt) =
n1ξ1 + ¯x1(kt)
n1 +
, ξ2(kt) =
n2ξ2 + (n − )¯x2(kt)
n2 + n −
,
s1(kt) = s2
1 + ˆs2
1(kt) +
n1
n1 +
(ξ1 − ¯x1(kt))2
,
s2(kt) = s2
2 + ˆs2
2(kt) +
n2(n − )
n2 + n −
(ξ2 − ¯x2(kt))2
,
posterior updates of the hyperparameters

Latent variables
Bayes estimator of θ:
δπ
(x1, . . . , xn) =
n
=0 (kt)
ω(kt)Eπ
[θ|x, (kt)]
Too costly: 2n terms

The AR(p) model
AR(p) model
Auto-regressive representation of a time series,
xt|xt−1, . . . ∼ N µ +
p
i=1
i(xt−i − µ), σ2

The AR(p) model
AR(p) model
Auto-regressive representation of a time series,
xt|xt−1, . . . ∼ N µ +
p
i=1
i(xt−i − µ), σ2
Generalisation of AR(1)
Among the most commonly used models in dynamic settings
More challenging than the static models (stationarity
constraints)
Diﬀerent models depending on the processing of the starting
value x0

The AR(p) model
Unknown stationarity constraints
Practical diﬃculty: for complex models, stationarity constraints get
quite involved to the point of being unknown in some cases
Example (AR(1))
Case of linear Markovian dependence on the last value
xt = µ + (xt−1 − µ) + t , t
i.i.d.
∼ N (0, σ2
)
If | | < 1, (xt)t∈Z can be written as
xt = µ +
∞
j=0
j
t−j
and this is a stationary representation.

The AR(p) model
Stationary but...
If | | > 1, alternative stationary representation
xt = µ −
∞
j=1
−j
t+j .

The AR(p) model
Stationary but...
If | | > 1, alternative stationary representation
xt = µ −
∞
j=1
−j
t+j .
This stationary solution is criticized as artiﬁcial because xt is
correlated with future white noises ( t)s>t, unlike the case when
| | < 1.
Non-causal representation...

The AR(p) model
Stationarity+causality
Stationarity constraints in the prior as a restriction on the values of
θ.
Theorem
AR(p) model second-order stationary and causal iﬀ the roots of the
polynomial
P(x) = 1 −
p
i=1
ixi
are all outside the unit circle

The AR(p) model
Stationarity constraints
Under stationarity constraints, complex parameter space: each
value of needs to be checked for roots of corresponding
polynomial with modulus less than 1

The AR(p) model
Stationarity constraints
Under stationarity constraints, complex parameter space: each
value of needs to be checked for roots of corresponding
polynomial with modulus less than 1
E.g., for an AR(2) process with
autoregressive polynomial
P(u) = 1 − 1u − 2u2, constraint is
1 + 2 < 1, 1 − 2 < 1
and | 2| < 1
q
−2 −1 0 1 2
−1.0−0.50.00.51.0 θ1
θ2

The MA(q) model
The MA(q) model
Alternative type of time series
xt = µ + t −
q
j=1
ϑj t−j , t ∼ N (0, σ2
)
Stationary but, for identiﬁability considerations, the polynomial
Q(x) = 1 −
q
j=1
ϑjxj
must have all its roots outside the unit circle.

The MA(q) model
Identiﬁability
Example
For the MA(1) model, xt = µ + t − ϑ1 t−1,
var(xt) = (1 + ϑ2
1)σ2
can also be written
xt = µ + ˜t−1 −
1
ϑ1
˜t, ˜ ∼ N (0, ϑ2
1σ2
) ,
Both pairs (ϑ1, σ) & (1/ϑ1, ϑ1σ) lead to alternative
representations of the same model.

The MA(q) model
Properties of MA models
Non-Markovian model (but special case of hidden Markov)
Autocovariance γx(s) is null for |s| > q

The MA(q) model
Representations
x1:T is a normal random variable with constant mean µ and
covariance matrix
Σ =





σ2
γ1 γ2 . . . γq 0 . . . 0 0
γ1 σ2
γ1 . . . γq−1 γq . . . 0 0
...
0 0 0 . . . 0 0 . . . γ1 σ2





,
with (|s| ≤ q)
γs = σ2
q−|s|
i=0
ϑiϑi+|s|
Not manageable in practice [large T’s]

The MA(q) model
Representations (contd.)
Conditional on past ( 0, . . . , −q+1),
L(µ, ϑ1, . . . , ϑq, σ|x1:T , 0, . . . , −q+1) ∝
σ−T
T
t=1
exp



−

xt − µ +
q
j=1
ϑjˆt−j


2
2σ2



,
where (t > 0)
ˆt = xt − µ +
q
j=1
ϑjˆt−j, ˆ0 = 0, . . . , ˆ1−q = 1−q
Recursive deﬁnition of the likelihood, still costly O(T × q)

The MA(q) model
Encompassing approach for general time series models
State-space representation
xt = Gyt + εt , (1)
yt+1 = Fyt + ξt , (2)
(1) is the observation equation and (2) is the state equation

The MA(q) model
Encompassing approach for general time series models
State-space representation
xt = Gyt + εt , (1)
yt+1 = Fyt + ξt , (2)
(1) is the observation equation and (2) is the state equation
Note
This is a special case of hidden Markov model

The MA(q) model
MA(q) state-space representation
For the MA(q) model, take
yt = ( t−q, . . . , t−1, t)
and then
yt+1 =






0 1 0 . . . 0
0 0 1 . . . 0
. . .
0 0 0 . . . 1
0 0 0 . . . 0






yt + t+1







0
0
...
0
1







xt = µ − ϑq ϑq−1 . . . ϑ1 −1 yt .

The MA(q) model
MA(q) state-space representation (cont’d)
Example
For the MA(1) model, observation equation
xt = (1 0)yt
with
yt = (y1t y2t)
directed by the state equation
yt+1 =
0 1
0 0
yt + t+1
1
ϑ1
.

The MA(q) model
c A typology of Bayes computational problems
(i). latent variable models in general

The MA(q) model
(ii). use of a complex parameter space, as for instance in
constrained parameter sets like those resulting from imposing
stationarity constraints in dynamic models;

The MA(q) model
(iii). use of a complex sampling model with an intractable
likelihood, as for instance in some graphical models;

The MA(q) model
(iv). use of a huge dataset;

The MA(q) model
(v). use of a complex prior distribution (which may be the
posterior distribution associated with an earlier sample);

The MA(q) model
(v). use of a complex prior distribution (which may be the
posterior distribution associated with an earlier sample);
(vi). use of a particular inferential procedure as for instance, Bayes
factors
Bπ
01(x) =
P(θ ∈ Θ0 | x)
P(θ ∈ Θ1 | x)
π(θ ∈ Θ0)
π(θ ∈ Θ1)
.

Monte Carlo basics
Importance Sampling
Monte Carlo Methods based on Markov Chains
The Metropolis–Hastings algorithm
Random-walk Metropolis-Hastings algorithms
Extensions
The Gibbs Sampler

Monte Carlo basics
General purpose
Given a density π known up to a normalizing constant, and an
integrable function h, compute
Π(h) = h(x)π(x)µ(dx) =
h(x)˜π(x)µ(dx)
˜π(x)µ(dx)
when h(x)˜π(x)µ(dx) is intractable.

Monte Carlo basics
Monte Carlo 101
Generate an iid sample x1, . . . , xN from π and estimate Π(h) by
ˆΠMC
N (h) = N−1
N
i=1
h(xi).
LLN: ˆΠMC
N (h)
as
−→ Π(h)
If Π(h2) = h2(x)π(x)µ(dx) < ∞,
CLT:
√
N ˆΠMC
N (h) − Π(h)
L
N 0, Π [h − Π(h)]2
.

Monte Carlo basics
Monte Carlo 101
Generate an iid sample x1, . . . , xN from π and estimate Π(h) by
ˆΠMC
N (h) = N−1
N
i=1
h(xi).
LLN: ˆΠMC
N (h)
as
−→ Π(h)
If Π(h2) = h2(x)π(x)µ(dx) < ∞,
CLT:
√
N ˆΠMC
N (h) − Π(h)
L
N 0, Π [h − Π(h)]2
.
Caveat announcing MCMC
Often impossible or ineﬃcient to simulate directly from Π

Importance Sampling
Importance Sampling
For Q proposal distribution such that Q(dx) = q(x)µ(dx),
alternative representation
Π(h) = h(x){π/q}(x)q(x)µ(dx).

Importance Sampling
Importance Sampling
For Q proposal distribution such that Q(dx) = q(x)µ(dx),
alternative representation
Π(h) = h(x){π/q}(x)q(x)µ(dx).
Principle of importance
Generate an iid sample x1, . . . , xN ∼ Q and estimate Π(h) by
ˆΠIS
Q,N (h) = N−1
N
i=1
h(xi){π/q}(xi).
return to pMC

Importance Sampling
Properties of importance
Then
LLN: ˆΠIS
Q,N (h)
as
−→ Π(h) and if Q((hπ/q)2) < ∞,
CLT:
√
N(ˆΠIS
Q,N (h) − Π(h))
L
N 0, Q{(hπ/q − Π(h))2
} .

Importance Sampling
Properties of importance
Then
LLN: ˆΠIS
Q,N (h)
as
−→ Π(h) and if Q((hπ/q)2) < ∞,
CLT:
√
N(ˆΠIS
Q,N (h) − Π(h))
L
N 0, Q{(hπ/q − Π(h))2
} .
Caveat
If normalizing constant of π unknown, impossible to use ˆΠIS
Q,N
Generic problem in Bayesian Statistics: π(θ|x) ∝ f(x|θ)π(θ).

Importance Sampling
Self-Normalised Importance Sampling
Self normalized version
ˆΠSNIS
Q,N (h) =
N
i=1
{π/q}(xi)
−1 N
i=1
h(xi){π/q}(xi).

Importance Sampling
ˆΠSNIS
Q,N (h) =
N
i=1
{π/q}(xi)
−1 N
i=1
h(xi){π/q}(xi).
LLN : ˆΠSNIS
Q,N (h)
as
−→ Π(h)
and if Π((1 + h2)(π/q)) < ∞,
CLT :
√
N(ˆΠSNIS
Q,N (h) − Π(h))
L
N 0, π {(π/q)(h − Π(h)}2
) .

Importance Sampling
ˆΠSNIS
Q,N (h) =
N
i=1
{π/q}(xi)
−1 N
i=1
h(xi){π/q}(xi).
LLN : ˆΠSNIS
Q,N (h)
as
−→ Π(h)
and if Π((1 + h2)(π/q)) < ∞,
CLT :
√
N(ˆΠSNIS
Q,N (h) − Π(h))
L
N 0, π {(π/q)(h − Π(h)}2
) .
c The quality of the SNIS approximation depends on the
choice of Q

Running Monte Carlo via Markov Chains (MCMC)
It is not necessary to use a sample from the distribution f to
approximate the integral
I = h(x)f(x)dx ,

Running Monte Carlo via Markov Chains (MCMC)
It is not necessary to use a sample from the distribution f to
approximate the integral
I = h(x)f(x)dx ,
We can obtain X1, . . . , Xn ∼ f (approx) without directly
simulating from f, using an ergodic Markov chain with
stationary distribution f

Running Monte Carlo via Markov Chains (2)
Idea
For an arbitrary starting value x(0), an ergodic chain (X(t)) is
generated using a transition kernel with stationary distribution f

Idea
Insures the convergence in distribution of (X(t)) to a random
variable from f.
For a “large enough” T0, X(T0) can be considered as
distributed from f
Produce a dependent sample X(T0), X(T0+1), . . ., which is
generated from f, suﬃcient for most approximation purposes.

Idea
Insures the convergence in distribution of (X(t)) to a random
variable from f.
For a “large enough” T0, X(T0) can be considered as
distributed from f
Produce a dependent sample X(T0), X(T0+1), . . ., which is
generated from f, suﬃcient for most approximation purposes.
Problem: How can one build a Markov chain with a given
stationary distribution?

Basics
The algorithm uses the objective (target) density
f
and a conditional density
q(y|x)
called the instrumental (or proposal) distribution

The MH algorithm
Algorithm (Metropolis–Hastings)
Given x(t),
1. Generate Yt ∼ q(y|x(t)).
2. Take
X(t+1)
=
Yt with prob. ρ(x(t), Yt),
x(t) with prob. 1 − ρ(x(t), Yt),
where
ρ(x, y) = min
f(y)
f(x)
q(x|y)
q(y|x)
, 1 .

Features
Independent of normalizing constants for both f and q(·|x)
(ie, those constants independent of x)
Never move to values with f(y) = 0
The chain (x(t))t may take the same value several times in a
row, even though f is a density wrt Lebesgue measure
The sequence (yt)t is usually not a Markov chain

Convergence properties
1. The M-H Markov chain is reversible, with
invariant/stationary density f since it satisﬁes the detailed
balance condition
f(y) K(y, x) = f(x) K(x, y)

balance condition
f(y) K(y, x) = f(x) K(x, y)
2. As f is a probability measure, the chain is positive recurrent

balance condition
f(y) K(y, x) = f(x) K(x, y)
2. As f is a probability measure, the chain is positive recurrent
3. If
Pr
f(Yt) q(X(t)|Yt)
f(X(t)) q(Yt|X(t))
≥ 1 < 1. (1)
that is, the event {X(t+1) = X(t)} is possible, then the chain
is aperiodic

Convergence properties (2)
4. If
q(y|x) > 0 for every (x, y), (2)
the chain is irreducible

4. If
5. For M-H, f-irreducibility implies Harris recurrence

4. If
5. For M-H, f-irreducibility implies Harris recurrence
6. Thus, for M-H satisfying (1) and (2)
(i) For h, with Ef |h(X)| < ∞,
lim
T →∞
1
T
T
t=1
h(X(t)
) = h(x)df(x) a.e. f.
(ii) and
lim
n→∞
Kn
(x, ·)µ(dx) − f
T V
= 0
for every initial distribution µ, where Kn
(x, ·) denotes the
kernel for n transitions.

Random walk Metropolis–Hastings
Use of a local perturbation as proposal
Yt = X(t)
+ εt,
where εt ∼ g, independent of X(t).
The instrumental density is of the form g(y − x) and the Markov
chain is a random walk if we take g to be symmetric g(x) = g(−x)

Algorithm (Random walk Metropolis)
Given x(t)
1. Generate Yt ∼ g(y − x(t))
2. Take
X(t+1)
=



Yt with prob. min 1,
f(Yt)
f(x(t))
,
x(t) otherwise.

Example (Random walk and normal target)
forget History! Generate N(0, 1) based on the uniform proposal [−δ, δ]
[Hastings (1970)]
The probability of acceptance is then
ρ(x(t)
, yt) = exp{(x(t)2
− y2
t )/2} ∧ 1.

Example (Random walk & normal (2))
Sample statistics
δ 0.1 0.5 1.0
mean 0.399 -0.111 0.10
variance 0.698 1.11 1.06
c As δ ↑, we get better histograms and a faster exploration of the
support of f.

-1 0 1 2
050100150200250
(a)
-1.5-1.0-0.50.00.5
-2 0 2
0100200300400
(b) -1.5-1.0-0.50.00.5
-3 -2 -1 0 1 2 3
0100200300400
(c)
-1.5-1.0-0.50.00.5
Three samples based on U[−δ, δ] with (a) δ = 0.1, (b) δ = 0.5
and (c) δ = 1.0, superimposed with the convergence of the
means (15, 000 simulations).

π(θ|x) ∝
n
j=1
k
=1
p f(xj|µ , σ ) π(θ)

π(θ|x) ∝
n
j=1
k
=1
p f(xj|µ , σ ) π(θ)
Metropolis-Hastings proposal:
θ(t+1)
=
θ(t) + ωε(t) if u(t) < ρ(t)
θ(t) otherwise
where
ρ(t)
=
π(θ(t) + ωε(t)|x)
π(θ(t)|x)
∧ 1
and ω scaled for good acceptance rate

p
theta
0.0 0.2 0.4 0.6 0.8 1.0
-1012
tau
theta
0.2 0.4 0.6 0.8 1.0 1.2
-1012
p
tau
0.0 0.2 0.4 0.6 0.8 1.0
0.20.40.60.81.01.2
-1 0 1 2
0.01.02.0
theta
0.2 0.4 0.6 0.8
024
tau
0.0 0.2 0.4 0.6 0.8 1.0
0123456
p
Random walk sampling (50000 iterations)
General case of a 3 component normal mixture
[Celeux & al., 2000]

−1 0 1 2 3
−10123
µ1
µ2
X
Random walk MCMC output for .7N(µ1, 1) + .3N(µ2, 1)

Uniform ergodicity prohibited by random walk structure

Uniform ergodicity prohibited by random walk structure
At best, geometric ergodicity:
Theorem (Suﬃcient ergodicity)
For a symmetric density f, log-concave in the tails, and a positive
and symmetric density g, the chain (X(t)) is geometrically ergodic.
[Mengersen & Tweedie, 1996]
no tail eﬀect

Example (Comparison of tail
eﬀects)
Random-walk
Metropolis–Hastings algorithms
based on a N (0, 1) instrumental
for the generation of (a) a
N(0, 1) distribution and (b) a
distribution with density
ψ(x) ∝ (1 + |x|)−3
(a)
0 50 100 150 200
-1.5-1.0-0.50.00.51.01.5
(a)
0 50 100 150 200
-1.5-1.0-0.50.00.51.01.5
0 50 100 150 200
-1.5-1.0-0.50.00.51.01.5
0 50 100 150 200
-1.5-1.0-0.50.00.51.01.5
(b)
0 50 100 150 200
-1.5-1.0-0.50.00.51.01.5
0 50 100 150 200
-1.5-1.0-0.50.00.51.01.5
0 50 100 150 200
-1.5-1.0-0.50.00.51.01.5
90% conﬁdence envelopes of
the means, derived from 500
parallel independent chains
1 + ξ2
1 + (ξ )2
∧ 1 ,

Further convergence properties
Under assumptions skip detailed convergence
(A1) f is super-exponential, i.e. it is positive with positive
continuous ﬁrst derivative such that
lim|x|→∞ n(x) log f(x) = −∞ where n(x) := x/|x|.
In words : exponential decay of f in every direction with rate
tending to ∞
(A2) lim sup|x|→∞ n(x) m(x) < 0, where
m(x) = f(x)/| f(x)|.
In words: non degeneracy of the countour manifold
Cf(y) = {y : f(y) = f(x)}
Q is geometrically ergodic, and
V (x) ∝ f(x)−1/2 veriﬁes the drift condition
[Jarner & Hansen, 2000]

Further [further] convergence properties
skip hyperdetailed convergence
If P ψ-irreducible and aperiodic, for r = (r(n))n∈N real-valued non
decreasing sequence, such that, for all n, m ∈ N,
r(n + m) ≤ r(n)r(m),
and r(0) = 1, for C a small set, τC = inf{n ≥ 1, Xn ∈ C}, and
h ≥ 1, assume
sup
x∈C
Ex
τC −1
k=0
r(k)h(Xk) < ∞,

then,
S(f, C, r) := x ∈ X, Ex
τC −1
k=0
r(k)h(Xk) < ∞
is full and absorbing and for x ∈ S(f, C, r),
lim
n→∞
r(n) Pn
(x, .) − f h = 0.
[Tuominen & Tweedie, 1994]

Comments
[CLT, Rosenthal’s inequality...] h-ergodicity implies CLT
for additive (possibly unbounded) functionals of the chain,
Rosenthal’s inequality and so on...
[Control of the moments of the return-time] The
condition implies (because h ≥ 1) that
sup
x∈C
Ex[r0(τC)] ≤ sup
x∈C
Ex
τC −1
k=0
r(k)h(Xk) < ∞,
where r0(n) = n
l=0 r(l) Can be used to derive bounds for
the coupling time, an essential step to determine computable
bounds, using coupling inequalities
[Roberts & Tweedie, 1998; Fort & Moulines, 2000]

Alternative conditions
The condition is not really easy to work with...
[Possible alternative conditions]
(a) [Tuominen, Tweedie, 1994] There exists a sequence
(Vn)n∈N, Vn ≥ r(n)h, such that
(i) supC V0 < ∞,
(ii) {V0 = ∞} ⊂ {V1 = ∞} and
(iii) PVn+1 ≤ Vn − r(n)h + br(n)IC.

(b) [Fort 2000] ∃V ≥ f ≥ 1 and b < ∞, such that supC V < ∞
and
PV (x) + Ex
σC
k=0
∆r(k)f(Xk) ≤ V (x) + bIC(x)
where σC is the hitting time on C and
∆r(k) = r(k) − r(k − 1), k ≥ 1 and ∆r(0) = r(0).
Result (a) ⇔ (b) ⇔ supx∈C Ex
τC −1
k=0 r(k)f(Xk) < ∞.

Extensions
Langevin Algorithms
Proposal based on the Langevin diffusion Lt is defined by the
stochastic differential equation
dLt = dBt +
1
2
log f(Lt)dt,
where Bt is the standard Brownian motion
Theorem
The Langevin diffusion is the only non-explosive diffusion which is
reversible with respect to f.

Extensions
Discretization
Instead, consider the sequence
x(t+1)
= x(t)
+
σ2
2
log f(x(t)
) + σεt, εt ∼ Np(0, Ip)
where σ2 corresponds to the discretization step

Extensions
Discretization
Instead, consider the sequence
x(t+1)
= x(t)
+
σ2
2
log f(x(t)
) + σεt, εt ∼ Np(0, Ip)
where σ2 corresponds to the discretization step
Unfortunately, the discretized chain may be be transient, for
instance when
lim
x→±∞
σ2
log f(x)|x|−1
> 1

Extensions
MH correction
Accept the new value Yt with probability
f(Yt)
f(x(t))
·
exp − Yt − x(t) − σ2
2 log f(x(t))
2
2σ2
exp − x(t) − Yt − σ2
2 log f(Yt)
2
2σ2
∧ 1 .
Choice of the scaling factor σ
Should lead to an acceptance rate of 0.574 to achieve optimal
convergence rates (when the components of x are uncorrelated)
[Roberts & Rosenthal, 1998; Girolami & Calderhead, 2011]

Extensions
Optimizing the Acceptance Rate
Problem of choice of the transition kernel from a practical point of
view
Most common alternatives:
(a) a fully automated algorithm like ARMS;
(b) an instrumental density g which approximates f, such that
f/g is bounded for uniform ergodicity to apply;
(c) a random walk
In both cases (b) and (c), the choice of g is critical,

Extensions
Case of the random walk
Diﬀerent approach to acceptance rates
A high acceptance rate does not indicate that the algorithm is
moving correctly since it indicates that the random walk is moving
too slowly on the surface of f.

Extensions
Case of the random walk
Diﬀerent approach to acceptance rates
A high acceptance rate does not indicate that the algorithm is
moving correctly since it indicates that the random walk is moving
too slowly on the surface of f.
If x(t) and yt are close, i.e. f(x(t)) f(yt) y is accepted with
probability
min
f(yt)
f(x(t))
, 1 1 .
For multimodal densities with well separated modes, the negative
eﬀect of limited moves on the surface of f clearly shows.

Extensions
Case of the random walk (2)
If the average acceptance rate is low, the successive values of f(yt)
tend to be small compared with f(x(t)), which means that the
random walk moves quickly on the surface of f since it often
reaches the “borders” of the support of f

Extensions
Rule of thumb
In small dimensions, aim at an average acceptance rate of 50%. In
large dimensions, at an average acceptance rate of 25%.
[Gelman,Gilks and Roberts, 1995]

Extensions
Rule of thumb
In small dimensions, aim at an average acceptance rate of 50%. In
large dimensions, at an average acceptance rate of 25%.
[Gelman,Gilks and Roberts, 1995]
This rule is to be taken with a pinch of salt!

Extensions
Example (Noisy AR(1))
Hidden Markov chain from a regular AR(1) model,
xt+1 = ϕxt + t+1 t ∼ N (0, τ2
)
and observables
yt|xt ∼ N (x2
t , σ2
)

Extensions
Example (Noisy AR(1))
Hidden Markov chain from a regular AR(1) model,
xt+1 = ϕxt + t+1 t ∼ N (0, τ2
)
and observables
yt|xt ∼ N (x2
t , σ2
)
The distribution of xt given xt−1, xt+1 and yt is
exp
−1
2τ2
(xt − ϕxt−1)2
+ (xt+1 − ϕxt)2
+
τ2
σ2
(yt − x2
t )2
.

Extensions
Example (Noisy AR(1) continued)
For a Gaussian random walk with scale ω small enough, the
random walk never jumps to the other mode. But if the scale ω is
suﬃciently large, the Markov chain explores both modes and give a
satisfactory approximation of the target distribution.

Extensions
Markov chain based on a random walk with scale ω = .1.

Extensions
Markov chain based on a random walk with scale ω = .5.

Extensions
MA(2)
Since the constraints on (ϑ1, ϑ2) are well-deﬁned, use of a ﬂat
prior over the triangle as prior.
Simple representation of the likelihood
library(mnormt)
ma2like=function(theta){
n=length(y)
sigma = toeplitz(c(1 +theta[1]^2+theta[2]^2,
theta[1]+theta[1]*theta[2],theta[2],rep(0,n-3)))
dmnorm(y,rep(0,n),sigma,log=TRUE)
}

Extensions
Basic RWHM for MA(2)
Algorithm 1 RW-HM-MA(2) sampler
set ω and ϑ(1)
for i = 2 to T do
generate ˜ϑj ∼ U(ϑ
(i−1)
j − ω, ϑ
(i−1)
j + ω)
set p = 0 and ϑ(i) = ϑ(i−1)
if ˜ϑ within the triangle then
p = exp(ma2like(˜ϑ) − ma2like(ϑ(i−1)))
end if
if U < p then
ϑ(i) = ˜ϑ
end if
end for

Extensions
Outcome
Result with a simulated sample of 100 points and ϑ1 = 0.6,
ϑ2 = 0.2 and scale ω = 0.2
q q
−2 −1 0 1 2
−1.0−0.50.00.51.0
θ1
θ2
q
qqq
q
qqq
qqqqq
q
qqq
qq qqqq
qq
qqqqqqqqqq
q qqqqqqqq
qqq
q
qqqqqqqq
q
qq
qqqqq
qq
qqq
qqqqqqqqqqq
q
qqqqq
q
qq
qqqq
qqqqqqq
qqqqq
qqq
qqqqqqqqq
q
q
qqqqqqqqqqqq
qq
qq
q
qqq
qqqqqqqq
qqqqqqq
qqqqq
qqqq
q
qqqqqq
qqqqqqq
qqqqq
qqq
qq
q
qqq
q
q
qq
qqq
qqq
qq
qq
qqqq
qq
q
qqq
qqq
q
q
qq
qqq
qqq
qq
qq
qqqqq
qqq
qqqqq
q
qqq
qqqqq
qq
qqqq
q
qqq
q
q
qq
qqqqqq
qqqqqqqqq
qq
q
qq
q
q
qqqqqqqq
qqqqqqqq
qqq
qqqqqqq
qqqqq
q
qqq
qq
qqqqqqqqqqqqq
qqqq
qqq
q
qq
q
qq
qqqq
q
qq
qq
q
qqqq
q
qq q
q
qqqqq
q
qqq
q
qq
qqqqqqq
q
qq
q
q
q
q
qqq
qqqq
q
qq
q
q
q q
qq
qq
qqqqqqq
qqq
q
qqqq
q
qqqqq
qq
qqq
qqq
q
q
q
qq
qq
qqqqqqqq
qqqqq
qqqqqq
qq
qq
qqqqqqq
q
qqq
q
qqqqqq
qqq
qq
qq
qqq
qqq
qq
qqqqqqq
q
qqq
qqqq
q
qqq
qq
q
qqqqqqqqqqq
q
qq
qq
q
qqqq
q
qqq
qqqqq
q
qqqqqq
qq
qqqq
qq
qqq
qqq
qq
qq
q
qqqq
qq
q
qqq
qq
qq
qqqqqqq
q
qqqq
qqq
q
q
qqqqqqqqqqq
q
q
qqq
qqqq
q
qqqq
qq
qq
qqqqq
q
qqqqq
qqqqqq
qq
q
q
q
q
qqq
q
qqqq
qq
qq
qqqqqqqqqqq
qqqq
q
qq
qqqq
qqqqqqqq
qq
qqq
qqq
q
q
qq
qq
qq
qqq
qqqqqq
q
q
q
q
qqqq
qq
qq
qqqq
qqq
qq
q
qqqqq
qqqqq
qqq
qq
qqq
q
q
qq
qqqq
qqqq
qq
qq
qqqqqqqqqqq
q
qqq
q
q
qqq
q
qq
qq
qq
q
qqq
q
q
qqqqqqqqqqqq
qqq
qqqqq
q
q
qq
q
qqqqq
q
q
qqqqq
q q
q
qqq
qqqqqqqqqqqqqqqqqqq
qq
qq
q
q
q
qq
qq
q
qq
qq
q
q
q
qqqqq
q
q
qqqqq
q
qq
q
qqq
q
qqq
q
q
q
q
q
qq
qqqq
q
qq
q
q
q
qqqqqqqqqq
q
qqqqqqqqq
qqqqqqqqqqqqqq
q
qq
qqqqqqqqqqqqqqqqqqqqqqqqq
qq
q
qq
qqqqqqqqq
qqqqqqqqqqqqqq
qqq
qq q
q
q
q
qqqqqqqqq
q
qq
q
qqqq
q
q
qqq
qqqqq
q
q
q
q
qqqq
qq
qq
qqq
q
qq
qqqq
q
qqqqqqq
qq
q
qq
qqqqqqqqqq
q
qqqqq
qqqqq
qqqq
qqqq
q
qqqqq
qq
qqqqqq
q
q
qq
q
qq
qq
qqq
qq
qqqq
qq
q
qq
qq
q
q
q qq
qq
qqq
q
qq
q
q
qq
q
qqqq
qqqqqqqqqqqqqqqqq
qqq
q
qq
qqq
qqq
qqqq
qqqq
qqq
qqqqqq
qq
qqqq
qqqq
q
qqq
q
qq
qqqqqqq
q
qq
qq
q
qqqq
qq
qq
q
qq
qqqq
qq
qq
qq
qq
qq
qqqqqqq
qqqqqqqqqq
qq
q
qq
qqqqq
qq
qqq
qq
qqq
q
qq
qq
qq
qqq
qq
q
q
q
qqqq
qqqqqqqq
qqq
qq
q
q
qqqqq
qq
qqqq
q
qqqqqqqq
qqq
q
q
qqqq
qq
qq
q
qqqqqqq
qqq
qq
q
q
qq
q
qq
q
qqq
q
q
q
qq
qq
qqqqqq
qqqq
q
qq
qqqq
qqqq
qqqq
q
q
qqq
q
q
qqqqq
q
qqq
qq
q
qq
q q
qqq
q
qqq
qqq
q
qqqqqq
q
q
qq
q
q
qqq
q
qqqq
qq
qqq
q
qq
qq
q
qqqqq
q
q
qqq
qqqq
qqq
qqq
qqqqqqqqqq
qqqqq
q
q
q
qqqq
qqqqqqqqqqqqqqqqqq
qqq
qq
qq
q
qqqqq
qqq
qqq
qqq
qqqqq
q
qqqqqqqqqqqq
q
qq
qqqqqqqqq
qqqqq
qqq
qqqq
qq
qqqq
q
qqqq
q
qq
q
qq
q
q
q
qqq
q
q
qq
q
qqqqq
q
q
q
q
qq
qqqqq
qqq
q
q
qqqqq
qq
qqqqq
q
qq
qqqqqqqqq
qqqqqqqq
qq
qqqqq
qq
qqqqqq
qqqqqqqqqqqqqq
qqq
qqqqqqqq qq
qqq
qqqqqqqqq
q
q
q
q
q
qqq
qq
qq qqq
q
q
qq
q
qq
q
qqqqq
qqqq
qqqq q
q
q
q
qq
q
qqq
q
qqqqqq
q
q q
qqqqqqqqq
qqq q
qqqq
qqqqqqqqqq
q
q
q
q qqqq
q
qqqq
qq
qq
qqqqq
qq
q
q
q
q
q
qqq
qq
qqqq
qq
q q q
qqqq
qqq
q
qqqq
qq
q
qq
qq
qqqqq
q
qq
q
q
q
qqq
qqqqqq
q
q
qqqqqqqqqqqqqq qq
qq
qq
qqqq
qqqqq
qqq
qqq
q
q
qq
qqqqqqq
q
qqq
q qq
q
qq
q
qq
qqqqqqqqqqqqq
qq
q
q
qqqqqqqqqq
qqq
qqq
qqqqqq
qq
qq
qqqqqqq
qqqqqqqqqqqqq
q
qqqq
qq
qq
q
qq
q
q
qqqq
qq
q
q
qqqq
q
qqqq
qq
qqqq
qq
qq
q
q
qqqqq
q
qq
qqqqqqqq
q
qqqqqqqq
q
qq
qqqqqqq
qqqqq
qqqq
qqqqq
qq
q
q
qqqqq
qqqqqqq
qqqqq
q
qq
qq
qq
qqqqqqq
q
qqqqqqq
qqqq
qqq
q
q
qqqq
qq
q
q
qq
qqq
q
qqq
qq
qqqqqqqqqqqqqqqqqq
q
qqq
qqqqqq
qqqq
qqqqqq
qqq
q
q
qqq
q
q
q
qqq
q
q
qqqqqqqqq
qqqq
qq
qqq
qq
qq
qqq
qq
q
qq
qqqqqqqqq
q
qq
q
qqqqqq
q
qqqqqq
qqqqqqqqqqqqqq
qqqq
q
qqqq
qqq
qqqqqq
qqq
q
qq
q
q
qq
q
q
qq
qqqq
q
q
qqq
qqq
q
q
qqqqqqq
q
q
qqqqqqqqq
q
qq
qq
qqqq
qq
qqqq
qqqqq
qq
qqq
q
qqq
q
qq
qqq
q
qqq
q
qq
q
qqq
qqq
qqq
q
qqq
q
qq
qq
qq
q
q
qqq
q
qqqqq
qq
qq
qq q
q
qq
qqq
qqqqqq
qq
qqq
q
qq
q
q
qqqqq
q
qqqqqqqqqqqqqq
qqqqq
qqq
q
qqq
qq
qqqqq
q
qq
q
qq
qqq
qqq
q
q
qq
qq
qq
qq
qq
q
qqqqqqq
qqqq q
qqqq
qq
qqqq
q
q
qq
qqq
qqq
qqq
qqqqq
qq
qq
q
qq
q
qqq
qq
qqq
qqq
qq
q
q
qq
qq
q
q
q
qq
qqqq
q
qq
q
qq
q
qqqqq
q
qqqqqqqqq
qq
q
qqq
qq
qq
qqq
qqqqqq
q
qqq
qqqq
q
qqqq
qqqqqqqqq
q
q
qq
q
q
qqqq
qqqqqq
qq
qq
q
qq
qqqqqqqqqqq
qq
qq
q
q
qqqq
qq
q
qqq
q
q
qq
qq
qq
qqq
qq
qqqq
qq
q
qq
q
qqqqq
qqqqq
q
q
qqq qqq
qqq qq
qqqqqqqqqqqqqqqqqqqq
q
q
q
q
qq
q
qqqqqq
qq
q
q
q
qqq
qq
q
qq
qq
q
qq
qqqq
qq
qq
qqqqqqq
qq
q
q
qqqq
q
qqq qq
q
qqqq
q
qqqqqq
qqqqqq
qqqqq
qqqqqqq q
qqqq
qqqqq
qq
qq
q
q
qqqqqqq
qqqqq
q
qqq
q
q
qqqqqq
qqq
qqq q
qq
qq
qq
qqqq
q
qqqqqqqqqq
qq
q
qqq
qqq
qqq
q
q
qq
qqq
qq
qqqqqqq
q
qq
qqqqqq
q
qqqqq
qq
qq
q
qqqq
qqqq
q
qqqqq
q
q
q
qqqq
q
qq
q
qqqqq
q
qqq
q
qqq
qqqqqqq
qq
qq
q
q
q
qqq
q
q
qq
q
qq
qqq
q
qqqq
qqqqqq
qqq
qqqq
qqqqqqqqq
qqqq
q
qqqq
qqqq
q
qqq
qqqqq
qqqq
q
q
qq
qqqqqqq
q
q
qqq
q q
qqq
qqqqqqqqq
q
qqq
qq
qqqq
qq
qqq
qqq
qqqq
qq
qqqq
q
q
qqq
q
qqqqqq
qqqqqq
qqq
qq
qqqqqqqqqqq
qq q
qq
qqqqqqqqqqqqq
qqqq
q
q
qq
qqqq
qqq
qqqq
qqq
q
qqqq
qq
qq
q
qq
qq
qqq
qqqqq
q
qqqqqqqqqqqqqqq
qq
q
qq
qqq
q
qq
qqqqqqqq
q
qqqqqq
qq
qq
qq
q
qqqqqqqqqqq
q
qqqqqqq
qq
qqqqq
q
qqqq
qq
q
q
qqqqqq
qq
qqqqq
q
qqq
qq
qqqqq
qqqqqqq
qq
qqq
q
q
q
qq
qq
qq
qqqqq
qqqq
q
q
qqqqq
q
qqq
q
q
qqq
qq
qqqq
qqqqqqq
qqq
qqq qq
qq
qq
q
qq
qqq
qqqqqq
qq
q
q
q
qqqqqqqqqq
qqqqqq
q
q
qqq
qqqqqq
qqqqqq
qqqqqqqq
qq
qqq
qq
qqqq
qqq
qqqq
qqqqq
qq
qqqqq
qqq
qqqq
q
qqq
qqqq qq
qqqqqqqqqqqqq
qqqq
qq
qqqqq
qqqqq
qq
q
qqqqqqqq
q
q
qqq
qqqqqq
qq
qq
q
qq
qqqqq
qqqq
qqqq
q
qqqqq
qq
qq
qqq
q
qq
qqqqqq
qqqq
q
qq
qqqqq
qqq
qqq
q
qq
q
q
qq
q
qq
qq
qqqq
q
q
qqqqq
qqq
qqqq
qqqqqq
q
qqqqqq
q
qqq
qqq
qqqqq
qq
qqqq
qqqqq
qq
qq
qq
q
q
qqq
q
qq
qq
qq
q
q
qq
q
qqqqq
q
qq
qq
q
qqqqqqqq
qqq
qq
qqqqq
qqq
qqqqq
qqqqqqqqqqqqqq
qqq
qqq
qqq
qq
q
qq
qqq
q
q
q
qq
q
qqqq
qqq
q
qqqqq
q
qq
qq
qq
q
qqqqq
qq
q
q
q
q
qq
qq
q
q
qq
q
qq
qq
qq
qqqqq
qq
q q
qq
q
qqqqqqqqqqqq
qq
q
qq
qqqqqq
q
q
qq
qq
qqqqqqqqq
qq
q
q
qq
q
qq
qqqqqq
qq
qqqqqqqq
q
qq
q
q
q
qqq
qq
q
q
qqq
qq
qqq
qq
qqq
qqq
q
q
qqq
qqqqq
qqqqqq
qqqqqq
qq
qqqqqqq
q
q
q
qqqq
qqqqqq
qqqqqqq
q
q
q
q
qq
qqq
q
qq
q
qqq
q
qq qq
q
qqqqq
q
qqqqqqqqqqq
q qqqq
qq
q
qqqq
qqq
qq
qqq
q
q
qq
qq
q
qq
qqq
q
qqq
qqq
qqqqqqqqqqqq
qqqqq
qqqq
qqqqqq qqqqqqq
qq
qqq
q
qq
qq
qq
qq
q
q
qqqqqq
q
q
q
qq
qqqq
qqqqq
qqqqqqqqqq
qqqq
qqqqqq
q
qqq
q
qq
qq
q
qq
qq
qq
q
q
qqqqq
qq
qqq
qq
qq
qqq
q
q
qq
qqqqqqqqq
q
qqqq
qqq
q
qqq
qq
qqqqq
q
qqqqqqqqqqqqqq qq qq
qqq
qqqq
q
qq
q
q
q
qqq
q
qq
qqqqq
qq
qqq
qqqqqqqqq
qq
q
q
q
q
qqq
qqqqq
q
q
qq
qqqqq
qqqq
qqqqq
q
qq
q
qqqq
qq
qqq
q
q
qqqqqqqqqqqqqq
qqqq
qqqqq
qq
qqq
qqq
qqqq
qq
qqqqq
q
q
q
qq
qqq
qqqqq
qqqqq
qqqqqqqq
qq
q
qqqqq
q
qqqqqqq
qqqqqqq
qq
qq
q
q
q
qq
q
q
qqqqq
qqqqqqq
q
qq
qq
qqqqqqq
q
qqqqqqqqqq
q
q
qqq
qq
qqq
qqqqqqqqqqqqqqqqqq
q
qqq
q
qqqqq
q
qqq
qq
q
q
qqqq
qq
q
qq
q
q
qqqq
q
qqqqqq
qqqqq
qqq
qqq
qq
q
q
qq
qq
qq
q
qq
q
q
q
qqq
q
q
qq
qqq
qqq
qq
qq
q
q
qq
qqq
qqqqqqqqqqqq
q
q
qqq
qqqqq
qq
q
qq
q
qqq
q
qq
qq
q
q
qqqqq
qqqqq
qqq
qq
q
q
qqq
q
qq
q
q
q
q
qqq
qq
q
qqqq
q
q
qqqq
q
qqqqq
qqq
qq
qqqq
q
qq
q
q
q
q
q
qqqq
q
q
q
qqq
qqqq
qqq
qqqq
q
q
q
qqq
qq
qqqqqq
qqq
qq
qqqqqq
qqqq
q
qqq
q
q
q
q
qqqqqqq
qqqq
qqqqq
q
qqqqq
qqqqqqqqqqqqqq
qqq
q
qq
qqqqqqqqq
qqq
qqq
qq
q
qqqq
qqqqq
qqq
qqqqqqq
qq
qq
qqqq
qq
qqqqqq
qqq
qqq
qqq
qqqqq
q
q
qq
qqq
q
qqqq
qqqq
qqqqqqqqq
qq
q
q
qqqq
qq
q
qqqq
qq
qqq
qq
qqq
q
qqqqq
q
qqqqqq
q
q
q
qqqq
q
q
qqq
qqqqq
q
q
qqqq qq
qqqq
qq
q
qq
qqqq
qq
qqq
qqqqq
q
qq
qq
qqqqq
qq
qqqq
qqqqqq
qqq
q
qqqqq
qq
qq
q
qqq
qqq
qqq
qqqqqq
qqqq
qqq
qqqqq
q
qqq
qqq q
qq
qq
q
qq
qq
q
q
q
qqq
qqqqqqqq
qq
qqqq
qqqqq qq
qq
q
qqq
qq
q
q
qqqq
qqq
qqqqqq
qqqqqq
qq
q
qqq
q
qqqqqqq
qq
q
q
qqqq
qqqq
q
q
qqqqq
q
qqqq
q
q
qqq
q
q
q
qqqqqqqqqq
qqqqq
qqqqqq
qq
qqq
qqqq
qqqq
qqqq
qq
qqqq
q
qqqqqqq
q
qqqqq
qq
q
qqq
q
qqqq
qqq
q
q
q
q
qqq
qq
qqqqqqqqqqqqq
qqq
q qqqqqq
qq
qqq
q
qq
q
qqqq
q
qq
qqqqq
qq
qqqqqqqq
qqq
q
q
qq
q
qqqq
qqq
qqqqq
qq
qqqqqqqqqq
qq
qqqqqqq
q
qq
qqqqqq
q
qqqqq
q
qqq
q
q
qq
q
qqq
qq
qq
q
q
q
qq
q
qq
q
qqq
q
qq
qqqqq
q
qqq
qqq
q
qqqqqqq
q
qqq
q
q
qqqqq
q
qqqqqq
q
qqqq
qqq
qqqqq
qqqqq
q
qqqqqq
qqqqq
qqq
q
qqqqq
qqqqqq
qq
q
qqqqqq
qqqq
qqqqqq
q
q
q
q
qqq
qq
qqq
qqqqqq
qqq
q
qqqqq
q
q
q
q
qqqq
qq
qq
qq
qqqqqq
q
q
q
qq
qq
q
qqq
qq
qqqqq
q
qqqqqqqqqqqqq
qqqqqqqq
q
qq
q
qq
qqq
qqq
q
q
qq
qq
qqqq
q
qqqqqqqq
q
q
q
qq
qqqq
qqq
qqqqqqqq
q
qqq
q
q
qq
qqqq
qqqq
qqqq q
q
qqq
qqqqqqqq
qqqqq
q
qqqqq
q
qqq
q
qqq
q
q
qq
q
qqqq qq
qq
qqqqq
qqq
qqqq
qqqqqqq
qq
qqqq
qqq
q
qq
qq
q
qqqqq
qqqqqqq
qqqqqq
qqqqqq
qq
qqqq
q
q
qqqqqqqqq qqqq
qqqqqq
q
q
qq
qq
q
qqqq
qqq
q
q
q
q
qq
q
qqqqqqq
q
qqqq
qq
q
qqqqqqq
qqq
qqqqqq
q
qqqqqqqqq
qqq
qqq
q
qqqq
qqqqqq
qqqqqq
qq q
qqqqq
qqq
q
qq
qq
qqqq
q
q
qqqqqq
q q
q
q
q
q
qqqq
q
qqqqqqq
qqq
qq
qqqqq
q
qqqq
qqq
qqqqq
q
qq
qq
qqq q
qqqq
qqqqqqqq
qqq
q
qqq
qqqqqqqqqqq
q
qq
qqq
qqqqqq
qqq
qqqq
qqq
qq
qqqqqqqqqq
qq
qq
qqqqqqqq
qqqqq
qq
qqqq qq
q
qq
q
q
q
qq
qqqqq
qqqqqqqqqq
q
qqq
qqq
qqqq qq
qqqq
qqqq
qq
qqqq
qqq
qqq
qq
qqq
qqqqq
qqqqqq
qq
qqqqqqqqqqqqq
qq
qq
qqqq
q
q
qqqqqq
q
q
qq
qqqqqq
qqqqqq
qqqqq
q qqq
qqqqq
q
qqqqq
qq
qqqq
q
qqqq
qqqq
qq
q
qqqqqqq
qq
q
q
qqqq
q
qqqqq
qqqqq
qqq
qqq
q
q
qqq
qq
qqq
qqqqqqq
qqq
q
q
qqqqqqq
q
q
qqqqq
qqq
qq
qqqqqqqqqq
qq
q
qqqq
q
qqqqqqqq
qq
qqqq
qqq
qq
qqq
qqqq
qqqq
qqqqqq
qqq
qqqq
qqqqq
qqq
qqqqq q
q
q
qqqq
q
qqqqqqq
q
q
q
q
q
qqqqq
qqq
qq
qq
qq
q
qqq
qq
q
qqqqqqq
qqqq
qqq
qqq
q
q
qqq
q
q
q
q
qq
qqq
q
qqqqqqq
qqqqqq
qqqqqqq
qqqqqqqq
qq
qqq
qq
q
qqqq
q
qqqq
qq
q
qqq
q
qq
q
q
qqqqqq
q
qq
q
q
q
q
q
qq
q
q
qqqqq
q
q
qq
q qqqq
qq
q
qqq
qqqq
qq
q
qq
qqq
qqqqqqqqqq
q
qqq
qqq
qqqq
q
qqqqqq
q
q
qqqqq
qq
qqq
qqqqqqq
qqqqq
qqq
qq
q
qqqqqqqqqq
qqqqq
q
qq
qqqq
q
q
qqqqq
qq
qq
qq
qqqqqq
qq
qqqq
q
qqqqqqqq
qq
qqq
qq
qq
qq q
q
q
qq
qqqqq
q
q
qqqqqqqqq
qq
q
q
q
qqq
qqqqq
qq
q
qqqq
qqq
qqqqqqqqqqq
q
q
qqqq
q
qqqq
qq
qq
qq
qq
qqqqq
q
qq
q
qqqqq
qqqq
qqqq
q
qqqqqqqqq qq
qqq
qq
q
qq
qqqqqqqqq
q
qqq
qqq
qqqq
qqq
qqqq
qqqqq
qqqq
q
qqqq
q
q
q
qq
qqqqqqq
qq
q
qq
q
q
q
q
qq
qqqqq
q
qqqqq
q
qqq
qqqqq q
q
qq
qq
qq
qq
qqqq
q
q
qq
qqqq
qq
q
qq
q
qqqqqqqqqqq
qqqqqqq
qqqqq
qq
qq
qqq
qqqq
qqqq
q
q
qqq
q
qqq
q
qq
q
qqqqqq
qqq
qq
qqqqqqqqqq
qqq
qq
q
q
qq
qq
qqqq
q
qq
q
qqq
q
qqqq
q
q
qq
qqqqq
qqqqqq
qqqqq
q
qqq q
qqqq
q
qq
qq
q
q
qqq
q
qq
qqqqqq
q
qqq
qqq
q
q
q
qqqqqqq
qq
q
q
qq
qqqqqq
qqqqqqqqqqqq
q
qq
qqq
qq
qqq
qq
qqq
qq
qqqq qqqqq
qqqqqqqq
qqq
qqqqqq
qqq
qqqqq
q
qqqq
q qqqqq
qqq
q
q
q
qqqqqq
qq
q
q
qq
qq
q q
qqqq
q
q
qq
q
qqqqqq
qqqqqq
qqq
qq
qq
qqq
q qqqqqq
q
qqqqqqqqqq
q
qqqqqqqqq
q
qq
qqqqq
qqqqqq
qq
qqq
qqqqqq
qqqq
qq qqqqqq
q
qq
qqqq
qqqqqqq
q
q
q
q
qqqqqqq
qq
q
q
qq
qqqq
qqqqq qqq
qqqqqqq
q
qqqqq
qqqqqqq
qq
q
q
qq
qqq
q
qqq
q
q qqqqq
qq
q
q
q
qqqqq
qqq
qqqq
q
qqq
qq
qq
qq
qqqqq
qq
qq
q
qqqq
qq
qqqqq qqqqqqqq
qq
qq
qqqq
q
q
qq
qqq
qqqq
qqqqq
q
qqqqq
qqq
q
qqqqqqq
q
qqq
q q
q
qqq
q
q
qq
qqqqqqqqqqqqqqqqqq
qqq
q
q
qqqqqq
qqqq
qqqqqqqqqqqqqqqq
qqq
q
q
qqqq
qqqq
qq
q
q
qq
qqqq
q
q
q
qqqqqqq
qqq
qqqqqqqq
q
q
qqqqq
q
qq
q
qqq
qq
qqqqqq
q
qqqqq
q
q
qq
qq
q
qqq
qq
q
qq
q
q
qq
qqqqqqq
q
qqqqqqqq
q
q
q
qqqq
q
q
q
q
qqq
qqqq
qq
q
qqqqqqqq
qqqqqqqq
qq
qqq
qq
q
qqq
qq
qqq
qqq
q
qqqq
qqqq
qqqqqq
qqqqq
qqq
q
q
qqq
qqqqqqq
q
qqq
qqqq
qqq
qq
qqq
qqqq
q
q
qqqqqqqqqq
qq
q
qq
qqqqqqq
qqqq
qq
qq
q
qq
qqqqqqq
qqq
q
qq
qqq
qqqqqqq
qqqqq
qqqqqq
q
qq
qqqqqqq
qq
q
q
qqq
qqqqqqq
qqqqq
qqqq
q
q
q
qq
qqqq
qq
q
q
qq
qqq
qq
qqqqqq
qqq
q
qqq
qqqqq
qq
q
qqqqqqqqq
q
qq
q
qqqq
qqqqqqqqq
q
qq
qqqqq
qq
q
qq
q
qqq
q
qqqqqqqq
qq
qqqqqqqq
qqqqqqqqqqqq
qqqq
q
qqq
qqqqq
q
qqqqqq
q
q
qq
qqqq
qq
q
qqqq
q
qqqq
qqqqq
q
q
q
qqq
qq
q
qqq
qq
q
q
qq
qq
q
qq
qq
qqqq
q
q
qq
qq
qqqq
qqq
q
qqq
qqqqq
q
qq
q
qqqq
q
qq
qqqqq
q
q
qqq
q
q
qqq
qqqqq
qqqqqqqqqqqqq
q
q
qqq
qq
q
qq
qqqqq
qqqq
q
qq
qq
qqq
q
q
q
q
qq
qq
qqqqqqqq
qqq
q
q
qq
qqqqqqq
q q
q
q
qq
q
qq
qqqq
qq
qqqq
q
qqq
qqqqqqqqq
qqqqqqqqqq
q
q
q
qq
qqqqqqq qqqq
qqq
qqqqqq
q
q
qqq
qq
q
qqqqq
qqqqq
q
qq
q
q
q
qqq
qq
q
q q
qqqqqqqqqq
qq
qq
qqqqq
q
qq
q
qqqq
qq
qqqq
qqq
qq
qqq
qqqq
qq
q
q
q
qqqqq
qqq
q
qq
qqqqqqqqqqq
qqqqqq
qqqqqqqq
qq
qq
qq
qqqqqq
qqq
qqq
q
q
q
qqq
qqq
qqqqqqqq
qqqqqqqq
q
qqq
q
q
qqq
qqqqqqqq
qqq
q
qq
qq
qqqqq
qqqq
qqqqq
q
qqqq
qqqqqqqqqqq
qq
qqqq
q
qq
q
qqq
q
q
q
qq
qqqq
qq
qqq
q
q
q
q
q
q
qqqqq
q
qq
qqqqqqqqq
qq
q
q
qq
qqqqq
qqqqqqqqqq
qq
qqq
qq
qqqq
qq
qq
q
qqq
qq
qq
qqqqqq
qqqqq
qqq
qqqqqqq
qqqqqq
qq
q
q
qqqqqqqqq
qqqqq
q
qqqqqq
q
qqqqqqqqqqqq
qqqq
q
q
qq
qq
qqqqqqq
qqqqqqqqq
qq
q
qq
q
qq
q
q
qq
q
q
q
qqqqqqqqq
qq
q
qqqq
qq
qq
q
qqqq
qq
qqqqqqqqqqq
q
qq
qqqqqq
q
q qqqqqqq
qqq
q
q
qqqqq
q
qq
q
qqqq
q
qq
qqqqqqqq
qqq
qqq
q
qq
qq
q
q
q
q
qq
qqq
qqq
q
qqqqqqq
qqqqq
qq
q
q
qqqq
qq
qqq
q
q
qqq
qq
qq
qq
q
qq
qqq
qq
q
qq
q
q
qqq
qqq
qq
qqq
qqqqqqqqq
q
qqqqq
q
qqqqqqqqqqqq
qqq
q
qq
qqqqqqqq
q
qqq
qqq
qqqqq
q
qqq
qq
qqqqqq
qqqqqq
qq
q
qqqq
qq
qq
q
qq
q
qqqqqqqq
qqq
qqqqqqqqq
qq
qqqqq
qqqq
q
q
qqqqq
qqqqqqq
q
qqq
qqqq
q
qq
q
qq
q
qqqqqqqqq
qqq
qqqqq
q
q
q
qqqqqqq
q
qqqq
qqqq
q
q
q
q
qqqq
qqqqqq
q
q
q
qq
qqqq
qq
qq
qqqqq
qq
q
q
q
qq
qq
qqqqqqq
qqqqq
qqqqqqq
q
qqq
qqq
q
qq
qqq
q
qq
qqqqq
q
qqq
q
q
q
q qq
qqqqqqq
qqqqq
q
q
qqqq
qqqq
q
qqq
q
q
qqqq
q
qqq q
qqqqqqq
qqqq
qqqqq
q
qq
qq
qqq
q
q
qqqq
q
qq
qq
q
q
q
qq
qq
qqqqq
qq
q
qq
q
qq
qqq
qqqq
qqq
qqqq
qqqqqqqqq
q
qqqqqqqqqqqqq
q
qqq
qqqqqqqqqqqq
q
q
q
qqq
q
qqqq
qqq
qq
qq
qq
qq
q
q
qqq
q
qqqq
qqqqqq
q
q
qqqqqq
qq
qq
q
q
qqqqq
q
qqqqqqq
q
q
qq
qqq
qqq
qqqqqqqqqqqqqq
qqqqq
qqqqqqqq
qqqq
qq
q
qqqqqqqq
q
qq
qqqqq
q
q
qq
q
qq
q
q
qq
qqqq
q
q
q
qq
qqqq
qqqq
qqqq
q
q
qqq
q
qqqqqqq
qqqq
q
qqqqqqqq qq
q
q
qqq
qq
qqqqq
qqqqqqq
qq
qqqqqq
qqq
qq
q
q
qqqqqq
q
qqqqq
q
qqqqqqq
qq
q
qq
q
qqqqq
q
q
qqqqqqq
qqqqq
q
qq
qqq
qq
q
q
qq
qq
q
q
qqq
qqqq
qqq
qqq
qqq
qqqqq
qq
qq
qq
q
qq
qqq
q
q
q
qqqqqqqq
qqq
q
qqqqqqqqqq
qq
qqqqq
qqqqq
qq
qq
qqq
qqq
q
q
q
q
qq
q
q
qqqqqqq
qq
q
q
qqq
q
qqqqqqqqqqqqq
q
qq
qqqqq
qq
qqqqqqqqqq
qqq
qqqq
qqqqqq
qq
qqqqqqqqq
q
q
qqq
q
qq
qq
qqqqqqq qqqqq
q
qqqq qq
q
qq
q
q
q
q qqqq
qqq
q
q
qqqq
q
q
q
qq
q
qqq
qqq
qqqq
qqqqqqqqqqqqqq
qqq
q
qqq
q
q
qq
qqqqq
qqqqqqqqqq
qqqqqqqqqqq
qq
qqq
qq
qqq
qqq
qqqqqq
qqqq
qq
qq
q qq q
qq
qqqqq
q
qqqqq
qq
qq
qqqqqqq
qqqqqq
qq
q
qq
qqqq
qqq qqq
qqqq
qqq
qqqqq
q
q
qqqqqqqq
q
q
q
q
q
qq
qqq
q
qq
qqqqqq
q
qqqqq
qqq
qqqq
qqqq
q
q
qqqqqq
qqqq
qqqqqq
qqqqqq
qq
q
q
qq
q
q qq
qqqqqqqqqqqqqq
qq
qqq
q

Extensions
Outcome
ϑ2 = 0.2 and scale ω = 0.5
q q
−2 −1 0 1 2
−1.0−0.50.00.51.0
θ1
θ2
qqqqqqq
qqqq
qqqqq
qq
qqq
q
q
q
qqqqqqqqqqqqqqqqqqq
qq
qq
qqqqqqqqq
qqqqqqqqqqqqq
qq
qqqqqqqqq
q
qq
q
q
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqq
qqqqqqqqqqqqqqq
qqqqqqq
qqqqqqqqq
qqqqqq
qqqq
qqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqq
qqqqqqqqqq
q
qqqqqqqqqq
qqqqqq
qqqq
q
qqqqqqqq
qqq
qqqqqqqqqq
qqqqqq
qqqqqqqqqqqqqq
qqq
q
qq
qqqqqqqqqqqqqqqq
qqqq
qqqqq
qq
qqq
qqqqqqqq
qqqqqqqqq
qqqqqqqqqq
qqqqq
q
qqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqq
qqqqqqqqqqqqq
qqqqqqqqqq
qqqq
qq
qqqqq
qqqqqq
qqqq
qqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqq
qqqqqqqqq
qqqqqqq
qqq
qqqqqqqqqqqqqq
qqqqqq
qqqqqq
qqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqq
q
qqqqqqqqq
qqqqqq
q
qqqqqqqqqqq
qqqqqqqqqq
qqq
qqqqqqqqqqqqqqqq
qqq
qqqq
qqq
qqqqqqqqqqqqqqqqqqqqqqqqqqq
q
q
qq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqq
qqq
qqqqqqqq
qqqqqqqqq
qqqq
qqqqqq
qqq
qqqqq
qqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqq
qqqq
qqqqqq
qqqqq
qqqqq
q
qq
qqqqqqqq
q
qqqqqqq
qq
qqqqqqqqqqqqq
qq
qqq
qqqqqqqqqq
qqq
qqqqqq
qqq
qqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqq
qqqqq
qq
qqq
q
qqqqq
qq
qqqqqqqqqqqq
qqq
qqqq
q
qqqq
q
qqqqq
qqqqqqqqqq
qq
qqqq
q
qqqqqq
qqq
qqq
q
qqqqq
qqq
qqqqqqqqqqqqq
qqqq
qqqqqq
qq
qqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqq
qq
q
q
qqq
qqqqqqqqqqqqq
qqqqqq
qqqqq
qqqqqq
qqq
qqqqqqq
qqqq
qqqqqqqqqqqqqqqqqqq
qqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqq
qqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqq
q
qqqqqqqqqqqqqqqqqq
qqqqq
qqqqqqqqqq
qqqqqqqqqqqqqqqqq
q
qq
qqqq
qqqqqqqq
qqqqqqqqqqqqqqq
qq
qqqqqqqqqq
qq
qqqqqq
qq
qqqqqqqqqqq
qq
qqqqq
qqqq
q
qqq
qqq
qqqqqqqqqqqqqqqqqqq
q
qqqq
q
qqqqqqqqqqqqqqq
qqqqqqq
qqqqqqqqq
qqqqqqqqqq
qqq
qqqqqqqqqqqqq
qqqqq
qqqqqqqqqqqqqq
qqqq
qqqqqqqqqqqqqqqqqqqqqqqq
qq
qqqq
qqq
qqqqq
qqqqqqqqqqqqqq
qq
qqqq
qqq
qqq
qqqqqqqqqqqqqqqqqqqqqqq
qqq
qqqqqqq
qqqqqqqqqqqqqqq
qqq
q
q
q qqqqqqqq
qqqqqqq
qqqqqqqqqqq
qqqqq
qq
qqqqqq
qq
q
qqqqq
qqqqqq
qqqqqqqqqq
qqqqqqqqq
qqqqqqqq
qqqqqqq
qqq
qqqqqqqq
qqqqqqqqqqqqq
qqqqqqqqqqqqqqq
qqq
qqqqqq
qqq
qqqqqqqq
qqqqqqqqqqqqq
qqqqq
qqqq
qqqqqqqqqqqqqq
qqq
q
qq
qqq
q
qqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqq
qqqqqq
qqqq
qqqqqq
qqq
qqqqqqq
qqqqq
qqqqqqqqqq
q
qqqqqqqqqqqqqqqqq qqqq
qqqqqqqqqq
qqqqqqq
qq
qqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqq
qqq
qqq
qqq
qqqqqqqqqqq
qqqqq
qqqqqqqq
qqq
qq
qqqqq
qq
qqqq
qqqqq
qqqqqqq
qqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqq
qqqqqqqqqqqqqqqq
qq qqq
qqqqq
qqqqqqqqqqq
q
qqqqqqq
qqqqqqqqqqqqq
qqq
qqq
qqqqqqqq
qq
qqqq
qqqqqqqqqqqqq
qqqqqqqqq
qq
qq
qqqq
qqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqq
qq
qqqqqq
qqqqqqqqqqqqqqqqqq
qqqqq
qqqqqqqqqqqqqqqqqq
qqqq
q
qqqqq
q
q
qq
q
qqqqqq
qqqqq
qq
qq
q
q
qqqqqqq
qqq
qqqqqqqqqqqqqqqqqqq
q
qqq
qqqqqqqqqqqqqq
qqqqqqqqq
q
qqq
qqqqqqq
qqqqqqq
qqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqq
qq
q
qqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqq
qqqqq
qqq
qqqqqqqqqqqqq
qqqqqqqqqqqqqqqqq
qqqq
q
qqqq
q
qqqqqqqqqqqqq
qqqqqqqqqqq
qqqqqqqq
qqqqqqqqq
qqqqqqqqqqq
qqqqqqqqqqqqqqq
q
qqqqqqq
qq
qqqqqqqqqq
qqq
qqqqqqqqqqqq
qqqqqqqqqqqq
qqqqqqqqqqq
qq
qqqqq
qqq
qq
q
qqqqq
qqq
qqqqq
qqqqqq
qqqqqqqqqqqqq
qqqqqqq
qq
qqqqqqqqqqq
qqqqq
qqqqqqqq
qqqqq
qqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqq
qqqqqqq
q
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqq
qqqq
q
qqq
qqqqqqqqqq
q
qq
qqqqqqqqqq
qq
qqqqqqqqqqqqqq
qq
qqqqqqqqqqqqqqqqq
qqq
q
qqqqq
qqqq
qqqq
qq
q
q
q
qqqqqqqq
qqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qq
q
qqqqqqq
qqqqqqq
qqqq
qqqqqqqqq
qq
qq
qqqq
q
qqq
qqqqqqq
qqqqq
qqq
qqq
qqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqq
qqqqqqqq
qqqqqqqqqqqqq
qqqq
qq
qqq
qqqq
qqqqqq
qqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqq
qqqqqqq
qqq
qq
qqqq
qqq
qqqqqqqqqqqqq
qqqqqqq
qq
qqqqqq
qqqqqqq
qqqqqqqqqqqqq q
qqqqqqqq
qqqqqqqqqqqqqqq
qq
qqq
q
qq
qqqqqqqq
qqqqqqq
qqqqqqqqqqqq
qqqqqqqqqqqqqqqqqq
qqqq
q
qqqqqq
qqqqqqqqqqqq
qqq
q
qq
qqqq
qqqqq
qqq
qq
qqqqqq
qqq
q
qq
qqqq
qqqqq
qqq
qqqqqqqqqqq
qq
qqqqqqqqqqqqqq
qqqqqqqqqqqqqqqq
qq
q
qqqqq
qqqq
qq
qqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqq
qq
qqqqqqqqqq q
qqqqqqqqqqqqqqq
qq
qqq
qqqqqqqqqqqqqqqqq
qq
qqqqqq
qqqqqqqqqqqqqqqqqqq
qqq
qqqqqq
qqqqqqqqqqqqqq
qqqq
qqqqqqq
q
qqqqq
q
qqqqqqqqq
qqq
qqq
qqqqqqqq
qqqqqqqqqqqqqqqqqqq
q
qqqq
qqqqq
qqqq
qqq
qq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qq
q
qqqqqqqqq
qqqqqqq
qqqqqqqqqqq
qq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
q
qq
q
qq
qqq
qqq
qqqqqqqqq
qqqqqqqqq
q
qqqqqqq
qqqqq
qqq
qqqqqqqq
qqqqqqqqqqqqqqqqqqq
q
qqq
q
qqqqqqqqqqqqqq
qq
qqqqqqq
qqqqq
qqqqqqqqqq
qq
qqqqqq
qqqqq
qqqqqqqqqqqqq
qqqqqq
qqqqqq
qqqqq
qqqqqqqqqq
qqqqqqqqqqqq
qqqq
q
qqqqqqq
q
qq
qqqqqqqqqqqqqqq
qqq
qqqqqqqqqq
qqqq
qq
qq
qqqqqqqqqqqqqqqqqq
qqqqqqq
qqqq
qqqqqqqqqqq
qq
qqqqqqqqqqqqqqq
qqqqqqqqqq qqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqq
qqq
qqqqqqq
q
qqqq
qqqq
qq
qqqqqq
qqqqqq
qqq
q
qqqqqqq
qqqqqqqqqqqqqqq
qqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqq
qqqq
qqqqqqqqqq
qqq
qq
qqqq
qqqqqqqqq
qqqqqqq
qqqqqq
qq
q
qqq
qqqqqqq
qqqqqqqqq
qqqq
qqq
qqqqqqqqq
qqqq q
qqqqqqq
qqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqq
qq
qqqqqq
qqqqq
q
qqqqq
qqq
qqqqqqqq
qqq qqq
qqqqqqqqqqqqqqqqqqqqqqqqq qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
q
qqqqqqqqqqqqqqqq
qqqqqq
qq
qqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqq
qq
qqqq
qqqq
q
qqqqqqqqq
qqqqqqqq
qq
qqq
qqqqqqqq
qqqqqqqq
qqqqqqq
qq
qqqqqqqqqqq
qqqqqqqqqqqqqqqq
qqqqqqqqqqq
qqqqqqqqqq
qq
qq
qqqqqqqqqqqqqq
qqqq
qq
q
qqqqq
qq
qqqqqqqqqqqqq
qqqqqqqqqqqqqq
q
q
qqq
qqqqqq
qqqqq
qq
qqqqqqq
qqqqq
qqqqq
qqqqqqqqqqqqq
qqqqqqqqqq
qqqqq
q
qqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqq
qqqqqqq
qqq
q qq
qq
q
qqq
q
q
qqqqqqq
qq
qqqqqqqqq
q
qqq
qqqqq
qqqqq
qqqqqqqqqq
q
qqqq
qqqqqqqqqqqqqqqq
qqqq
qqqqq
qqqqq
qqqqqqqqqqqq
qqqqqq
qqqqqq
qqqqq
qqqqqqqq
qqq
qqq
qqqqqqqqqqqqqq
q
qqqq
qqqq
qqqqqq
qqqqqqqqqqqqqqqqq
qqqqqqqq
qqqqqqqqqqqqq
qqqqqqqqqq
qqqqqqqqqqqqqqqqqq
qq
qqqqq
qqqqqq
qqqqqqqqqq
qq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqq
qqq
qqqqqqqqqqq
qqqqq
qqqqq
qqq
qqqqqqqqqqqq
qqqqqqqqq
qq
qqqqqqqqqq
qqqqqqqqqqqqqq
qqqqq
qq
qqqqq
qqq
qqqqqq
qqqqqqqqqqqq
q
qqqqq
qqqqqq
qqqqqqq
qqqq
q
q
qqqq
qqqqqqqq
q
qqqqqqq
qqqqqqqq
q
qqqqqqqqq
qqqqqqqqq
qqqqqqq
qqqqqqqqq
qqqqqqqqqqqqqqq
qq
qqqqqqqqqqqqq
qqqqq
qq
qqqqqqqqq
qqqqqq
qqqqq
q
qqqqqq
qqqqqqqq
qqqq
qqqqqqqqqqqqqq
qqq
qq qqqqqqqqqqqqq
qq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqq
qqqq
qqqqqqqq
qqqqq
qqqqqqq
qqqq
qqqqq
qq
qqqqqqqqqqqqqq
qqqqqqqq
qqqqqqqqqqqqq
qqqqqqqqq
qqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqq
qqqqqqqqqqqqqqqq
qqqqqq
qqqqqqq
q
qqqqqqqq
q
qqqqqqqq
qqqqqqq
qqqqqq
qqqq
qqq
qqqq
qqq
qqqqqqqqq
q
qqqq
qq
qqqqqqqqqqq
qqqq
qqqq
qq
qqqqqqq
qqqqqqqqqq
q
q
qq
qqq
qqqq
q
qqqq
qqqqqq
qqq
q
qqqqq
qqq
qqq
qqqqq
qqqqqqqqqqq
qqqqqqqqqq
qqqqqqqqqq
qqqqq
qqqqq
q
qq
qqqqqqq
qq
qqqqqqqqqqqqqqqqqq
qqqqqqqq
q
q
q
qqqqq
qqqq
q
qqqq
qqqqq qqqqqqqq
qqqq
qqqqqqq
q
q
qqqqqqq
qqqqqqqq
qqqqqq
qq
qqqq
q qqqq
qqqqqqq
qqqqqqq
qq
qqqqqqqqqq
qqqqqq
q
qqqqqqqqqqqqqqqqqqq
q
qqqq
q
qqqqqqqqqq
qqqqqqqqqqqqq
qqq
qq
q
qqqqq
q
qqqqqqqqqqqqqq
qqqq
qqq
qqqqqqq
qqqqqqq
qq
qqqqqqqq qqqqqq
qqqqqqqqqq
q
qqqqq
qqqqqq
qqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
q
qqqqqqqqqqqqqqqqq
q
qq
qqqqqqqqq
qqqqqqqq
qqqqqqq
qq
qqqqqqqq
qqqqqq
q
qqqq
qqqqqq
qqqq
qq
qqq
q
q
qqqqqqq
qqqqqqqq
qqqqq
qqqqq
qqqqqq
qqqqq
qqqq
q
qqqq qqqqq
qqqqqqqqqqqqq
qqqqqqqqqqqqqqq
qqqq
qqqqqqqqqqqqqqqqqq
qqqqqqq
q
qqq
qqqqqqqqqqqqqqqqq
q
q
qqqqqq
qqqqqqqqq
qqqqqqqqqqqqq
qqq
qqqq
qqqqqqqqqqqqqqqq
q
qqqqqqq
qqqqqq
qqqqqqq
qqqqqq
qqqqq
qqqqqqq
qqqqqq
qqqqq
q
qqqqqq
q
qqqqqqqqqqq
q
qqqqqqq
qqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqq
qqqqqqqq
qqq
qqqq
qqqqqqqqqqqqqqqqqqq
qqqqqqqqq
qqqq
qqqqqqqq
qqq
qq
qqq
qqqqqqqqqqqqqq
qq
qqqqqqqqqqq
qqq
qqqqqqqqqqqq
qqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqq
q
qqq
qqq
qqqq
qqqqqqq
qqq
qqqqqqqqq
qqqqqqq
qqqq
q
q
qqqqqqqqqqqqqqqqq
qqqqqqqqqqqqq
q
q
qq
qqqqqqqqqqqqqqqqqqq
qq
qq
qqqqq
qqqqqqqqq
qq
qqqqqqqqqqqq
qqqqqqqqqq
q
qq
qqqqqqqqq
q
qqqqqqq
qqqqqqqqqqqqqqq
qqqq
qq
qqq
qqqqqqqq
qq
qqq
qqqqqqqqq
qqqqqqqqqqqqqqqq
qqqqqq
q
q
q
qq
qqqqqq
qq
q
qqqqqqqqqqqqq
qqq
qqqqqqqqqqqqq
qqqqq
qqqqqqq
qqqqqqqqqq
qqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqq
qq
qqqqqq
qqqqqqqqqqqqqqqqq
q
qqqqq
qqqqqqqqqqqqqqqqqq
qqqqqq
qqqqqqq
q
qqqqqq
qqqqqq
q
qqqq
qqqqqqqqqqq
qqqqqqqqqq
qqq
qqqqqqqqqq
qqqqq
qq
qqq
qqq
qqqqqqq
q

Extensions
Outcome
ϑ2 = 0.2 and scale ω = 2.0
q q
−2 −1 0 1 2
−1.0−0.50.00.51.0
θ1
θ2
qqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
q
qqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq

The Gibbs Sampler
The Gibbs Sampler
skip to population Monte Carlo
The Gibbs Sampler
General Principles
Completion
Convergence
The Hammersley-Cliﬀord theorem

The Gibbs Sampler
General Principles
General Principles
A very speciﬁc simulation algorithm based on the target
distribution f:
1. Uses the conditional densities f1, . . . , fp from f

The Gibbs Sampler
General Principles
General Principles
distribution f:
2. Start with the random variable X = (X1, . . . , Xp)

The Gibbs Sampler
General Principles
General Principles
distribution f:
2. Start with the random variable X = (X1, . . . , Xp)
3. Simulate from the conditional densities,
Xi|x1, x2, . . . , xi−1, xi+1, . . . , xp
∼ fi(xi|x1, x2, . . . , xi−1, xi+1, . . . , xp)
for i = 1, 2, . . . , p.

The Gibbs Sampler
General Principles
Algorithm (Gibbs sampler)
Given x(t) = (x
(t)
1 , . . . , x
(t)
p ), generate
1. X
(t+1)
1 ∼ f1(x1|x
(t)
2 , . . . , x
(t)
p );
2. X
(t+1)
2 ∼ f2(x2|x
(t+1)
1 , x
(t)
3 , . . . , x
(t)
p ),
. . .
p. X
(t+1)
p ∼ fp(xp|x
(t+1)
1 , . . . , x
(t+1)
p−1 )
X(t+1) → X ∼ f

The Gibbs Sampler
General Principles
Properties
The full conditionals densities f1, . . . , fp are the only densities used
for simulation. Thus, even in a high dimensional problem, all of
the simulations may be univariate

The Gibbs Sampler
General Principles
Properties
The full conditionals densities f1, . . . , fp are the only densities used
for simulation. Thus, even in a high dimensional problem, all of
the simulations may be univariate
The Gibbs sampler is not reversible with respect to f. However,
each of its p components is. Besides, it can be turned into a
reversible sampler, either using the Random Scan Gibbs sampler
see section or running instead the (double) sequence
f1 · · · fp−1fpfp−1 · · · f1

The Gibbs Sampler
General Principles
A Very Simple Example: Independent N(µ, σ2
)
Observations
When Y1, . . . , Yn
iid
∼ N(y|µ, σ2) with both µ and σ unknown, the
posterior in (µ, σ2) is conjugate outside a standard familly

The Gibbs Sampler
General Principles
A Very Simple Example: Independent N(µ, σ2
)
Observations
When Y1, . . . , Yn
iid
∼ N(y|µ, σ2) with both µ and σ unknown, the
posterior in (µ, σ2) is conjugate outside a standard familly
But...
µ|Y 0:n, σ2 ∼ N µ 1
n
n
i=1 Yi, σ2
n )
σ2|Y 1:n, µ ∼ IG σ2 n
2 − 1, 1
2
n
i=1(Yi − µ)2
assuming constant (improper) priors on both µ and σ2
Hence we may use the Gibbs sampler for simulating from the
posterior of (µ, σ2)

The Gibbs Sampler
General Principles
R Gibbs Sampler for Gaussian posterior
n = length(Y);
S = sum(Y);
mu = S/n;
for (i in 1:500)
S2 = sum((Y-mu)^2);
sigma2 = 1/rgamma(1,n/2-1,S2/2);
mu = S/n + sqrt(sigma2/n)*rnorm(1);

The Gibbs Sampler
General Principles
Example of results with n = 10 observations from the
N(0, 1) distribution
Number of Iterations 1

The Gibbs Sampler
General Principles
Number of Iterations 1, 2

The Gibbs Sampler
General Principles
Number of Iterations 1, 2, 3

The Gibbs Sampler
General Principles
Number of Iterations 1, 2, 3, 4

The Gibbs Sampler
General Principles
Number of Iterations 1, 2, 3, 4, 5

The Gibbs Sampler
General Principles
Number of Iterations 1, 2, 3, 4, 5, 10

The Gibbs Sampler
General Principles
Number of Iterations 1, 2, 3, 4, 5, 10, 25

The Gibbs Sampler
General Principles
Number of Iterations 1, 2, 3, 4, 5, 10, 25, 50

The Gibbs Sampler
General Principles
Number of Iterations 1, 2, 3, 4, 5, 10, 25, 50, 100

The Gibbs Sampler
General Principles
Number of Iterations 1, 2, 3, 4, 5, 10, 25, 50, 100, 500

The Gibbs Sampler
General Principles
Limitations of the Gibbs sampler
Formally, a special case of a sequence of 1-D M-H kernels, all with
acceptance rate uniformly equal to 1.
The Gibbs sampler
1. limits the choice of instrumental distributions

The Gibbs Sampler
General Principles
The Gibbs sampler
2. requires some knowledge of f

The Gibbs Sampler
General Principles
The Gibbs sampler
3. is, by construction, multidimensional

The Gibbs Sampler
General Principles
The Gibbs sampler
3. is, by construction, multidimensional
4. does not apply to problems where the number of parameters
varies as the resulting chain is not irreducible.

The Gibbs Sampler
Completion
Latent variables are back
The Gibbs sampler can be generalized in much wider generality
A density g is a completion of f if
Z
g(x, z) dz = f(x)

The Gibbs Sampler
Completion
Latent variables are back
The Gibbs sampler can be generalized in much wider generality
A density g is a completion of f if
Z
g(x, z) dz = f(x)
Note
The variable z may be meaningless for the problem

The Gibbs Sampler
Completion
Purpose
g should have full conditionals that are easy to simulate for a
Gibbs sampler to be implemented with g rather than f
For p > 1, write y = (x, z) and denote the conditional densities of
g(y) = g(y1, . . . , yp) by
Y1|y2, . . . , yp ∼ g1(y1|y2, . . . , yp),
Y2|y1, y3, . . . , yp ∼ g2(y2|y1, y3, . . . , yp),
. . . ,
Yp|y1, . . . , yp−1 ∼ gp(yp|y1, . . . , yp−1).

The Gibbs Sampler
Completion
The move from Y (t) to Y (t+1) is deﬁned as follows:
Algorithm (Completion Gibbs sampler)
Given (y
(t)
1 , . . . , y
(t)
p ), simulate
1. Y
(t+1)
1 ∼ g1(y1|y
(t)
2 , . . . , y
(t)
p ),
2. Y
(t+1)
2 ∼ g2(y2|y
(t+1)
1 , y
(t)
3 , . . . , y
(t)
p ),
. . .
p. Y
(t+1)
p ∼ gp(yp|y
(t+1)
1 , . . . , y
(t+1)
p−1 ).

The Gibbs Sampler
Completion
Example (Mixtures all over again)
Hierarchical missing data structure:
If
X1, . . . , Xn ∼
k
i=1
pif(x|θi),
then
X|Z ∼ f(x|θZ), Z ∼ p1I(z = 1) + . . . + pkI(z = k),
Z is the component indicator associated with observation x

The Gibbs Sampler
Completion
Example (Mixtures (2))
Conditionally on (Z1, . . . , Zn) = (z1, . . . , zn) :
π(p1, . . . , pk, θ1, . . . , θk|x1, . . . , xn, z1, . . . , zn)
∝ pα1+n1−1
1 . . . pαk+nk−1
k
×π(θ1|y1 + n1¯x1, λ1 + n1) . . . π(θk|yk + nk ¯xk, λk + nk),
with
ni =
j
I(zj = i) and ¯xi =
j; zj=i
xj/ni.

The Gibbs Sampler
Completion
Algorithm (Mixture Gibbs sampler)
1. Simulate
θi ∼ π(θi|yi + ni¯xi, λi + ni) (i = 1, . . . , k)
(p1, . . . , pk) ∼ D(α1 + n1, . . . , αk + nk)
2. Simulate (j = 1, . . . , n)
Zj|xj, p1, . . . , pk, θ1, . . . , θk ∼
k
i=1
pijI(zj = i)
with (i = 1, . . . , k)
pij ∝ pif(xj|θi)
and update ni and ¯xi (i = 1, . . . , k).

The Gibbs Sampler
Completion
A wee problem
−1 0 1 2 3 4
−101234
µ1
µ2
Gibbs started at random

The Gibbs Sampler
Completion
A wee problem
−1 0 1 2 3 4
−101234
µ1
µ2
Gibbs started at random
Gibbs stuck at the wrong mode
−1 0 1 2 3
−10123
µ1
µ2

The Gibbs Sampler
Completion
Slice sampler as generic Gibbs
If f(θ) can be written as a product
k
i=1
fi(θ),

The Gibbs Sampler
Completion
Slice sampler as generic Gibbs
If f(θ) can be written as a product
k
i=1
fi(θ),
it can be completed as
k
i=1
I0≤ωi≤fi(θ),
leading to the following Gibbs algorithm:

The Gibbs Sampler
Completion
Algorithm (Slice sampler)
Simulate
1. ω
(t+1)
1 ∼ U[0,f1(θ(t))];
. . .
k. ω
(t+1)
k ∼ U[0,fk(θ(t))];
k+1. θ(t+1) ∼ UA(t+1) , with
A(t+1)
= {y; fi(y) ≥ ω
(t+1)
i , i = 1, . . . , k}.

The Gibbs Sampler
Completion
Example of results with a truncated N(−3, 1) distribution
0.0 0.2 0.4 0.6 0.8 1.0
0.0000.0020.0040.0060.0080.010
x
y
Number of Iterations 2

The Gibbs Sampler
Completion
0.0 0.2 0.4 0.6 0.8 1.0
0.0000.0020.0040.0060.0080.010
x
y
Number of Iterations 2, 3

The Gibbs Sampler
Completion
0.0 0.2 0.4 0.6 0.8 1.0
0.0000.0020.0040.0060.0080.010
x
y
Number of Iterations 2, 3, 4

The Gibbs Sampler
Completion
0.0 0.2 0.4 0.6 0.8 1.0
0.0000.0020.0040.0060.0080.010
x
y
Number of Iterations 2, 3, 4, 5

The Gibbs Sampler
Completion
0.0 0.2 0.4 0.6 0.8 1.0
0.0000.0020.0040.0060.0080.010
x
y
Number of Iterations 2, 3, 4, 5, 10

The Gibbs Sampler
Completion
0.0 0.2 0.4 0.6 0.8 1.0
0.0000.0020.0040.0060.0080.010
x
y
Number of Iterations 2, 3, 4, 5, 10, 50

The Gibbs Sampler
Completion
0.0 0.2 0.4 0.6 0.8 1.0
0.0000.0020.0040.0060.0080.010
x
y
Number of Iterations 2, 3, 4, 5, 10, 50, 100

The Gibbs Sampler
Completion
Good slices
The slice sampler usually enjoys good theoretical properties (like
geometric ergodicity and even uniform ergodicity under bounded f
and bounded X ).
As k increases, the determination of the set A(t+1) may get
increasingly complex.

The Gibbs Sampler
Completion
Example (Stochastic volatility core distribution)
Diﬃcult part of the stochastic volatility model
π(x) ∝ exp − σ2
(x − µ)2
+ β2
exp(−x)y2
+ x /2 ,
simpliﬁed in exp − x2 + α exp(−x)

The Gibbs Sampler
Completion
Example (Stochastic volatility core distribution)
Diﬃcult part of the stochastic volatility model
π(x) ∝ exp − σ2
(x − µ)2
+ β2
exp(−x)y2
+ x /2 ,
simpliﬁed in exp − x2 + α exp(−x)
Slice sampling means simulation from a uniform distribution on
A = x; exp − x2
+ α exp(−x) /2 ≥ u
= x; x2
+ α exp(−x) ≤ ω
if we set ω = −2 log u.
Note Inversion of x2 + α exp(−x) = ω needs to be done by
trial-and-error.

The Gibbs Sampler
Completion
0 10 20 30 40 50 60 70 80 90 100
−0.1
−0.05
0
0.05
0.1
Lag
Correlation
−1 −0.5 0 0.5 1 1.5 2 2.5 3 3.5
0
0.2
0.4
0.6
0.8
1
Density
Histogram of a Markov chain produced by a slice sampler
and target distribution in overlay.

The Gibbs Sampler
Convergence
Properties of the Gibbs sampler
Theorem (Convergence)
For
(Y1, Y2, · · · , Yp) ∼ g(y1, . . . , yp),
if either
[Positivity condition]
(i) g(i)(yi) > 0 for every i = 1, · · · , p, implies that
g(y1, . . . , yp) > 0, where g(i) denotes the marginal distribution
of Yi, or
(ii) the transition kernel is absolutely continuous with respect to g,
then the chain is irreducible and positive Harris recurrent.

The Gibbs Sampler
Convergence
Properties of the Gibbs sampler (2)
Consequences
(i) If h(y)g(y)dy < ∞, then
lim
nT→∞
1
T
T
t=1
h1(Y (t)
) = h(y)g(y)dy a.e. g.
(ii) If, in addition, (Y (t)) is aperiodic, then
lim
n→∞
Kn
(y, ·)µ(dx) − f
TV
= 0
for every initial distribution µ.

The Gibbs Sampler
Convergence
Slice sampler
fast on that slice
For convergence, the properties of Xt and of f(Xt) are identical
Theorem (Uniform ergodicity)
If f is bounded and suppf is bounded, the simple slice sampler is
uniformly ergodic.
[Mira & Tierney, 1997]

The Gibbs Sampler
Convergence
A small set for a slice sampler
no slice detail
For > ,
C = {x ∈ X; < f(x) < }
is a small set:
Pr(x, ·) ≥ µ(·)
where
µ(A) =
1
0
λ(A ∩ L( ))
λ(L( ))
d
if L( ) = {x ∈ X; f(x) > }‘
[Roberts & Rosenthal, 1998]

The Gibbs Sampler
Convergence
Slice sampler: drift
Under diﬀerentiability and monotonicity conditions, the slice
sampler also veriﬁes a drift condition with V (x) = f(x)−β, is
geometrically ergodic, and there even exist explicit bounds on the
total variation distance

The Gibbs Sampler
Convergence
Slice sampler: drift
Under diﬀerentiability and monotonicity conditions, the slice
sampler also veriﬁes a drift condition with V (x) = f(x)−β, is
geometrically ergodic, and there even exist explicit bounds on the
total variation distance
Example (Exponential Exp(1))
For n > 23,
||Kn
(x, ·) − f(·)||TV ≤ .054865 (0.985015)n
(n − 15.7043)

The Gibbs Sampler
Convergence
Slice sampler: convergence
no more slice detail
Theorem
For any density such that
∂
∂
λ ({x ∈ X; f(x) > }) is non-increasing
then
||K523
(x, ·) − f(·)||TV ≤ .0095

The Gibbs Sampler
Convergence
A poor slice sampler
Example
Consider
f(x) = exp {−||x||} x ∈ Rd
Slice sampler equivalent to
one-dimensional slice sampler on
π(z) = zd−1
e−z
z > 0
or on
π(u) = e−u1/d
u > 0
Poor performances when d large
(heavy tails)
0 200 400 600 800 1000
-2-101
1 dimensional run
correlation
0 10 20 30 40
0.00.20.40.60.81.0
1 dimensional acf
0 200 400 600 800 1000
1015202530
10 dimensional run
correlation
0 10 20 30 40
0.00.20.40.60.81.0
10 dimensional acf
0 200 400 600 800 1000
0204060
20 dimensional run
correlation
0 10 20 30 40
0.00.20.40.60.81.0
20 dimensional acf
0 200 400 600 800 1000
0100200300400
100 dimensional run
correlation
0 10 20 30 40
0.00.20.40.60.81.0
100 dimensional acf
Sample runs of log(u) and
ACFs for log(u) (Roberts

The Gibbs Sampler
Hammersley-Cliﬀord theorem
An illustration that conditionals determine the joint distribution
Theorem
If the joint density g(y1, y2) have conditional distributions
g1(y1|y2) and g2(y2|y1), then
g(y1, y2) =
g2(y2|y1)
g2(v|y1)/g1(y1|v) dv
.
[Hammersley & Cliﬀord, circa 1970]

The Gibbs Sampler
General HC decomposition
Under the positivity condition, the joint distribution g satisﬁes
g(y1, . . . , yp) ∝
p
j=1
g j
(y j
|y 1 , . . . , y j−1
, y j+1
, . . . , y p
)
g j
(y j
|y 1 , . . . , y j−1
, y j+1
, . . . , y p
)
for every permutation on {1, 2, . . . , p} and every y ∈ Y .

Sequential importance sampling
The Gibbs Sampler

Importance sampling (revisited)
basic importance
Approximation of integrals
I = h(x)π(x)dx
by unbiased estimators
ˆI =
1
n
n
i=1
ih(xi)
when
x1, . . . , xn
iid
∼ q(x) and i
def
=
π(xi)
q(xi)

Iterated importance sampling
As in Markov Chain Monte Carlo (MCMC) algorithms,
introduction of a temporal dimension :
x
(t)
i ∼ qt(x|x
(t−1)
i ) i = 1, . . . , n, t = 1, . . .
and
ˆIt =
1
n
n
i=1
(t)
i h(x
(t)
i )
is still unbiased for
(t)
i =
πt(x
(t)
i )
qt(x
(t)
i |x
(t−1)
i )
, i = 1, . . . , n

Fundamental importance equality
Preservation of unbiasedness
E h(X(t)
)
π(X(t))
qt(X(t)|X(t−1))
= h(x)
π(x)
qt(x|y)
qt(x|y) g(y) dx dy
= h(x) π(x) dx
for any distribution g on X(t−1)

Sequential variance decomposition
Furthermore,
var ˆIt =
1
n2
n
i=1
var
(t)
i h(x
(t)
i ) ,
if var
(t)
i exists, because the x
(t)
i ’s are conditionally uncorrelated
Note
This decomposition is still valid for correlated [in i] x
(t)
i ’s when
incorporating weights
(t)
i

Simulation of a population
The importance distribution of the sample (a.k.a. particles) x(t)
qt(x(t)
|x(t−1)
)
can depend on the previous sample x(t−1) in any possible way as
long as marginal distributions
qit(x) = qt(x(t)
) dx
(t)
−i
can be expressed to build importance weights
it =
π(x
(t)
i )
qit(x
(t)
i )

Special case of the product proposal
If
qt(x(t)
|x(t−1)
) =
n
i=1
qit(x
(t)
i |x(t−1)
)
[Independent proposals]
then
var ˆIt =
1
n2
n
i=1
var
(t)
i h(x
(t)
i ) ,

Validation
skip validation
E
(t)
i h(X
(t)
i )
(t)
j h(X
(t)
j )
= h(xi)
π(xi)
qit(xi|x(t−1))
π(xj)
qjt(xj|x(t−1))
h(xj)
qit(xi|x(t−1)
) qjt(xj|x(t−1)
) dxi dxj g(x(t−1)
)dx(t−1)
= Eπ [h(X)]2
whatever the distribution g on x(t−1)

Self-normalised version
In general, π is unscaled and the weight
(t)
i ∝
π(x
(t)
i )
qit(x
(t)
i )
, i = 1, . . . , n ,
is scaled so that
i
(t)
i = 1

Self-normalised version properties
Loss of the unbiasedness property and the variance
decomposition
Normalising constant can be estimated by
t =
1
tn
t
τ=1
n
i=1
π(x
(τ)
i )
qiτ (x
(τ)
i )
Variance decomposition (approximately) recovered if t−1 is
used instead

Sampling importance resampling
Importance sampling from g can also produce samples from the
target π
[Rubin, 1987]

target π
[Rubin, 1987]
Theorem (Bootstraped importance sampling)
If a sample (xi )1≤i≤m is derived from the weighted sample
(xi, i)1≤i≤n by multinomial sampling with weights i, then
xi ∼ π(x)

target π
[Rubin, 1987]
Theorem (Bootstraped importance sampling)
If a sample (xi )1≤i≤m is derived from the weighted sample
(xi, i)1≤i≤n by multinomial sampling with weights i, then
xi ∼ π(x)
Note
Obviously, the xi ’s are not iid

Iterated sampling importance resampling
This principle can be extended to iterated importance sampling:
After each iteration, resampling produces a sample from π
[Again, not iid!]

Iterated sampling importance resampling
This principle can be extended to iterated importance sampling:
After each iteration, resampling produces a sample from π
[Again, not iid!]
Incentive
Use previous sample(s) to learn about π and q

Generic Population Monte Carlo
Algorithm (Population Monte Carlo Algorithm)
For t = 1, . . . , T
For i = 1, . . . , n,
1. Select the generating distribution qit(·)
2. Generate ˜x
(t)
i ∼ qit(x)
3. Compute
(t)
i = π(˜x
(t)
i )/qit(˜x
(t)
i )
Normalise the
(t)
i ’s into ¯
(t)
i ’s
Generate Ji,t ∼ M((¯
(t)
i )1≤i≤N ) and set xi,t = ˜x
(t)
Ji,t

D-kernels in competition
A general adaptive construction:
Construct qi,t as a mixture of D diﬀerent transition kernels
depending on x
(t−1)
i
qi,t =
D
=1
pt, K (x
(t−1)
i , x),
D
=1
pt, = 1 ,
and adapt the weights pt, .

D-kernels in competition
A general adaptive construction:
Construct qi,t as a mixture of D diﬀerent transition kernels
depending on x
(t−1)
i
qi,t =
D
=1
pt, K (x
(t−1)
i , x),
D
=1
pt, = 1 ,
and adapt the weights pt, .
Darwinian example
Take pt, proportional to the survival rate of the points
(a.k.a. particles) x
(t)
i generated from K

Implementation
Algorithm (D-kernel PMC)
For t = 1, . . . , T
generate (Ki,t)1≤i≤N ∼ M ((pt,k)1≤k≤D)
for 1 ≤ i ≤ N, generate
˜xi,t ∼ KKi,t (x)
compute and renormalize the importance weights ωi,t
generate (Ji,t)1≤i≤N ∼ M ((ωi,t)1≤i≤N )
take xi,t = ˜xJi,t,t and pt+1,d = N
i=1 ¯ωi,tId(Ki,t)

Links with particle ﬁlters
Sequential setting where π = πt changes with t: Population
Monte Carlo also adapts to this case
Can be traced back all the way to Hammersley and Morton
(1954) and the self-avoiding random walk problem
Gilks and Berzuini (2001) produce iterated samples with (SIR)
resampling steps, and add an MCMC step: this step must use
a πt invariant kernel
Chopin (2001) uses iterated importance sampling to handle
large datasets: this is a special case of PMC where the qit’s
are the posterior distributions associated with a portion kt of
the observed dataset

Links with particle ﬁlters (2)
Rubinstein and Kroese’s (2004) cross-entropy method is
parameterised importance sampling targeted at rare events
Stavropoulos and Titterington’s (1999) smooth bootstrap and
Warnes’ (2001) kernel coupler use nonparametric kernels on
the previous importance sample to build an improved
proposal: this is a special case of PMC
West (1992) mixture approximation is a precursor of smooth
bootstrap
Mengersen and Robert (2002) “pinball sampler” is an MCMC
attempt at population sampling
Del Moral, Doucet and Jasra (2006, JRSS B) sequential
Monte Carlo samplers also relates to PMC, with a Markovian
dependence on the past sample x(t) but (limited) stationarity
constraints

Things can go wrong
Unexpected behaviour of the mixture weights when the number of
particles increases
N
i=1
¯ωi,tIKi,t=d−→P
1
D

Things can go wrong
Unexpected behaviour of the mixture weights when the number of
particles increases
N
i=1
¯ωi,tIKi,t=d−→P
1
D
Conclusion
At each iteration, every weight converges to 1/D:
the algorithm fails to learn from experience!!

Saved by Rao-Blackwell!!
Modiﬁcation: Rao-Blackwellisation (=conditioning)

Saved by Rao-Blackwell!!
Modiﬁcation: Rao-Blackwellisation (=conditioning)
Use the whole mixture in the importance weight:
ωi,t = π(˜xi,t)
D
d=1
pt,dKd(xi,t−1, ˜xi,t)
instead of
ωi,t =
π(˜xi,t)
KKi,t (xi,t−1, ˜xi,t)

Adapted algorithm
Algorithm (Rao-Blackwellised D-kernel PMC)
At time t (t = 1, . . . , T),
Generate
(Ki,t)1≤i≤N
iid
∼ M((pt,d)1≤d≤D);
Generate
(˜xi,t)1≤i≤N
ind
∼ KKi,t (xi,t−1, x)
and set ωi,t = π(˜xi,t) D
d=1 pt,dKd(xi,t−1, ˜xi,t);
Generate
(Ji,t)1≤i≤N
iid
∼ M((¯ωi,t)1≤i≤N )
and set xi,t = ˜xJi,t,t and pt+1,d = N
i=1 ¯ωi,tpt,d.

Theorem (LLN)
Under regularity assumptions, for h ∈ L1
Π and for every t ≥ 1,
1
N
N
k=1
¯ωi,th(xi,t)
N→∞
−→P Π(h)
and
pt,d
N→∞
−→P αt
d
The limiting coeﬃcients (αt
d)1≤d≤D are deﬁned recursively as
αt
d = αt−1
d
Kd(x, x )
D
j=1 αt−1
j Kj(x, x )
Π ⊗ Π(dx, dx ).

Recursion on the weights
Set F as
F(α) = αd
Kd(x, x )
D
j=1 αjKj(x, x )
Π ⊗ Π(dx, dx )
1≤d≤D
on the simplex
S = α = (α1, . . . , αD); ∀d ∈ {1, . . . , D}, αd ≥ 0 and
D
d=1
αd = 1 .
and deﬁne the sequence
αt+1
= F(αt
)

Kullback divergence
Deﬁnition (Kullback divergence)
For α ∈ S,
KL(α) = log
π(x)π(x )
π(x) D
d=1 αdKd(x, x )
Π ⊗ Π(dx, dx ).
Kullback divergence between Π and the mixture.
Goal: Obtain the mixture closest to Π, i.e., that minimises KL(α)

Connection with RBDPMCA ??
Theorem
Under the assumption
∀d ∈ {1, . . . , D}, −∞ < log(Kd(x, x ))Π ⊗ Π(dx, dx ) < ∞
for every α ∈ SD,
KL(F(α)) ≤ KL(α).

Connection with RBDPMCA ??
Theorem
Under the assumption
∀d ∈ {1, . . . , D}, −∞ < log(Kd(x, x ))Π ⊗ Π(dx, dx ) < ∞
for every α ∈ SD,
KL(F(α)) ≤ KL(α).
Conclusion
The Kullback divergence decreases at every iteration of RBDPMCA

An integrated EM interpretation
skip interpretation
We have
αmin
= arg min
α∈S
KL(α) = arg max
α∈S
log pα(¯x)Π ⊗ Π(d¯x)
= arg max
α∈S
log pα(¯x, K)dK Π ⊗ Π(d¯x)
for ¯x = (x, x ) and K ∼ M((αd)1≤d≤D). Then αt+1 = F(αt)
means
αt+1
= arg max
α
Eαt (log pα( ¯X, K)| ¯X = ¯x)Π ⊗ Π(d¯x)
and
lim
t→∞
αt
= αmin

Illustration
Example (A toy example)
Take the target
1/4N (−1, 0.3)(x) + 1/4N (0, 1)(x) + 1/2N (3, 2)(x)
and use 3 proposals: N (−1, 0.3), N (0, 1) and N (3, 2)
[Surprise!!!]

Illustration
Example (A toy example)
Take the target
1/4N (−1, 0.3)(x) + 1/4N (0, 1)(x) + 1/2N (3, 2)(x)
and use 3 proposals: N (−1, 0.3), N (0, 1) and N (3, 2)
[Surprise!!!]
Then
1 0.0500000 0.05000000 0.9000000
2 0.2605712 0.09970292 0.6397259
6 0.2740816 0.19160178 0.5343166
10 0.2989651 0.19200904 0.5090259
16 0.2651511 0.24129039 0.4935585
Weight evolution

Target and mixture evolution

c Learning scheme
The eﬃciency of the SNIS approximation depends on the choice of
Q, ranging from optimal
q(x) ∝ |h(x) − Π(h)|π(x)
to useless
var ˆΠSNIS
Q,N (h) = +∞

c Learning scheme
The efficiency of the SNIS approximation depends on the choice of
Q, ranging from optimal
q(x) ∝ |h(x) − Π(h)|π(x)
to useless
var ˆΠSNIS
Q,N (h) = +∞
Example (PMC=adaptive importance sampling)
Population Monte Carlo is producing a sequence of proposals Qt
aiming at improving efficiency
Kull(π, qt) ≤ Kull(π, qt−1) or var ˆΠSNIS
Qt,∞ (h) ≤ var ˆΠSNIS
Qt−1,∞(h)
[Cappé, Douc, Guillin, Marin, Robert, 04, 07a, 07b, 08]

The Gibbs Sampler
ABC basics
Alphabet soup
Calibration of ABC

ABC basics
Untractable likelihoods
Cases when the likelihood function f(y|θ) is unavailable and when
the completion step
f(y|θ) =
Z
f(y, z|θ) dz
is impossible or too costly because of the dimension of z
c MCMC cannot be implemented!

ABC basics
Illustrations
Example
Stochastic volatility model: for
t = 1, . . . , T,
yt = exp(zt) t , zt = a+bzt−1+σηt ,
T very large makes it diﬃcult to
include z within the simulated
parameters
0 200 400 600 800 1000
−0.4−0.20.00.20.4 t
Highest weight trajectories

ABC basics
Illustrations
Example
Potts model: if y takes values on a grid Y of size kn and
f(y|θ) ∝ exp θ
l∼i
Iyl=yi
where l∼i denotes a neighbourhood relation, n moderately large
prohibits the computation of the normalising constant

ABC basics
Illustrations
Example
Inference on CMB: in cosmology, study of the Cosmic Microwave
Background via likelihoods immensely slow to computate (e.g
WMAP, Plank), because of numerically costly spectral transforms
[Data is a Fortran program]
[Kilbinger et al., 2010, MNRAS]

ABC basics
Illustrations
Example
Phylogenetic tree: in population
genetics, reconstitution of a common
ancestor from a sample of genes via
a phylogenetic tree that is close to
impossible to integrate out
[100 processor days with 4
parameters]
[Cornuet et al., 2009, Bioinformatics]

ABC basics
The ABC method
Bayesian setting: target is π(θ)f(x|θ)

ABC basics
The ABC method
When likelihood f(x|θ) not in closed form, likelihood-free rejection
technique:

ABC basics
The ABC method
When likelihood f(x|θ) not in closed form, likelihood-free rejection
technique:
ABC algorithm
For an observation y ∼ f(y|θ), under the prior π(θ), keep jointly
simulating
θ ∼ π(θ) , z ∼ f(z|θ ) ,
until the auxiliary variable z is equal to the observed value, z = y.
[Tavar´e et al., 1997]

ABC basics
Why does it work?!
The proof is trivial:
f(θi) ∝
z∈D
π(θi)f(z|θi)Iy(z)
∝ π(θi)f(y|θi)
= π(θi|y) .
[Accept–Reject 101]

ABC basics
Earlier occurrence
‘Bayesian statistics and Monte Carlo methods are ideally
suited to the task of passing many models over one
dataset’
[Don Rubin, Annals of Statistics, 1984]
Note Rubin (1984) does not promote this algorithm for
likelihood-free simulation but frequentist intuition on posterior
distributions: parameters from posteriors are more likely to be
those that could have generated the data.

ABC basics
A as approximative
When y is a continuous random variable, equality z = y is replaced
with a tolerance condition,
(y, z) ≤
where is a distance

ABC basics
A as approximative
When y is a continuous random variable, equality z = y is replaced
with a tolerance condition,
(y, z) ≤
where is a distance
Output distributed from
π(θ) Pθ{ (y, z) < } ∝ π(θ| (y, z) < )

ABC basics
ABC algorithm
Algorithm 2 Likelihood-free rejection sampler 2
for i = 1 to N do
repeat
generate θ from the prior distribution π(·)
generate z from the likelihood f(·|θ )
until ρ{η(z), η(y)} ≤
set θi = θ
end for
where η(y) deﬁnes a (not necessarily suﬃcient) statistic

ABC basics
Output
The likelihood-free algorithm samples from the marginal in z of:
π (θ, z|y) =
π(θ)f(z|θ)IA ,y (z)
A ,y×Θ π(θ)f(z|θ)dzdθ
,
wheere A ,y = {z ∈ D|ρ(η(z), η(y)) < }.

ABC basics
Output
The likelihood-free algorithm samples from the marginal in z of:
π (θ, z|y) =
π(θ)f(z|θ)IA ,y (z)
A ,y×Θ π(θ)f(z|θ)dzdθ
,
wheere A ,y = {z ∈ D|ρ(η(z), η(y)) < }.
The idea behind ABC is that the summary statistics coupled with a
small tolerance should provide a good approximation of the
posterior distribution:
π (θ|y) = π (θ, z|y)dz ≈ π(θ|y) .

ABC basics
MA example
Back to the MA(q) model
xt = t +
q
i=1
ϑi t−i
Simple prior: uniform over the inverse [real and complex] roots in
Q(u) = 1 −
q
i=1
ϑiui
under the identiﬁability conditions

ABC basics
MA example
Back to the MA(q) model
xt = t +
q
i=1
ϑi t−i
Simple prior: uniform prior over the identiﬁability zone, e.g.
triangle for MA(2)

ABC basics
MA example (2)
ABC algorithm thus made of
1. picking a new value (ϑ1, ϑ2) in the triangle
2. generating an iid sequence ( t)−q<t≤T
3. producing a simulated series (xt)1≤t≤T

ABC basics
MA example (2)
ABC algorithm thus made of
1. picking a new value (ϑ1, ϑ2) in the triangle
2. generating an iid sequence ( t)−q<t≤T
3. producing a simulated series (xt)1≤t≤T
Distance: basic distance between the series
ρ((xt)1≤t≤T , (xt)1≤t≤T ) =
T
t=1
(xt − xt)2
or distance between summary statistics like the q autocorrelations
τj =
T
t=j+1
xtxt−j

ABC basics
Comparison of distance impact
Evaluation of the tolerance on the ABC sample against both
distances ( = 100%, 10%, 1%, 0.1%) for an MA(2) model

ABC basics
Comparison of distance impact
0.0 0.2 0.4 0.6 0.8
01234
θ1
−2.0 −1.0 0.0 0.5 1.0 1.5
0.00.51.01.5
θ2
Evaluation of the tolerance on the ABC sample against both
distances ( = 100%, 10%, 1%, 0.1%) for an MA(2) model

ABC basics
Homonomy
The ABC algorithm is not to be confused with the ABC algorithm
The Artificial Bee Colony algorithm is a swarm based meta-heuristic
algorithm that was introduced by Karaboga in 2005 for optimizing
numerical problems. It was inspired by the intelligent foraging
behavior of honey bees. The algorithm is specifically based on the
model proposed by Tereshko and Loengarov (2005) for the foraging
behaviour of honey bee colonies. The model consists of three
essential components: employed and unemployed foraging bees, and
food sources. The first two components, employed and unemployed
foraging bees, search for rich food sources (...) close to their hive.
The model also defines two leading modes of behaviour (...):
recruitment of foragers to rich food sources resulting in positive
feedback and abandonment of poor sources by foragers causing
negative feedback.
[Karaboga, Scholarpedia]

ABC basics
ABC advances
Simulating from the prior is often poor in eﬃciency

ABC basics
ABC advances
Either modify the proposal distribution on θ to increase the density
of x’s within the vicinity of y...
[Marjoram et al, 2003; Bortot et al., 2007, Sisson et al., 2007]

ABC basics
ABC advances
...or by viewing the problem as a conditional density estimation
and by developing techniques to allow for larger
[Beaumont et al., 2002]

ABC basics
ABC advances
...or by viewing the problem as a conditional density estimation
and by developing techniques to allow for larger
[Beaumont et al., 2002]
.....or even by including in the inferential framework [ABCµ]
[Ratmann et al., 2009]

Alphabet soup
ABC-NP
Better usage of [prior] simulations by
adjustement: instead of throwing away
θ such that ρ(η(z), η(y)) > , replace
θs with locally regressed
θ∗
= θ − {η(z) − η(y)}T ˆβ
[Csill´ery et al., TEE, 2010]
where ˆβ is obtained by [NP] weighted least square regression on
(η(z) − η(y)) with weights
Kδ {ρ(η(z), η(y))}
[Beaumont et al., 2002, Genetics]

Alphabet soup
ABC-MCMC
Markov chain (θ(t)) created via the transition function
θ(t+1)
=



θ ∼ Kω(θ |θ(t)) if x ∼ f(x|θ ) is such that x = y
and u ∼ U(0, 1) ≤ π(θ )Kω(θ(t)|θ )
π(θ(t))Kω(θ |θ(t))
,
θ(t) otherwise,

Alphabet soup
ABC-MCMC
Markov chain (θ(t)) created via the transition function
θ(t+1)
=



θ ∼ Kω(θ |θ(t)) if x ∼ f(x|θ ) is such that x = y
and u ∼ U(0, 1) ≤ π(θ )Kω(θ(t)|θ )
π(θ(t))Kω(θ |θ(t))
,
θ(t) otherwise,
has the posterior π(θ|y) as stationary distribution
[Marjoram et al, 2003]

Alphabet soup
ABC-MCMC (2)
Algorithm 3 Likelihood-free MCMC sampler
Use Algorithm 2 to get (θ(0), z(0))
for t = 1 to N do
Generate θ from Kω ·|θ(t−1) ,
Generate z from the likelihood f(·|θ ),
Generate u from U[0,1],
if u ≤ π(θ )Kω(θ(t−1)|θ )
π(θ(t−1)Kω(θ |θ(t−1))
IA ,y (z ) then
set (θ(t), z(t)) = (θ , z )
else
(θ(t), z(t))) = (θ(t−1), z(t−1)),
end if
end for

Alphabet soup
ABCµ
[Ratmann, Andrieu, Wiuf and Richardson, 2009, PNAS]
Use of a joint density
f(θ, |y) ∝ ξ( |y, θ) × πθ(θ) × π ( )
where y is the data, and ξ( |y, θ) is the prior predictive density of
ρ(η(z), η(y)) given θ and x when z ∼ f(z|θ)

Alphabet soup
ABCµ
[Ratmann, Andrieu, Wiuf and Richardson, 2009, PNAS]
Use of a joint density
f(θ, |y) ∝ ξ( |y, θ) × πθ(θ) × π ( )
where y is the data, and ξ( |y, θ) is the prior predictive density of
ρ(η(z), η(y)) given θ and x when z ∼ f(z|θ)
Warning! Replacement of ξ( |y, θ) with a non-parametric kernel
approximation.

Alphabet soup
ABCµ details
Multidimensional distances ρk (k = 1, . . . , K) and errors
k = ρk(ηk(z), ηk(y)), with
k ∼ ξk( |y, θ) ≈ ˆξk( |y, θ) =
1
Bhk
b
K[{ k−ρk(ηk(zb), ηk(y))}/hk]
then used in replacing ξ( |y, θ) with mink
ˆξk( |y, θ)

Alphabet soup
ABCµ details
Multidimensional distances ρk (k = 1, . . . , K) and errors
k = ρk(ηk(z), ηk(y)), with
k ∼ ξk( |y, θ) ≈ ˆξk( |y, θ) =
1
Bhk
b
K[{ k−ρk(ηk(zb), ηk(y))}/hk]
then used in replacing ξ( |y, θ) with mink
ˆξk( |y, θ)
ABCµ involves acceptance probability
π(θ , )
π(θ, )
q(θ , θ)q( , )
q(θ, θ )q( , )
mink
ˆξk( |y, θ )
mink
ˆξk( |y, θ)

Alphabet soup
ABCµ multiple errors
[ c Ratmann et al., PNAS, 2009]

Alphabet soup
ABCµ for model choice
[ c Ratmann et al., PNAS, 2009]

Alphabet soup
Questions about ABCµ
For each model under comparison, marginal posterior on used to
assess the ﬁt of the model (HPD includes 0 or not).

Alphabet soup
Questions about ABCµ
For each model under comparison, marginal posterior on used to
assess the ﬁt of the model (HPD includes 0 or not).
Is the data informative about ? [Identiﬁability]
How is the prior π( ) impacting the comparison?
How is using both ξ( |x0, θ) and π ( ) compatible with a
standard probability model? [remindful of Wilkinson]
Where is the penalisation for complexity in the model
comparison?
[X, Mengersen & Chen, 2010, PNAS]

Alphabet soup
ABC-PRC
Another sequential version producing a sequence of Markov
transition kernels Kt and of samples (θ
(t)
1 , . . . , θ
(t)
N ) (1 ≤ t ≤ T)

Alphabet soup
ABC-PRC
Another sequential version producing a sequence of Markov
transition kernels Kt and of samples (θ
(t)
1 , . . . , θ
(t)
N ) (1 ≤ t ≤ T)
ABC-PRC Algorithm
1. Pick a θ is selected at random among the previous θ
(t−1)
i ’s
with probabilities ω
(t−1)
i (1 ≤ i ≤ N).
2. Generate
θ
(t)
i ∼ Kt(θ|θ ) , x ∼ f(x|θ
(t)
i ) ,
3. Check that (x, y) < , otherwise start again.
[Sisson et al., 2007]

Alphabet soup
Why PRC?
Partial rejection control: Resample from a population of weighted
particles by pruning away particles with weights below threshold C,
replacing them by new particles obtained by propagating an
existing particle by an SMC step and modifying the weights
accordinly.
[Liu, 2001]

Alphabet soup
Why PRC?
Partial rejection control: Resample from a population of weighted
particles by pruning away particles with weights below threshold C,
replacing them by new particles obtained by propagating an
existing particle by an SMC step and modifying the weights
accordinly.
[Liu, 2001]
PRC justiﬁcation in ABC-PRC:
Suppose we then implement the PRC algorithm for some
c > 0 such that only identically zero weights are smaller
than c
Trouble is, there is no such c...

Alphabet soup
ABC-PRC weight
Probability ω
(t)
i computed as
ω
(t)
i ∝ π(θ
(t)
i )Lt−1(θ |θ
(t)
i ){π(θ )Kt(θ
(t)
i |θ )}−1
,
where Lt−1 is an arbitrary transition kernel.

Alphabet soup
ABC-PRC weight
Probability ω
(t)
i computed as
ω
(t)
i ∝ π(θ
(t)
i )Lt−1(θ |θ
(t)
i ){π(θ )Kt(θ
(t)
i |θ )}−1
,
In case
Lt−1(θ |θ) = Kt(θ|θ ) ,
all weights are equal under a uniform prior.

Alphabet soup
ABC-PRC weight
Probability ω
(t)
i computed as
ω
(t)
i ∝ π(θ
(t)
i )Lt−1(θ |θ
(t)
i ){π(θ )Kt(θ
(t)
i |θ )}−1
,
In case
Lt−1(θ |θ) = Kt(θ|θ ) ,
all weights are equal under a uniform prior.
Inspired from Del Moral et al. (2006), who use backward kernels
Lt−1 in SMC to achieve unbiasedness

Alphabet soup
ABC-PRC bias
Lack of unbiasedness of the method

Alphabet soup
A mixture example (1)
Toy model of Sisson et al. (2007): if
θ ∼ U(−10, 10) , x|θ ∼ 0.5 N(θ, 1) + 0.5 N(θ, 1/100) ,
then the posterior distribution associated with y = 0 is the normal
mixture
θ|y = 0 ∼ 0.5 N(0, 1) + 0.5 N(0, 1/100)
restricted to [−10, 10].
Furthermore, true target available as
π(θ||x| < ) ∝ Φ( −θ)−Φ(− −θ)+Φ(10( −θ))−Φ(−10( +θ)) .

Alphabet soup
“Ugly, squalid graph...”
θθ
−3 −1 1 3
0.00.20.40.60.81.0
θθ
−3 −1 1 3
0.00.20.40.60.81.0
θθ
−3 −1 1 3
0.00.20.40.60.81.0
θθ
−3 −1 1 3
0.00.20.40.60.81.0
θθ
−3 −1 1 3
0.00.20.40.60.81.0
θθ
−3 −1 1 3
0.00.20.40.60.81.0
θθ
−3 −1 1 3
0.00.20.40.60.81.0
θθ
−3 −1 1 3
0.00.20.40.60.81.0
θθ
−3 −1 1 3
0.00.20.40.60.81.0
θθ
−3 −1 1 3
0.00.20.40.60.81.0
Comparison of τ = 0.15 and τ = 1/0.15 in Kt

Alphabet soup
A PMC version
Use of the same kernel idea as ABC-PRC but with IS correction
Generate a sample at iteration t by
ˆπt(θ(t)
) ∝
N
j=1
ω
(t−1)
j Kt(θ(t)
|θ
(t−1)
j )
modulo acceptance of the associated xt, and use an importance
weight associated with an accepted simulation θ
(t)
i
ω
(t)
i ∝ π(θ
(t)
i ) ˆπt(θ
(t)
i ) .
c Still likelihood free
[Beaumont et al., 2008, arXiv:0805.2256]

Alphabet soup
The ABC-PMC algorithm
Given a decreasing sequence of approximation levels 1 ≥ . . . ≥ T ,
1. At iteration t = 1,
For i = 1, ..., N
Simulate θ
(1)
i ∼ π(θ) and x ∼ f(x|θ
(1)
i ) until (x, y) < 1
Set ω
(1)
i = 1/N
Take τ2
as twice the empirical variance of the θ
(1)
i ’s
2. At iteration 2 ≤ t ≤ T,
For i = 1, ..., N, repeat
Pick θi from the θ
(t−1)
j ’s with probabilities ω
(t−1)
j
generate θ
(t)
i |θi ∼ N(θi , σ2
t ) and x ∼ f(x|θ
(t)
i )
until (x, y) < t
Set ω
(t)
i ∝ π(θ
(t)
i )/
N
j=1 ω
(t−1)
j ϕ σ−1
t θ
(t)
i − θ
(t−1)
j )
Take τ2
t+1 as twice the weighted empirical variance of the θ
(t)
i ’s

Alphabet soup
Sequential Monte Carlo
SMC is a simulation technique to approximate a sequence of
related probability distributions πn with π0 “easy” and πT as
target.
Iterated IS as PMC : particles moved from time n to time n via
kernel Kn and use of a sequence of extended targets ˜πn
˜πn(z0:n) = πn(zn)
n
j=0
Lj(zj+1, zj)
where the Lj’s are backward Markov kernels [check that πn(zn) is
a marginal]
[Del Moral, Doucet & Jasra, Series B, 2006]

Alphabet soup
Sequential Monte Carlo (2)
Algorithm 4 SMC sampler
sample z
(0)
i ∼ γ0(x) (i = 1, . . . , N)
compute weights w
(0)
i = π0(z
(0)
i ))/γ0(z
(0)
i )
for t = 1 to N do
if ESS(w(t−1)) < NT then
resample N particles z(t−1) and set weights to 1
end if
generate z
(t−1)
i ∼ Kt(z
(t−1)
i , ·) and set weights to
w
(t)
i = W
(t−1)
i−1
πt(z
(t)
i ))Lt−1(z
(t)
i ), z
(t−1)
i ))
πt−1(z
(t−1)
i ))Kt(z
(t−1)
i ), z
(t)
i ))
end for

Alphabet soup
ABC-SMC
[Del Moral, Doucet & Jasra, 2009]
True derivation of an SMC-ABC algorithm
Use of a kernel Kn associated with target π n and derivation of the
backward kernel
Ln−1(z, z ) =
π n (z )Kn(z , z)
πn(z)
Update of the weights
win ∝ wi(n−1)
M
m=1 IA n
(xm
in)
M
m=1 IA n−1
(xm
i(n−1))
when xm
in ∼ K(xi(n−1), ·)

Alphabet soup
ABC-SMCM
Modiﬁcation: Makes M repeated simulations of the pseudo-data z
given the parameter, rather than using a single [M = 1]
simulation, leading to weight that is proportional to the number of
accepted zis
ω(θ) =
1
M
M
i=1
Iρ(η(y),η(zi))<
[limit in M means exact simulation from (tempered) target]

Alphabet soup
Properties of ABC-SMC
The ABC-SMC method properly uses a backward kernel L(z, z ) to
simplify the importance weight and to remove the dependence on
the unknown likelihood from this weight. Update of importance
weights is reduced to the ratio of the proportions of surviving
particles
Major assumption: the forward kernel K is supposed to be
invariant against the true target [tempered version of the true
posterior]

Alphabet soup
Properties of ABC-SMC
The ABC-SMC method properly uses a backward kernel L(z, z ) to
simplify the importance weight and to remove the dependence on
the unknown likelihood from this weight. Update of importance
weights is reduced to the ratio of the proportions of surviving
particles
Major assumption: the forward kernel K is supposed to be
invariant against the true target [tempered version of the true
posterior]
Adaptivity in ABC-SMC algorithm only found in on-line
construction of the thresholds t, slowly enough to keep a large
number of accepted transitions

Alphabet soup
A mixture example (2)
Recovery of the target, whether using a ﬁxed standard deviation of
τ = 0.15 or τ = 1/0.15, or a sequence of adaptive τt’s.
θθ
−3 −2 −1 0 1 2 3
0.00.20.40.60.81.0
θθ
−3 −2 −1 0 1 2 3
0.00.20.40.60.81.0
θθ
−3 −2 −1 0 1 2 3
0.00.20.40.60.81.0
θθ
−3 −2 −1 0 1 2 3
0.00.20.40.60.81.0
θθ
−3 −2 −1 0 1 2 3
0.00.20.40.60.81.0

Alphabet soup
Wilkinson’s exact BC
Wilkinson (2008) replaces the ABC approximation error
(i.e. non-zero tolerance) in with an exact simulation from a
controlled approximation to the target, a convolution of the true
posterior with an arbitrary kernel function
π (θ, z|y) =
π(θ)f(z|θ)K (y − z)
π(θ)f(z|θ)K (y − z)dzdθ
,
where K is a kernel parameterised by a bandwidth .

Alphabet soup
Wilkinson’s exact BC
Wilkinson (2008) replaces the ABC approximation error
(i.e. non-zero tolerance) in with an exact simulation from a
controlled approximation to the target, a convolution of the true
posterior with an arbitrary kernel function
π (θ, z|y) =
π(θ)f(z|θ)K (y − z)
π(θ)f(z|θ)K (y − z)dzdθ
,
where K is a kernel parameterised by a bandwidth .
Requires K to be bounded
True approximation error never assessed
Requires a modiﬁcation of the standard ABC algorithms

Alphabet soup
Semi-automatic ABC
Fearnhead and Prangle (2010) study ABC and the selection of the
summary statistic in close proximity to Wilkinson’s proposal
ABC then considered from a purely inferential viewpoint and
calibrated for estimation purposes.
Use of a randomised (or ‘noisy’) version of the summary statistics
˜η(y) = η(y) + τ
Derivation of a well-calibrated version of ABC, i.e. an algorithm
that gives proper predictions for the distribution associated with
this randomised summary statistic.

Alphabet soup
Semi-automatic ABC
Fearnhead and Prangle (2010) study ABC and the selection of the
summary statistic in close proximity to Wilkinson’s proposal
ABC then considered from a purely inferential viewpoint and
calibrated for estimation purposes.
Use of a randomised (or ‘noisy’) version of the summary statistics
˜η(y) = η(y) + τ
Derivation of a well-calibrated version of ABC, i.e. an algorithm
that gives proper predictions for the distribution associated with
this randomised summary statistic. [calibration constraint: ABC
approximation with same posterior mean as the true randomised
posterior.]

Alphabet soup
Summary statistics
Optimality of the posterior expectations of the parameters of
interest as summary statistics!

Alphabet soup
Summary statistics
Optimality of the posterior expectations of the parameters of
interest as summary statistics!
Use of the standard quadratic loss function
(θ − θ0)T
A(θ − θ0) .

Calibration of ABC
Which summary?
Fundamental difficulty of the choice of the summary statistic when
there is no non-trivial sufficient statistics [except when done by the
experimenters in the field]

Calibration of ABC
Which summary?
Starting from a large collection of summary statistics is available,
Joyce and Marjoram (2008) consider the sequential inclusion into
the ABC target, with a stopping rule based on a likelihood ratio
test.

Calibration of ABC
Which summary?
Starting from a large collection of summary statistics is available,
Joyce and Marjoram (2008) consider the sequential inclusion into
the ABC target, with a stopping rule based on a likelihood ratio
test.
Does not taking into account the sequential nature of the tests
Depends on parameterisation
Order of inclusion matters.

Calibration of ABC
A connected Monte Carlo study
Repeating simulations of the pseudo-data per simulated parameter
does not improve approximation
Tolerance level does not seem to be highly inﬂuential
Choice of distance / summary statistics / calibration factors
are paramount to successful approximation
ABC-SMC outperforms ABC-MCMC
[Mckinley, Cook, Deardon, 2009]

Calibration of ABC
A Brave New World?!

The Gibbs Sampler
Model choice
Gibbs random ﬁelds

Model choice
Bayesian model choice
Several models M1, M2, . . . are considered simultaneously for a
dataset y and the model index M is part of the inference.
Use of a prior distribution. π(M = m), plus a prior distribution on
the parameter conditional on the value m of the model index,
πm(θm)
Goal is to derive the posterior distribution of M, challenging
computational target when models are complex.

Model choice
Generic ABC for model choice
Algorithm 5 Likelihood-free model choice sampler (ABC-MC)
for t = 1 to T do
repeat
Generate m from the prior π(M = m)
Generate θm from the prior πm(θm)
Generate z from the model fm(z|θm)
until ρ{η(z), η(y)} <
Set m(t) = m and θ(t)
= θm
end for

Model choice
ABC estimates
Posterior probability π(M = m|y) approximated by the frequency
of acceptances from model m
1
T
T
t=1
Im(t)=m .
Issues with implementation:
should tolerances be the same for all models?
should summary statistics vary across models (incl. their
dimension)?
should the distance measure ρ vary as well?

Model choice
ABC estimates
Posterior probability π(M = m|y) approximated by the frequency
of acceptances from model m
1
T
T
t=1
Im(t)=m .
Issues with implementation:
should tolerances be the same for all models?
should summary statistics vary across models (incl. their
dimension)?
should the distance measure ρ vary as well?
Extension to a weighted polychotomous logistic regression estimate
of π(M = m|y), with non-parametric kernel weights
[Cornuet et al., DIYABC, 2009]

Model choice
The Great ABC controversy
On-going controvery in phylogeographic genetics about the validity
of using ABC for testing
Against: Templeton, 2008,
2009, 2010a, 2010b, 2010c
argues that nested hypotheses
cannot have higher probabilities
than nesting hypotheses (!)

Model choice
The Great ABC controversy
On-going controvery in phylogeographic genetics about the validity
of using ABC for testing
Against: Templeton, 2008,
2009, 2010a, 2010b, 2010c
argues that nested hypotheses
cannot have higher probabilities
than nesting hypotheses (!)
Replies: Fagundes et al., 2008,
Beaumont et al., 2010, Berger et
al., 2010, Csill`ery et al., 2010
point out that the criticisms are
addressed at [Bayesian]
model-based inference and have
nothing to do with ABC...

Gibbs distribution
The rv y = (y1, . . . , yn) is a Gibbs random ﬁeld associated with
the graph G if
f(y) =
1
Z
exp −
c∈C
Vc(yc) ,
where Z is the normalising constant, C is the set of cliques of G
and Vc is any function also called potential
U(y) = c∈C Vc(yc) is the energy function

Gibbs distribution
The rv y = (y1, . . . , yn) is a Gibbs random ﬁeld associated with
the graph G if
f(y) =
1
Z
exp −
c∈C
Vc(yc) ,
where Z is the normalising constant, C is the set of cliques of G
and Vc is any function also called potential
U(y) = c∈C Vc(yc) is the energy function
c Z is usually unavailable in closed form

Potts model
Potts model
Vc(y) is of the form
Vc(y) = θS(y) = θ
l∼i
δyl=yi
where l∼i denotes a neighbourhood structure

Potts model
Potts model
Vc(y) is of the form
Vc(y) = θS(y) = θ
l∼i
δyl=yi
where l∼i denotes a neighbourhood structure
In most realistic settings, summation
Zθ =
x∈X
exp{θT
S(x)}
involves too many terms to be manageable and numerical
approximations cannot always be trusted
[Cucala, Marin, CPR & Titterington, 2009]

Model choice via ABC
Bayesian Model Choice
Comparing a model with potential S0 taking values in Rp0 versus a
model with potential S1 taking values in Rp1 can be done through
the Bayes factor corresponding to the priors π0 and π1 on each
parameter space
Bm0/m1
(x) =
exp{θT
0 S0(x)}/Zθ0,0π0(dθ0)
exp{θT
1 S1(x)}/Zθ1,1π1(dθ1)

Bayesian Model Choice
Comparing a model with potential S0 taking values in Rp0 versus a
model with potential S1 taking values in Rp1 can be done through
the Bayes factor corresponding to the priors π0 and π1 on each
parameter space
Bm0/m1
(x) =
exp{θT
0 S0(x)}/Zθ0,0π0(dθ0)
exp{θT
1 S1(x)}/Zθ1,1π1(dθ1)
Use of Jeﬀreys’ scale to select most appropriate model

Neighbourhood relations
Choice to be made between M neighbourhood relations
i
m
∼ i (0 ≤ m ≤ M − 1)
with
Sm(x) =
i
m
∼i
I{xi=xi }
driven by the posterior probabilities of the models.

Model index
Formalisation via a model index M that appears as a new
parameter with prior distribution π(M = m) and
π(θ|M = m) = πm(θm)

Model index
Formalisation via a model index M that appears as a new
parameter with prior distribution π(M = m) and
π(θ|M = m) = πm(θm)
Computational target:
P(M = m|x) ∝
Θm
fm(x|θm)πm(θm) dθm π(M = m) ,

Sufficient statistics
By definition, if S(x) sufficient statistic for the joint parameters
(M, θ0, . . . , θM−1),
P(M = m|x) = P(M = m|S(x)) .

(M, θ0, . . . , θM−1),
P(M = m|x) = P(M = m|S(x)) .
For each model m, own suﬃcient statistic Sm(·) and
S(·) = (S0(·), . . . , SM−1(·)) also suﬃcient.

(M, θ0, . . . , θM−1),
P(M = m|x) = P(M = m|S(x)) .
For each model m, own sufficient statistic Sm(·) and
S(·) = (S0(·), . . . , SM−1(·)) also sufficient.
For Gibbs random fields,
x|M = m ∼ fm(x|θm) = f1
m(x|S(x))f2
m(S(x)|θm)
=
1
n(S(x))
f2
m(S(x)|θm)
where
n(S(x)) = {˜x ∈ X : S(˜x) = S(x)}
c S(x) is therefore also sufficient for the joint parameters
[Specific to Gibbs random fields!]

ABC model choice Algorithm
ABC-MC
Generate m∗ from the prior π(M = m).
Generate θ∗
m∗ from the prior πm∗ (·).
Generate x∗ from the model fm∗ (·|θ∗
m∗ ).
Compute the distance ρ(S(x0), S(x∗)).
Accept (θ∗
m∗ , m∗) if ρ(S(x0), S(x∗)) < .
Note When = 0 the algorithm is exact

ABC approximation to the Bayes factor
Frequency ratio:
BFm0/m1
(x0
) =
ˆP(M = m0|x0)
ˆP(M = m1|x0)
×
π(M = m1)
π(M = m0)
=
{mi∗ = m0}
{mi∗ = m1}
×
π(M = m1)
π(M = m0)
,

ABC approximation to the Bayes factor
Frequency ratio:
BFm0/m1
(x0
) =
ˆP(M = m0|x0)
ˆP(M = m1|x0)
×
π(M = m1)
π(M = m0)
=
{mi∗ = m0}
{mi∗ = m1}
×
π(M = m1)
π(M = m0)
,
replaced with
BFm0/m1
(x0
) =
1 + {mi∗ = m0}
1 + {mi∗ = m1}
×
π(M = m1)
π(M = m0)
to avoid indeterminacy (also Bayes estimate).

Illustrations
Toy example
iid Bernoulli model versus two-state ﬁrst-order Markov chain, i.e.
f0(x|θ0) = exp θ0
n
i=1
I{xi=1} {1 + exp(θ0)}n
,
versus
f1(x|θ1) =
1
2
exp θ1
n
i=2
I{xi=xi−1} {1 + exp(θ1)}n−1
,
with priors θ0 ∼ U(−5, 5) and θ1 ∼ U(0, 6) (inspired by “phase
transition” boundaries).

Illustrations
Toy example (2)
−40 −20 0 10
−505
BF01
BF
^
01
−40 −20 0 10−10−50510
BF01
BF
^
01
(left) Comparison of the true BFm0/m1
(x0) with BFm0/m1
(x0)
(in logs) over 2, 000 simulations and 4.106 proposals from the
prior. (right) Same when using tolerance corresponding to the
1% quantile on the distances.

Illustrations
Protein folding
Superposition of the native structure (grey) with the ST1
structure (red.), the ST2 structure (orange), the ST3 structure
(green), and the DT structure (blue).

Illustrations
Protein folding (2)
% seq . Id. TM-score FROST score
1i5nA (ST1) 32 0.86 75.3
1ls1A1 (ST2) 5 0.42 8.9
1jr8A (ST3) 4 0.24 8.9
1s7oA (DT) 10 0.08 7.8
Characteristics of dataset. % seq. Id.: percentage of identity with
the query sequence. TM-score: similarity between predicted and
native structure (uncertainty between 0.17 and 0.4) FROST score:
quality of alignment of the query onto the candidate structure
(uncertainty between 7 and 9).

Illustrations
Protein folding (3)
NS/ST1 NS/ST2 NS/ST3 NS/DT
BF 1.34 1.22 2.42 2.76
P(M = NS|x0) 0.573 0.551 0.708 0.734
Estimates of the Bayes factors between model NS and models
ST1, ST2, ST3, and DT, and corresponding posterior
probabilities of model NS based on an ABC-MC algorithm using
1.2 106 simulations and a tolerance equal to the 1% quantile of
the distances.

MCMC and likelihood-free methods

More Related Content

What's hot

Viewers also liked

Similar to MCMC and likelihood-free methods

More from Christian Robert

Recently uploaded

MCMC and likelihood-free methods