The document discusses Markov chain Monte Carlo (MCMC) methods for posterior simulation. MCMC methods generate dependent samples from the posterior distribution using iterative sampling algorithms like the Metropolis algorithm and Gibbs sampler. The Metropolis algorithm uses an accept-reject rule to propose new samples from a jumping distribution and either accepts or rejects them, while the Gibbs sampler directly samples from conditional posterior distributions one parameter at a time. Both algorithms are proven to converge to the true posterior distribution given enough iterations. The document provides details on how to implement the Metropolis and Gibbs sampling algorithms.

1. MAST90125: Bayesian Statistical Learning
Semester 2 2021
Lecture 9: Posterior simulation
John Holmes
1 / 18

2. What do we want to achieve
In the previous lecture, we looked at various techniques for approximating a posterior
distribution. These methods were either
I Deterministic,
I Based on generating samples of i.i.d. random numbers.
However as would have become apparent in the examples we looked at, these
techniques are at least one of
I Do not generalise well in multi-dimensional problems,
I Computationally wasteful.
In this lecture we will introduce a much more computationally efficient technique based
on iterative sampling called Markov Chain Monte Carlo (MCMC) which comes at the
cost of having to generate dependent samples.
2 / 18

3. Markov Chains
Before we look at MCMC algorithms, we need to define what a Markov chain is, and
what properties a Markov chain should have in the context of posterior simulation.
I A Markov chain is a stochastic process where the distribution of the present state
(at time t) θ(t) is only dependent on the immediately preceding state, that is
p(θ(t)
|θ(1)
, . . . , θ(t−1)
) = p(θ(t)
|θ(t−1)
).
I While there are many possible characteristics we can use to categorise a Markov
chain, for posterior simulation, we are most interested that the Markov chain has
the following properties:
I That all possible θ ∈ Θ can eventually be reached by the Markov chain.
I That the chain is aperiodic and not transient.
I While the conditions above imply a unique stationary distribution exists,
I we still need to show the stationary distribution is p(θ|y).
3 / 18

4. Metropolis algorithm
Like rejection sampling, the Metropolis algorithm is an example of a accept-reject rule.
The steps of the algorithm are:
I Define a proposal (or jumping) distribution, J(θa|θb), that is symmetric, that is
J() satisfies
J(θa|θb) = J(θb|θa).
In addition, J(·|·) must be able to eventually reach all possible θ ∈ Θ to ensure a
stationary distribution exists.
I Draw a starting point θ(0) from a starting distribution p(0)(θ) such that
p(θ(0)|y) > 0.
I Then we iterate,
4 / 18

5. Metropolis algorithm
I For t = 1, 2, . . .,
I Sample θ∗
from the proposal distribution J(θ∗
|θ(t−1)
)
I Calculate
r =
p(θ∗
|y)
p(θ(t−1)|y)
=
p(θ∗
, y)/p(y)
p(θ(t−1), y)/p(y)
=
p(y|θ∗
)p(θ∗
)
p(y|θ(t−1))p(θ(t−1))
I Set
θ(t)
=
(
θ∗
if u ≤ min(r, 1) where u ∼ U(0, 1).
θ(t−1)
otherwise
I Some notes:
I If a jump θ∗
is rejected, that is θ(t)
= θ(t−1)
, it is still counted as an iteration.
I The transition distribution is a mixture distribution.
5 / 18

6. Metropolis-Hastings algorithm
As might be guessed from the name, the Metropolis-Hastings algorithm is an extension
of the Metropolis algorithm. Therefore Metropolis Hastings is also a case of an
accept-reject algorithm.
I Like the Metropolis algorithm, we first need to define a proposal (or jumping)
distribution, J(θa|θb). Also like the Metropolis algorithm, J(·|·) must be chosen in
such a way as to ensure a stationary distribution exists.
I However there is now no requirement that the jumping distribution is symmetric.
This also means the Metropolis algorithm is a special case of the
Metropolis-Hastings algorithm.
I Draw a starting point θ(0) from a starting distribution p(0)(θ) such that
p(θ(0)|y) > 0.
I Then we must iterate,
6 / 18

7. Metropolis-Hastings algorithm
I For t = 1, 2, . . .,
I Sample θ∗
from the proposal distribution J(θ∗
|θ(t−1)
)
I Calculate
r =
p(θ∗
|y)/J(θ∗
|θ(t−1)
)
p(θ(t−1)|y)/J(θ(t−1)|θ∗)
=
p(θ∗
,y)
p(y)J(θ∗|θ(t−1))
p(θ(t−1),y)
p(y)J(θ(t−1)|θ∗)
=
p(y|θ∗
)p(θ∗
)/J(θ∗
|θ(t−1)
)
p(y|θ(t−1))p(θ(t−1))/J(θ(t−1)|θ∗)
I Set
θ(t)
=
(
θ∗
if u ≤ min(r, 1) where u ∼ U(0, 1).
θ(t−1)
otherwise
I Some notes:
I If a jump θ∗
is rejected, that is θ(t)
= θ(t−1)
, it is still counted as an iteration.
I The transition distribution is again a mixture distribution.
7 / 18

8. Motivation to consider a Gibbs sampler
I In most real life problems, we will not be attempting to make inference on a single
parameter only, as we have done in most examples to date. Rather, we will want
to make inference on a set of parameters, θ = (θ1, . . . , θK ).
I In this case, it may be possible to integrate out the parameters that are not of
immediate interest. We did this when estimating the parameters of the normal
distribution in Assignment 1 and Lab 2. Usually integrating out parameters is not
a straight forward task.
I More commonly though, we simply cannot analytically determine the posterior.
What we can easily obtain is the joint distribution of parameters and data,
p(θ, y) = p(θ1, . . . θK , y) = p(y|θ1, . . . θK )p(θ1, . . . θK )
,
8 / 18

9. Conditional posteriors
I Normally, we would try to directly determine p(θ|y) from p(θ, y). However the
rules of probability means there is nothing to stop us from instead finding
p(θ1 | θ2 . . . θK , y)
.
.
.
p(θK | θ1 . . . θK−1, y),
the set of conditional posteriors. Note the conditional here refers to the fact we
are finding the posterior of θi conditional on all other parameters as well as data.
I The appeal of conditional posteriors is
I A sequence of draws from low-dimensional spaces may be less computationally
intensive than a single draw from a high-dimensional space.
I Even if p(θ|y) is unknown analytically, p(θi |θ−i , y) could be.
,
9 / 18

10. The Gibbs sampler
I In the Gibbs sampler, it is assumed that is possible to directly sample from the set
of conditional posteriors p(θi |θ−i , y), 1 ≤ i ≤ K.
I First, we define a starting point θ(0) = (θ
(0)
1 , . . . , θ
(0)
K ) from some starting
distribution.
I Then we iterate. However unlike in the Metropolis(-Hastings) algorithm, we need
to iterate over the components of θ within each iteration t.
I For t = 1, 2, . . .,
I For i = 1, . . . K, draw θ
(t)
i from the conditional posterior
p(θ
(t)
i |θ∗
−i , y),
where θ∗
−i = (θ
(t)
1 , . . . , θ
(t)
i−1, θ
(t−1)
i+1 , . . . , θ
(t−1)
K ).
10 / 18

11. How do we know that we will converge to p(θ|y)
I While we have introduced various MCMC algorithms, we have not shown that any
will converge to p(θ|y). To prove convergence, consider the joint distribution of
two successive draws θa, θb from a Metropolis-Hastings algorithm. To help,
assume p(θb|y)/J(θb|θa) ≥ p(θa|y)/J(θa|θb).
I First assume θ(t−1) = θa, θ(t) = θb. Then the joint distribution is,
p(θ(t−1)
= θa, θ(t)
= θb) = p(θ(t−1)
= θa)p(θ(t)
= θb|θ(t−1)
= θa)
= p(θ(t−1)
= θa)J(θb|θa),
since as r ≥ 1, θb ∼ J(θb|θa) is accepted with probability 1.
I Now change the order of events.
11 / 18

12. How do we know that we will converge to p(θ|y)
I The joint distribution of θ(t) = θa, θ(t−1) = θb is,
p(θ(t)
= θa, θ(t−1)
= θb) = p(θ(t−1)
= θb)p(θ(t)
= θa|θ(t−1)
= θb)
= p(θ(t−1)
= θb)J(θa|θb)
p(θa|y)/J(θa|θb)
p(θb|y)/J(θb|θa)
,
= p(θa|y)J(θb|θa)
p(θ(t−1) = θb)
p(θb|y)
,
since flipping the event order is equivalent to flipping the density ratio so r must
now be ≤ 1.
12 / 18

13. How do we know that we will converge to p(θ|y)
I From our choice of J(·|·), we know a unique stationary distribution exists. Now
assume θ(t−1) is drawn from the posterior. This means that
p(θ(t−1)
= θa, θ(t)
= θb) = p(θ(t)
= θa, θ(t−1)
= θb),
and
p(θ(t)
= θa) =
Z
p(θ(t)
= θa, θ(t−1)
= θb)dθb
= p(θa|y)
Z
J(θb|θa)
p(θb|y)
p(θb|y)
dθb
= p(θa|y)
Z
J(θb|θa)dθb = p(θa|y)
meaning we can conclude θ(t) is also drawn from the posterior and thus p(θ|y) is
the stationary distribution.
13 / 18

14. But what about the convergence of the Gibbs sampler?
I We have shown the stationary distribution of the Metropolis-Hastings algorithm is
the posterior, p(θ|y).
I Implicitly, we have shown this is also true for the Metropolis algorithm, as the
Metropolis algorithm is just a special case of the Metropolis-Hastings algorithm.
I But what about the Gibbs sampler?
I It turns out the Gibbs sampler is another special case of the Metropolis-Hastings
algorithm. If we view iteration t(i) as the iteration, t0
= tK + i − K, rather than an
iteration within iteration, with candidate θ∗
= (θ
(t)
1 , . . . , θ
(t)
i−1, θ∗
i , θ
(t−1)
i+1 , . . . , θ
(t−1)
K )
and current draw θ(t0
−1)
= (θ
(t)
1 , . . . , θ
(t)
i−1, θ
(t−1)
i , θ
(t−1)
i+1 , . . . , θ
(t−1)
K ), then p(θ∗
i |·) =
J(θ∗
|θ(t−1)
) is a valid and usually non-symmetric jumping distribution.
14 / 18

15. The Gibbs sampler is a special case of Metropolis-Hastings
I Moreover, the Gibbs sampler is an example of the Metropolis-Hastings algorithm
where all moves will be accepted as shown below,
r =
p(θ∗|y)/J(θ∗|θ(t0−1))
p(θ(t0−1)|y)/J(θ(t0−1)|θ∗)
=
p(θ
(t)
1 ,...,θ
(t)
i−1,θ∗
i ,θ
(t−1)
i+1 ,...,θ
(t−1)
K |y)
p(θ∗
i |θ
(t)
1 ,...,θ
(t)
i−1,θ
(t−1)
i+1 ,...,θ
(t−1)
K ,y)
p(θ
(t)
1 ,...,θ
(t)
i−1,θ
(t−1)
i ,θ
(t−1)
i+1 ,...,θ
(t−1)
K |y)
p(θ
(t−1)
i |θ
(t)
1 ,...,θ
(t)
i−1,θ
(t−1)
i+1 ,...,θ
(t−1)
K ,y)
=
p(θ
(t)
1 , . . . , θ
(t)
i−1, θ
(t−1)
i+1 , . . . , θ
(t−1)
K |y)
p(θ
(t)
1 , . . . , θ
(t)
i−1, θ
(t−1)
i+1 , . . . , θ
(t−1)
K |y)
= 1
15 / 18

16. Comments
I As we are using computationally intensive techniques, what might we want to do?
Minimise computational cost wherever possible.
I For example, in previous lectures we have encountered the sufficiency principle.
Therefore you may want to compress the data down to just the sufficient statistics.
I Addition/subtraction are easier operations for a computer than multiplication/
division. Rejection sampling, importance sampling, and Metropolis (-Hastings)
algorithms all require density ratios. If we use (natural) log-densities instead, these
ratios become differences and quicker to compute. Then exponentiate when needed.
16 / 18

17. Some more comments
I Note there is nothing to stop us mixing techniques for a particular problem.
For example, you may be faced with the situation where the parameters split
θ = (θC θNC ) such that p(θi |θ−i , y) can be directly sampled from only if θi ⊆ θC .
However, you can combine a Gibbs sampler with Metropolis(-Hastings), such that
θ
(t)
i ∼ p(θ
(t)
i |θ∗
−i , y) if θi ⊆ θC
θ
(t)
i =
(
θ∗
i if u ≤ min(r, 1) where u ∼ U(0, 1), θ∗
i ∼ g(θ∗
i |·)
θ
(t−1)
i otherwise
if θi 6⊆ θC
with g(θ∗
i |·) being a jumping rule and r defined as in the Metropolis(-Hastings)
algorithm.
17 / 18

18. Conclusion
I In the next lecture, we will look at examples of the Metropolis,
Metropolis-Hastings and Gibbs sampling algorithms in R.
I The code used in these examples will be put up before the lecture,
so you can run the code during the lecture.
18 / 18