This document provides an introduction to Hamiltonian Monte Carlo (HMC), a Markov chain Monte Carlo method for sampling from distributions. It begins with an overview and motivation, then covers preliminary concepts. The main section explains how HMC constructs a Markov chain using Hamiltonian dynamics to efficiently explore complex, high-dimensional distributions. A demonstration compares HMC to the random walk Metropolis algorithm on a challenging "banana-shaped" distribution. The document concludes with a discussion of future work and applications of HMC.

1. Introduction To
Hamiltonian Monte Carlo
Kota Takeda
Department of Mathematics
Kyoto University

2. Program
1. Introduction
2. Preliminary
3. Hamiltonian Monte Carlo
4. Demonstration
5. Discussion

3. 1. Introduction
Introduction to today’s presentation and Hamiltonian Monte Carlo

4. Introduction
This is intuitive introduction of Hamiltonian Monte
Carlo.
Almost all parts of this presentation is from
“A Conceptual Introduction to Hamiltonian Monte Carlo”, Michael
Betancourt .

5. Motivation
Computing expectations with respect to the posterior distribution in
Bayesian inference
𝔼𝜋 𝑓 =
𝑄
𝑓 𝑞 𝜋 𝑞 𝑑𝑞

6. 2. Preliminary
Preparing to constructing Hamiltonian Monte Carlo

7. Notation
 Sample space 𝑄 = ℝ𝑑
.
 Denote 𝑀 > 0, if M is symmetric positive
definite matrix.
 𝑁 0, Σ ; for mean zero covariance Σ Gaussian
distribution.
 𝑞 ~ 𝜋; 𝑞 is distributed by 𝜋.
 Use “≡” for definition

8. Monte Carlo Method
Approximate distribution 𝜋(𝑞) with large samples
𝔼𝜋 𝑓 ≈
1
𝑁
𝑛=1
𝑁
𝑓(𝑞𝑛)

9. Sampling Problem
How do we make efficient sampling from given
target distribution 𝜋 𝑞 ?
Define sample problem we are interested in

10. Markov Chain Monte Carlo: MCMC
Desired transition kernel 𝕋 𝑞′
𝑞 to satisfy
reversibility
Construct Markov chain that converges to target distribution by Random
proposal and Acceptance
𝜋 𝑞 𝕋 𝑞′
𝑞 = 𝜋 𝑞′ 𝕋 𝑞 𝑞′

11. Random Walk Metropolis: RWM
One simple implementation
of Metropolis-Hasting
algorithm
 Require: target 𝜋 𝑞
 proposal covariance Σ > 0
1. Random proposal
2. Accept
given qn, draw qprop ~ N qn, Σ
accept qn+1 = qprop with probability
min{1,
π(qprop)
π(qn)
}

12. Issue of MCMC
Poor performance with high dimension and
complex target distributions

13. Hamilton Dynamics
Hamilton equation on phase
space
 preserve volume in phase
space(Liouville’s Theorem)
 preserve total energy in phase
space, which is Hamiltonian
 time reversal symmetry
𝑑𝑞
𝑑𝑡
=
𝑑𝐻
𝑑𝑝
𝑑𝑝
𝑑𝑡
= −
𝑑𝐻
𝑑𝑞

14. 3. Hamiltonian Monte Carlo
Constructing Hamiltonian Monte Carlo

15. Hamiltonian Monte Carlo: HMC
A. One approach is using geometric information
of target and constructing conservative transition
kernel by Hamilton flow.
Q. How do we make efficient sampling from given target distribution
𝝅𝑼 𝒒 ?

16. Information of Gradient
1. Consider 𝜋𝑈 𝑞 = 𝑒−𝑈(𝑞)
, where 𝑈 𝑞 ≡
− log(𝜋𝑈 𝑞 )
2. But gradient
dU
dq
pulls us the mode of density!
3. → Need to introduce momentum 𝑝
Using geometric information of target density

17. Expand Sample Space
1. Expand to phase space 𝑞 → 𝑞, 𝑝 with 𝑝
2. Choose conditional distribution 𝜋𝐾(𝑝|𝑞)
3. Lift 𝜋𝑈 𝑞 to 𝜋𝐻 𝑞, 𝑝 ≡ 𝜋𝐾 𝑝|𝑞 𝜋𝑈 𝑞
Expand sample space to phase space,
We can always gain sample q by projection(marginalization).

18. Choice of Kinetic Energy
1. Choose Kinetic Energy 𝐾(𝑞, 𝑝)
2. Conditional distribution of momentum determined
by 𝜋𝐾 𝑝 𝑞 ≡ 𝑒−𝐾 𝑞,𝑝
To define conditional distribution of momentum 𝜋𝐾 𝑝 𝑞 , a user choose kinetic
energy.
In simple case, let 𝐾 𝑞, 𝑝 =
1
2
𝑝𝑇
𝑀−1
𝑝.

19. Hamiltonian
1. 𝐻 𝑞, 𝑝 ≡ 𝐾 𝑞, 𝑝 + 𝑈(𝑞)
2. 𝜋𝐻 𝑞, 𝑝 ≡ 𝜋𝐾 𝑝|𝑞 𝜋𝑈 𝑞 = 𝑒−𝐻(𝑞,𝑝)
Hamiltonian 𝐻 and canonical distribution 𝜋𝐻 are defined as below.

20. Symplectic integrator
Scheme exactly
preserving volume
 Assume: 𝐾 𝑞, 𝑝 ≡
1
2
𝑝𝑇
𝑀−1
𝑝
with Mass matrix 𝑀 > 0,
 𝑈(𝑞) is differentiable.
 Require: step size 𝜀 > 0
𝑝
𝑛+
1
2
= 𝑝𝑛 −
𝜀
2
𝑑𝑈
𝑑𝑞
(𝑞𝑛)
𝑞𝑛+1 = 𝑞𝑛 + 𝜀𝑀−1
𝑝
𝑛+
1
2
𝑝𝑛+1 = 𝑝
𝑛+
1
2
−
𝜀
2
𝑑𝑈
𝑑𝑞
(𝑞𝑛+1)
𝜑𝜀 𝑞𝑛, 𝑝𝑛 ≡ 𝑞𝑛+1, 𝑝𝑛+1

21. Numerical Hamilton Flow
1. 𝜑𝜀 𝑞𝑛, 𝑝𝑛 ≡ 𝑞𝑛+1, 𝑝𝑛+1
2. 𝜑𝜀
𝐿
≡ 𝜑𝜀 ∘ ⋯ ∘ 𝜑𝜀 for integer 𝐿.
Define numerical Hamilton flow by symplectic integrator on previous page
and define 𝐿 times composition

22. HMC Algorithm
Hybrid of deterministic and
stochastic transitions
 𝐻 𝑞, 𝑝 ≡
1
2
𝑝𝑇
𝑀−1
𝑝 + 𝑈(𝑞)
 Require: 𝑀 > 0, 𝐿 ∈ ℕ, 𝜀 > 0
1. Energy Lift
2. Hamilton flow
3. Accept
given 𝑞𝑛, draw 𝑝𝑛 ~ 𝑁 0, 𝑀
𝑞𝑝𝑟𝑜𝑝, 𝑝𝑝𝑟𝑜𝑝 = 𝜑𝜀
𝐿(𝑞𝑛, 𝑝𝑛)
accept 𝑞𝑛+1 = 𝑞prop with probability
min{1, exp(𝐻(𝑞𝑛, 𝑝𝑛) − 𝐻(𝑞𝑝𝑟𝑜𝑝, −𝑝𝑝𝑟𝑜𝑝))}

23. Conceptual Animation of HMC Algorithm
1. Energy Lift
2. Hamilton flow
3. (Accept)
 Sample space 𝑄 = ℝ1
 target 𝜋𝑈 𝑞 = 𝑒−
1
2
𝑞2
, 𝜋𝐾 𝑝|𝑞 =
𝑒−
1
2
𝑝2
⟹ 𝐻 𝑞, 𝑝 =
1
2
𝑞2 +
1
2
𝑝2
Phase space

24. Advantage of Hamiltonian Monte Carlo
 Rich Theoretical Support
effective for wider class of target than non-gradient method
 Computational Efficiency
Fast exploration and large acceptance probability

25. 4. Demonstration
Demonstration of efficient Hamiltonian Monte Carlo compared with
Random Walk Metropolis

26. Strongly Nonlinear Banana Gaussian
Test Target distribution is
Strongly banana gaussian
 Sample space 𝑄 = ℝ2
 target 𝜋𝑈 𝑞1, 𝑞2 = 𝑔 ∘ 𝜓𝑏=0.1 𝑞1, 𝑞2
where
 𝑔 𝑞1, 𝑞2 = 𝑒−
1
200
𝑞1
2−
1
2
𝑞2
2
 𝜓𝑏: 𝑞1, 𝑞2 ⟼ (𝑞1, 𝑞2 + 𝒃𝒒𝟏
𝟐
− 100𝑏)

27. RWM vs HMC after 10 iterations
RWM: Σ = 2𝐼 HMC: 𝜀 = 0.5, 𝐿 = 10, 𝑀 = 𝐼

28. RWM vs HMC after 100 iterations
RWM: Σ = 2𝐼 HMC: 𝜀 = 0.5, 𝐿 = 10, 𝑀 = 𝐼

29. RWM vs HMC after 1000 iterations
RWM: Σ = 2𝐼 HMC: 𝜀 = 0.5, 𝐿 = 10, 𝑀 = 𝐼

30. Stats*
RWM: Σ = 2𝐼 HMC: 𝜀 = 0.5, 𝐿 = 10, 𝑀 = 𝐼
Try Sample Time*(ms) Acceptance
Probability
10 3.02 0.300
100 25.8 0.260
1000 179 0.288
Try Sample Time*(ms) Acceptance
Probability
10 7.43 0.900
100 34.3 0.970
1000 474 0.940
*Not guaranteed value, just a reference.
*Time is measured by jupyter magic command `%%time`.

31. 5. Discussion
Discussion about future work or application of Hamiltonian Monte Carlo

32. Future work
 Studying mathematical guarantee and guideline
 Adaptive tuning of parameters
 Selecting Integrators
 Generalizing to infinite-dimensional sample
space
 Introducing inverse temperature

33. Discussion
 Particle Filter
 Variational method
 Inverse problem
 “Estimation of Hydraulic Conductivity from Steady
State Seepage Flow Using Hamiltonian Monte
Carlo”- 01/02/2021

34. For Your Information…
“Remember that using Bayes' Theorem doesn't make
you a Bayesian. Quantifying uncertainty with probability
makes you a Bayesian.” - Michael Betancourt
https://twitter.com/betanalpha/status/81701286064363
5204
This is pinned tweet of Michael Betancourt.

35. References
 “A Conceptual Introduction to Hamiltonian Monte Carlo”
 “The Geometric Foundations of Hamiltonian Monte Carlo”
 “The Adaptive proposal distribution for Random Walk
Metropolis Algorithm” – only for banana Gaussian
 “Estimation of Hydraulic Conductivity from Steady State
Seepage Flow Using Hamiltonian Monte Carlo” -
http://soil.en.a.u-tokyo.ac.jp/jsidre/search/PDFs/20/%5B1-
52%5D.pdf

36. Documents
 Detail documents on my site
https://kotatakeda.github.io/math/2021/01/03/
hamiltonian-monte-carlo.html