MLPI Lecture 3: Advanced Sampling Techniques

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Lecture'3
Advanced(Sampling(Techniques
Dahua%Lin
The$Chinese$University$of$Hong$Kong
1

Overview
• Collapsed*Gibbs*Sampling
• Sampling*with*Auxiliary*Variables
• Slice*Sampling
• Simulated*Tempering*&*Parallel*...

Collapsed)Gibbs)Sampling
3

Mo#va#ng(Example
with
We#want#to#sample#from# .
4

Gibbs%Sampling
Draw% %where% :
with% .
5

Gibbs%Sampling%(cont'd)
Draw% :
• How%well%can%this%sampler%perform%when%
?
6

Collapsed)Gibbs)Sampling
• Basic&idea:"replace"the"original"condi0onal"
distribu0on"with"a"condi0onal"distribu0on"of"a"
ma...

Collapsed)Gibbs)Sampling)(cont'd)
• Draw& ,&with& &marginalized&out,&as:
• Draw&
• Can&we&exchange&the&order&of&these&two&...

Basic&Guidelines
• Order%of%steps%ma-ers!
• Generally,*one*can*move*components*from*"being'
sampled"*to*"being'condi0oned'...

Why$do$collapsed$samplers$o/en$perform$be3er$than$
full6ﬂedged$Gibbs$samplers?
10

Rao$Blackwell+Theorem
Consider)an)example) )and)we)want)to)
es1mate) .)Suppose)we)have)two)tractable)
ways)to)do)so:
(1)$d...

Rao$Blackwell+Theorem+(cont'd)
(2)$draw$ $where$ $is$the$
marginal$distribu4on,$and$compute
• Both&are&correct.&By&Strong&...

Rao$Blackwell+Theorem+(cont'd)
• (Rao%Blackwell,Theorem)"Sample"variance"will"be"
reduced"when"some"components"are"margina...

Sampling)with)Auxiliary)Variables
• The%Rao$Blackwell$Theorem%suggests%that%in%order%to%
achieve%be3er%performance,%one%sh...

Use$Auxiliary$Variables
• Specify)an)auxiliary)variable) )and)the)joint)
distribu8on) )such)that)
)for)certain) .
• Design...

Slice&Sampling
16

Slice&Sampler
• Sampling* *is*equivalent*to*sampling*
uniformly*from*the*area*under* :*
.
• Gibbs*sampling*based*on*the*un...

Slice&Sampler&(Illustra0on)
18

Slice&Sampler&(Discussion)
• Slice&sampler"can"mix"very"rapidly,"as"it"will"not"be"
locally"trapped.
• Slice&sampler"is"o7...

Simulated*Tempering
20

Gibbs%Measure
A"Gibbs%measure"is"a"probability"
measure"with"a"density"in"the"
following"form:
Here,% %is%called%the%energ...

Gibbs%Measure%(cont'd)
In#literature#of#MCMC#sampling,#we#o5en#
parameterize#a#Gibbs#measure#using#the#temperature(
parame...

Tempered'MCMC
Typical(MCMC(methods(usually(rely(on(local%moves(to(
explore(the(state(space.(What(is(the(problem?
23

Tempered'MCMC'(cont'd)
Local&traps&o+en&leads&to&very&poor&mixing.&Can&we&
improve&this?
24

Simulated*Tempering
Suppose'we'intend'to'sample'from'
Basic&idea:!Augment!the!target!distribu0on!by!
including!a!temperatu...

Simulated*Tempering*(cont'd)
• We$only$collect$samples$at$the$lowest'temperature,$
.
• The$chain$mixes$much$faster$at$high...

Simulated*Tempering*(Algorithm)
One$itera)on$of$Simulated*Tempering$has$two$steps:
• (Base&transi+on):#update# #at#the#sam...

Simulated*Tempering*(Discussion)
• Set% .%Given% ,%we%should%set% %such%that%
uphill%moves%from%( )%should%have%
a%conside...

Simulated*Tempering*(Discussion)
• All$temperature$levels$play$an$important$role.$So$it$
is$desirable$to$spend$comparable$...

Parallel&Tempering
(Basic'idea)!rather!than!jumping!between!
temperatures,!it!simultaneously!simulate!mul3ple!
chains,!eac...

Parallel&Tempering&(Algorithm)
Each%itera*on%consists%of%the%following%steps:
• (Parallel'update):"simulate"each"replica"w...

Parallel&Tempering&(Algorithm)
• The%proposal%is%accepted%with%probability%
,%where
• We$collect$samples$from$the$base$rep...

Parallel&Tempering&(Jus1ﬁca1on)
Let$ .$We$deﬁne
Obviously,+the+step+of+parallel&update+preserves+the+
invariant+distribu5o...

Parallel&Tempering&(Jus1ﬁca1on)
Note%that%the%step%of%replica(exchange%is%symmetric,%i.e.%
the%probabili0es%of%going%up%an...

Parallel&Tempering&(Discussion)
• It$is$eﬃcient$and$very$easy$to$implement,$especially$
in$a$parallel$compu6ng$environment...

Swendsen'Wang+Algorithm
The$Swendsen'Wang$algorithm$(R.-Swendsen$and$J.-
Wang,$1987)$is$an$eﬃcient$Gibbs$sampling$algorith...

Standard'Ising'Model
The$standard$Ising&model$is$deﬁned$as
where% %for%each% %is%called%a%spin,%and%
.
• Gibbs&sampling&is...

Extended'Ising'Model
• We$extend$the$model$by$introducing$addi5onal$
bond%variables$ ,$each$for$an$edge.$Each$bond$
has$tw...

Extended'Ising'Model'(cont'd)
Here,% %is%described%as%below:
• When& ,& &for&every&
se.ng&of&
• When& ,&
39

Extended'Ising'Model'(cont'd)
With%this%se(ng,% %can%be%wri1en%as:
where% :
• when& ,& &must&be&
• when& ,& &is&set&to&zer...

Swendsen'Wang+Algorithm
!Each!itera*on!consists!of!two!steps:
• (Clustering):"condi(oned"on"the"spins" ,"draw"the"
bonds" ...

Swendsen'Wang+Algorithm
• (Swapping):"condi(oned"on"the"bonds" ,"draw"the"
spins" .
• For"each"connected"component,"draw" ...

Swendsen'Wang+Algorithm+
(Illustra7on)In the case of a rectangular grid, this Gibbs sampling algorithm mixes very rapidly....

Swendsen'Wang+Algorithm+
(Discussion)
• When& &is&large,& &has&a&high&probability&of&
being&set&to&one,&i.e.& &and& &are&l...

Swendsen'Wang+Algorithm+
(Discussion)
• The%Swendsen'Wang%algorithm%can%be%generalized%
to%Po4s%models%(nodes%can%take%val...

Hamiltonian)Monte)Carlo
• An$MCMC$method$based$on$Hamiltonian)Dynamics.$
It$was$originally$devised$for$molecular)simula1on...

Mo#va#ng(Example:(Free(Fall
47

Mo#va#ng(Example:(Free(Fall
• The%change%of%momentum% %is%caused%by%the%
accumula5on/release%of%the%poten(al+energy:
• The...

Hamiltonian)Dynamics
• Hamiltonian)Dynamics"is"a"generalized"theory"of"the"
classical)mechanics,"which"provides"a"elegant"...

Hamilton's+Equa/ons
The$dynamics$of$the$system$is$characterized$by$the$
Hamilton's+Equa/ons:
Here,% %is%called%the%Hamilto...

Hamilton's+Equa/ons+(cont'd)
• The%Hamiltonian% %is%o)en%formulated%as%the%sum%
of%the%poten+al,energy% %and%the%kine+c,en...

Conserva)on*of*Hamiltonian
The$Hamiltonian$is$conserved,$i.e.,$it$is$invariant$over$
,me:
Intui&vely,,this,reﬂects,the,law...

Hamiltonian)Reversibility
• The%Hamiltonian)dynamics%is%reversible
• Let%the%ini+al%states%be% %and%the%states%at%
+me% %b...

Simula'on*of*Hamiltonian*Dynamics
A"natural"idea"to"simulate"Hamiltonian)dynamics"is"to"
use"Euler's)method"over"discre1ze...

Leapfrog)Method
Be#er%results%can%be%obtained%with%leapfrog:
More%importantly,%the%leapfrog%update%is%reversible.
55

Leapfrog)Method)(cont'd)
56

Example
Consider)a)Hamiltonian)system:
Write&down&the&Hamilton's+Equa/ons:
Derive&the&solu-on:
57

Example((Simula-on)
58

Hamiltonian)Monte)Carlo
(Basic'idea):!Consider!the!poten&al)energy!as!the!Gibbs)
energy,!and!introduce!the!"momentums"!as!...

Hamiltonian)Monte)Carlo)(cont'd)
Suppose'the'target&distribu,on'is'
,'then'we'form'an'augmented&
distribu,on'as
Here,%the%...

Hamiltonian)Monte)Carlo)(cont'd)
In#prac(ce,#the#kine%c#energy#is#o2en#formalized#as
61

Hamiltonian)Monte)Carlo)(Algorithm)
Each%itera*on%of%HMC%comprises%two%steps:
• Gibbs%update:#sample#the#momentums# #from#...

Hamiltonian)Monte)Carlo)(Algorithm)
• Metropolis*update:#using#Hamiltonian#dynamics#to#
propose#a#new#state.#Star8ng#from#...

HMC$(Discussion)
• If$the$simula.on$is$exact,$we$will$have$
,$and$thus$the$proposed$state$
should$always$be$accepted.$
• I...

HMC$(Discussion)
• HMC%has%a%high%acceptance%
rate%while%allowing%large%
moves%along%less6constrained%
direc8ons%at%each%i...

Tuning&HMC
• For%eﬃcient%simula1on,%it%is%important%to%choose%
appropriate%values%for%both%the%leapfrog%step%size% %
and%t...

Tuning&HMC&(cont'd)
• For%most%cases,% %and% %can%be%tuned%independently.
• Too%small%a%stepsize%would%waste%computa8on%
8...

Generic'Sampling'Systems
A"number"of"so,ware"systems"are"available"for"
sampling"from"models"speciﬁed"by"the"user
• WinBUG...

Generic'Sampling'Systems'(cont'd)
• JAGS:'"Just'Another'Gibbs'Sampler"
• Cross8pla9orm'support
• Use'a'dialect'of'BUGS
• E...

Generic'Sampling'Systems'(cont'd)
• Stan:'"Sampling'Through'Adap5ve'Neighborhoods"
• Core'wri=en'in'C++,'and'ports'availab...

Stan%Example
data {
int<lower=0> N;
vector[N] x;
vector[N] y;
}
parameters {
real alpha;
real beta;
real<lower=0> sigma;
}...

Generic'Sampling'System'vs.'
Dedicated'Algorithms
Generic' Dedicated'
Easy%to%use% Require%knowledge%and%experience%
High%...

Share Slideshare

LinkedIn
Facebook
Twitter

Embed

Size (px)

Show related Slideshows at end

WordPress Shortcode

Link

Share
Email

MLPI Lecture 1: Maths for Machine L... by Dahua Lin 2064 views
Lecture 5: Variational Estimation a... by Dahua Lin 964 views
MLPI Lecture 0: Overview by Dahua Lin 1103 views
MLPI Lecture 4: Graphical Model and... by Dahua Lin 1064 views
MLPI Lecture 2: Monte Carlo Methods... by Dahua Lin 2203 views
Replica exchange MCMC by . . 13880 views

View on Slideshare

1 of 72 Ad

View on Slideshare

1 of 72 Ad