Discussion of Faming Liang's talk

  • 794 views
Uploaded on

Discussion at Adap'skiii, Park City, Utah, Jan. 3-4 2011

Discussion at Adap'skiii, Park City, Utah, Jan. 3-4 2011

More in: Education
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
794
On Slideshare
0
From Embeds
0
Number of Embeds
1

Actions

Shares
Downloads
5
Comments
0
Likes
0

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Adaptive MCMH for Bayesian inference of spatial autologistic models discussion by Pierre E. Jacob & Christian P. Robert Universit´ Paris-Dauphine and CREST e http://statisfaction.wordpress.com Adap’skiii, Park City, Utah, Jan. 03, 2011 Adaptive MCMH for Bayesian inference of spatial autologistic modiscussion by Pierre E. Jacob & Christian P. Robert 1/ 7
  • 2. Context: untractable likelihoods Cases when the likelihood function f (y|θ) is unavailable and when the completion step f (y|θ) = f (y, z|θ) dz Z is impossible or too costly because of the dimension of z Adaptive MCMH for Bayesian inference of spatial autologistic mo discussion by Pierre E. Jacob & Christian P. Robert 2/ 7
  • 3. Context: untractable likelihoods Cases when the likelihood function f (y|θ) is unavailable and when the completion step f (y|θ) = f (y, z|θ) dz Z is impossible or too costly because of the dimension of z c MCMC cannot be implemented! Adaptive MCMH for Bayesian inference of spatial autologistic mo discussion by Pierre E. Jacob & Christian P. Robert 2/ 7
  • 4. Context: untractable likelihoods Cases when the likelihood function f (y|θ) is unavailable and when the completion step f (y|θ) = f (y, z|θ) dz Z is impossible or too costly because of the dimension of z c MCMC cannot be implemented! =⇒ adaptive MCMH Adaptive MCMH for Bayesian inference of spatial autologistic mo discussion by Pierre E. Jacob & Christian P. Robert 2/ 7
  • 5. Illustrations Potts model: if y takes values on a grid Y of size k n and f (y|θ) ∝ exp θ Iyl =yi l∼i where l∼i denotes a neighbourhood relation, n moderately large prohibits the computation of the normalising constant Adaptive MCMH for Bayesian inference of spatial autologistic mo discussion by Pierre E. Jacob & Christian P. Robert 3/ 7
  • 6. Compare ABC with adaptive MCMH? ABC algorithm For an observation y ∼ f (y|θ), under the prior π(θ), keep jointly simulating θ ∼ π(θ) , z ∼ f (z|θ ) , until the auxiliary variable z is equal to the observed value, z = y. [Tavar´ et al., 1997] e ABC can also be applied when the normalizing constant is unknown, hence it could be compared to adaptive MCMH, e.g. for a given number of y’s drawn from f , or in terms of execution time. Adaptive MCMH for Bayesian inference of spatial autologistic mo discussion by Pierre E. Jacob & Christian P. Robert 4/ 7
  • 7. ˆSome questions on the estimator R(θt , ϑ) ˆ 1 R(θt , ϑ) = m0 + m0 θi ∈St {θt } I(||θi − ϑ|| ≤ η)     m0 (i) m0 (t)  g(zj , θt ) g(zj , θt )  × I(||θi − ϑ|| ≤ η) (i) + (t)  θi ∈St {θt } j=1 g(zj , ϑ) j=1 g(zj , ϑ)  Why does it bypass the number of repetitions of the θi ’s? Adaptive MCMH for Bayesian inference of spatial autologistic mo discussion by Pierre E. Jacob & Christian P. Robert 5/ 7
  • 8. ˆSome questions on the estimator R(θt , ϑ) ˆ 1 R(θt , ϑ) = m0 + m0 θi ∈St {θt } I(||θi − ϑ|| ≤ η)     m0 (i) m0 (t)  g(zj , θt ) g(zj , θt )  × I(||θi − ϑ|| ≤ η) (i) + (t)  θi ∈St {θt } j=1 g(zj , ϑ) j=1 g(zj , ϑ)  Why does it bypass the number of repetitions of the θi ’s? (i) Why resample the yj ’s only to inverse-weight them later? (i) Why not stick to the original sample and weight the yj ’s directly? This would avoid the necessity to choose m0 . Adaptive MCMH for Bayesian inference of spatial autologistic mo discussion by Pierre E. Jacob & Christian P. Robert 5/ 7
  • 9. Combine adaptations? Adaptive MCMH looks great, but it adds new tuning parameters compared to standard MCMC! It would be nice to be able to adapt the proposal Q(θt , ϑ), e.g. to control the acceptance rate, as in other adaptive MCMC algorithms. Adaptive MCMH for Bayesian inference of spatial autologistic mo discussion by Pierre E. Jacob & Christian P. Robert 6/ 7
  • 10. Combine adaptations? Adaptive MCMH looks great, but it adds new tuning parameters compared to standard MCMC! It would be nice to be able to adapt the proposal Q(θt , ϑ), e.g. to control the acceptance rate, as in other adaptive MCMC algorithms. Then could η be adaptive, since it does as well depend on the (unknown) scale of the posterior distribution of θ ? Adaptive MCMH for Bayesian inference of spatial autologistic mo discussion by Pierre E. Jacob & Christian P. Robert 6/ 7
  • 11. Combine adaptations? Adaptive MCMH looks great, but it adds new tuning parameters compared to standard MCMC! It would be nice to be able to adapt the proposal Q(θt , ϑ), e.g. to control the acceptance rate, as in other adaptive MCMC algorithms. Then could η be adaptive, since it does as well depend on the (unknown) scale of the posterior distribution of θ ? How would the current theoretical framework cope with these additional adaptations? Adaptive MCMH for Bayesian inference of spatial autologistic mo discussion by Pierre E. Jacob & Christian P. Robert 6/ 7
  • 12. Combine adaptations? Adaptive MCMH looks great, but it adds new tuning parameters compared to standard MCMC! It would be nice to be able to adapt the proposal Q(θt , ϑ), e.g. to control the acceptance rate, as in other adaptive MCMC algorithms. Then could η be adaptive, since it does as well depend on the (unknown) scale of the posterior distribution of θ ? How would the current theoretical framework cope with these additional adaptations? In the same spirit, could m (and m0 ) be considered as algorithmic parameters and be adapted as well? What criterion would be used to adapt them? Adaptive MCMH for Bayesian inference of spatial autologistic mo discussion by Pierre E. Jacob & Christian P. Robert 6/ 7
  • 13. Sample size and acceptance rate Another MH algorithm where the “true” acceptance ratio is replaced by a ratio with an unbiased estimator is the Particle MCMC algorithm. Adaptive MCMH for Bayesian inference of spatial autologistic mo discussion by Pierre E. Jacob & Christian P. Robert 7/ 7
  • 14. Sample size and acceptance rate Another MH algorithm where the “true” acceptance ratio is replaced by a ratio with an unbiased estimator is the Particle MCMC algorithm. In PMCMC, the acceptance rate is growing with the number of particles used to estimate the likelihood, ie when the estimation is more precise, the acceptance rate increases and converges towards the acceptance rate of an idealized standard MH algorithm. Adaptive MCMH for Bayesian inference of spatial autologistic mo discussion by Pierre E. Jacob & Christian P. Robert 7/ 7
  • 15. Sample size and acceptance rate Another MH algorithm where the “true” acceptance ratio is replaced by a ratio with an unbiased estimator is the Particle MCMC algorithm. In PMCMC, the acceptance rate is growing with the number of particles used to estimate the likelihood, ie when the estimation is more precise, the acceptance rate increases and converges towards the acceptance rate of an idealized standard MH algorithm. In adaptive MCMH, is there such a link betweeen m, m0 and the number of iterations on one side and the acceptance rate of the algorithm on the other side? Should it be growing when the estimation becomes more and more precise? Adaptive MCMH for Bayesian inference of spatial autologistic mo discussion by Pierre E. Jacob & Christian P. Robert 7/ 7