Discussion of ABC talk by Stefano Cabras, Padova, March 21, 2013
Upcoming SlideShare
Loading in...5
×
 

Discussion of ABC talk by Stefano Cabras, Padova, March 21, 2013

on

  • 2,073 views

This discussion was given after my talk by Stefano Cabras, at the Padova workshop on recent advances in statistical inference

This discussion was given after my talk by Stefano Cabras, at the Padova workshop on recent advances in statistical inference

Statistics

Views

Total Views
2,073
Views on SlideShare
415
Embed Views
1,658

Actions

Likes
0
Downloads
6
Comments
0

6 Embeds 1,658

http://xianblog.wordpress.com 1650
http://newsblur.com 2
http://dev.newsblur.com 2
https://xianblog.wordpress.com 2
http://www.newsblur.com 1
http://webcache.googleusercontent.com 1

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Discussion of ABC talk by Stefano Cabras, Padova, March 21, 2013 Discussion of ABC talk by Stefano Cabras, Padova, March 21, 2013 Presentation Transcript

  • DISCUSSION ofBayesian Computation via empirical likelihood Stefano Cabras, stefano.cabras@uc3m.es Universidad Carlos III de Madrid (Spain) Universit` di Cagliari (Italy) a Padova, 21-Mar-2013
  • Summary ◮ Problem:
  • Summary ◮ Problem: ◮ a statistical model f (y | θ); ◮ a prior π(θ) on θ;
  • Summary ◮ Problem: ◮ a statistical model f (y | θ); ◮ a prior π(θ) on θ; ◮ we want to obtain the posterior πN (θ | y ) ∝ LN (θ)π(θ).
  • Summary ◮ Problem: ◮ a statistical model f (y | θ); ◮ a prior π(θ) on θ; ◮ we want to obtain the posterior πN (θ | y ) ∝ LN (θ)π(θ). ◮ BUT
  • Summary ◮ Problem: ◮ a statistical model f (y | θ); ◮ a prior π(θ) on θ; ◮ we want to obtain the posterior πN (θ | y ) ∝ LN (θ)π(θ). ◮ BUT ◮ IF LN (θ) is not available: ◮ THEN all life ABC;
  • Summary ◮ Problem: ◮ a statistical model f (y | θ); ◮ a prior π(θ) on θ; ◮ we want to obtain the posterior πN (θ | y ) ∝ LN (θ)π(θ). ◮ BUT ◮ IF LN (θ) is not available: ◮ THEN all life ABC; ◮ IF it is not even possible to simulate from f (y | θ):
  • Summary ◮ Problem: ◮ a statistical model f (y | θ); ◮ a prior π(θ) on θ; ◮ we want to obtain the posterior πN (θ | y ) ∝ LN (θ)π(θ). ◮ BUT ◮ IF LN (θ) is not available: ◮ THEN all life ABC; ◮ IF it is not even possible to simulate from f (y | θ): ◮ THEN replace LN (θ) with LEL (θ) (the proposed BCel procedure): π(θ|y ) ∝ LEL (θ) × π(θ). .
  • ... what remains about the f (y | θ) ?
  • ... what remains about the f (y | θ) ? ◮ Recall that the Empirical Likelihood is defined, for iid sample, by means of a set of constraints: Ef (y |θ) [h(Y , θ)] = 0.
  • ... what remains about the f (y | θ) ? ◮ Recall that the Empirical Likelihood is defined, for iid sample, by means of a set of constraints: Ef (y |θ) [h(Y , θ)] = 0. ◮ The relation between θ and obs. Y is model conditioned and expressed by h(Y , θ);
  • ... what remains about the f (y | θ) ? ◮ Recall that the Empirical Likelihood is defined, for iid sample, by means of a set of constraints: Ef (y |θ) [h(Y , θ)] = 0. ◮ The relation between θ and obs. Y is model conditioned and expressed by h(Y , θ); ◮ Constraints are model driven and so there is still a timid trace of f (y | θ) in BCel .
  • ... what remains about the f (y | θ) ? ◮ Recall that the Empirical Likelihood is defined, for iid sample, by means of a set of constraints: Ef (y |θ) [h(Y , θ)] = 0. ◮ The relation between θ and obs. Y is model conditioned and expressed by h(Y , θ); ◮ Constraints are model driven and so there is still a timid trace of f (y | θ) in BCel . ◮ Examples:
  • ... what remains about the f (y | θ) ? ◮ Recall that the Empirical Likelihood is defined, for iid sample, by means of a set of constraints: Ef (y |θ) [h(Y , θ)] = 0. ◮ The relation between θ and obs. Y is model conditioned and expressed by h(Y , θ); ◮ Constraints are model driven and so there is still a timid trace of f (y | θ) in BCel . ◮ Examples: ◮ The coalescent model example is illuminating in suggesting the score of the pairwise likelihood;
  • ... what remains about the f (y | θ) ? ◮ Recall that the Empirical Likelihood is defined, for iid sample, by means of a set of constraints: Ef (y |θ) [h(Y , θ)] = 0. ◮ The relation between θ and obs. Y is model conditioned and expressed by h(Y , θ); ◮ Constraints are model driven and so there is still a timid trace of f (y | θ) in BCel . ◮ Examples: ◮ The coalescent model example is illuminating in suggesting the score of the pairwise likelihood; ◮ The residuals in GARCH models.
  • ... a suggestion What if we do not even known h(·) ?
  • ... how to elicit h(·) automatically
  • ... how to elicit h(·) automatically
  • ... how to elicit h(·) automatically ◮ Set h(Y , θ) = Y − g (θ), where g (θ) = Ef (y |θ) (Y |θ), is the regression function of Y |θ;
  • ... how to elicit h(·) automatically ◮ Set h(Y , θ) = Y − g (θ), where g (θ) = Ef (y |θ) (Y |θ), is the regression function of Y |θ; ◮ g (θ) should be replaced by an estimator g (θ).
  • How to estimate g (θ) ? 1 ... similar to Fearnhead, P. and D. Prangle (JRRS-B, 2012) or Cabras, Castellanos, Ruli (Ercim-2012, Oviedo).
  • How to estimate g (θ) ? ◮ Use a once forever pilot-run simulation study: 1 1 ... similar to Fearnhead, P. and D. Prangle (JRRS-B, 2012) or Cabras, Castellanos, Ruli (Ercim-2012, Oviedo).
  • How to estimate g (θ) ? ◮ Use a once forever pilot-run simulation study: 1 1. Consider a grid (or regular lattice) of θ made by M points: θ1 , . . . , θM 1 ... similar to Fearnhead, P. and D. Prangle (JRRS-B, 2012) or Cabras, Castellanos, Ruli (Ercim-2012, Oviedo).
  • How to estimate g (θ) ? ◮ Use a once forever pilot-run simulation study: 1 1. Consider a grid (or regular lattice) of θ made by M points: θ1 , . . . , θM 2. Simulate the corresponding y1 , . . . , yM 1 ... similar to Fearnhead, P. and D. Prangle (JRRS-B, 2012) or Cabras, Castellanos, Ruli (Ercim-2012, Oviedo).
  • How to estimate g (θ) ? ◮ Use a once forever pilot-run simulation study: 1 1. Consider a grid (or regular lattice) of θ made by M points: θ1 , . . . , θM 2. Simulate the corresponding y1 , . . . , yM 3. Regress y1 , . . . , yM on θ 1 , . . . , θ M obtaining g (θ). 1 ... similar to Fearnhead, P. and D. Prangle (JRRS-B, 2012) or Cabras, Castellanos, Ruli (Ercim-2012, Oviedo).
  • ... example: y ∼ N(|θ|, 1) For a pilot run of M = 1000 we have g (θ) = |θ|. ˆ Pilot−Run s.s. g(θ) 10 y 5 0 −10 −5 0 5 10 θ
  • ... example: y ∼ N(|θ|, 1) Suppose to draw a n = 100 sample from θ = 2: Histogram of y 20 15 Frequency 10 5 0 0 1 2 3 4 y
  • ... example: y ∼ N(|θ|, 1) The Empirical Likelihood is this 2.5 2.0 Emp. Lik. 1.5 1.0 −4 −2 0 2 4 θ
  • 1st Point: Do we need necessarily have to use f (y | θ) ?
  • 1st Point: Do we need necessarily have to use f (y | θ) ? ◮ The above data maybe drawn from a (e.g.) a Half Normal;
  • 1st Point: Do we need necessarily have to use f (y | θ) ? ◮ The above data maybe drawn from a (e.g.) a Half Normal; ◮ How this is reflected in the BCel ?
  • 1st Point: Do we need necessarily have to use f (y | θ) ? ◮ The above data maybe drawn from a (e.g.) a Half Normal; ◮ How this is reflected in the BCel ? ◮ For a given data y;
  • 1st Point: Do we need necessarily have to use f (y | θ) ? ◮ The above data maybe drawn from a (e.g.) a Half Normal; ◮ How this is reflected in the BCel ? ◮ For a given data y; ◮ and h(Y , θ) fixed;
  • 1st Point: Do we need necessarily have to use f (y | θ) ? ◮ The above data maybe drawn from a (e.g.) a Half Normal; ◮ How this is reflected in the BCel ? ◮ For a given data y; ◮ and h(Y , θ) fixed; ◮ the LEL (θ) is the same regardless of f (y | θ).
  • 1st Point: Do we need necessarily have to use f (y | θ) ? ◮ The above data maybe drawn from a (e.g.) a Half Normal; ◮ How this is reflected in the BCel ? ◮ For a given data y; ◮ and h(Y , θ) fixed; ◮ the LEL (θ) is the same regardless of f (y | θ). Can we ignore f (y | θ) ?
  • 2nd Point: Sample free vs Simulation free
  • 2nd Point: Sample free vs Simulation free ◮ The Empirical Likelihood is ”simulation free” but not ”sample free”, i.e.
  • 2nd Point: Sample free vs Simulation free ◮ The Empirical Likelihood is ”simulation free” but not ”sample free”, i.e. ◮ LEL (θ) → LN (θ) for n → ∞, ◮ implying π(θ|y) → πN (θ | y ) asymptotically in n.
  • 2nd Point: Sample free vs Simulation free ◮ The Empirical Likelihood is ”simulation free” but not ”sample free”, i.e. ◮ LEL (θ) → LN (θ) for n → ∞, ◮ implying π(θ|y) → πN (θ | y ) asymptotically in n. ◮ The ABC is ”sample free” but not ”simulation free”, i.e.
  • 2nd Point: Sample free vs Simulation free ◮ The Empirical Likelihood is ”simulation free” but not ”sample free”, i.e. ◮ LEL (θ) → LN (θ) for n → ∞, ◮ implying π(θ|y) → πN (θ | y ) asymptotically in n. ◮ The ABC is ”sample free” but not ”simulation free”, i.e. ◮ π(θ|ρ(s(y), so bs) < ǫ) → πN (θ | y ) as ǫ → 0 ◮ implying convergence in the number of simulations if s(y ) were sufficient.
  • 2nd Point: Sample free vs Simulation free ◮ The Empirical Likelihood is ”simulation free” but not ”sample free”, i.e. ◮ LEL (θ) → LN (θ) for n → ∞, ◮ implying π(θ|y) → πN (θ | y ) asymptotically in n. ◮ The ABC is ”sample free” but not ”simulation free”, i.e. ◮ π(θ|ρ(s(y), so bs) < ǫ) → πN (θ | y ) as ǫ → 0 ◮ implying convergence in the number of simulations if s(y ) were sufficient. A quick answer recommends use BCel BUT a small sample would recommend ABC ?
  • 3nd Point: How to validate a pseudo-posteriorπ(θ|y ) ∝ LEL (θ) × π(θ) ?
  • 3nd Point: How to validate a pseudo-posteriorπ(θ|y ) ∝ LEL (θ) × π(θ) ? ◮ The use of pseudo-likelihoods is not new in the Bayesian setting:
  • 3nd Point: How to validate a pseudo-posteriorπ(θ|y ) ∝ LEL (θ) × π(θ) ? ◮ The use of pseudo-likelihoods is not new in the Bayesian setting: ◮ Empirical Likelihoods:
  • 3nd Point: How to validate a pseudo-posteriorπ(θ|y ) ∝ LEL (θ) × π(θ) ? ◮ The use of pseudo-likelihoods is not new in the Bayesian setting: ◮ Empirical Likelihoods: ◮ Lazar (Biometrika, 2003) ◮ Mengersen et al. (PNAS, 2012) ◮ ...
  • 3nd Point: How to validate a pseudo-posteriorπ(θ|y ) ∝ LEL (θ) × π(θ) ? ◮ The use of pseudo-likelihoods is not new in the Bayesian setting: ◮ Empirical Likelihoods: ◮ Lazar (Biometrika, 2003) ◮ Mengersen et al. (PNAS, 2012) ◮ ... ◮ Modified-Likelihoods:
  • 3nd Point: How to validate a pseudo-posteriorπ(θ|y ) ∝ LEL (θ) × π(θ) ? ◮ The use of pseudo-likelihoods is not new in the Bayesian setting: ◮ Empirical Likelihoods: ◮ Lazar (Biometrika, 2003) ◮ Mengersen et al. (PNAS, 2012) ◮ ... ◮ Modified-Likelihoods: ◮ Ventura et al. (JASA, 2009) ◮ Chang and Mukerjee (Stat. & Prob. Letters 2006) ◮ ...
  • 3nd Point: How to validate a pseudo-posteriorπ(θ|y ) ∝ LEL (θ) × π(θ) ? ◮ The use of pseudo-likelihoods is not new in the Bayesian setting: ◮ Empirical Likelihoods: ◮ Lazar (Biometrika, 2003) ◮ Mengersen et al. (PNAS, 2012) ◮ ... ◮ Modified-Likelihoods: ◮ Ventura et al. (JASA, 2009) ◮ Chang and Mukerjee (Stat. & Prob. Letters 2006) ◮ ... ◮ Quasi-Likelihoods:
  • 3nd Point: How to validate a pseudo-posteriorπ(θ|y ) ∝ LEL (θ) × π(θ) ? ◮ The use of pseudo-likelihoods is not new in the Bayesian setting: ◮ Empirical Likelihoods: ◮ Lazar (Biometrika, 2003) ◮ Mengersen et al. (PNAS, 2012) ◮ ... ◮ Modified-Likelihoods: ◮ Ventura et al. (JASA, 2009) ◮ Chang and Mukerjee (Stat. & Prob. Letters 2006) ◮ ... ◮ Quasi-Likelihoods: ◮ Lin (Statist. Methodol., 2006) ◮ Greco et al. (JSPI, 2008) ◮ Ventura et al. (JSPI, 2010) ◮ ...
  • 3nd Point: How to validate a pseudo-posteriorπ(θ|y ) ∝ LEL (θ) × π(θ) ? ◮ The use of pseudo-likelihoods is not new in the Bayesian setting: ◮ Empirical Likelihoods: ◮ Lazar (Biometrika, 2003) : examples and coverages of C.I. ◮ Mengersen et al. (PNAS, 2012) ◮ ... ◮ Modified-Likelihoods: ◮ Ventura et al. (JASA, 2009) ◮ Chang and Mukerjee (Stat. & Prob. Letters 2006) ◮ ... ◮ Quasi-Likelihoods: ◮ Lin (Statist. Methodol., 2006) ◮ Greco et al. (JSPI, 2008) ◮ Ventura et al. (JSPI, 2010) ◮ ...
  • 3nd Point: How to validate a pseudo-posteriorπ(θ|y ) ∝ LEL (θ) × π(θ) ? ◮ The use of pseudo-likelihoods is not new in the Bayesian setting: ◮ Empirical Likelihoods: ◮ Lazar (Biometrika, 2003) : examples and coverages of C.I. ◮ Mengersen et al. (PNAS, 2012) : examples and coverages of C.I. ◮ ... ◮ Modified-Likelihoods: ◮ Ventura et al. (JASA, 2009) ◮ Chang and Mukerjee (Stat. & Prob. Letters 2006) ◮ ... ◮ Quasi-Likelihoods: ◮ Lin (Statist. Methodol., 2006) ◮ Greco et al. (JSPI, 2008) ◮ Ventura et al. (JSPI, 2010) ◮ ...
  • 3nd Point: How to validate a pseudo-posteriorπ(θ|y ) ∝ LEL (θ) × π(θ) ? ◮ The use of pseudo-likelihoods is not new in the Bayesian setting: ◮ Empirical Likelihoods: ◮ Lazar (Biometrika, 2003) : examples and coverages of C.I. ◮ Mengersen et al. (PNAS, 2012) : examples and coverages of C.I. ◮ ... ◮ Modified-Likelihoods: ◮ Ventura et al. (JASA, 2009) : second order matching properties; ◮ Chang and Mukerjee (Stat. & Prob. Letters 2006) ◮ ... ◮ Quasi-Likelihoods: ◮ Lin (Statist. Methodol., 2006) ◮ Greco et al. (JSPI, 2008) ◮ Ventura et al. (JSPI, 2010) ◮ ...
  • 3nd Point: How to validate a pseudo-posteriorπ(θ|y ) ∝ LEL (θ) × π(θ) ? ◮ The use of pseudo-likelihoods is not new in the Bayesian setting: ◮ Empirical Likelihoods: ◮ Lazar (Biometrika, 2003) : examples and coverages of C.I. ◮ Mengersen et al. (PNAS, 2012) : examples and coverages of C.I. ◮ ... ◮ Modified-Likelihoods: ◮ Ventura et al. (JASA, 2009) : second order matching properties; ◮ Chang and Mukerjee (Stat. & Prob. Letters 2006) : examples; ◮ ... ◮ Quasi-Likelihoods: ◮ Lin (Statist. Methodol., 2006) ◮ Greco et al. (JSPI, 2008) ◮ Ventura et al. (JSPI, 2010) ◮ ...
  • 3nd Point: How to validate a pseudo-posteriorπ(θ|y ) ∝ LEL (θ) × π(θ) ? ◮ The use of pseudo-likelihoods is not new in the Bayesian setting: ◮ Empirical Likelihoods: ◮ Lazar (Biometrika, 2003) : examples and coverages of C.I. ◮ Mengersen et al. (PNAS, 2012) : examples and coverages of C.I. ◮ ... ◮ Modified-Likelihoods: ◮ Ventura et al. (JASA, 2009) : second order matching properties; ◮ Chang and Mukerjee (Stat. & Prob. Letters 2006) : examples; ◮ ... ◮ Quasi-Likelihoods: ◮ Lin (Statist. Methodol., 2006) : examples; ◮ Greco et al. (JSPI, 2008) : robustness properties; ◮ Ventura et al. (JSPI, 2010) : examples and coverages of C.I.; ◮ ...
  • 3nd Point: How to validate a pseudo-posteriorπ(θ|y ) ∝ LEL (θ) × π(θ) ? ◮ Monahan & Boos (Biometrika, 1992) proposed a notion of validity:
  • 3nd Point: How to validate a pseudo-posteriorπ(θ|y ) ∝ LEL (θ) × π(θ) ? ◮ Monahan & Boos (Biometrika, 1992) proposed a notion of validity: π(θ|y ) should obey the laws of probability in a fashion that is consistent with statements derived from Bayes’rule.
  • 3nd Point: How to validate a pseudo-posteriorπ(θ|y ) ∝ LEL (θ) × π(θ) ? ◮ Monahan & Boos (Biometrika, 1992) proposed a notion of validity: π(θ|y ) should obey the laws of probability in a fashion that is consistent with statements derived from Bayes’rule. ◮ Very difficult!
  • 3nd Point: How to validate a pseudo-posteriorπ(θ|y ) ∝ LEL (θ) × π(θ) ? ◮ Monahan & Boos (Biometrika, 1992) proposed a notion of validity: π(θ|y ) should obey the laws of probability in a fashion that is consistent with statements derived from Bayes’rule. ◮ Very difficult! How to validate the pseudo-posterior π(θ|y ) when this is not possible ?
  • ... Last point: the ABC is still a terrific tool
  • ... Last point: the ABC is still a terrific tool ◮ ... a lot of references:
  • ... Last point: the ABC is still a terrific tool ◮ ... a lot of references: ◮ Statistical Journals;
  • ... Last point: the ABC is still a terrific tool ◮ ... a lot of references: ◮ Statistical Journals; ◮ Twitter;
  • ... Last point: the ABC is still a terrific tool ◮ ... a lot of references: ◮ Statistical Journals; ◮ Twitter; ◮ Xiang’s blog ( xianblog.wordpress.com )
  • ... Last point: the ABC is still a terrific tool ◮ ... a lot of references: ◮ Statistical Journals; ◮ Twitter; ◮ Xiang’s blog ( xianblog.wordpress.com ) ◮ ... it is tailored to Approximate LN (θ).
  • ... Last point: the ABC is still a terrific tool ◮ ... a lot of references: ◮ Statistical Journals; ◮ Twitter; ◮ Xiang’s blog ( xianblog.wordpress.com ) ◮ ... it is tailored to Approximate LN (θ). Where is the A in BCel ?