Upcoming SlideShare
×

# Discussion of ABC talk by Stefano Cabras, Padova, March 21, 2013

2,307 views

Published on

This discussion was given after my talk by Stefano Cabras, at the Padova workshop on recent advances in statistical inference

Published in: Education
• Full Name
Comment goes here.

Are you sure you want to Yes No
• Be the first to comment

• Be the first to like this

### Discussion of ABC talk by Stefano Cabras, Padova, March 21, 2013

1. 1. DISCUSSION ofBayesian Computation via empirical likelihood Stefano Cabras, stefano.cabras@uc3m.es Universidad Carlos III de Madrid (Spain) Universit` di Cagliari (Italy) a Padova, 21-Mar-2013
2. 2. Summary ◮ Problem:
3. 3. Summary ◮ Problem: ◮ a statistical model f (y | θ); ◮ a prior π(θ) on θ;
4. 4. Summary ◮ Problem: ◮ a statistical model f (y | θ); ◮ a prior π(θ) on θ; ◮ we want to obtain the posterior πN (θ | y ) ∝ LN (θ)π(θ).
5. 5. Summary ◮ Problem: ◮ a statistical model f (y | θ); ◮ a prior π(θ) on θ; ◮ we want to obtain the posterior πN (θ | y ) ∝ LN (θ)π(θ). ◮ BUT
6. 6. Summary ◮ Problem: ◮ a statistical model f (y | θ); ◮ a prior π(θ) on θ; ◮ we want to obtain the posterior πN (θ | y ) ∝ LN (θ)π(θ). ◮ BUT ◮ IF LN (θ) is not available: ◮ THEN all life ABC;
7. 7. Summary ◮ Problem: ◮ a statistical model f (y | θ); ◮ a prior π(θ) on θ; ◮ we want to obtain the posterior πN (θ | y ) ∝ LN (θ)π(θ). ◮ BUT ◮ IF LN (θ) is not available: ◮ THEN all life ABC; ◮ IF it is not even possible to simulate from f (y | θ):
8. 8. Summary ◮ Problem: ◮ a statistical model f (y | θ); ◮ a prior π(θ) on θ; ◮ we want to obtain the posterior πN (θ | y ) ∝ LN (θ)π(θ). ◮ BUT ◮ IF LN (θ) is not available: ◮ THEN all life ABC; ◮ IF it is not even possible to simulate from f (y | θ): ◮ THEN replace LN (θ) with LEL (θ) (the proposed BCel procedure): π(θ|y ) ∝ LEL (θ) × π(θ). .
9. 9. ... what remains about the f (y | θ) ?
10. 10. ... what remains about the f (y | θ) ? ◮ Recall that the Empirical Likelihood is deﬁned, for iid sample, by means of a set of constraints: Ef (y |θ) [h(Y , θ)] = 0.
11. 11. ... what remains about the f (y | θ) ? ◮ Recall that the Empirical Likelihood is deﬁned, for iid sample, by means of a set of constraints: Ef (y |θ) [h(Y , θ)] = 0. ◮ The relation between θ and obs. Y is model conditioned and expressed by h(Y , θ);
12. 12. ... what remains about the f (y | θ) ? ◮ Recall that the Empirical Likelihood is deﬁned, for iid sample, by means of a set of constraints: Ef (y |θ) [h(Y , θ)] = 0. ◮ The relation between θ and obs. Y is model conditioned and expressed by h(Y , θ); ◮ Constraints are model driven and so there is still a timid trace of f (y | θ) in BCel .
13. 13. ... what remains about the f (y | θ) ? ◮ Recall that the Empirical Likelihood is deﬁned, for iid sample, by means of a set of constraints: Ef (y |θ) [h(Y , θ)] = 0. ◮ The relation between θ and obs. Y is model conditioned and expressed by h(Y , θ); ◮ Constraints are model driven and so there is still a timid trace of f (y | θ) in BCel . ◮ Examples:
14. 14. ... what remains about the f (y | θ) ? ◮ Recall that the Empirical Likelihood is deﬁned, for iid sample, by means of a set of constraints: Ef (y |θ) [h(Y , θ)] = 0. ◮ The relation between θ and obs. Y is model conditioned and expressed by h(Y , θ); ◮ Constraints are model driven and so there is still a timid trace of f (y | θ) in BCel . ◮ Examples: ◮ The coalescent model example is illuminating in suggesting the score of the pairwise likelihood;
15. 15. ... what remains about the f (y | θ) ? ◮ Recall that the Empirical Likelihood is deﬁned, for iid sample, by means of a set of constraints: Ef (y |θ) [h(Y , θ)] = 0. ◮ The relation between θ and obs. Y is model conditioned and expressed by h(Y , θ); ◮ Constraints are model driven and so there is still a timid trace of f (y | θ) in BCel . ◮ Examples: ◮ The coalescent model example is illuminating in suggesting the score of the pairwise likelihood; ◮ The residuals in GARCH models.
16. 16. ... a suggestion What if we do not even known h(·) ?
17. 17. ... how to elicit h(·) automatically
18. 18. ... how to elicit h(·) automatically
19. 19. ... how to elicit h(·) automatically ◮ Set h(Y , θ) = Y − g (θ), where g (θ) = Ef (y |θ) (Y |θ), is the regression function of Y |θ;
20. 20. ... how to elicit h(·) automatically ◮ Set h(Y , θ) = Y − g (θ), where g (θ) = Ef (y |θ) (Y |θ), is the regression function of Y |θ; ◮ g (θ) should be replaced by an estimator g (θ).
21. 21. How to estimate g (θ) ? 1 ... similar to Fearnhead, P. and D. Prangle (JRRS-B, 2012) or Cabras, Castellanos, Ruli (Ercim-2012, Oviedo).
22. 22. How to estimate g (θ) ? ◮ Use a once forever pilot-run simulation study: 1 1 ... similar to Fearnhead, P. and D. Prangle (JRRS-B, 2012) or Cabras, Castellanos, Ruli (Ercim-2012, Oviedo).
23. 23. How to estimate g (θ) ? ◮ Use a once forever pilot-run simulation study: 1 1. Consider a grid (or regular lattice) of θ made by M points: θ1 , . . . , θM 1 ... similar to Fearnhead, P. and D. Prangle (JRRS-B, 2012) or Cabras, Castellanos, Ruli (Ercim-2012, Oviedo).
24. 24. How to estimate g (θ) ? ◮ Use a once forever pilot-run simulation study: 1 1. Consider a grid (or regular lattice) of θ made by M points: θ1 , . . . , θM 2. Simulate the corresponding y1 , . . . , yM 1 ... similar to Fearnhead, P. and D. Prangle (JRRS-B, 2012) or Cabras, Castellanos, Ruli (Ercim-2012, Oviedo).
25. 25. How to estimate g (θ) ? ◮ Use a once forever pilot-run simulation study: 1 1. Consider a grid (or regular lattice) of θ made by M points: θ1 , . . . , θM 2. Simulate the corresponding y1 , . . . , yM 3. Regress y1 , . . . , yM on θ 1 , . . . , θ M obtaining g (θ). 1 ... similar to Fearnhead, P. and D. Prangle (JRRS-B, 2012) or Cabras, Castellanos, Ruli (Ercim-2012, Oviedo).
26. 26. ... example: y ∼ N(|θ|, 1) For a pilot run of M = 1000 we have g (θ) = |θ|. ˆ Pilot−Run s.s. g(θ) 10 y 5 0 −10 −5 0 5 10 θ
27. 27. ... example: y ∼ N(|θ|, 1) Suppose to draw a n = 100 sample from θ = 2: Histogram of y 20 15 Frequency 10 5 0 0 1 2 3 4 y
28. 28. ... example: y ∼ N(|θ|, 1) The Empirical Likelihood is this 2.5 2.0 Emp. Lik. 1.5 1.0 −4 −2 0 2 4 θ
29. 29. 1st Point: Do we need necessarily have to use f (y | θ) ?
30. 30. 1st Point: Do we need necessarily have to use f (y | θ) ? ◮ The above data maybe drawn from a (e.g.) a Half Normal;
31. 31. 1st Point: Do we need necessarily have to use f (y | θ) ? ◮ The above data maybe drawn from a (e.g.) a Half Normal; ◮ How this is reﬂected in the BCel ?
32. 32. 1st Point: Do we need necessarily have to use f (y | θ) ? ◮ The above data maybe drawn from a (e.g.) a Half Normal; ◮ How this is reﬂected in the BCel ? ◮ For a given data y;
33. 33. 1st Point: Do we need necessarily have to use f (y | θ) ? ◮ The above data maybe drawn from a (e.g.) a Half Normal; ◮ How this is reﬂected in the BCel ? ◮ For a given data y; ◮ and h(Y , θ) ﬁxed;
34. 34. 1st Point: Do we need necessarily have to use f (y | θ) ? ◮ The above data maybe drawn from a (e.g.) a Half Normal; ◮ How this is reﬂected in the BCel ? ◮ For a given data y; ◮ and h(Y , θ) ﬁxed; ◮ the LEL (θ) is the same regardless of f (y | θ).
35. 35. 1st Point: Do we need necessarily have to use f (y | θ) ? ◮ The above data maybe drawn from a (e.g.) a Half Normal; ◮ How this is reﬂected in the BCel ? ◮ For a given data y; ◮ and h(Y , θ) ﬁxed; ◮ the LEL (θ) is the same regardless of f (y | θ). Can we ignore f (y | θ) ?
36. 36. 2nd Point: Sample free vs Simulation free
37. 37. 2nd Point: Sample free vs Simulation free ◮ The Empirical Likelihood is ”simulation free” but not ”sample free”, i.e.
38. 38. 2nd Point: Sample free vs Simulation free ◮ The Empirical Likelihood is ”simulation free” but not ”sample free”, i.e. ◮ LEL (θ) → LN (θ) for n → ∞, ◮ implying π(θ|y) → πN (θ | y ) asymptotically in n.
39. 39. 2nd Point: Sample free vs Simulation free ◮ The Empirical Likelihood is ”simulation free” but not ”sample free”, i.e. ◮ LEL (θ) → LN (θ) for n → ∞, ◮ implying π(θ|y) → πN (θ | y ) asymptotically in n. ◮ The ABC is ”sample free” but not ”simulation free”, i.e.
40. 40. 2nd Point: Sample free vs Simulation free ◮ The Empirical Likelihood is ”simulation free” but not ”sample free”, i.e. ◮ LEL (θ) → LN (θ) for n → ∞, ◮ implying π(θ|y) → πN (θ | y ) asymptotically in n. ◮ The ABC is ”sample free” but not ”simulation free”, i.e. ◮ π(θ|ρ(s(y), so bs) < ǫ) → πN (θ | y ) as ǫ → 0 ◮ implying convergence in the number of simulations if s(y ) were suﬃcient.
41. 41. 2nd Point: Sample free vs Simulation free ◮ The Empirical Likelihood is ”simulation free” but not ”sample free”, i.e. ◮ LEL (θ) → LN (θ) for n → ∞, ◮ implying π(θ|y) → πN (θ | y ) asymptotically in n. ◮ The ABC is ”sample free” but not ”simulation free”, i.e. ◮ π(θ|ρ(s(y), so bs) < ǫ) → πN (θ | y ) as ǫ → 0 ◮ implying convergence in the number of simulations if s(y ) were suﬃcient. A quick answer recommends use BCel BUT a small sample would recommend ABC ?
42. 42. 3nd Point: How to validate a pseudo-posteriorπ(θ|y ) ∝ LEL (θ) × π(θ) ?
43. 43. 3nd Point: How to validate a pseudo-posteriorπ(θ|y ) ∝ LEL (θ) × π(θ) ? ◮ The use of pseudo-likelihoods is not new in the Bayesian setting:
44. 44. 3nd Point: How to validate a pseudo-posteriorπ(θ|y ) ∝ LEL (θ) × π(θ) ? ◮ The use of pseudo-likelihoods is not new in the Bayesian setting: ◮ Empirical Likelihoods:
45. 45. 3nd Point: How to validate a pseudo-posteriorπ(θ|y ) ∝ LEL (θ) × π(θ) ? ◮ The use of pseudo-likelihoods is not new in the Bayesian setting: ◮ Empirical Likelihoods: ◮ Lazar (Biometrika, 2003) ◮ Mengersen et al. (PNAS, 2012) ◮ ...
46. 46. 3nd Point: How to validate a pseudo-posteriorπ(θ|y ) ∝ LEL (θ) × π(θ) ? ◮ The use of pseudo-likelihoods is not new in the Bayesian setting: ◮ Empirical Likelihoods: ◮ Lazar (Biometrika, 2003) ◮ Mengersen et al. (PNAS, 2012) ◮ ... ◮ Modiﬁed-Likelihoods:
47. 47. 3nd Point: How to validate a pseudo-posteriorπ(θ|y ) ∝ LEL (θ) × π(θ) ? ◮ The use of pseudo-likelihoods is not new in the Bayesian setting: ◮ Empirical Likelihoods: ◮ Lazar (Biometrika, 2003) ◮ Mengersen et al. (PNAS, 2012) ◮ ... ◮ Modiﬁed-Likelihoods: ◮ Ventura et al. (JASA, 2009) ◮ Chang and Mukerjee (Stat. & Prob. Letters 2006) ◮ ...
48. 48. 3nd Point: How to validate a pseudo-posteriorπ(θ|y ) ∝ LEL (θ) × π(θ) ? ◮ The use of pseudo-likelihoods is not new in the Bayesian setting: ◮ Empirical Likelihoods: ◮ Lazar (Biometrika, 2003) ◮ Mengersen et al. (PNAS, 2012) ◮ ... ◮ Modiﬁed-Likelihoods: ◮ Ventura et al. (JASA, 2009) ◮ Chang and Mukerjee (Stat. & Prob. Letters 2006) ◮ ... ◮ Quasi-Likelihoods:
49. 49. 3nd Point: How to validate a pseudo-posteriorπ(θ|y ) ∝ LEL (θ) × π(θ) ? ◮ The use of pseudo-likelihoods is not new in the Bayesian setting: ◮ Empirical Likelihoods: ◮ Lazar (Biometrika, 2003) ◮ Mengersen et al. (PNAS, 2012) ◮ ... ◮ Modiﬁed-Likelihoods: ◮ Ventura et al. (JASA, 2009) ◮ Chang and Mukerjee (Stat. & Prob. Letters 2006) ◮ ... ◮ Quasi-Likelihoods: ◮ Lin (Statist. Methodol., 2006) ◮ Greco et al. (JSPI, 2008) ◮ Ventura et al. (JSPI, 2010) ◮ ...
50. 50. 3nd Point: How to validate a pseudo-posteriorπ(θ|y ) ∝ LEL (θ) × π(θ) ? ◮ The use of pseudo-likelihoods is not new in the Bayesian setting: ◮ Empirical Likelihoods: ◮ Lazar (Biometrika, 2003) : examples and coverages of C.I. ◮ Mengersen et al. (PNAS, 2012) ◮ ... ◮ Modiﬁed-Likelihoods: ◮ Ventura et al. (JASA, 2009) ◮ Chang and Mukerjee (Stat. & Prob. Letters 2006) ◮ ... ◮ Quasi-Likelihoods: ◮ Lin (Statist. Methodol., 2006) ◮ Greco et al. (JSPI, 2008) ◮ Ventura et al. (JSPI, 2010) ◮ ...
51. 51. 3nd Point: How to validate a pseudo-posteriorπ(θ|y ) ∝ LEL (θ) × π(θ) ? ◮ The use of pseudo-likelihoods is not new in the Bayesian setting: ◮ Empirical Likelihoods: ◮ Lazar (Biometrika, 2003) : examples and coverages of C.I. ◮ Mengersen et al. (PNAS, 2012) : examples and coverages of C.I. ◮ ... ◮ Modiﬁed-Likelihoods: ◮ Ventura et al. (JASA, 2009) ◮ Chang and Mukerjee (Stat. & Prob. Letters 2006) ◮ ... ◮ Quasi-Likelihoods: ◮ Lin (Statist. Methodol., 2006) ◮ Greco et al. (JSPI, 2008) ◮ Ventura et al. (JSPI, 2010) ◮ ...
52. 52. 3nd Point: How to validate a pseudo-posteriorπ(θ|y ) ∝ LEL (θ) × π(θ) ? ◮ The use of pseudo-likelihoods is not new in the Bayesian setting: ◮ Empirical Likelihoods: ◮ Lazar (Biometrika, 2003) : examples and coverages of C.I. ◮ Mengersen et al. (PNAS, 2012) : examples and coverages of C.I. ◮ ... ◮ Modiﬁed-Likelihoods: ◮ Ventura et al. (JASA, 2009) : second order matching properties; ◮ Chang and Mukerjee (Stat. & Prob. Letters 2006) ◮ ... ◮ Quasi-Likelihoods: ◮ Lin (Statist. Methodol., 2006) ◮ Greco et al. (JSPI, 2008) ◮ Ventura et al. (JSPI, 2010) ◮ ...
53. 53. 3nd Point: How to validate a pseudo-posteriorπ(θ|y ) ∝ LEL (θ) × π(θ) ? ◮ The use of pseudo-likelihoods is not new in the Bayesian setting: ◮ Empirical Likelihoods: ◮ Lazar (Biometrika, 2003) : examples and coverages of C.I. ◮ Mengersen et al. (PNAS, 2012) : examples and coverages of C.I. ◮ ... ◮ Modiﬁed-Likelihoods: ◮ Ventura et al. (JASA, 2009) : second order matching properties; ◮ Chang and Mukerjee (Stat. & Prob. Letters 2006) : examples; ◮ ... ◮ Quasi-Likelihoods: ◮ Lin (Statist. Methodol., 2006) ◮ Greco et al. (JSPI, 2008) ◮ Ventura et al. (JSPI, 2010) ◮ ...
54. 54. 3nd Point: How to validate a pseudo-posteriorπ(θ|y ) ∝ LEL (θ) × π(θ) ? ◮ The use of pseudo-likelihoods is not new in the Bayesian setting: ◮ Empirical Likelihoods: ◮ Lazar (Biometrika, 2003) : examples and coverages of C.I. ◮ Mengersen et al. (PNAS, 2012) : examples and coverages of C.I. ◮ ... ◮ Modiﬁed-Likelihoods: ◮ Ventura et al. (JASA, 2009) : second order matching properties; ◮ Chang and Mukerjee (Stat. & Prob. Letters 2006) : examples; ◮ ... ◮ Quasi-Likelihoods: ◮ Lin (Statist. Methodol., 2006) : examples; ◮ Greco et al. (JSPI, 2008) : robustness properties; ◮ Ventura et al. (JSPI, 2010) : examples and coverages of C.I.; ◮ ...
55. 55. 3nd Point: How to validate a pseudo-posteriorπ(θ|y ) ∝ LEL (θ) × π(θ) ? ◮ Monahan & Boos (Biometrika, 1992) proposed a notion of validity:
56. 56. 3nd Point: How to validate a pseudo-posteriorπ(θ|y ) ∝ LEL (θ) × π(θ) ? ◮ Monahan & Boos (Biometrika, 1992) proposed a notion of validity: π(θ|y ) should obey the laws of probability in a fashion that is consistent with statements derived from Bayes’rule.
57. 57. 3nd Point: How to validate a pseudo-posteriorπ(θ|y ) ∝ LEL (θ) × π(θ) ? ◮ Monahan & Boos (Biometrika, 1992) proposed a notion of validity: π(θ|y ) should obey the laws of probability in a fashion that is consistent with statements derived from Bayes’rule. ◮ Very diﬃcult!
58. 58. 3nd Point: How to validate a pseudo-posteriorπ(θ|y ) ∝ LEL (θ) × π(θ) ? ◮ Monahan & Boos (Biometrika, 1992) proposed a notion of validity: π(θ|y ) should obey the laws of probability in a fashion that is consistent with statements derived from Bayes’rule. ◮ Very diﬃcult! How to validate the pseudo-posterior π(θ|y ) when this is not possible ?
59. 59. ... Last point: the ABC is still a terriﬁc tool
60. 60. ... Last point: the ABC is still a terriﬁc tool ◮ ... a lot of references:
61. 61. ... Last point: the ABC is still a terriﬁc tool ◮ ... a lot of references: ◮ Statistical Journals;
62. 62. ... Last point: the ABC is still a terriﬁc tool ◮ ... a lot of references: ◮ Statistical Journals; ◮ Twitter;
63. 63. ... Last point: the ABC is still a terriﬁc tool ◮ ... a lot of references: ◮ Statistical Journals; ◮ Twitter; ◮ Xiang’s blog ( xianblog.wordpress.com )
64. 64. ... Last point: the ABC is still a terriﬁc tool ◮ ... a lot of references: ◮ Statistical Journals; ◮ Twitter; ◮ Xiang’s blog ( xianblog.wordpress.com ) ◮ ... it is tailored to Approximate LN (θ).
65. 65. ... Last point: the ABC is still a terriﬁc tool ◮ ... a lot of references: ◮ Statistical Journals; ◮ Twitter; ◮ Xiang’s blog ( xianblog.wordpress.com ) ◮ ... it is tailored to Approximate LN (θ). Where is the A in BCel ?