Your SlideShare is downloading. ×
0
Approximate Bayesian Computation (ABC):    model choice and empirical likelihood                        Christian P. Rober...
OutlineIntroductionABCABC as an inference machineABC for model choiceModel choice consistencyABCel
Intractable likelihood   Case of a well-defined statistical model where the likelihood   function                         (...
Intractable likelihood   Case of a well-defined statistical model where the likelihood   function                         (...
Different perspectives on abc   What is the (most) fundamental issue?       a mere computational issue (that will eventuall...
Different perspectives on abc   What is the (most) fundamental issue?       a mere computational issue (that will eventuall...
Different perspectives on abc   What is the (most) fundamental issue?       a mere computational issue (that will eventuall...
Econom’ections   Similar exploration of simulation-based and approximation   techniques in Econometrics       Simulated me...
Econom’ections   Similar exploration of simulation-based and approximation   techniques in Econometrics       Simulated me...
Indirect inference                                                 ^   Minimise [in θ] a distance between estimators β bas...
Indirect inference (PML vs. PSE)   Example of the pseudo-maximum-likelihood (PML)                ^                β(y) = a...
Indirect inference (PML vs. PSE)   Example of the pseudo-score-estimator (PSE)                                            ...
Consistent indirect inference           “...in order to get a unique solution the dimension of       the auxiliary paramet...
Consistent indirect inference           “...in order to get a unique solution the dimension of       the auxiliary paramet...
Choice of pseudo-model   Arbitrariness of pseudo-model   Pick model such that        ^    1. β(θ) not flat (i.e. sensitive ...
Approximate Bayesian computation Introduction ABC   Genesis of ABC   ABC basics   Advances and interpretations   ABC as kn...
Genetic background of ABC    skip genetics   ABC is a recent computational technique that only requires being   able to sa...
Demo-genetic inference   Each model is characterized by a set of parameters θ that cover   historical (time divergence, ad...
Demo-genetic inference   Each model is characterized by a set of parameters θ that cover   historical (time divergence, ad...
Neutral model at a given microsatellite locus, in a closedpanmictic population at equilibrium                             ...
Neutral model at a given microsatellite locus, in a closedpanmictic population at equilibrium                             ...
Neutral model at a given microsatellite locus, in a closedpanmictic population at equilibrium                             ...
Neutral model at a given microsatellite locus, in a closedpanmictic population at equilibrium                             ...
Much more interesting models. . .       several independent locus       Independent gene genealogies and mutations       d...
Intractable likelihood   Missing (too missing!) data structure:                     f (y|θ) =         f (y|G , θ)f (G |θ)d...
Intractable likelihood   Missing (too missing!) data structure:                     f (y|θ) =         f (y|G , θ)f (G |θ)d...
not-so-obvious ancestry...                     You went to school to learn, girl (. . . )                     Why 2 plus 2...
A?B?C?    A stands for approximate    [wrong likelihood /    picture]    B stands for Bayesian    C stands for computation...
A?B?C?    A stands for approximate    [wrong likelihood /    picture]    B stands for Bayesian    C stands for computation...
A?B?C?                                                        ESS=108.9                                                   ...
How Bayesian is aBc?  Could we turn the resolution into a Bayesian answer?      ideally so (not meaningfull: requires ∞-ly...
Untractable likelihoodBack to stage zero: what can we dowhen a likelihood function f (y|θ) iswell-defined but impossible / ...
Untractable likelihoodBack to stage zero: what can we dowhen a likelihood function f (y|θ) iswell-defined but impossible / ...
ABC methodology  Bayesian setting: target is π(θ)f (x|θ)  When likelihood f (x|θ) not in closed form, likelihood-free reje...
ABC methodology  Bayesian setting: target is π(θ)f (x|θ)  When likelihood f (x|θ) not in closed form, likelihood-free reje...
ABC methodology  Bayesian setting: target is π(θ)f (x|θ)  When likelihood f (x|θ) not in closed form, likelihood-free reje...
A as A...pproximative   When y is a continuous random variable, strict equality z = y is   replaced with a tolerance zone ...
A as A...pproximative   When y is a continuous random variable, strict equality z = y is   replaced with a tolerance zone ...
ABC algorithm  In most implementations, further degree of A...pproximation:  Algorithm 1 Likelihood-free rejection sampler...
Output  The likelihood-free algorithm samples from the marginal in z of:                                   π(θ)f (z|θ)IA ,...
Output  The likelihood-free algorithm samples from the marginal in z of:                                   π(θ)f (z|θ)IA ,...
Output  The likelihood-free algorithm samples from the marginal in z of:                                   π(θ)f (z|θ)IA ,...
Output  The likelihood-free algorithm samples from the marginal in z of:                                        π(θ)f (z|θ...
Convergence of ABC  What happens when                     → 0?  For B ⊂ Θ, we have               A         f (z|θ)dz      ...
Convergence of ABC  What happens when                     → 0?  For B ⊂ Θ, we have               A         f (z|θ)dz      ...
Convergence (do not attempt!)   ...and the above does not apply to insufficient statistics:   If η(y) is not a sufficient stat...
Convergence (do not attempt!)   ...and the above does not apply to insufficient statistics:   If η(y) is not a sufficient stat...
Convergence (do not attempt!)   ...and the above does not apply to insufficient statistics:   If η(y) is not a sufficient stat...
Convergence (do not attempt!)   ...and the above does not apply to insufficient statistics:   If η(y) is not a sufficient stat...
MA example  Inference on the parameters of a MA(q) model                                       q                        xt...
MA example  Inference on the parameters of a MA(q) model                                       q                        xt...
MA example (2)  ABC algorithm thus made of    1. picking a new value (ϑ1 , ϑ2 ) in the triangle    2. generating an iid se...
MA example (2)  ABC algorithm thus made of    1. picking a new value (ϑ1 , ϑ2 ) in the triangle    2. generating an iid se...
Comparison of distance impact   Impact of tolerance on ABC sample against either distance   ( = 100%, 10%, 1%, 0.1%) for a...
Comparison of distance impact            4                                              1.5            3                  ...
Comparison of distance impact            4                                              1.5            3                  ...
Comments     Role of distance paramount (because     = 0)     Scaling of components of η(y) is also determinant       matt...
ABC (simul’) advances                         how approximative is ABC?                ABC as knn   Simulating from the pr...
ABC (simul’) advances                         how approximative is ABC?                ABC as knn   Simulating from the pr...
ABC (simul’) advances                         how approximative is ABC?                ABC as knn   Simulating from the pr...
ABC (simul’) advances                         how approximative is ABC?                ABC as knn   Simulating from the pr...
ABC-NP  Better usage of [prior] simulations by adjustement: instead of throwing away θ such that ρ(η(z), η(y)) > , replace...
ABC-NP (regression)   Also found in the subsequent literature, e.g. in        Fearnhead-Prangle (2012)   :   weight direct...
ABC-NP (regression)   Also found in the subsequent literature, e.g. in        Fearnhead-Prangle (2012)   :   weight direct...
ABC-NP (density estimation)   Use of the kernel weights                             Kδ {ρ(η(z(θ)), η(y))}   leads to the N...
ABC-NP (density estimation)   Use of the kernel weights                            Kδ {ρ(η(z(θ)), η(y))}   leads to the NP...
ABC-NP (density estimations)   Other versions incorporating regression adjustments                         i              ...
ABC-NP (density estimations)   Other versions incorporating regression adjustments                         i              ...
ABC-NP (density estimations)   Other versions incorporating regression adjustments                         i              ...
ABC-NCH  Incorporating non-linearities and heterocedasticities:                                                 σ(η(y))   ...
ABC-NCH  Incorporating non-linearities and heterocedasticities:                                                 σ(η(y))   ...
ABC as knn                                    [Biau et al., 2012, arxiv:1207.6461]  Practice of ABC: determine tolerance  ...
ABC as knn                                    [Biau et al., 2012, arxiv:1207.6461]  Practice of ABC: determine tolerance  ...
ABC as knn                                    [Biau et al., 2012, arxiv:1207.6461]  Practice of ABC: determine tolerance  ...
ABC consistency   Provided                 kN / log log N −→ ∞ and kN /N −→ 0    as N → ∞, for almost all s0 (with respect...
ABC consistency   Provided                 kN / log log N −→ ∞ and kN /N −→ 0    as N → ∞, for almost all s0 (with respect...
Rates of convergence   Further assumptions (on target and kernel) allow for precise   (integrated mean square) convergence...
Rates of convergence   Further assumptions (on target and kernel) allow for precise   (integrated mean square) convergence...
ABC inference machine Introduction ABC ABC as an inference machine   Error inc.   Exact BC and approximate   targets   sum...
How much Bayesian aBc is..?      maybe a convergent method of inference (meaningful?      sufficient? foreign?)      approxi...
How much Bayesian aBc is..?      maybe a convergent method of inference (meaningful?      sufficient? foreign?)      approxi...
How much Bayesian aBc is..?      maybe a convergent method of inference (meaningful?      sufficient? foreign?)      approxi...
How much Bayesian aBc is..?      maybe a convergent method of inference (meaningful?      sufficient? foreign?)      approxi...
ABCµ  Idea Infer about the error as well as about the parameter:  Use of a joint density                f (θ, |y) ∝ ξ( |y,...
ABCµ  Idea Infer about the error as well as about the parameter:  Use of a joint density                f (θ, |y) ∝ ξ( |y,...
ABCµ  Idea Infer about the error as well as about the parameter:  Use of a joint density                f (θ, |y) ∝ ξ( |y,...
ABCµ details   Multidimensional distances ρk (k = 1, . . . , K ) and errors    k = ρk (ηk (z), ηk (y)), with              ...
ABCµ details   Multidimensional distances ρk (k = 1, . . . , K ) and errors    k = ρk (ηk (z), ηk (y)), with              ...
ABCµ multiple errors                       [ c Ratmann et al., PNAS, 2009]
ABCµ for model choice                        [ c Ratmann et al., PNAS, 2009]
Wilkinson’s exact BC (not exactly!)   ABC approximation error (i.e. non-zero tolerance) replaced with   exact simulation f...
Wilkinson’s exact BC (not exactly!)   ABC approximation error (i.e. non-zero tolerance) replaced with   exact simulation f...
How exact a BC?      “Using to represent measurement error is      straightforward, whereas using to model the model      ...
How exact a BC?  Pros         Pseudo-data from true model and observed data from noisy         model         Interesting p...
ABC for HMMs  Specific case of a hidden     Markov model                             Xt+1 ∼ Qθ (Xt , ·)                    ...
ABC for HMMs  Specific case of a hidden     Markov model                             Xt+1 ∼ Qθ (Xt , ·)                    ...
ABC-MLE for HMMs  ABC-MLE defined by         θn = arg max Pθ Y1 ∈ B(y1 , ), . . . , Yn ∈ B(yn , )         ^                ...
ABC-MLE for HMMs  ABC-MLE defined by         θn = arg max Pθ Y1 ∈ B(y1 , ), . . . , Yn ∈ B(yn , )         ^                ...
ABC-MLE is biased      ABC-MLE is asymptotically (in n) biased with target                      l (θ) = Eθ∗ [log pθ (Y1 |Y...
ABC-MLE is biased      ABC-MLE is asymptotically (in n) biased with target                      l (θ) = Eθ∗ [log pθ (Y1 |Y...
Noisy ABC-MLE  Idea: Modify instead the data from the start                        0                      (y1 + ζ1 , . . ....
Consistent noisy ABC-MLE      Degrading the data improves the estimation performances:          Noisy ABC-MLE is asymptoti...
SMC for ABC likelihood   Algorithm 2 SMC ABC for HMMs     Given θ     for k = 1, . . . , n do                             ...
Which summary?  Fundamental difficulty of the choice of the summary statistic when  there is no non-trivial sufficient statist...
Which summary?  Fundamental difficulty of the choice of the summary statistic when  there is no non-trivial sufficient statist...
Which summary?  Fundamental difficulty of the choice of the summary statistic when  there is no non-trivial sufficient statist...
Which summary for model choice?   Depending on the choice of η(·), the Bayes factor based on this   insufficient statistic, ...
Which summary for model choice?   Depending on the choice of η(·), the Bayes factor based on this   insufficient statistic, ...
Semi-automatic ABC  Fearnhead and Prangle (2010) study ABC and the selection of the  summary statistic in close proximity ...
Summary [of F&P/statistics)      optimality of the posterior expectation                                  E[θ|y]      of t...
Summary [of F&P/statistics)      optimality of the posterior expectation                                  E[θ|y]      of t...
Conclusion      Choice of summary statistics is paramount for ABC      validation/performance      At best, ABC approximat...
Conclusion      Choice of summary statistics is paramount for ABC      validation/performance      At best, ABC approximat...
Conclusion      Choice of summary statistics is paramount for ABC      validation/performance      At best, ABC approximat...
Conclusion      Choice of summary statistics is paramount for ABC      validation/performance      At best, ABC approximat...
Conclusion      Choice of summary statistics is paramount for ABC      validation/performance      At best, ABC approximat...
ABC for model choiceIntroductionABCABC as an inference machineABC for model choice  BMC Principle  Gibbs random fields (cou...
Bayesian model choice   BMC Principle   Several models                              M1 , M2 , . . .   are considered simul...
Bayesian model choice   BMC Principle   Several models                             M1 , M2 , . . .   are considered simult...
Generic ABC for model choice   Algorithm 3 Likelihood-free model choice sampler (ABC-MC)     for t = 1 to T do       repea...
ABC estimates  Posterior probability π(M = m|y) approximated by the frequency  of acceptances from model m                ...
ABC estimates  Posterior probability π(M = m|y) approximated by the frequency  of acceptances from model m                ...
Potts model    Skip MRFs   Potts model   Distribution with an energy function of the form                          θS(y) =...
Potts model    Skip MRFs   Potts model   Distribution with an energy function of the form                          θS(y) =...
Neighbourhood relations   Setup   Choice to be made between M neighbourhood relations                        m            ...
Model index   Computational target:          P(M = m|x) ∝          fm (x|θm )πm (θm ) dθm π(M = m)                        ...
Model index   Computational target:          P(M = m|x) ∝          fm (x|θm )πm (θm ) dθm π(M = m)                        ...
Sufficient statistics in Gibbs random fields   Each model m has its own sufficient statistic Sm (·) and   S(·) = (S0 (·), . . ....
Sufficient statistics in Gibbs random fields   Each model m has its own sufficient statistic Sm (·) and   S(·) = (S0 (·), . . ....
Toy example  iid Bernoulli model versus two-state first-order Markov chain, i.e.                                       n   ...
About sufficiency  If η1 (x) sufficient statistic for model m = 1 and parameter θ1 and  η2 (x) sufficient statistic for model m ...
About sufficiency  If η1 (x) sufficient statistic for model m = 1 and parameter θ1 and  η2 (x) sufficient statistic for model m ...
Poisson/geometric example   Sample                            x = (x1 , . . . , xn )   from either a Poisson P(λ) or from ...
Limiting behaviour of B12 (T → ∞)   ABC approximation                                T                                t=1 ...
Limiting behaviour of B12 (T → ∞)   ABC approximation                                T                                t=1 ...
Limiting behaviour of B12 ( → 0)   When    goes to zero,                   η            π1 (θ1 )f1η (η(y)|θ1 ) dθ1        ...
Limiting behaviour of B12 ( → 0)   When    goes to zero,                   η            π1 (θ1 )f1η (η(y)|θ1 ) dθ1        ...
Limiting behaviour of B12 (under sufficiency)   If η(y) sufficient statistic in both models,                        fi (y|θi )...
Limiting behaviour of B12 (under sufficiency)   If η(y) sufficient statistic in both models,                        fi (y|θi )...
Poisson/geometric example (back)   Sample                           x = (x1 , . . . , xn )   from either a Poisson P(λ) or...
Poisson/geometric discrepancy                            η   Range of B12 (x) versus B12 (x): The values produced have not...
Formal recovery   Creating an encompassing exponential family   f (x|θ1 , θ2 , α1 , α2 ) ∝ exp{θT η1 (x) + θT η2 (x) + α1 ...
Formal recovery   Creating an encompassing exponential family   f (x|θ1 , θ2 , α1 , α2 ) ∝ exp{θT η1 (x) + θT η2 (x) + α1 ...
Formal recovery   Creating an encompassing exponential family   f (x|θ1 , θ2 , α1 , α2 ) ∝ exp{θT η1 (x) + θT η2 (x) + α1 ...
Meaning of the ABC-Bayes factor   The ABC approximation to the Bayes Factor is based solely on the   summary statistics......
Meaning of the ABC-Bayes factor   The ABC approximation to the Bayes Factor is based solely on the   summary statistics......
MA example                        1.0                                      1.0                                            ...
MA example                        1.0                                      1.0                                            ...
The only safe cases??? [circa April 2011]   Besides specific models like Gibbs random fields,   using distances over the dat...
The only safe cases??? [circa April 2011]   Besides specific models like Gibbs random fields,   using distances over the dat...
The only safe cases??? [circa April 2011]   Besides specific models like Gibbs random fields,   using distances over the dat...
ABC model choice consistencyIntroductionABCABC as an inference machineABC for model choiceModel choice consistency  Formal...
The starting point   Central question to the validation of ABC for model choice:   When is a Bayes factor based on an insu...
The starting point   Central question to the validation of ABC for model choice:   When is a Bayes factor based on an insu...
A benchmark if toy example   Comparison suggested by referee of PNAS paper [thanks!]:                             [X, Corn...
A benchmark if toy example   Comparison suggested by referee of PNAS paper [thanks!]:                              [X, Cor...
A benchmark if toy example   Comparison suggested by referee of PNAS paper [thanks!]:                             [X, Corn...
ABC in Venezia
ABC in Venezia
ABC in Venezia
ABC in Venezia
ABC in Venezia
ABC in Venezia
ABC in Venezia
ABC in Venezia
ABC in Venezia
ABC in Venezia
ABC in Venezia
ABC in Venezia
ABC in Venezia
ABC in Venezia
ABC in Venezia
ABC in Venezia
ABC in Venezia
ABC in Venezia
ABC in Venezia
ABC in Venezia
ABC in Venezia
ABC in Venezia
ABC in Venezia
ABC in Venezia
ABC in Venezia
ABC in Venezia
ABC in Venezia
ABC in Venezia
ABC in Venezia
ABC in Venezia
ABC in Venezia
ABC in Venezia
ABC in Venezia
ABC in Venezia
ABC in Venezia
ABC in Venezia
ABC in Venezia
ABC in Venezia
ABC in Venezia
ABC in Venezia
ABC in Venezia
ABC in Venezia
ABC in Venezia
ABC in Venezia
ABC in Venezia
ABC in Venezia
ABC in Venezia
ABC in Venezia
ABC in Venezia
ABC in Venezia
ABC in Venezia
ABC in Venezia
ABC in Venezia
ABC in Venezia
ABC in Venezia
ABC in Venezia
ABC in Venezia
ABC in Venezia
ABC in Venezia
ABC in Venezia
ABC in Venezia
ABC in Venezia
ABC in Venezia
ABC in Venezia
ABC in Venezia
ABC in Venezia
ABC in Venezia
ABC in Venezia
ABC in Venezia
ABC in Venezia
ABC in Venezia
ABC in Venezia
Upcoming SlideShare
Loading in...5
×

ABC in Venezia

3,617

Published on

Slides of a tutorial I gave in Venezia Ca' Foscari on October 8, 2012

Published in: Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
3,617
On Slideshare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
25
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Transcript of "ABC in Venezia"

  1. 1. Approximate Bayesian Computation (ABC): model choice and empirical likelihood Christian P. Robert Venezia, Oct. 8, 2012 Universit´ Paris-Dauphine, IuF, & CREST eJoint works with J.-M. Cornuet, J.-M. Marin, K.L. Mengersen, N. Pillai, and P. Pudlo
  2. 2. OutlineIntroductionABCABC as an inference machineABC for model choiceModel choice consistencyABCel
  3. 3. Intractable likelihood Case of a well-defined statistical model where the likelihood function (θ|y) = f (y1 , . . . , yn |θ) is (really!) not available in closed form can (easily!) be neither completed nor demarginalised cannot be estimated by an unbiased estimator c Prohibits direct implementation of a generic MCMC algorithm like Metropolis–Hastings
  4. 4. Intractable likelihood Case of a well-defined statistical model where the likelihood function (θ|y) = f (y1 , . . . , yn |θ) is (really!) not available in closed form can (easily!) be neither completed nor demarginalised cannot be estimated by an unbiased estimator c Prohibits direct implementation of a generic MCMC algorithm like Metropolis–Hastings
  5. 5. Different perspectives on abc What is the (most) fundamental issue? a mere computational issue (that will eventually end up being solved by more powerful computers, &tc, even if too costly in the short term) an inferential issue (opening opportunities for new inference machine, with different legitimity than classical B approach) a Bayesian conundrum (while inferential methods available, how closely related to the B approach?)
  6. 6. Different perspectives on abc What is the (most) fundamental issue? a mere computational issue (that will eventually end up being solved by more powerful computers, &tc, even if too costly in the short term) an inferential issue (opening opportunities for new inference machine, with different legitimity than classical B approach) a Bayesian conundrum (while inferential methods available, how closely related to the B approach?)
  7. 7. Different perspectives on abc What is the (most) fundamental issue? a mere computational issue (that will eventually end up being solved by more powerful computers, &tc, even if too costly in the short term) an inferential issue (opening opportunities for new inference machine, with different legitimity than classical B approach) a Bayesian conundrum (while inferential methods available, how closely related to the B approach?)
  8. 8. Econom’ections Similar exploration of simulation-based and approximation techniques in Econometrics Simulated method of moments Method of simulated moments Simulated pseudo-maximum-likelihood Indirect inference [Gouri´roux & Monfort, 1996] e even though motivation is partly-defined models rather than complex likelihoods
  9. 9. Econom’ections Similar exploration of simulation-based and approximation techniques in Econometrics Simulated method of moments Method of simulated moments Simulated pseudo-maximum-likelihood Indirect inference [Gouri´roux & Monfort, 1996] e even though motivation is partly-defined models rather than complex likelihoods
  10. 10. Indirect inference ^ Minimise [in θ] a distance between estimators β based on a pseudo-model for genuine observations and for observations simulated under the true model and the parameter θ. [Gouri´roux, Monfort, & Renault, 1993; e Smith, 1993; Gallant & Tauchen, 1996]
  11. 11. Indirect inference (PML vs. PSE) Example of the pseudo-maximum-likelihood (PML) ^ β(y) = arg max log f (yt |β, y1:(t−1) ) β t leading to arg min ||β(yo ) − β(y1 (θ), . . . , yS (θ))||2 ^ ^ θ when ys (θ) ∼ f (y|θ) s = 1, . . . , S
  12. 12. Indirect inference (PML vs. PSE) Example of the pseudo-score-estimator (PSE) 2 ∂ log f ^ β(y) = arg min (yt |β, y1:(t−1) ) β t ∂β leading to arg min ||β(yo ) − β(y1 (θ), . . . , yS (θ))||2 ^ ^ θ when ys (θ) ∼ f (y|θ) s = 1, . . . , S
  13. 13. Consistent indirect inference “...in order to get a unique solution the dimension of the auxiliary parameter β must be larger than or equal to the dimension of the initial parameter θ. If the problem is just identified the different methods become easier...” Consistency depending on the criterion and on the asymptotic identifiability of θ [Gouri´roux & Monfort, 1996, p. 66] e
  14. 14. Consistent indirect inference “...in order to get a unique solution the dimension of the auxiliary parameter β must be larger than or equal to the dimension of the initial parameter θ. If the problem is just identified the different methods become easier...” Consistency depending on the criterion and on the asymptotic identifiability of θ [Gouri´roux & Monfort, 1996, p. 66] e
  15. 15. Choice of pseudo-model Arbitrariness of pseudo-model Pick model such that ^ 1. β(θ) not flat (i.e. sensitive to changes in θ) ^ 2. β(θ) not dispersed (i.e. robust agains changes in ys (θ)) [Frigessi & Heggland, 2004]
  16. 16. Approximate Bayesian computation Introduction ABC Genesis of ABC ABC basics Advances and interpretations ABC as knn ABC as an inference machine ABC for model choice Model choice consistency ABCel
  17. 17. Genetic background of ABC skip genetics ABC is a recent computational technique that only requires being able to sample from the likelihood f (·|θ) This technique stemmed from population genetics models, about 15 years ago, and population geneticists still contribute significantly to methodological developments of ABC. [Griffith & al., 1997; Tavar´ & al., 1999] e
  18. 18. Demo-genetic inference Each model is characterized by a set of parameters θ that cover historical (time divergence, admixture time ...), demographics (population sizes, admixture rates, migration rates, ...) and genetic (mutation rate, ...) factors The goal is to estimate these parameters from a dataset of polymorphism (DNA sample) y observed at the present time Problem: most of the time, we cannot calculate the likelihood of the polymorphism data f (y|θ)...
  19. 19. Demo-genetic inference Each model is characterized by a set of parameters θ that cover historical (time divergence, admixture time ...), demographics (population sizes, admixture rates, migration rates, ...) and genetic (mutation rate, ...) factors The goal is to estimate these parameters from a dataset of polymorphism (DNA sample) y observed at the present time Problem: most of the time, we cannot calculate the likelihood of the polymorphism data f (y|θ)...
  20. 20. Neutral model at a given microsatellite locus, in a closedpanmictic population at equilibrium Mutations according to the Simple stepwise Mutation Model (SMM) • date of the mutations ∼ Poisson process with intensity θ/2 over the branches • MRCA = 100 • independent mutations: ±1 with pr. 1/2 Sample of 8 genes
  21. 21. Neutral model at a given microsatellite locus, in a closedpanmictic population at equilibrium Kingman’s genealogy When time axis is normalized, T (k) ∼ Exp(k(k − 1)/2) Mutations according to the Simple stepwise Mutation Model (SMM) • date of the mutations ∼ Poisson process with intensity θ/2 over the branches • MRCA = 100 • independent mutations: ±1 with pr. 1/2
  22. 22. Neutral model at a given microsatellite locus, in a closedpanmictic population at equilibrium Kingman’s genealogy When time axis is normalized, T (k) ∼ Exp(k(k − 1)/2) Mutations according to the Simple stepwise Mutation Model (SMM) • date of the mutations ∼ Poisson process with intensity θ/2 over the branches • MRCA = 100 • independent mutations: ±1 with pr. 1/2
  23. 23. Neutral model at a given microsatellite locus, in a closedpanmictic population at equilibrium Kingman’s genealogy When time axis is normalized, T (k) ∼ Exp(k(k − 1)/2) Mutations according to the Simple stepwise Mutation Model (SMM) • date of the mutations ∼ Poisson process with intensity θ/2 over the branches Observations: leafs of the tree • MRCA = 100 ^ θ=? • independent mutations: ±1 with pr. 1/2
  24. 24. Much more interesting models. . . several independent locus Independent gene genealogies and mutations different populations linked by an evolutionary scenario made of divergences, admixtures, migrations between populations, etc. larger sample size usually between 50 and 100 genes MRCA τ2 τ1 A typical evolutionary scenario: POP 0 POP 1 POP 2
  25. 25. Intractable likelihood Missing (too missing!) data structure: f (y|θ) = f (y|G , θ)f (G |θ)dG G cannot be computed in a manageable way... The genealogies are considered as nuisance parameters This modelling clearly differs from the phylogenetic perspective where the tree is the parameter of interest.
  26. 26. Intractable likelihood Missing (too missing!) data structure: f (y|θ) = f (y|G , θ)f (G |θ)dG G cannot be computed in a manageable way... The genealogies are considered as nuisance parameters This modelling clearly differs from the phylogenetic perspective where the tree is the parameter of interest.
  27. 27. not-so-obvious ancestry... You went to school to learn, girl (. . . ) Why 2 plus 2 makes four Now, now, now, I’m gonna teach you (. . . ) All you gotta do is repeat after me! A, B, C! It’s easy as 1, 2, 3! Or simple as Do, Re, Mi! (. . . )
  28. 28. A?B?C? A stands for approximate [wrong likelihood / picture] B stands for Bayesian C stands for computation [producing a parameter sample]
  29. 29. A?B?C? A stands for approximate [wrong likelihood / picture] B stands for Bayesian C stands for computation [producing a parameter sample]
  30. 30. A?B?C? ESS=108.9 ESS=81.48 ESS=105.2 3.0 2.0 2.0 Density Density Density 1.5 1.0 1.0 0.0 0.0 0.0 A stands for approximate −0.4 0.0 0.2 ESS=133.3 θ 0.4 0.6 −0.8 −0.6 −0.4 ESS=87.75 θ −0.2 0.0 −0.2 0.0 0.2 ESS=72.89 θ 0.4 0.6 0.8 3.0 2.0 4 [wrong likelihood / 2.0 Density Density Density 1.0 1.0 2 0.0 0.0 0 picture] −0.8 −0.4 ESS=116.5 θ 0.0 0.2 0.4 −0.2 0.0 0.2 ESS=103.9 θ 0.4 0.6 0.8 −0.2 0.0 ESS=126.9 θ 0.2 0.4 3.0 2.0 3.0 2.0 Density Density Density 1.0 1.5 1.0 B stands for Bayesian 0.0 0.0 0.0 −0.4 0.0 0.2 0.4 0.6 −0.4 −0.2 0.0 0.2 0.4 −0.8 −0.4 0.0 0.4 ESS=113.3 θ ESS=92.99 θ ESS=121.4 θ 3.0 2.0 C stands for computation 2.0 Density Density Density 1.5 1.0 1.0 0.0 0.0 0.0 [producing a parameter −0.6 −0.2 ESS=133.6 θ 0.2 0.6 −0.2 0.0 0.2 ESS=116.4 θ 0.4 0.6 −0.5 ESS=131.6 θ 0.0 0.5 0.0 1.0 2.0 3.0 2.0 sample] Density Density Density 1.0 1.0 0.0 0.0 −0.6 −0.2 0.2 0.4 0.6 −0.4 −0.2 0.0 0.2 0.4 −0.5 0.0 0.5
  31. 31. How Bayesian is aBc? Could we turn the resolution into a Bayesian answer? ideally so (not meaningfull: requires ∞-ly powerful computer asymptotically so (when sample size goes to ∞: meaningfull?) approximation error unknown (w/o costly simulation) true Bayes for wrong model (formal and artificial) true Bayes for estimated likelihood (back to econometrics?)
  32. 32. Untractable likelihoodBack to stage zero: what can we dowhen a likelihood function f (y|θ) iswell-defined but impossible / toocostly to compute...? MCMC cannot be implemented! shall we give up Bayesian inference altogether?! or settle for an almost Bayesian inference/picture...?
  33. 33. Untractable likelihoodBack to stage zero: what can we dowhen a likelihood function f (y|θ) iswell-defined but impossible / toocostly to compute...? MCMC cannot be implemented! shall we give up Bayesian inference altogether?! or settle for an almost Bayesian inference/picture...?
  34. 34. ABC methodology Bayesian setting: target is π(θ)f (x|θ) When likelihood f (x|θ) not in closed form, likelihood-free rejection technique: Foundation For an observation y ∼ f (y|θ), under the prior π(θ), if one keeps jointly simulating θ ∼ π(θ) , z ∼ f (z|θ ) , until the auxiliary variable z is equal to the observed value, z = y, then the selected θ ∼ π(θ|y) [Rubin, 1984; Diggle & Gratton, 1984; Tavar´ et al., 1997] e
  35. 35. ABC methodology Bayesian setting: target is π(θ)f (x|θ) When likelihood f (x|θ) not in closed form, likelihood-free rejection technique: Foundation For an observation y ∼ f (y|θ), under the prior π(θ), if one keeps jointly simulating θ ∼ π(θ) , z ∼ f (z|θ ) , until the auxiliary variable z is equal to the observed value, z = y, then the selected θ ∼ π(θ|y) [Rubin, 1984; Diggle & Gratton, 1984; Tavar´ et al., 1997] e
  36. 36. ABC methodology Bayesian setting: target is π(θ)f (x|θ) When likelihood f (x|θ) not in closed form, likelihood-free rejection technique: Foundation For an observation y ∼ f (y|θ), under the prior π(θ), if one keeps jointly simulating θ ∼ π(θ) , z ∼ f (z|θ ) , until the auxiliary variable z is equal to the observed value, z = y, then the selected θ ∼ π(θ|y) [Rubin, 1984; Diggle & Gratton, 1984; Tavar´ et al., 1997] e
  37. 37. A as A...pproximative When y is a continuous random variable, strict equality z = y is replaced with a tolerance zone ρ(y, z) where ρ is a distance Output distributed from def π(θ) Pθ {ρ(y, z) < } ∝ π(θ|ρ(y, z) < ) [Pritchard et al., 1999]
  38. 38. A as A...pproximative When y is a continuous random variable, strict equality z = y is replaced with a tolerance zone ρ(y, z) where ρ is a distance Output distributed from def π(θ) Pθ {ρ(y, z) < } ∝ π(θ|ρ(y, z) < ) [Pritchard et al., 1999]
  39. 39. ABC algorithm In most implementations, further degree of A...pproximation: Algorithm 1 Likelihood-free rejection sampler for i = 1 to N do repeat generate θ from the prior distribution π(·) generate z from the likelihood f (·|θ ) until ρ{η(z), η(y)} set θi = θ end for where η(y) defines a (not necessarily sufficient) statistic
  40. 40. Output The likelihood-free algorithm samples from the marginal in z of: π(θ)f (z|θ)IA ,y (z) π (θ, z|y) = , A ,y ×Θ π(θ)f (z|θ)dzdθ where A ,y = {z ∈ D|ρ(η(z), η(y)) < }. The idea behind ABC is that the summary statistics coupled with a small tolerance should provide a good approximation of the posterior distribution: π (θ|y) = π (θ, z|y)dz ≈ π(θ|y) . ...does it?!
  41. 41. Output The likelihood-free algorithm samples from the marginal in z of: π(θ)f (z|θ)IA ,y (z) π (θ, z|y) = , A ,y ×Θ π(θ)f (z|θ)dzdθ where A ,y = {z ∈ D|ρ(η(z), η(y)) < }. The idea behind ABC is that the summary statistics coupled with a small tolerance should provide a good approximation of the posterior distribution: π (θ|y) = π (θ, z|y)dz ≈ π(θ|y) . ...does it?!
  42. 42. Output The likelihood-free algorithm samples from the marginal in z of: π(θ)f (z|θ)IA ,y (z) π (θ, z|y) = , A ,y ×Θ π(θ)f (z|θ)dzdθ where A ,y = {z ∈ D|ρ(η(z), η(y)) < }. The idea behind ABC is that the summary statistics coupled with a small tolerance should provide a good approximation of the posterior distribution: π (θ|y) = π (θ, z|y)dz ≈ π(θ|y) . ...does it?!
  43. 43. Output The likelihood-free algorithm samples from the marginal in z of: π(θ)f (z|θ)IA ,y (z) π (θ, z|y) = , A ,y ×Θ π(θ)f (z|θ)dzdθ where A ,y = {z ∈ D|ρ(η(z), η(y)) < }. The idea behind ABC is that the summary statistics coupled with a small tolerance should provide a good approximation of the restricted posterior distribution: π (θ|y) = π (θ, z|y)dz ≈ π(θ|η(y)) . Not so good..! skip convergence details!
  44. 44. Convergence of ABC What happens when → 0? For B ⊂ Θ, we have A f (z|θ)dz f (z|θ)π(θ)dθ ,y B π(θ)dθ = dz B A ,y ×Θ π(θ)f (z|θ)dzdθ A ,y A ,y ×Θ π(θ)f (z|θ)dzdθ B f (z|θ)π(θ)dθ m(z) = dz A ,y m(z) A ,y ×Θ π(θ)f (z|θ)dzdθ m(z) = π(B|z) dz A ,y A ,y ×Θ π(θ)f (z|θ)dzdθ which indicates convergence for a continuous π(B|z).
  45. 45. Convergence of ABC What happens when → 0? For B ⊂ Θ, we have A f (z|θ)dz f (z|θ)π(θ)dθ ,y B π(θ)dθ = dz B A ,y ×Θ π(θ)f (z|θ)dzdθ A ,y A ,y ×Θ π(θ)f (z|θ)dzdθ B f (z|θ)π(θ)dθ m(z) = dz A ,y m(z) A ,y ×Θ π(θ)f (z|θ)dzdθ m(z) = π(B|z) dz A ,y A ,y ×Θ π(θ)f (z|θ)dzdθ which indicates convergence for a continuous π(B|z).
  46. 46. Convergence (do not attempt!) ...and the above does not apply to insufficient statistics: If η(y) is not a sufficient statistics, the best one can hope for is π(θ|η(y)) , not π(θ|y) If η(y) is an ancillary statistic, the whole information contained in y is lost!, the “best” one can “hope” for is π(θ|η(y)) = π(θ) Bummer!!!
  47. 47. Convergence (do not attempt!) ...and the above does not apply to insufficient statistics: If η(y) is not a sufficient statistics, the best one can hope for is π(θ|η(y)) , not π(θ|y) If η(y) is an ancillary statistic, the whole information contained in y is lost!, the “best” one can “hope” for is π(θ|η(y)) = π(θ) Bummer!!!
  48. 48. Convergence (do not attempt!) ...and the above does not apply to insufficient statistics: If η(y) is not a sufficient statistics, the best one can hope for is π(θ|η(y)) , not π(θ|y) If η(y) is an ancillary statistic, the whole information contained in y is lost!, the “best” one can “hope” for is π(θ|η(y)) = π(θ) Bummer!!!
  49. 49. Convergence (do not attempt!) ...and the above does not apply to insufficient statistics: If η(y) is not a sufficient statistics, the best one can hope for is π(θ|η(y)) , not π(θ|y) If η(y) is an ancillary statistic, the whole information contained in y is lost!, the “best” one can “hope” for is π(θ|η(y)) = π(θ) Bummer!!!
  50. 50. MA example Inference on the parameters of a MA(q) model q xt = t + ϑi t−i t−i i.i.d.w.n. i=1 bypass MA illustration Simple prior: uniform over the inverse [real and complex] roots in q Q(u) = 1 − ϑi u i i=1 under the identifiability conditions
  51. 51. MA example Inference on the parameters of a MA(q) model q xt = t + ϑi t−i t−i i.i.d.w.n. i=1 bypass MA illustration Simple prior: uniform prior over the identifiability zone in the parameter space, i.e. triangle for MA(2)
  52. 52. MA example (2) ABC algorithm thus made of 1. picking a new value (ϑ1 , ϑ2 ) in the triangle 2. generating an iid sequence ( t )−q<t T 3. producing a simulated series (xt )1 t T Distance: basic distance between the series T ρ((xt )1 t T , (xt )1 t T) = (xt − xt )2 t=1 or distance between summary statistics like the q = 2 autocorrelations T τj = xt xt−j t=j+1
  53. 53. MA example (2) ABC algorithm thus made of 1. picking a new value (ϑ1 , ϑ2 ) in the triangle 2. generating an iid sequence ( t )−q<t T 3. producing a simulated series (xt )1 t T Distance: basic distance between the series T ρ((xt )1 t T , (xt )1 t T) = (xt − xt )2 t=1 or distance between summary statistics like the q = 2 autocorrelations T τj = xt xt−j t=j+1
  54. 54. Comparison of distance impact Impact of tolerance on ABC sample against either distance ( = 100%, 10%, 1%, 0.1%) for an MA(2) model
  55. 55. Comparison of distance impact 4 1.5 3 1.0 2 0.5 1 0.0 0 0.0 0.2 0.4 0.6 0.8 −2.0 −1.0 0.0 0.5 1.0 1.5 θ1 θ2 Impact of tolerance on ABC sample against either distance ( = 100%, 10%, 1%, 0.1%) for an MA(2) model
  56. 56. Comparison of distance impact 4 1.5 3 1.0 2 0.5 1 0.0 0 0.0 0.2 0.4 0.6 0.8 −2.0 −1.0 0.0 0.5 1.0 1.5 θ1 θ2 Impact of tolerance on ABC sample against either distance ( = 100%, 10%, 1%, 0.1%) for an MA(2) model
  57. 57. Comments Role of distance paramount (because = 0) Scaling of components of η(y) is also determinant matters little if “small enough” representative of “curse of dimensionality” small is beautiful! the data as a whole may be paradoxically weakly informative for ABC
  58. 58. ABC (simul’) advances how approximative is ABC? ABC as knn Simulating from the prior is often poor in efficiency Either modify the proposal distribution on θ to increase the density of x’s within the vicinity of y ... [Marjoram et al, 2003; Bortot et al., 2007, Sisson et al., 2007] ...or by viewing the problem as a conditional density estimation and by developing techniques to allow for larger [Beaumont et al., 2002] .....or even by including in the inferential framework [ABCµ ] [Ratmann et al., 2009]
  59. 59. ABC (simul’) advances how approximative is ABC? ABC as knn Simulating from the prior is often poor in efficiency Either modify the proposal distribution on θ to increase the density of x’s within the vicinity of y ... [Marjoram et al, 2003; Bortot et al., 2007, Sisson et al., 2007] ...or by viewing the problem as a conditional density estimation and by developing techniques to allow for larger [Beaumont et al., 2002] .....or even by including in the inferential framework [ABCµ ] [Ratmann et al., 2009]
  60. 60. ABC (simul’) advances how approximative is ABC? ABC as knn Simulating from the prior is often poor in efficiency Either modify the proposal distribution on θ to increase the density of x’s within the vicinity of y ... [Marjoram et al, 2003; Bortot et al., 2007, Sisson et al., 2007] ...or by viewing the problem as a conditional density estimation and by developing techniques to allow for larger [Beaumont et al., 2002] .....or even by including in the inferential framework [ABCµ ] [Ratmann et al., 2009]
  61. 61. ABC (simul’) advances how approximative is ABC? ABC as knn Simulating from the prior is often poor in efficiency Either modify the proposal distribution on θ to increase the density of x’s within the vicinity of y ... [Marjoram et al, 2003; Bortot et al., 2007, Sisson et al., 2007] ...or by viewing the problem as a conditional density estimation and by developing techniques to allow for larger [Beaumont et al., 2002] .....or even by including in the inferential framework [ABCµ ] [Ratmann et al., 2009]
  62. 62. ABC-NP Better usage of [prior] simulations by adjustement: instead of throwing away θ such that ρ(η(z), η(y)) > , replace θ’s with locally regressed transforms θ∗ = θ − {η(z) − η(y)}T β ^ [Csill´ry et al., TEE, 2010] e ^ where β is obtained by [NP] weighted least square regression on (η(z) − η(y)) with weights Kδ {ρ(η(z), η(y))} [Beaumont et al., 2002, Genetics]
  63. 63. ABC-NP (regression) Also found in the subsequent literature, e.g. in Fearnhead-Prangle (2012) : weight directly simulation by Kδ {ρ(η(z(θ)), η(y))} or S 1 Kδ {ρ(η(zs (θ)), η(y))} S s=1 [consistent estimate of f (η|θ)] Curse of dimensionality: poor estimate when d = dim(η) is large...
  64. 64. ABC-NP (regression) Also found in the subsequent literature, e.g. in Fearnhead-Prangle (2012) : weight directly simulation by Kδ {ρ(η(z(θ)), η(y))} or S 1 Kδ {ρ(η(zs (θ)), η(y))} S s=1 [consistent estimate of f (η|θ)] Curse of dimensionality: poor estimate when d = dim(η) is large...
  65. 65. ABC-NP (density estimation) Use of the kernel weights Kδ {ρ(η(z(θ)), η(y))} leads to the NP estimate of the posterior expectation i θi Kδ {ρ(η(z(θi )), η(y))} i Kδ {ρ(η(z(θi )), η(y))} [Blum, JASA, 2010]
  66. 66. ABC-NP (density estimation) Use of the kernel weights Kδ {ρ(η(z(θ)), η(y))} leads to the NP estimate of the posterior conditional density i ˜ Kb (θi − θ)Kδ {ρ(η(z(θi )), η(y))} i Kδ {ρ(η(z(θi )), η(y))} [Blum, JASA, 2010]
  67. 67. ABC-NP (density estimations) Other versions incorporating regression adjustments i ˜ Kb (θ∗ − θ)Kδ {ρ(η(z(θi )), η(y))} i i Kδ {ρ(η(z(θi )), η(y))} In all cases, error E[^ (θ|y)] − g (θ|y) = cb 2 + cδ2 + OP (b 2 + δ2 ) + OP (1/nδd ) g c var(^ (θ|y)) = g (1 + oP (1)) nbδd
  68. 68. ABC-NP (density estimations) Other versions incorporating regression adjustments i ˜ Kb (θ∗ − θ)Kδ {ρ(η(z(θi )), η(y))} i i Kδ {ρ(η(z(θi )), η(y))} In all cases, error E[^ (θ|y)] − g (θ|y) = cb 2 + cδ2 + OP (b 2 + δ2 ) + OP (1/nδd ) g c var(^ (θ|y)) = g (1 + oP (1)) nbδd [Blum, JASA, 2010]
  69. 69. ABC-NP (density estimations) Other versions incorporating regression adjustments i ˜ Kb (θ∗ − θ)Kδ {ρ(η(z(θi )), η(y))} i i Kδ {ρ(η(z(θi )), η(y))} In all cases, error E[^ (θ|y)] − g (θ|y) = cb 2 + cδ2 + OP (b 2 + δ2 ) + OP (1/nδd ) g c var(^ (θ|y)) = g (1 + oP (1)) nbδd [standard NP calculations]
  70. 70. ABC-NCH Incorporating non-linearities and heterocedasticities: σ(η(y)) ^ θ∗ = m(η(y)) + [θ − m(η(z))] ^ ^ σ(η(z)) ^ where m(η) estimated by non-linear regression (e.g., neural network) ^ σ(η) estimated by non-linear regression on residuals ^ log{θi − m(ηi )}2 = log σ2 (ηi ) + ξi ^ [Blum & Fran¸ois, 2009] c
  71. 71. ABC-NCH Incorporating non-linearities and heterocedasticities: σ(η(y)) ^ θ∗ = m(η(y)) + [θ − m(η(z))] ^ ^ σ(η(z)) ^ where m(η) estimated by non-linear regression (e.g., neural network) ^ σ(η) estimated by non-linear regression on residuals ^ log{θi − m(ηi )}2 = log σ2 (ηi ) + ξi ^ [Blum & Fran¸ois, 2009] c
  72. 72. ABC as knn [Biau et al., 2012, arxiv:1207.6461] Practice of ABC: determine tolerance as a quantile on observed distances, say 10% or 1% quantile, = N = qα (d1 , . . . , dN ) Interpretation of ε as nonparametric bandwidth only approximation of the actual practice [Blum & Fran¸ois, 2010] c ABC is a k-nearest neighbour (knn) method with kN = N N [Loftsgaarden & Quesenberry, 1965]
  73. 73. ABC as knn [Biau et al., 2012, arxiv:1207.6461] Practice of ABC: determine tolerance as a quantile on observed distances, say 10% or 1% quantile, = N = qα (d1 , . . . , dN ) Interpretation of ε as nonparametric bandwidth only approximation of the actual practice [Blum & Fran¸ois, 2010] c ABC is a k-nearest neighbour (knn) method with kN = N N [Loftsgaarden & Quesenberry, 1965]
  74. 74. ABC as knn [Biau et al., 2012, arxiv:1207.6461] Practice of ABC: determine tolerance as a quantile on observed distances, say 10% or 1% quantile, = N = qα (d1 , . . . , dN ) Interpretation of ε as nonparametric bandwidth only approximation of the actual practice [Blum & Fran¸ois, 2010] c ABC is a k-nearest neighbour (knn) method with kN = N N [Loftsgaarden & Quesenberry, 1965]
  75. 75. ABC consistency Provided kN / log log N −→ ∞ and kN /N −→ 0 as N → ∞, for almost all s0 (with respect to the distribution of S), with probability 1, kN 1 ϕ(θj ) −→ E[ϕ(θj )|S = s0 ] kN j=1 [Devroye, 1982] Biau et al. (2012) also recall pointwise and integrated mean square error consistency results on the corresponding kernel estimate of the conditional posterior distribution, under constraints p kN → ∞, kN /N → 0, hN → 0 and hN kN → ∞,
  76. 76. ABC consistency Provided kN / log log N −→ ∞ and kN /N −→ 0 as N → ∞, for almost all s0 (with respect to the distribution of S), with probability 1, kN 1 ϕ(θj ) −→ E[ϕ(θj )|S = s0 ] kN j=1 [Devroye, 1982] Biau et al. (2012) also recall pointwise and integrated mean square error consistency results on the corresponding kernel estimate of the conditional posterior distribution, under constraints p kN → ∞, kN /N → 0, hN → 0 and hN kN → ∞,
  77. 77. Rates of convergence Further assumptions (on target and kernel) allow for precise (integrated mean square) convergence rates (as a power of the sample size N), derived from classical k-nearest neighbour regression, like 4 when m = 1, 2, 3, kN ≈ N (p+4)/(p+8) and rate N − p+8 4 when m = 4, kN ≈ N (p+4)/(p+8) and rate N − p+8 log N 4 when m > 4, kN ≈ N (p+4)/(m+p+4) and rate N − m+p+4 [Biau et al., 2012, arxiv:1207.6461] Only applies to sufficient summary statistics
  78. 78. Rates of convergence Further assumptions (on target and kernel) allow for precise (integrated mean square) convergence rates (as a power of the sample size N), derived from classical k-nearest neighbour regression, like 4 when m = 1, 2, 3, kN ≈ N (p+4)/(p+8) and rate N − p+8 4 when m = 4, kN ≈ N (p+4)/(p+8) and rate N − p+8 log N 4 when m > 4, kN ≈ N (p+4)/(m+p+4) and rate N − m+p+4 [Biau et al., 2012, arxiv:1207.6461] Only applies to sufficient summary statistics
  79. 79. ABC inference machine Introduction ABC ABC as an inference machine Error inc. Exact BC and approximate targets summary statistic ABC for model choice Model choice consistency ABCel
  80. 80. How much Bayesian aBc is..? maybe a convergent method of inference (meaningful? sufficient? foreign?) approximation error unknown (w/o simulation) pragmatic Bayes (there is no other solution!) many calibration issues (tolerance, distance, statistics)
  81. 81. How much Bayesian aBc is..? maybe a convergent method of inference (meaningful? sufficient? foreign?) approximation error unknown (w/o simulation) pragmatic Bayes (there is no other solution!) many calibration issues (tolerance, distance, statistics) ...should Bayesians care?!
  82. 82. How much Bayesian aBc is..? maybe a convergent method of inference (meaningful? sufficient? foreign?) approximation error unknown (w/o simulation) pragmatic Bayes (there is no other solution!) many calibration issues (tolerance, distance, statistics) yes they should!!!
  83. 83. How much Bayesian aBc is..? maybe a convergent method of inference (meaningful? sufficient? foreign?) approximation error unknown (w/o simulation) pragmatic Bayes (there is no other solution!) many calibration issues (tolerance, distance, statistics) to ABCel
  84. 84. ABCµ Idea Infer about the error as well as about the parameter: Use of a joint density f (θ, |y) ∝ ξ( |y, θ) × πθ (θ) × π ( ) where y is the data, and ξ( |y, θ) is the prior predictive density of ρ(η(z), η(y)) given θ and y when z ∼ f (z|θ) Warning! Replacement of ξ( |y, θ) with a non-parametric kernel approximation. [Ratmann, Andrieu, Wiuf and Richardson, 2009, PNAS]
  85. 85. ABCµ Idea Infer about the error as well as about the parameter: Use of a joint density f (θ, |y) ∝ ξ( |y, θ) × πθ (θ) × π ( ) where y is the data, and ξ( |y, θ) is the prior predictive density of ρ(η(z), η(y)) given θ and y when z ∼ f (z|θ) Warning! Replacement of ξ( |y, θ) with a non-parametric kernel approximation. [Ratmann, Andrieu, Wiuf and Richardson, 2009, PNAS]
  86. 86. ABCµ Idea Infer about the error as well as about the parameter: Use of a joint density f (θ, |y) ∝ ξ( |y, θ) × πθ (θ) × π ( ) where y is the data, and ξ( |y, θ) is the prior predictive density of ρ(η(z), η(y)) given θ and y when z ∼ f (z|θ) Warning! Replacement of ξ( |y, θ) with a non-parametric kernel approximation. [Ratmann, Andrieu, Wiuf and Richardson, 2009, PNAS]
  87. 87. ABCµ details Multidimensional distances ρk (k = 1, . . . , K ) and errors k = ρk (ηk (z), ηk (y)), with 1 k ∼ ξk ( |y, θ) ≈ ξk ( |y, θ) = ^ K [{ k −ρk (ηk (zb ), ηk (y))}/hk ] Bhk b then used in replacing ξ( |y, θ) with mink ξk ( |y, θ) ^ ABCµ involves acceptance probability π(θ , ) q(θ , θ)q( , ) mink ξk ( |y, θ ) ^ π(θ, ) q(θ, θ )q( , ) mink ξk ( |y, θ) ^
  88. 88. ABCµ details Multidimensional distances ρk (k = 1, . . . , K ) and errors k = ρk (ηk (z), ηk (y)), with 1 k ∼ ξk ( |y, θ) ≈ ξk ( |y, θ) = ^ K [{ k −ρk (ηk (zb ), ηk (y))}/hk ] Bhk b then used in replacing ξ( |y, θ) with mink ξk ( |y, θ) ^ ABCµ involves acceptance probability π(θ , ) q(θ , θ)q( , ) mink ξk ( |y, θ ) ^ π(θ, ) q(θ, θ )q( , ) mink ξk ( |y, θ) ^
  89. 89. ABCµ multiple errors [ c Ratmann et al., PNAS, 2009]
  90. 90. ABCµ for model choice [ c Ratmann et al., PNAS, 2009]
  91. 91. Wilkinson’s exact BC (not exactly!) ABC approximation error (i.e. non-zero tolerance) replaced with exact simulation from a controlled approximation to the target, convolution of true posterior with kernel function π(θ)f (z|θ)K (y − z) π (θ, z|y) = , π(θ)f (z|θ)K (y − z)dzdθ with K kernel parameterised by bandwidth . [Wilkinson, 2008] Theorem The ABC algorithm based on the assumption of a randomised observation y = y + ξ, ξ ∼ K , and an acceptance probability of ˜ K (y − z)/M gives draws from the posterior distribution π(θ|y).
  92. 92. Wilkinson’s exact BC (not exactly!) ABC approximation error (i.e. non-zero tolerance) replaced with exact simulation from a controlled approximation to the target, convolution of true posterior with kernel function π(θ)f (z|θ)K (y − z) π (θ, z|y) = , π(θ)f (z|θ)K (y − z)dzdθ with K kernel parameterised by bandwidth . [Wilkinson, 2008] Theorem The ABC algorithm based on the assumption of a randomised observation y = y + ξ, ξ ∼ K , and an acceptance probability of ˜ K (y − z)/M gives draws from the posterior distribution π(θ|y).
  93. 93. How exact a BC? “Using to represent measurement error is straightforward, whereas using to model the model discrepancy is harder to conceptualize and not as commonly used” [Richard Wilkinson, 2008]
  94. 94. How exact a BC? Pros Pseudo-data from true model and observed data from noisy model Interesting perspective in that outcome is completely controlled Link with ABCµ and assuming y is observed with a measurement error with density K Relates to the theory of model approximation [Kennedy & O’Hagan, 2001] Cons Requires K to be bounded by M True approximation error never assessed Requires a modification of the standard ABC algorithm
  95. 95. ABC for HMMs Specific case of a hidden Markov model Xt+1 ∼ Qθ (Xt , ·) Yt+1 ∼ gθ (·|xt ) where only y0 is observed. 1:n [Dean, Singh, Jasra, & Peters, 2011] Use of specific constraints, adapted to the Markov structure: y1 ∈ B(y1 , ) × · · · × yn ∈ B(yn , ) 0 0
  96. 96. ABC for HMMs Specific case of a hidden Markov model Xt+1 ∼ Qθ (Xt , ·) Yt+1 ∼ gθ (·|xt ) where only y0 is observed. 1:n [Dean, Singh, Jasra, & Peters, 2011] Use of specific constraints, adapted to the Markov structure: y1 ∈ B(y1 , ) × · · · × yn ∈ B(yn , ) 0 0
  97. 97. ABC-MLE for HMMs ABC-MLE defined by θn = arg max Pθ Y1 ∈ B(y1 , ), . . . , Yn ∈ B(yn , ) ^ 0 0 θ Exact MLE for the likelihood same basis as Wilkinson! 0 pθ (y1 , . . . , yn ) corresponding to the perturbed process (xt , yt + zt )1 t n zt ∼ U(B(0, 1) [Dean, Singh, Jasra, & Peters, 2011]
  98. 98. ABC-MLE for HMMs ABC-MLE defined by θn = arg max Pθ Y1 ∈ B(y1 , ), . . . , Yn ∈ B(yn , ) ^ 0 0 θ Exact MLE for the likelihood same basis as Wilkinson! 0 pθ (y1 , . . . , yn ) corresponding to the perturbed process (xt , yt + zt )1 t n zt ∼ U(B(0, 1) [Dean, Singh, Jasra, & Peters, 2011]
  99. 99. ABC-MLE is biased ABC-MLE is asymptotically (in n) biased with target l (θ) = Eθ∗ [log pθ (Y1 |Y−∞:0 )] but ABC-MLE converges to true value in the sense l n (θn ) → l (θ) for all sequences (θn ) converging to θ and n
  100. 100. ABC-MLE is biased ABC-MLE is asymptotically (in n) biased with target l (θ) = Eθ∗ [log pθ (Y1 |Y−∞:0 )] but ABC-MLE converges to true value in the sense l n (θn ) → l (θ) for all sequences (θn ) converging to θ and n
  101. 101. Noisy ABC-MLE Idea: Modify instead the data from the start 0 (y1 + ζ1 , . . . , yn + ζn ) [ see Fearnhead-Prangle ] noisy ABC-MLE estimate arg max Pθ Y1 ∈ B(y1 + ζ1 , ), . . . , Yn ∈ B(yn + ζn , ) 0 0 θ [Dean, Singh, Jasra, & Peters, 2011]
  102. 102. Consistent noisy ABC-MLE Degrading the data improves the estimation performances: Noisy ABC-MLE is asymptotically (in n) consistent under further assumptions, the noisy ABC-MLE is asymptotically normal increase in variance of order −2 likely degradation in precision or computing time due to the lack of summary statistic [curse of dimensionality]
  103. 103. SMC for ABC likelihood Algorithm 2 SMC ABC for HMMs Given θ for k = 1, . . . , n do 1 1 N N generate proposals (xk , yk ), . . . , (xk , yk ) from the model weigh each proposal with ωk l =I l B(yk + ζk , ) (yk ) 0 l renormalise the weights and sample the xk ’s accordingly end for approximate the likelihood by n N ωlk N k=1 l=1 [Jasra, Singh, Martin, & McCoy, 2010]
  104. 104. Which summary? Fundamental difficulty of the choice of the summary statistic when there is no non-trivial sufficient statistics Starting from a large collection of summary statistics is available, Joyce and Marjoram (2008) consider the sequential inclusion into the ABC target, with a stopping rule based on a likelihood ratio test Not taking into account the sequential nature of the tests Depends on parameterisation Order of inclusion matters likelihood ratio test?!
  105. 105. Which summary? Fundamental difficulty of the choice of the summary statistic when there is no non-trivial sufficient statistics Starting from a large collection of summary statistics is available, Joyce and Marjoram (2008) consider the sequential inclusion into the ABC target, with a stopping rule based on a likelihood ratio test Not taking into account the sequential nature of the tests Depends on parameterisation Order of inclusion matters likelihood ratio test?!
  106. 106. Which summary? Fundamental difficulty of the choice of the summary statistic when there is no non-trivial sufficient statistics Starting from a large collection of summary statistics is available, Joyce and Marjoram (2008) consider the sequential inclusion into the ABC target, with a stopping rule based on a likelihood ratio test Not taking into account the sequential nature of the tests Depends on parameterisation Order of inclusion matters likelihood ratio test?!
  107. 107. Which summary for model choice? Depending on the choice of η(·), the Bayes factor based on this insufficient statistic, η π1 (θ1 )f1η (η(y)|θ1 ) dθ1 B12 (y) = , π2 (θ2 )f2η (η(y)|θ2 ) dθ2 is consistent or not. [X, Cornuet, Marin, & Pillai, 2012] Consistency only depends on the range of Ei [η(y)] under both models. [Marin, Pillai, X, & Rousseau, 2012]
  108. 108. Which summary for model choice? Depending on the choice of η(·), the Bayes factor based on this insufficient statistic, η π1 (θ1 )f1η (η(y)|θ1 ) dθ1 B12 (y) = , π2 (θ2 )f2η (η(y)|θ2 ) dθ2 is consistent or not. [X, Cornuet, Marin, & Pillai, 2012] Consistency only depends on the range of Ei [η(y)] under both models. [Marin, Pillai, X, & Rousseau, 2012]
  109. 109. Semi-automatic ABC Fearnhead and Prangle (2010) study ABC and the selection of the summary statistic in close proximity to Wilkinson’s proposal ABC considered as inferential method and calibrated as such randomised (or ‘noisy’) version of the summary statistics ˜ η(y) = η(y) + τ derivation of a well-calibrated version of ABC, i.e. an algorithm that gives proper predictions for the distribution associated with this randomised summary statistic
  110. 110. Summary [of F&P/statistics) optimality of the posterior expectation E[θ|y] of the parameter of interest as summary statistics η(y)! use of the standard quadratic loss function (θ − θ0 )T A(θ − θ0 ) . recent extension to model choice, optimality of Bayes factor B12 (y) [F&P, ISBA 2012 talk]
  111. 111. Summary [of F&P/statistics) optimality of the posterior expectation E[θ|y] of the parameter of interest as summary statistics η(y)! use of the standard quadratic loss function (θ − θ0 )T A(θ − θ0 ) . recent extension to model choice, optimality of Bayes factor B12 (y) [F&P, ISBA 2012 talk]
  112. 112. Conclusion Choice of summary statistics is paramount for ABC validation/performance At best, ABC approximates π(. | η(y)) Model selection feasible with ABC [with caution!] For estimation, consistency if {θ; µ(θ) = µ0 } = θ0 For testing consistency if {µ1 (θ1 ), θ1 ∈ Θ1 } ∩ {µ2 (θ2 ), θ2 ∈ Θ2 } = ∅ [Marin et al., 2011]
  113. 113. Conclusion Choice of summary statistics is paramount for ABC validation/performance At best, ABC approximates π(. | η(y)) Model selection feasible with ABC [with caution!] For estimation, consistency if {θ; µ(θ) = µ0 } = θ0 For testing consistency if {µ1 (θ1 ), θ1 ∈ Θ1 } ∩ {µ2 (θ2 ), θ2 ∈ Θ2 } = ∅ [Marin et al., 2011]
  114. 114. Conclusion Choice of summary statistics is paramount for ABC validation/performance At best, ABC approximates π(. | η(y)) Model selection feasible with ABC [with caution!] For estimation, consistency if {θ; µ(θ) = µ0 } = θ0 For testing consistency if {µ1 (θ1 ), θ1 ∈ Θ1 } ∩ {µ2 (θ2 ), θ2 ∈ Θ2 } = ∅ [Marin et al., 2011]
  115. 115. Conclusion Choice of summary statistics is paramount for ABC validation/performance At best, ABC approximates π(. | η(y)) Model selection feasible with ABC [with caution!] For estimation, consistency if {θ; µ(θ) = µ0 } = θ0 For testing consistency if {µ1 (θ1 ), θ1 ∈ Θ1 } ∩ {µ2 (θ2 ), θ2 ∈ Θ2 } = ∅ [Marin et al., 2011]
  116. 116. Conclusion Choice of summary statistics is paramount for ABC validation/performance At best, ABC approximates π(. | η(y)) Model selection feasible with ABC [with caution!] For estimation, consistency if {θ; µ(θ) = µ0 } = θ0 For testing consistency if {µ1 (θ1 ), θ1 ∈ Θ1 } ∩ {µ2 (θ2 ), θ2 ∈ Θ2 } = ∅ [Marin et al., 2011]
  117. 117. ABC for model choiceIntroductionABCABC as an inference machineABC for model choice BMC Principle Gibbs random fields (counterexample) Generic ABC model choiceModel choice consistencyABCel
  118. 118. Bayesian model choice BMC Principle Several models M1 , M2 , . . . are considered simultaneously for dataset y and model index M central to inference. Use of prior π(M = m), plus prior distribution on the parameter conditional on the value m of the model index, πm (θm )
  119. 119. Bayesian model choice BMC Principle Several models M1 , M2 , . . . are considered simultaneously for dataset y and model index M central to inference. Goal is to derive the posterior distribution of M, π(M = m|data) a challenging computational target when models are complex.
  120. 120. Generic ABC for model choice Algorithm 3 Likelihood-free model choice sampler (ABC-MC) for t = 1 to T do repeat Generate m from the prior π(M = m) Generate θm from the prior πm (θm ) Generate z from the model fm (z|θm ) until ρ{η(z), η(y)} < Set m(t) = m and θ(t) = θm end for [Grelaud & al., 2009; Toni & al., 2009]
  121. 121. ABC estimates Posterior probability π(M = m|y) approximated by the frequency of acceptances from model m T 1 Im(t) =m . T t=1
  122. 122. ABC estimates Posterior probability π(M = m|y) approximated by the frequency of acceptances from model m T 1 Im(t) =m . T t=1 Extension to a weighted polychotomous logistic regression estimate of π(M = m|y), with non-parametric kernel weights [Cornuet et al., DIYABC, 2009]
  123. 123. Potts model Skip MRFs Potts model Distribution with an energy function of the form θS(y) = θ δyl =yi l∼i where l∼i denotes a neighbourhood structure In most realistic settings, summation Zθ = exp{θS(x)} x∈X involves too many terms to be manageable and numerical approximations cannot always be trusted
  124. 124. Potts model Skip MRFs Potts model Distribution with an energy function of the form θS(y) = θ δyl =yi l∼i where l∼i denotes a neighbourhood structure In most realistic settings, summation Zθ = exp{θS(x)} x∈X involves too many terms to be manageable and numerical approximations cannot always be trusted
  125. 125. Neighbourhood relations Setup Choice to be made between M neighbourhood relations m i ∼i (0 m M − 1) with Sm (x) = I{xi =xi } m i ∼i driven by the posterior probabilities of the models.
  126. 126. Model index Computational target: P(M = m|x) ∝ fm (x|θm )πm (θm ) dθm π(M = m) Θm If S(x) sufficient statistic for the joint parameters (M, θ0 , . . . , θM−1 ), P(M = m|x) = P(M = m|S(x)) .
  127. 127. Model index Computational target: P(M = m|x) ∝ fm (x|θm )πm (θm ) dθm π(M = m) Θm If S(x) sufficient statistic for the joint parameters (M, θ0 , . . . , θM−1 ), P(M = m|x) = P(M = m|S(x)) .
  128. 128. Sufficient statistics in Gibbs random fields Each model m has its own sufficient statistic Sm (·) and S(·) = (S0 (·), . . . , SM−1 (·)) is also (model-)sufficient. Explanation: For Gibbs random fields, x|M = m ∼ fm (x|θm ) = fm (x|S(x))fm (S(x)|θm ) 1 2 1 = f 2 (S(x)|θm ) n(S(x)) m where n(S(x)) = {x ∈ X : S(x) = S(x)} ˜ ˜ c S(x) is sufficient for the joint parameters
  129. 129. Sufficient statistics in Gibbs random fields Each model m has its own sufficient statistic Sm (·) and S(·) = (S0 (·), . . . , SM−1 (·)) is also (model-)sufficient. Explanation: For Gibbs random fields, x|M = m ∼ fm (x|θm ) = fm (x|S(x))fm (S(x)|θm ) 1 2 1 = f 2 (S(x)|θm ) n(S(x)) m where n(S(x)) = {x ∈ X : S(x) = S(x)} ˜ ˜ c S(x) is sufficient for the joint parameters
  130. 130. Toy example iid Bernoulli model versus two-state first-order Markov chain, i.e. n f0 (x|θ0 ) = exp θ0 I{xi =1} {1 + exp(θ0 )}n , i=1 versus n 1 f1 (x|θ1 ) = exp θ1 I{xi =xi−1 } {1 + exp(θ1 )}n−1 , 2 i=2 with priors θ0 ∼ U(−5, 5) and θ1 ∼ U(0, 6) (inspired by “phase transition” boundaries).
  131. 131. About sufficiency If η1 (x) sufficient statistic for model m = 1 and parameter θ1 and η2 (x) sufficient statistic for model m = 2 and parameter θ2 , (η1 (x), η2 (x)) is not always sufficient for (m, θm ) c Potential loss of information at the testing level
  132. 132. About sufficiency If η1 (x) sufficient statistic for model m = 1 and parameter θ1 and η2 (x) sufficient statistic for model m = 2 and parameter θ2 , (η1 (x), η2 (x)) is not always sufficient for (m, θm ) c Potential loss of information at the testing level
  133. 133. Poisson/geometric example Sample x = (x1 , . . . , xn ) from either a Poisson P(λ) or from a geometric G(p) Sum n S= xi = η(x) i=1 sufficient statistic for either model but not simultaneously
  134. 134. Limiting behaviour of B12 (T → ∞) ABC approximation T t=1 Imt =1 Iρ{η(zt ),η(y)} B12 (y) = T , t=1 Imt =2 Iρ{η(zt ),η(y)} where the (mt , z t )’s are simulated from the (joint) prior As T goes to infinity, limit Iρ{η(z),η(y)} π1 (θ1 )f1 (z|θ1 ) dz dθ1 B12 (y) = Iρ{η(z),η(y)} π2 (θ2 )f2 (z|θ2 ) dz dθ2 Iρ{η,η(y)} π1 (θ1 )f1η (η|θ1 ) dη dθ1 = , Iρ{η,η(y)} π2 (θ2 )f2η (η|θ2 ) dη dθ2 where f1η (η|θ1 ) and f2η (η|θ2 ) distributions of η(z)
  135. 135. Limiting behaviour of B12 (T → ∞) ABC approximation T t=1 Imt =1 Iρ{η(zt ),η(y)} B12 (y) = T , t=1 Imt =2 Iρ{η(zt ),η(y)} where the (mt , z t )’s are simulated from the (joint) prior As T goes to infinity, limit Iρ{η(z),η(y)} π1 (θ1 )f1 (z|θ1 ) dz dθ1 B12 (y) = Iρ{η(z),η(y)} π2 (θ2 )f2 (z|θ2 ) dz dθ2 Iρ{η,η(y)} π1 (θ1 )f1η (η|θ1 ) dη dθ1 = , Iρ{η,η(y)} π2 (θ2 )f2η (η|θ2 ) dη dθ2 where f1η (η|θ1 ) and f2η (η|θ2 ) distributions of η(z)
  136. 136. Limiting behaviour of B12 ( → 0) When goes to zero, η π1 (θ1 )f1η (η(y)|θ1 ) dθ1 B12 (y) = π2 (θ2 )f2η (η(y)|θ2 ) dθ2 c Bayes factor based on the sole observation of η(y)
  137. 137. Limiting behaviour of B12 ( → 0) When goes to zero, η π1 (θ1 )f1η (η(y)|θ1 ) dθ1 B12 (y) = π2 (θ2 )f2η (η(y)|θ2 ) dθ2 c Bayes factor based on the sole observation of η(y)
  138. 138. Limiting behaviour of B12 (under sufficiency) If η(y) sufficient statistic in both models, fi (y|θi ) = gi (y)fi η (η(y)|θi ) Thus Θ1 π(θ1 )g1 (y)f1η (η(y)|θ1 ) dθ1 B12 (y) = Θ2 π(θ2 )g2 (y)f2η (η(y)|θ2 ) dθ2 g1 (y) π1 (θ1 )f1η (η(y)|θ1 ) dθ1 g1 (y) η = = B (y) . g2 (y) π2 (θ2 )f2η (η(y)|θ2 ) dθ2 g2 (y) 12 [Didelot, Everitt, Johansen & Lawson, 2011] c No discrepancy only when cross-model sufficiency
  139. 139. Limiting behaviour of B12 (under sufficiency) If η(y) sufficient statistic in both models, fi (y|θi ) = gi (y)fi η (η(y)|θi ) Thus Θ1 π(θ1 )g1 (y)f1η (η(y)|θ1 ) dθ1 B12 (y) = Θ2 π(θ2 )g2 (y)f2η (η(y)|θ2 ) dθ2 g1 (y) π1 (θ1 )f1η (η(y)|θ1 ) dθ1 g1 (y) η = = B (y) . g2 (y) π2 (θ2 )f2η (η(y)|θ2 ) dθ2 g2 (y) 12 [Didelot, Everitt, Johansen & Lawson, 2011] c No discrepancy only when cross-model sufficiency
  140. 140. Poisson/geometric example (back) Sample x = (x1 , . . . , xn ) from either a Poisson P(λ) or from a geometric G(p) Discrepancy ratio g1 (x) S!n−S / i xi ! = g2 (x) 1 n+S−1 S
  141. 141. Poisson/geometric discrepancy η Range of B12 (x) versus B12 (x): The values produced have nothing in common.
  142. 142. Formal recovery Creating an encompassing exponential family f (x|θ1 , θ2 , α1 , α2 ) ∝ exp{θT η1 (x) + θT η2 (x) + α1 t1 (x) + α2 t2 (x)} 1 2 leads to a sufficient statistic (η1 (x), η2 (x), t1 (x), t2 (x)) [Didelot, Everitt, Johansen & Lawson, 2011]
  143. 143. Formal recovery Creating an encompassing exponential family f (x|θ1 , θ2 , α1 , α2 ) ∝ exp{θT η1 (x) + θT η2 (x) + α1 t1 (x) + α2 t2 (x)} 1 2 leads to a sufficient statistic (η1 (x), η2 (x), t1 (x), t2 (x)) [Didelot, Everitt, Johansen & Lawson, 2011] In the Poisson/geometric case, if i xi ! is added to S, no discrepancy
  144. 144. Formal recovery Creating an encompassing exponential family f (x|θ1 , θ2 , α1 , α2 ) ∝ exp{θT η1 (x) + θT η2 (x) + α1 t1 (x) + α2 t2 (x)} 1 2 leads to a sufficient statistic (η1 (x), η2 (x), t1 (x), t2 (x)) [Didelot, Everitt, Johansen & Lawson, 2011] Only applies in genuine sufficiency settings... c Inability to evaluate information loss due to summary statistics
  145. 145. Meaning of the ABC-Bayes factor The ABC approximation to the Bayes Factor is based solely on the summary statistics.... In the Poisson/geometric case, if E[yi ] = θ0 > 0, η (θ0 + 1)2 −θ0 lim B12 (y) = e n→∞ θ0
  146. 146. Meaning of the ABC-Bayes factor The ABC approximation to the Bayes Factor is based solely on the summary statistics.... In the Poisson/geometric case, if E[yi ] = θ0 > 0, η (θ0 + 1)2 −θ0 lim B12 (y) = e n→∞ θ0
  147. 147. MA example 1.0 1.0 1.0 1.0 0.8 0.8 0.8 0.8 0.6 0.6 0.6 0.6 0.4 0.4 0.4 0.4 0.2 0.2 0.2 0.2 0.0 0.0 0.0 0.0 1 2 1 2 1 2 1 2 Evolution [against ] of ABC Bayes factor, in terms of frequencies of visits to models MA(1) (left) and MA(2) (right) when equal to 10, 1, .1, .01% quantiles on insufficient autocovariance distances. Sample of 50 points from a MA(2) with θ1 = 0.6, θ2 = 0.2. True Bayes factor equal to 17.71.
  148. 148. MA example 1.0 1.0 1.0 1.0 0.8 0.8 0.8 0.8 0.6 0.6 0.6 0.6 0.4 0.4 0.4 0.4 0.2 0.2 0.2 0.2 0.0 0.0 0.0 0.0 1 2 1 2 1 2 1 2 Evolution [against ] of ABC Bayes factor, in terms of frequencies of visits to models MA(1) (left) and MA(2) (right) when equal to 10, 1, .1, .01% quantiles on insufficient autocovariance distances. Sample of 50 points from a MA(1) model with θ1 = 0.6. True Bayes factor B21 equal to .004.
  149. 149. The only safe cases??? [circa April 2011] Besides specific models like Gibbs random fields, using distances over the data itself escapes the discrepancy... [Toni & Stumpf, 2010; Sousa & al., 2009] ...and so does the use of more informal model fitting measures [Ratmann & al., 2009] ...or use another type of approximation like empirical likelihood [Mengersen et al., 2012, see Kerrie’s ASC 2012 talk]
  150. 150. The only safe cases??? [circa April 2011] Besides specific models like Gibbs random fields, using distances over the data itself escapes the discrepancy... [Toni & Stumpf, 2010; Sousa & al., 2009] ...and so does the use of more informal model fitting measures [Ratmann & al., 2009] ...or use another type of approximation like empirical likelihood [Mengersen et al., 2012, see Kerrie’s ASC 2012 talk]
  151. 151. The only safe cases??? [circa April 2011] Besides specific models like Gibbs random fields, using distances over the data itself escapes the discrepancy... [Toni & Stumpf, 2010; Sousa & al., 2009] ...and so does the use of more informal model fitting measures [Ratmann & al., 2009] ...or use another type of approximation like empirical likelihood [Mengersen et al., 2012, see Kerrie’s ASC 2012 talk]
  152. 152. ABC model choice consistencyIntroductionABCABC as an inference machineABC for model choiceModel choice consistency Formalised framework Consistency results Summary statisticsABCel
  153. 153. The starting point Central question to the validation of ABC for model choice: When is a Bayes factor based on an insufficient statistic T(y) consistent? T Note/warnin: c drawn on T(y) through B12 (y) necessarily differs from c drawn on y through B12 (y) [Marin, Pillai, X, & Rousseau, arXiv, 2012]
  154. 154. The starting point Central question to the validation of ABC for model choice: When is a Bayes factor based on an insufficient statistic T(y) consistent? T Note/warnin: c drawn on T(y) through B12 (y) necessarily differs from c drawn on y through B12 (y) [Marin, Pillai, X, & Rousseau, arXiv, 2012]
  155. 155. A benchmark if toy example Comparison suggested by referee of PNAS paper [thanks!]: [X, Cornuet, Marin, & Pillai, Aug. 2011] Model M1 : y ∼ N(θ1 , 1) opposed √ to model M2 : y ∼ L(θ2 , 1/ 2), Laplace distribution with mean θ2 √ and scale parameter 1/ 2 (variance one).
  156. 156. A benchmark if toy example Comparison suggested by referee of PNAS paper [thanks!]: [X, Cornuet, Marin, & Pillai, Aug. 2011] Model M1 : y ∼ N(θ1 , 1) opposed √ to model M2 : y ∼ L(θ2 , 1/ 2), Laplace distribution with mean θ2 √ and scale parameter 1/ 2 (variance one). Four possible statistics 1. sample mean y (sufficient for M1 if not M2 ); 2. sample median med(y) (insufficient); 3. sample variance var(y) (ancillary); 4. median absolute deviation mad(y) = med(|y − med(y)|);
  157. 157. A benchmark if toy example Comparison suggested by referee of PNAS paper [thanks!]: [X, Cornuet, Marin, & Pillai, Aug. 2011] Model M1 : y ∼ N(θ1 , 1) opposed √ to model M2 : y ∼ L(θ2 , 1/ 2), Laplace distribution with mean θ2 √ and scale parameter 1/ 2 (variance one). n=100 0.7 0.6 0.5 0.4 q 0.3 q q q 0.2 0.1 q q q q q 0.0 q q Gauss Laplace
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×