This document discusses challenges and recent advances in Approximate Bayesian Computation (ABC) methods. ABC methods are used when the likelihood function is intractable or unavailable in closed form. The core ABC algorithm involves simulating parameters from the prior and simulating data, retaining simulations where the simulated and observed data are close according to a distance measure on summary statistics. The document outlines key issues like scalability to large datasets, assessment of uncertainty, and model choice, and discusses advances such as modified proposals, nonparametric methods, and perspectives that include summary construction in the framework. Validation of ABC model choice and selection of summary statistics remains an open challenge.

1. Intractable likelihoods
large and larger datasets:
beyond memory
deeply hierarchical
models with no common
topology
highly structured and
numerous latent variables
black box models and
computer experiments
MCMC, pMCMC, SMC,
&tc. not really scalable
in datasize
uncertainty assessment
and calibration
ABC, indirect inference,
empirical likelihood face
dimension curse
harder cases when
simulating data too
costly
further approximations
require theoretical
validations
borrow from Bayesian
non-parametrics
towards cyber- sufficiency

2. ABC methods
Bayesian setting: target is π(θ)f (x|θ)
When likelihood f (x|θ) not in closed form, likelihood-free rejection
technique:
ABC algorithm
For an observation y ∼ f (y|θ), under the prior π(θ), keep jointly
simulating
θ ∼ π(θ) , z ∼ f (z|θ ) ,
until the auxiliary variable z is equal to the observed value, z = y.
[Tavaré et al., 1997]

3. ABC methods
Bayesian setting: target is π(θ)f (x|θ)
When likelihood f (x|θ) not in closed form, likelihood-free rejection
technique:
ABC algorithm
For an observation y ∼ f (y|θ), under the prior π(θ), keep jointly
simulating
θ ∼ π(θ) , z ∼ f (z|θ ) ,
until the auxiliary variable z is equal to the observed value, z = y.
[Tavaré et al., 1997]

4. ABC methods
Bayesian setting: target is π(θ)f (x|θ)
When likelihood f (x|θ) not in closed form, likelihood-free rejection
technique:
ABC algorithm
For an observation y ∼ f (y|θ), under the prior π(θ), keep jointly
simulating
θ ∼ π(θ) , z ∼ f (z|θ ) ,
until the auxiliary variable z is equal to the observed value, z = y.
[Tavaré et al., 1997]

5. ABC algorithm
Algorithm 1 Likelihood-free rejection sampler
for i = 1 to N do
repeat
generate θ from the prior distribution π(·)
generate z from the likelihood f (·|θ )
until ρ{η(z), η(y)}
set θi = θ
end for
where η(y) defines a (rarely sufficient) statistic

6. connections with Econometrics
Similar exploration of simulation-based techniques in Econometrics
Simulated method of moments
Method of simulated moments
Simulated pseudo-maximum-likelihood
Indirect inference
[Gouriéroux & Monfort, 1996]

7. ABC advances
Simulating from the prior is often poor in efficiency
modify the proposal distribution on θ to increase the density of
x’s within the vicinity of y.
[Marjoram et al, 2003; Bortot et al., 2007, Beaumont et al., 2007]
borrow from non-parametrics for conditional density estimation
and allow for larger
[Blum & François, 2010; Fearnhead & Prangle, 2012]
change inferential perspective by including in inferential
framework
[Ratmann et al., 2009; Del Moral et al., 2010]
incorporate summary construction as model choice
[Joyce & Marjoram, 2008; Pudlo & Sedki, 2012]

8. ABC advances
Simulating from the prior is often poor in efficiency
modify the proposal distribution on θ to increase the density of
x’s within the vicinity of y.
[Marjoram et al, 2003; Bortot et al., 2007, Beaumont et al., 2007]
borrow from non-parametrics for conditional density estimation
and allow for larger
[Blum & François, 2010; Fearnhead & Prangle, 2012]
change inferential perspective by including in inferential
framework
[Ratmann et al., 2009; Del Moral et al., 2010]
incorporate summary construction as model choice
[Joyce & Marjoram, 2008; Pudlo & Sedki, 2012]

9. ABC jams
difficulty in reproducing tall data
sheer dimension (Google, astronomy, genetics...)
summaries that cannot be directly simulated
sequential data
costly summary construction
precision of inference
assessing the approximation itself

10. ABC jams
difficulty in reproducing tall data
costly summary construction
implicit derivations
non parametric versions (e.g., random forest)
dimension of θ and storage
precision of inference
assessing the approximation itself

11. quality of inference: A, B, or C?
ABC output is
Monte Carlo approximation to ...
convolution or completion approximation to ...
partial information posterior
π(θ|η(y))
Issues of
reliability of approximations
consistency of inference
connection with other approximations (variational Bayes,
expectation propagation, composite and empirical likelihoods,
indirect inference, subsampling MCMC)
calibration

12. quality of inference: A, B, or C?
ABC output is
Monte Carlo approximation to ...
convolution or completion approximation to ...
partial information posterior
π(θ|η(y))
Issues of
reliability of approximations
consistency of inference
connection with other approximations (variational Bayes,
expectation propagation, composite and empirical likelihoods,
indirect inference, subsampling MCMC)
calibration

13. validation of ABC model choice
Amounts to solve
When is a Bayes factor based on an insufficient statistic
η(y)
consistent?

14. validation of ABC model choice
Amounts to solve
When is a Bayes factor based on an insufficient statistic
η(y)
consistent?
Answer: asymptotic behaviour of Bayes factor driven by the
asymptotic mean value of summary statistic under both models
[Marin et al., 2015]

15. validation of ABC model choice
Amounts to solve
When is a Bayes factor based on an insufficient statistic
η(y)
consistent?
does not solve the issue of selecting the summary statistic in
realistic settings
[Pudlo et al., 2015]