Bayesian Analysis discussion paper by Lewis, Mac Eachern and Lee, 09/02/2022

1. Bayesian Restricted Likelihood Methods:
A discussion
Christian P. Robert
(Paris Dauphine PSL & Warwick U.
&Università Ca’Foscari Venezia)
Bayesian Analysis webinar, 09/02/22

2. When you cannot trust the likelihood
I Model defined by moment conditions
I Use of empirical likelihood Bayesian tools
I Use of scoring rules
I Use of ABC as robustification
I Cut models
I non-parametric component
[Bissiri et al., 2016; Jacob et al., 2018; Frazier et al., 2019]
“...the prior distribution, the loss function, and the likelihood or
sampling density (...) a healthy skepticism encourages us to
question each of them”

3. When you cannot trust the likelihood
I Model defined by moment conditions
I Use of empirical likelihood Bayesian tools
I Use of scoring rules
I Use of ABC as robustification
I Cut models
I non-parametric component
[Bissiri et al., 2016; Jacob et al., 2018; Frazier et al., 2019]
“...the prior distribution, the loss function, and the likelihood or
sampling density (...) a healthy skepticism encourages us to
question each of them”

4. Empirical likelihood
Given dataset x1, . . . , xn and moment constraints
E[g(X, θ)] = 0
empirical likelihood defined as
`emp
(θ|y) =
n
Y
i=1
p̂i
with (p̂1, . . . , p̂n) minimising
n
X
i=1
pi log(pi)
under the constraint
n
X
i=1
pig(xi, θ) = 0
[Owen, 1988]

5. Bayesian empirical likelihood
Under prior π(θ), substitute empirical likelihood to true
likelihood
πemp
(θ|y) ∝ π(θ)`emp
(θ|y)
[Lazar, 2005; Mengersen, Pudlo, Robert, 2013]

6. Not all misspecified models are created outlying
I Convenient mixture representation
I Original model relevant for part of the sample
I Is there such a thing as ‘good data’?
I Lower likelihood input through ‘safe Bayes’
[Rousseau and Robert, 2000; Kamary et al., 2014; Grünwald, 2018]
“The literature on robust methods is replete with examples
described in terms of ‘outliers’ where the central problem is
model misspecification.”

7. Not all misspecified models are created outlying
I Convenient mixture representation
I Original model relevant for part of the sample
I Is there such a thing as ‘good data’?
I Lower likelihood input through ‘safe Bayes’
[Rousseau and Robert, 2000; Kamary et al., 2014; Grünwald, 2018]
“The literature on robust methods is replete with examples
described in terms of ‘outliers’ where the central problem is
model misspecification.”

8. Insufficient statistic is the solution?
”...for a variety of conditioning statistics with non-trivial
regularity conditions on prior, model, and likelihood, the
posterior distribution resembles the asymptotic sampling
distribution of the conditioning statistic.”
I Motivations for choice of T(·) and post-choice reassessment
I Sufficient and almost sufficiency irrelevant in misspecified
settings
I Degree of robustness to misspecification?
I Workhorse of ABC
I Sufficiency may prove an hindrance in ABC model choice
[Robert et al., 2011; Fearnhead & Prangle, 2021; Frazier et al.,2018]

9. On ABC
”Acceptance rates of this [ABC] algorithm can be intolerably low
(...) especially problematic in high-dimensional settings since
generating high-dimensional statistics that are close to the
observed values is difficult.”
I Comparison of MCMC and ABC rarely relevant
I Why ”ABC with = 0 ??
I Plus, = 0 is suboptimal for ABC
I Turner and van Zandt (2014) exploits likelihood-free
hierarchical conditionals to run semi-ABC
I Clarté et al. (2020) extend ABC-Gibbs to general setup
and point out potential inconsistencies
[Turner and van Zandt, 2014; Frazier et al., 2018; Clarté et al., 2020]

10. MCMC on manifolds
”...deliberate choice of an insufficient statistic T(y) guided by
targeted inference is sound practice.”
I Simulation of y conditional on T(y) not useful for inference
I Measure-theoretic difficulties with use of density on Rp
against density on manifold as in e.g.
Z
A
f(y|θ) dy
I Exploitation of the location scale structure to the uttermost
I What is a general strategy?
I Example of Bayesian empirical likelihood
[Byrne Girolami, 2013; Florens Simoni, 2015; Bornn al., 2019]