This document discusses Bayesian restricted likelihood methods for situations where the likelihood cannot be fully trusted. It presents several approaches including empirical likelihood, Bayesian empirical likelihood, using insufficient statistics, approximate Bayesian computation (ABC), and MCMC on manifolds. The key ideas are developing Bayesian tools that are robust to model misspecification by questioning the likelihood, prior, and other assumptions.

1. Bayesian Restricted Likelihood Methods:
A discussion
Christian P. Robert
(Paris Dauphine PSL & Warwick U.
&Università Ca’Foscari Venezia)
Bayesian Analysis webinar, 09/02/22

2. When you cannot trust the likelihood
I Model defined by moment conditions
I Use of empirical likelihood Bayesian tools
I Use of scoring rules
I Use of ABC as robustification
I Cut models
I non-parametric component
[Bissiri et al., 2016; Jacob et al., 2018; Frazier et al., 2019]
“...the prior distribution, the loss function, and the likelihood or
sampling density (...) a healthy skepticism encourages us to
question each of them”

3. When you cannot trust the likelihood
I Model defined by moment conditions
I Use of empirical likelihood Bayesian tools
I Use of scoring rules
I Use of ABC as robustification
I Cut models
I non-parametric component
[Bissiri et al., 2016; Jacob et al., 2018; Frazier et al., 2019]
“...the prior distribution, the loss function, and the likelihood or
sampling density (...) a healthy skepticism encourages us to
question each of them”

4. Empirical likelihood
Given dataset x1, . . . , xn and moment constraints
E[g(X, θ)] = 0
empirical likelihood defined as
`emp
(θ|y) =
n
Y
i=1
p̂i
with (p̂1, . . . , p̂n) minimising
n
X
i=1
pi log(pi)
under the constraint
n
X
i=1
pig(xi, θ) = 0
[Owen, 1988]

5. Bayesian empirical likelihood
Under prior π(θ), substitute empirical likelihood to true
likelihood
πemp
(θ|y) ∝ π(θ)`emp
(θ|y)
[Lazar, 2005; Mengersen, Pudlo, Robert, 2013]

6. Not all misspecified models are created outlying
I Convenient mixture representation
I Original model relevant for part of the sample
I Is there such a thing as ‘good data’?
I Lower likelihood input through ‘safe Bayes’
[Rousseau and Robert, 2000; Kamary et al., 2014; Grünwald, 2018]
“The literature on robust methods is replete with examples
described in terms of ‘outliers’ where the central problem is
model misspecification.”

7. Not all misspecified models are created outlying
I Convenient mixture representation
I Original model relevant for part of the sample
I Is there such a thing as ‘good data’?
I Lower likelihood input through ‘safe Bayes’
[Rousseau and Robert, 2000; Kamary et al., 2014; Grünwald, 2018]
“The literature on robust methods is replete with examples
described in terms of ‘outliers’ where the central problem is
model misspecification.”

8. Insufficient statistic is the solution?
”...for a variety of conditioning statistics with non-trivial
regularity conditions on prior, model, and likelihood, the
posterior distribution resembles the asymptotic sampling
distribution of the conditioning statistic.”
I Motivations for choice of T(·) and post-choice reassessment
I Sufficient and almost sufficiency irrelevant in misspecified
settings
I Degree of robustness to misspecification?
I Workhorse of ABC
I Sufficiency may prove an hindrance in ABC model choice
[Robert et al., 2011; Fearnhead & Prangle, 2021; Frazier et al.,2018]

9. On ABC
”Acceptance rates of this [ABC] algorithm can be intolerably low
(...) especially problematic in high-dimensional settings since
generating high-dimensional statistics that are close to the
observed values is difficult.”
I Comparison of MCMC and ABC rarely relevant
I Why ”ABC with = 0 ??
I Plus, = 0 is suboptimal for ABC
I Turner and van Zandt (2014) exploits likelihood-free
hierarchical conditionals to run semi-ABC
I Clarté et al. (2020) extend ABC-Gibbs to general setup
and point out potential inconsistencies
[Turner and van Zandt, 2014; Frazier et al., 2018; Clarté et al., 2020]

10. MCMC on manifolds
”...deliberate choice of an insufficient statistic T(y) guided by
targeted inference is sound practice.”
I Simulation of y conditional on T(y) not useful for inference
I Measure-theoretic difficulties with use of density on Rp
against density on manifold as in e.g.
Z
A
f(y|θ) dy
I Exploitation of the location scale structure to the uttermost
I What is a general strategy?
I Example of Bayesian empirical likelihood
[Byrne Girolami, 2013; Florens Simoni, 2015; Bornn al., 2019]