Bayesian restoration of high-dimensional photon-starved images

Bayesian restoration of high-dimensional
photon-starved images
J. Tachella1, Y. Altmann1, M. Pereyra1, J.-Y. Tourneret2
and S. McLaughlin1
1Heriot-Watt University, Edinburgh, UK
2INP-ENSEEHIT-IRIT-TéSA, Toulouse, France
European Signal Processing Conference, Rome
September 4, 2018
Julián Tachella (HWU-TéSA) Restoration of photon-starved images September 4, 2018 1 / 29

Outline
In this work, we compare 6 state-of-the-art Monte Carlo samplers in the
context of Poisson image restoration.
A Bayesian model for restoration of images with Poisson noise.
Appropriate posterior statistics.
We evaluate the performance of the samplers in a series of
experiments, varying the image size and the total number of collected
photons.

Photon-limited imaging
Photon-limited imaging arises in many applications where the light
ﬂux changes rapidly or is extremely limited.
The measurements generally follow Poisson statistics.
Examples:
Lidar [1]
Medical imaging [2]
Astronomy [3]
1 Altmann et al. (2016) Lidar waveform-based analysis of depth images constructed using sparse single-photon data, IEEE
Trans. on Image Proc., 25(5), 1935-1946
2 Willett et al. (2003) Platelets: a multiscale approach for recovering edges and surfaces in photon-limited medical
imaging, IEEE Trans. on Medical Imag., 22(3), 332-350.
3 Zhang et al. (2008) Wavelets, ridgelets, and curvelets for Poisson noise removal, IEEE Trans. on Image Proc., 17(7),
1093-1108.

Observational model
In this work, we study the following inverse problem:
y|z ∼ P(Az)
K discrete measurements: y ∈ ZK
+
N-pixel image to recover: z ∈ RN
+
Linear operator A ∈ RK×N
+ , generally rank-deﬁcient

A low-photon toy example (104
photons)
In this example, A is a 2 × 2 blur

Bayesian approach
In the Bayesian paradigm, we are interested in the posterior distribution
of the unknown image z given the measurements y, i.e.,
p(z|y, λ) ∝ p(y|z)p(z|λ), where
p(y|z) is the likelihood (or data ﬁdelity term)
p(z|λ) is the prior distribution, which contains the a priori
assumptions on the image z (sparsity under some basis, smoothness,
etc).

Bayesian model
Hierarchical formulation:
y|z ∼ P(Az)
z|λ ∼ LN(0, (λP)−1
)
The log-normal Gaussian Markov random ﬁeld prior with P = D D,
where D is the 2D laplacian ﬁlter promotes smooth and positive
images z.

Posterior distribution
Following Bayes theorem, the posterior distribution is
p(z|y, λ) =
p(y|z)p(z|λ)
p(y|z)p(z|λ)dz
∝ exp(−Uy (z))
where the negative log-posterior density (or energy) is
Uy (z) = 1K Az − y log(Az) +
λ
2
log(z) P log(z) + 1N log(z)

Inference
The convex optimization approach ﬁnds the MAP estimate
ˆzMAP = arg max
z
p(z|y, λ) = arg min
z
Uy (z). (1)
However, when we have very few photons...
0 1 2 3 4 5

Inference
To alleviate these problems, we consider other statistics of p(z|y, λ).
In particular, we compute numerical estimates of:
The minimum mean squared error estimator (MMSE) of zn [4]
E{zn|y, λ} =
+∞
0
znp(zn|y, λ)dzn
The marginal posterior variance
var{zn|y, λ} =
+∞
0
(zn − E{zn|y, λ})2
p(zn|y, λ)dzn.
4 Robert, C. (2007) The Bayesian choice: from decision-theoretic foundations to
computational implementation, Springer Science

Inference
How do we compute these posterior statistics?

Markov Chain Monte Carlo sampling
MCMC samplers construct a Markov chain of NMC samples
{x(s)|s = 1, . . . , NMC} that provide full insight of p(z|y, λ).
We compute the numerical estimates as
E{zn|y, λ} ≈
1
NMC
NMC
s=1
z(s)
n (2)
and
var{zn|y, λ} ≈
1
NMC
NMC
s=1
z(s)
n
2
− ˆµ2
n. (3)

Markov chain Monte Carlo sampling
We cannot obtain independent samples, as we are not using a
conjugate prior.
Many samplers are available in the statistics literature to overcome
this problem, but their behaviour is not well studied for
photon-limited imaging problems:
The dimension is very high
The number of data samples K is generally lower than the parameter
dimension N.
The likelihood term has longer tails as the photon-count reduces
(higher uncertainty).

Markov chain Monte Carlo sampling
In this work, we compare 6 state-of-the-art Monte Carlo samplers
Random walk Metropolis (RWM) [5]
Unadjusted Langevin algorithm (ULA) [5]
Metropolis adjusted Langevin algorithm (MALA) [5]
Hamiltonian Monte Carlo (HMC) [5]
No U-turn HMC (NUTS) [6]
Bouncy particle sampler (BPS) [7]
5 Brooks et al. (2011) Handbook of Markov chain Monte Carlo, CRC press.
6 Hoffman and Gelman (2014) The No-U-turn sampler: adaptively setting path lengths in
Hamiltonian Monte Carlo, Journal of Machine Learning Research, 15(1), 1593-1623.
7 Bouchard-Côté et al. (2018) The bouncy particle sampler: A nonreversible rejection-free
Markov chain Monte Carlo method, Journal of the American Statistical Association, 1-13.

Experiments
We evaluate the stochastic estimators as the dimension N of the
problem increases and the number of photons ( K
k=1 yk)
decreases
We assessed the bias and variance of the estimators as a function
of computing time.
Each algorithm was ran 10 times using diﬀerent random seeds.
The reference estimate was computed using a long chain.
The parameters of all samplers were adjusted adaptively to obtain the
best performance.

Posterior mean estimates vs image size
10 60 110 160 210 260
3
4
5
64x64
10 60 110 160 210 260
Computing time [sec]
0
0.05
0.1
0.15
0.2
256x256
RWM ULA MALA HMC NUTS BPS

Marginal posterior variance estimates vs image size
10 60 110 160 210 260
0
1
2
3
64x64
10 60 110 160 210 260
0
5
10
256x256
10
-3

Marginal posterior variance estimates vs total photons
10 70 130 190 250
0
10
20
30
40
100k
photons
10 70 130 190 250
0
0.01
0.02
1k
photons

Conclusions
HMC obtains the best scaling properties.
NUTS fails to provide good estimates in very high dimensions.
BPS achieves good mean estimates but shows slow convergence in
variance estimates

Conclusions
These results are in agreement with the theoretical scaling of the
eﬀective sample size (ESS):
Method
O(N) per independent sample
MC iterations Total computations1
RWM N1 N2 log N
ULA N0.33 N1.33 log N
MALA N0.33 N1.33 log N
HMC N0.25 N1.25 log N
NUTS N0.25 N1.25 log N
BPS N0.47 N1.47 log N
The scaling of the MCMC samplers is still worse than an optimization
alternative, which would scale as N log N.
1
Obtaining one sample involves order N log N computations in all methods

Thanks for your attention!
Contact: jat3@hw.ac.uk

Random Walk Metropolis (RWM)
A proposal is generated by
x∗
∼ N(x(s−1)
, δI)
x∗ is accepted with probability
ρ = min{1, exp Uy (x(s−1)
) − Uy (x∗
) }
The value of δ is adapted to achieve the optimal acceptance ratio of
23%.

Unadjusted Langevin algorithm (ULA)
A new sample is obtained using gradient information
x(s)
∼ N(x(s−1)
−
δ
2
Uy (x(s−1)
), δI)
As no acceptance rule is used, the samples converge to a distribution
that is close to p(x|y, λ)

Metropolis adjusted Langevin algorithm (MALA)
Similarly to ULA, a proposal is generated by
x∗
∼ N x(s−1)
−
δ
2
Uy (x(s−1)
), δI
To converge to the exact posterior, x∗ is accepted with probability
ρ = min{1, exp(α)}
α = Uy (x(s−1)
) − Uy (x∗
) +
1
2δ
||x∗
− x(s−1)
+
δ
2
Uy (x(s−1)
)||2
2
−
1
2δ
||x(s−1)
− x∗
+
δ
2
Uy (x∗
)||2
2
The value of δ is adapted to achieve the optimal acceptance ratio of
57%.

Hamiltonian Monte Carlo (HMC)
We augment the target distribution
log p(x, p|y, λ) ∝ −Uy (x) −
1
2
||p||2
2
Sampling (x∗, p∗) ∼ p(x, p|y, λ) is done by simulating Hamiltonian
dynamics starting from (x(s−1), p0) and using a leap-frog integrator
of L steps of size .
As the leap-frog integrator is an inexact approximation, we accept x
with probability
ρ = min{1, exp Uy (x(s−1)
) − Uy (x∗
) −
1
2
||p∗
||2
2 +
1
2
||p0
||2
2}

No U-turn HMC (NUTS)
Adjusting L steps of size in HMC is generally cumbersome and
requires many trial runs.
The no U-turn sampler [6] automatically adapts the values of L and ,
such that the optimal acceptance rate of ≈ 65% is attained.
6 Hoﬀman and Gelman (2014) The No-U-turn sampler: adaptively setting path lengths in
Hamiltonian Monte Carlo, Journal of Machine Learning Research, 15(1), 1593-1623.

Bouncy particle sampler (BPS)
Recently, a ’rejection-free’ MCMC algorithm was introduced in [7].
The algorithm simulates a particle x that bounces around the
posterior with constant velocity v.
Each bouncing time τ(s) is obtained by sampling V ∼ U(0, 1) and
solving the equations:
Uy (x(s)
+ v(s)
τ(s)
) − Uy (x(s)
+ v(s)
τ∗
) = − log V
τ∗
= arg min
t≥0
Uy (x(s)
+ v(s)
t)
that are 1D and can be efficiently solved using Newton’s method!
7 Bouchard-Côté et al. (2018) The bouncy particle sampler: A nonreversible
rejection-free Markov chain Monte Carlo method, Journal of the American
Statistical Association, 1-13.

Bouncy particle sampler (BPS)
As the particle trajectory is ergodic, we compute estimators as
time-averages:
ˆµn =
NB
s=1
1
v
(s)
n
exp x(s)
n + v(s)
n τ(s)
ˆσ2
n =
NB
s=1
1
2v
(s)
n
exp 2x(s)
n + 2v(s)
n τ(s)
− ˆµ2
n

Bayesian restoration of high-dimensional photon-starved images

Recommended

Recommended

More Related Content

What's hot

What's hot (15)

Similar to Bayesian restoration of high-dimensional photon-starved images

Similar to Bayesian restoration of high-dimensional photon-starved images (20)

More from Julián Tachella

More from Julián Tachella (6)

Recently uploaded

Recently uploaded (20)

Bayesian restoration of high-dimensional photon-starved images