This document discusses image denoising using total variation. It first introduces image formation models and types of noise such as Gaussian and Poisson noise. It then discusses conventional denoising methods like low pass filtering and their limitations. Total variation is introduced as a regularizer that can better preserve edges. The document formulates image denoising as an optimization problem with a data term and total variation regularization term. It describes implementing total variation denoising for Gaussian and Poisson noise using algorithms like MM, steepest descent, and conjugate gradient. Results show that total variation denoising achieves significant improvement in peak signal to noise ratio compared to conventional methods.

1. Image Denoising using Total
variation
P. M. V. D. Sai Baba (Intern)
Guide: Dr Renu M. Rameshan
Assistant professor
June 10, 2016

2. Contents
List of Figures 3
List of Tables 4
1 Introduction 7
1.1 Image formation model . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
1.2 Types of noise . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
1.3 Conventional denoising methods . . . . . . . . . . . . . . . . . . . . . . . 8
1.3.1 Denoising using Gaussian low pass filter . . . . . . . . . . . . . . 8
1.4 Total variation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
1.5 Optimization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
1.5.1 Steepest descent . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
1.5.2 Conjugate gradient . . . . . . . . . . . . . . . . . . . . . . . . . . 11
2 Literature survey 13
3 Total variation denoising for Gaussian noise 14
3.1 Total variation implementation . . . . . . . . . . . . . . . . . . . . . . . 15
3.1.1 MM Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
3.2 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
4 Total variation denoising for Poisson noise 19
4.1 Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
4.2 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
5 Conclusion 22
2

3. List of Figures
1.1 Image model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
1.2 Power spectral density for Gaussian noise . . . . . . . . . . . . . . . . . . 8
1.3 Image denoised with LPF . . . . . . . . . . . . . . . . . . . . . . . . . . 9
1.4 Total variation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
1.5 Steepest descent steps towards minimum . . . . . . . . . . . . . . . . . . 11
1.6 Steepest descent for large α . . . . . . . . . . . . . . . . . . . . . . . . . 11
1.7 Conjugate Direction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
3.1 Square root function upper bounded by tangent . . . . . . . . . . . . . . 15
3.2 Denoising using Total variation for Cameraman . . . . . . . . . . . . . . 17
3.3 Denoising using Total variation for Peppers . . . . . . . . . . . . . . . . . 18
3.4 Denoising using Total variation for Lena . . . . . . . . . . . . . . . . . . 18
4.1 Image created with intensities 5, 10, 70, 135, 200. . . . . . . . . . . . . . 20
4.2 Image corrupted by Poisson noise . . . . . . . . . . . . . . . . . . . . . . 21
3

4. List of Tables
3.1 Maximum PSNR(in dB) achieved for different images . . . . . . . . . . . 17
4.1 PSNR(in dB) for Poisson . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
4

5. Acknowledgement
I would like to express my deepest and sincere gratitude to Dr. Renu M. Rameshan,
Assistant professor, School of Computing and Electrical Engineering, IIT Mandi, for her
guidance and providing necessary facilities and ideal environment to carry out this work.
I extend my profound thanks to the ECE department, Amrita school of Engineering,
for allowing me to enhance my knowledge by working as a intern in IIT Mandi and for
supporting me all the time.
I am greatly indebted to my parents and friends for their wholehearted support and
prayers which made it possible for us to successfully complete this project. Above all I
am thankful to GOD Almighty for giving strength for completing this project.
- P. M. V. D. Sai Baba
5

6. Abstract
Practical image capturing system suffers from noise. Denoising is achieved using
different conceptual frameworks and computational tools. In this work denoising is
done in the optimisation framework with efficient regularizers. Total variation is used
as regularizer, which was first used by Rudin, Osher and Fatermi, due to its ability to
preserve strong edges. In this optimisation framework for solving denoising problem, the
data term is derived from noise distribution and total variation is used as a regularizer
for Gaussian and Poisson noise. Significant improvement in peak signal to noise ratio
(PSNR) was observed in both cases.
6

7. Chapter 1
Introduction
Noise gets introduced due to following factors: CCD/CMOS noise, quantisation,
and shot noise to list a few. In order to extract and use the original image, it is required
to denoise the image. Many good algorithms exist for natural images, BM3D being
the most prominent. The purpose of this work was to understand the image denoising
problem and to solve it in an optimisation framework.
1.1 Image formation model
The noisy image y can be represented as
Figure 1.1: Image model
where the block H is assumed to be linear shift invariant camera, n is the additive noise,
x is the original image. That is
y = Hρ(x) + n, (1.1)
where H is the convolution matrix which creates blur, ρ(x) is the image corrupted by
poisson noise and n is the additive white Gaussian noise.
In this work since we are dealing with with denoising H = I.
1.2 Types of noise
The noise is either signal dependent or independent. A brief description about some of
the noise models is given below.
7

8. • Gaussian noise: This is a additive white noise added at each pixel, irrespective
of the pixel intensity, during the image acquisition. The noise has a probability
density function of the Gaussian distribution. This is the most frequently used
model due to central limit theorem.
Figure 1.2: Power spectral density for Gaussian noise
From the above figure we can see that the effect of noise is constant for all frequen-
cies. The image spectrum and noisy spectrum have the same shape with different
SNR.
• Poisson noise or photon noise: This noise occurs when number of photons sensed
by the sensor is not sufficient to provide detectable statistical information. In prac-
tice the photon noise and other sensor based noise corrupt the signal at different
proportions.
• Salt and pepper noise or shot noise: This noise is generally caused by malfunction-
ing of camera’s sensor cells, by memory cell failure or by synchronization errors in
the image digitizing or transmission. Due to this clear black and white dots are
seen in the image.
1.3 Conventional denoising methods
Denoising is done using using both linear and non linear methods. Linear methods
include means filter, low pass filter (Gaussian filter) for image smoothing.
1.3.1 Denoising using Gaussian low pass filter
From the figure (1.3) it can be seen that after denoising the image the edges became
smooth.
From this it is clear that low pass filter doesn’t preserve the edges. This leads to
developing non linear methods like denoising in optimization framework using regular-
izers.
8

9. (a) Original image (b) Denoised image
Figure 1.3: Image denoised with LPF
1.4 Total variation
Isotropic total variation [3, 8] of an image is defined as sum of magnitude of gradient
at each pixel.
TV (x) =
MN
i=1
h
i x2 + v
i x2, (1.2)
where h
, v
are the horizantal and vertical first order differences respectively. Total
variation can also be defined as the integral of the norm of gradient.
TV (x) = | x(t)|dt (1.3)
The concept of total variation is made clear in the figure 1.4, total variation can be seen
as the distance travelled by the projective point, of the point moving along the curve,
on the y-axis. In other words, the length of path travelled by the projective point on
y-axis is the total variation of the function, corresponding to the point moving on the
curve.
9

10. Figure 1.4: Total variation
Image courtesy: wikipedia.com
1.5 Optimization
Posing denoising as an optimization problem, gives rise to a cost function of the
following form,
C(X) = D(x) + λR(x), (1.4)
where D(x) is the data term determined by the likelihood of y of the equation (1.1) and
R(x) is the regularizer and λ is the regularization factor. If λ is large then R(x) decides
the solution. The cost function (1.4) is usually a non linear. We have used unconstrained
optimization. The algorithms used are conjugate gradient and steepest descent.
1.5.1 Steepest descent
Steepest descent is an iterative method, where one starts with an initial vale and
updates the value by moving in a perpendicular direction opposite to the gradient. If
the function to be minimized is of the form
f(x) =
1
2
xT
Ax − xT
b
then for the minimum value we have to solve the gradient, f = 0, for minimum value
of x i.e., ˆx , which gives
Ax = b
and the solution of x gives the minimum or maximum value. The Hessian of f(x) is
2
= A. If A is a positive definite then gradient direction is towards the minimum.
10

11. Figure 1.5: Steepest descent steps towards minimum
Image courtesy: trond.hjorteland.com
Now here we have to assume initial solution x0 i.e., start as shown in figure 1.5, and
achieves the minimum value by converging towards it in the directions of the gradients
in each step. So the iterative equation for updating x is
xk+1 = xk + αkpk,
and in each iteration f(xk+1) < f(xk) and pk is the search direction with αk as the step
length in that direction. As our direction is negative gradient direction pk = − (fk)
and residual rk = (fk). Here the choosing the α is the very important.
Figure 1.6: Steepest descent for large α
Image courtesy: ats.cs.ut.ee
As shown in the figure 1.6 If the α is very high or very low it will take long time as
oscillates at the minimum value. Optimal α can be found out using
αk =
rT
k rk
rT
k Ark
.
1.5.2 Conjugate gradient
In this method we move in the conjugate direction[7] earlier where we moved
in the direction of the gradient. A-conjugacy means that a set of nonzero vectors
11

12. {p0, p1, ...., pn−1} are conjugate with respect to the matrix A. Now we represent
x − x0 = α0p0 + α1p1 + ..... + αn−1pn−1,
where x is the exact solution. But we don’t know the the conjugate vectors initially. We
will get the conjugate directions similar to Gram-Schmidt orthogonalization procedure.
The new direction will be orthogonal to the previous direction as shown in figure 1.7.
That is
pk = −rk + βpk−1,
and residual rk is similar as we discussed earlier. Where,
βk =
rT
k rk
rT
k−1rk−1
,
Of both steepest descent and conjugate gradient, conjugate gradient is the preferable
because the iterations taken by conjugate gradient is less compared with steepest descent.
Figure 1.7: Conjugate Direction.
Image courtesy: wikipedia
12

13. Chapter 2
Literature survey
The intension of this work is to study about the denoising problem. In this work,
we got familiarized with the optmization framework for denoising images using total
variation as regularizer, first used by Rudin, Osher and Fatemi[8]. Learnt about one
of the optimization algorithm named majorization minimisation ( MM algorithm) [3].
We used BM3D’s [2], which is efficent denoising algorithms, results as reference. In this
work, we minimisation is achived using conjugate gradient and steepest descent methods
[7].
13

14. Chapter 3
Total variation denoising for
Gaussian noise
From the equation (1.1) the observed image y can be represented as
y = x + n, (3.1)
where x is the original image, n is the additive white Gaussian noise. According to
Bayesian theorem we know that,
P(x/y) =
P(y/x) ∗ P(x)
P(y)
, (3.2)
where P(x) is the probability distribution of x. P(x/y) is called the posterior, P(y/x)
is the likelihood and P(x) is the prior distribution. In optimization P(y) doesn’t play a
role. At ith
pixel the value yi = xi + ni. Given x, y has the statistics of the noise n and
has mean x and variance σ2
. yi are uncorrelated which means they are independent,
since yi is Gaussian.
P(yi/xi) = exp
−(xi−yi)2
2σ2
. (3.3)
Let the size of the image be M ∗ N, where M, N are number of rows and columns
respectively. Then
P(y/x) ∝
MN
i=1
P(yi/xi), (3.4)
i varies from 1 to MN. Substituting equation (3.3) in (3.4) we get
P(y/x) ∝ e
−
MN
i=1
(yi−xi)2
2σ2
. (3.5)
On applying negative ln on both sides,
− ln(P(y/x)) ∝
||y − x||2
2
2σ2
− ln(P(x)), (3.6)
where
||y−x||2
2
2σ2 is similar to D(x) and − ln(P(x)) is similar to R(x) in (1.4). x can be
estimated by maximizing the posterior, which implies that − ln(P(y/x)) should be min-
imized. First derivative, is used to find the maximum or minimum value. Second
14

15. derivative decided whether it is maximum or minimum. First derivative is calculated
using total variation, which is described in [8].
3.1 Total variation implementation
The resulting denoising criteria is to get optimal ˆx using minimisation algorithm [1, 6].
The resulting denoising criteria [3] is
ˆx = arg min
x
{||x − y||2
+ λ TV (x)}, (3.7)
where λ is the weight of the prior or regularizer. Majorization minimisation(MM) [4]
algorithm was used for solving equation 3.7.
3.1.1 MM Algorithm
Let L be the some function to be minimised. In our case we have to minimise
equation (3.7). Let us rewrite as
x(t+1)
= arg minxQ(x, ˆx(t)
). (3.8)
In this algorithm, first the TV penalty is majorized. We know TV is a concave
in nature, because of square root function, thus TV is always upper bounded by its
tangents, as shown in the figure (3.1).
Figure 3.1: Square root function upper bounded by tangent
From the figure(3.1),
√
a ≤
√
a +
a − a
2
√
a
. (3.9)
15

16. Using this tangent as the upper bound, the majorizer for TV
QTV (x, x ) = TV (x ) +
λ
2 i
( h
i x)2
− ( h
i x )2
( h
i x )2 + ( v
i x )2
+
( v
i x)2
− ( v
i x )2
( h
i x )2 + ( v
i x )2
. (3.10)
Finally, notice that the terms h
i x and v
i x in the numerators are simply additive
constants which can be ignored as they do not affect the resulting MM algorithm[3].
Let Dh
and Dv
denote matrices such that Dh
x and Dv
x are the vectors of all horizontal
and vertical (respectively) first order differences. Define also the vector w(t) whose i-th
element is
wi(t) = λ(2 ( h
i x(t))2 + ( v
i x(t))2)−1
, (3.11)
the diagonal matrix
Λ(t)
= diag(w(t)
, w(t)
), (3.12)
and the matrix D = [(Dh
)T
(Dv
)T
]T
. With these notations, QTV (x, x(t)
) can be written
as a quadratic form
QTV (x, x(t)
) = xT
DT
Λ(t)
Dx + K(x(t)
), (3.13)
where K(x(t)
) is a constant, which doesn’t effect the optimisation. Adding QTV (x, x(t)
)
to the data term ||x − y||2
, we obtain
Q(x, x(t)
) = XT
(DT
Λ(t)
D + I)x − 2xT
y. (3.14)
Since this is a quadratic function, minimization w.r.t x leads to
ˆx(t+1)
= solutionx{(DT
Λ(t)
D + I)x = y}. (3.15)
Equation (3.11) can go to infinity, we need numerically stable way to handle this
matrix. We sidestep this difficulty by invoking the well known matrix inversion lemma,
DT
Λ(t)
D + I = [I − DT
(DDT
+ (Λ(t)
)−1
]−1
, (3.16)
which leads to
ˆx(t+1)
= y − DT
z(t)
, (3.17)
z(t)
= solutionz{[DDT
+ (Λ(t)
)−1
]z = Dy, (3.18)
where t = 1, 2, ..... In summary, it is two step algorithm. Starting with initial estimate
x(1)
, iteratively compute x(t+1)
using the equation (3.17). Solution for z in equation
(3.18) is found using the CG algorithm discussed in section 1.5.2.
The above algorithm from [3] is implemented and results are verified.
16

17. 3.2 Results
σ C.man C.man Peppers Peppers Lena Lena
. noisy denoisy noisy denoisy noisy denoisy
5 34.11 34.81 34.18 34.87 34.12 36.53
10 28.10 30.15 28.12 30.30 28.12 32.71
20 22.08 26.31 22.11 26.47 22.17 28.31
50 14.13 20.34 14.13 20.46 14.14 20.96
100 8.14 14.84 8.12 14.98 8.11 15.02
Table 3.1: Maximum PSNR(in dB) achieved for different images
(a) Noisy image with σ = 15 (b) Denoised image
Figure 3.2: Denoising using Total variation for Cameraman
17

18. (a) Noisy image with σ = 15 (b) Denoised image
Figure 3.3: Denoising using Total variation for Peppers
(a) Noisy image with σ = 15 (b) Denoised image
Figure 3.4: Denoising using Total variation for Lena
18

19. Chapter 4
Total variation denoising for Poisson
noise
Unlike Gaussian noise Poisson noise is intensity dependent. Poisson distribution
with mean and standard deviation µ
Pµ(n) =
e−µ
µn
n!
. (4.1)
We wish to determine the image u that is most likely given the observed image f. Using
Baye’s Law (3.2) it can be written as
P(u|f) =
P(f|u)P(u)
P(f)
. (4.2)
Thus, we wish to maximize P(f|u)P(u). We have
P(f(x)|u) = Pu(x)(f(x)) =
e−u(x)
u(x)f(x)
f(x)!
. (4.3)
It is known that the values of f at the pixels xi are independent. Then
P(f|u) =
i
e−u(xi)
u(xi)f(xi)
f(xi)!
. (4.4)
The total-variation is used as the regularizer. So the probability of the orignial image
u is written as
P(u) = exp(−β | u|), (4.5)
where β is a regularization paramter. Instead of maximizing P(f|u)P(u), we minimize
−log(P(f|u)P(u)). The result is that we seek a minimizer of
i
(u(xi) − f(xi)logu(xi)) + β | u| (4.6)
We have to minimise equation (4.6). The Euler-Lagrange equation for minimizing it
is
0 = div(
u
| u|
) +
1
βu
(f − u) (4.7)
19

20. The equation (4.7) is the differential equation which is
u(t+1)
− u(t)
δt
= div(
u(t)
| u(t)|
) +
1
βu(t)
(f − u(t)
) (4.8)
From the above equation u(t+1)
is computed using steepest descent which we discussed
in 1.5.1. The above mentioned algorithm is from [5].
4.1 Implementation
The differential equation (4.7) is implemented using the steepest descent as discussed in
the above section. Gradient is computed as we discussed earlier in the section 3.1 i.e.,
directional gradient. Then the divergence of the unit gradient is computed and used as
a regularizer. The simplified form of the divergence of unit gradient is
div(
u
| u|
) =
uxx(u2
y) + uyy(u2
x) − 2uxuyuxy
(u2
x + u2
y)
3
2
(4.9)
where uxx, uyy, uxy are the double partial derivatives and ux, uy are the partial derivatives.
Here the parameters β determines the regularization weightage.
As mentioned in [5] image of intensities 5, 10, 70, 135 and 200 are taken varying
from boundary to the center of the image respectively. The above discussed denoising
procedure is done and its results are mentioned below.
Figure 4.1: Image created with intensities 5, 10, 70, 135, 200.
20

21. 4.2 Results
(a) Noisy image (b) Output image
Figure 4.2: Image corrupted by Poisson noise
Intensity Noisy PSNR Obtained PSNR
5 41.1672 32.0741
10 29.4033 32.2218
70 27.8687 31.1377
135 26.3324 31.2620
200 25.0957 31.0055
Table 4.1: PSNR(in dB) for Poisson
21

22. Chapter 5
Conclusion
Significant denoising is obtained for both Gaussian and Poisson denoising in this
work. We notice that regularization factor i.e., λ for Gaussian depends on noise standard
deviation. While for Poisson depending on the intensity the regularization factor has be
choosen. Using optimization based techniques gives better results than the conventional
filtering.
22

23. Reference
[1] A. Chambolle and P.-L. Lions. Image recovery via total variation minimization and
related problems. Numerische Mathematik, 76(2):167–188, 1997.
[2] K. Dabov, A. Foi, V. Katkovnik, and K. Egiazarian. Image denoising by sparse 3-d
transform-domain collaborative filtering. Image Processing, IEEE Transactions on,
16(8):2080–2095, 2007.
[3] M. A. Figueiredo, J. B. Dias, J. P. Oliveira, and R. D. Nowak. On total varia-
tion denoising: A new majorization-minimization algorithm and an experimental
comparisonwith wavalet denoising. In Image Processing, 2006 IEEE International
Conference on, pages 2633–2636. IEEE, 2006.
[4] D. R. Hunter and K. Lange. A tutorial on mm algorithms. The American Statistician,
58(1):30–37, 2004.
[5] T. Le, R. Chartrand, and T. J. Asaki. A variational approach to reconstructing
images corrupted by poisson noise. Journal of mathematical imaging and vision,
27(3):257–263, 2007.
[6] S. Osher, A. Sol´e, and L. Vese. Image decomposition and restoration using total
variation minimization and the h 1. Multiscale Modeling & Simulation, 1(3):349–
370, 2003.
[7] R. H. Refsns. A brief introduction to the conjugate gradient method. Technical
report, NTNU Fall 2009.
[8] L. I. Rudin, S. Osher, and E. Fatemi. Nonlinear total variation based noise removal
algorithms. Physica D: Nonlinear Phenomena, 60(1):259–268, 1992.
23