Image processing , dsp, dip, matlab, DFT, FFT, FOURIER SERIES , FILTER DESIGN , Low pass, high Pass, Gaussian ,

1. Limitation of Imaging
Technology
 Two plagues in image acquisition
 Noise interference
 Blur (motion, out-of-focus, hazy weather)
 Difficult to obtain high-quality images as
imaging goes
 Beyond visible spectrum
 Micro-scale (microscopic imaging)
 Macro-scale (astronomical imaging)

2. What is Noise?
 Wiki definition: noise means any
unwanted signal
 One person’s signal is another one’s
noise
 Noise is not always random and
randomness is an artificial term
 Noise is not always bad (see stochastic
resonance example in the next slide)

3. Stochastic Resonance
no noise heavy noise
light noise

4. Image Denoising
 Where does noise come from?
 Sensor (e.g., thermal or electrical
interference)
 Environmental conditions (rain, snow etc.)
 Why do we want to denoise?
 Visually unpleasant
 Bad for compression
 Bad for analysis

5. electrical interference
Noisy Image Examples
thermal imaging
ultrasound imaging
physical interference

6. (Ad-hoc) Noise Modeling
 Simplified assumptions
 Noise is independent of signal
 Noise types
 Independent of spatial location
 Impulse noise
 Additive white Gaussian noise
 Spatially dependent
 Periodic noise

7. Noise Removal Techniques
 Linear filtering
 Nonlinear filtering
Recall
Linear system

8. Image Denoising
 Introduction
 Impulse noise removal
 Median filtering
 Additive white Gaussian noise removal
 2D convolution and DFT
 Periodic noise removal
 Band-rejection and Notch filter

9. Impulse Noise (salt-pepper
Noise)
Definition
Each pixel in an image has the probability of p/2 (0<p<1) being
contaminated by either a white dot (salt) or a black dot (pepper)
with probability of p/2
with probability of p/2
with probability of 1-p
noisy pixels
clean pixels
X: noise-free image, Y: noisy image
Note: in some applications, noisy pixels are not simply black or white,
which makes the impulse noise removal problem more difficult
WjHi
jiX
jiY
≤≤≤≤





=
1,1
),(
0
255
),(

10. Numerical Example
P=0.1
128 128 128 128 128 128 128 128 128 128
128 128 128 128 128 128 128 128 128 128
128 128 128 128 128 128 128 128 128 128
128 128 128 128 128 128 128 128 128 128
128 128 128 128 128 128 128 128 128 128
128 128 128 128 128 128 128 128 128 128
128 128 128 128 128 128 128 128 128 128
128 128 128 128 128 128 128 128 128 128
128 128 128 128 128 128 128 128 128 128
128 128 128 128 128 128 128 128 128 128
X
128 128 255 0 128 128 128 128 128 128
128 128 128 128 0 128 128 128 128 0
128 128 128 128 128 128 128 128 128 128
128 128 0 128 128 128 128 128 128 128
128 128 128 128 128 128 128 128 128 128
128 128 128 128 128 128 128 128 128 128
128 128 128 128 128 128 128 128 128 128
0 128 128 128 128 255 128 128 128 128
128 128 128 128 128 128 128 128 128 255
128 128 128 128 128 128 128 255 128 128
Y
Noise level p=0.1 means that approximately 10% of pixels are contaminated by
salt or pepper noise (highlighted by red color)

11. MATLAB Command
>Y = IMNOISE(X,'salt & pepper',p)
Notes:
• The intensity of input images is assumed to be normalized to [0,1].
If X is double, you need to do normalization first, i.e., X=X/255;
If X is uint8, MATLAB would do the normalization automatically
• The default value of p is 0.05 (i.e., 5 percent of pixels are contaminated)
• imnoise function can produce other types of noise as well (you need to
change the noise type ‘salt & pepper’)

12. Impulse Noise Removal Problem
Noisy image Y
filtering
algorithm
Can we make the denoised image X as close
to the noise-free image X as possible?
^
X^denoised
image

13. Median Operator
 Given a sequence of numbers {y1,
…,yN}
 Mean: average of N numbers
 Min: minimum of N numbers
 Max: maximum of N numbers
 Median: half-way of N numbersExample
sorted
]56,55,54,255,52,0,50[=y

54)( =ymedian

]255,56,55,54,52,50,0[=y


14. y(n)
W=2T+1
1D Median Filtering
… …
Note: median operator is nonlinear
MATLAB command: x=median(y(n-T:n+T));
)](),...,(),...,([)(ˆ TnynyTnymediannx +−=

15. Numerical Example
T=1:
Boundary
Padding
]56,55,54,255,52,0,50[=y

],56,55,54,255,52,0,50,[ 5650=y

]55,55,54,54,52,50,50[ˆ =x


16. x(m,n)
W: (2T+1)-by-(2T+1) window
2D Median Filtering
MATLAB command: x=medfilt2(y,[2*T+1,2*T+1]);
)],(),...,,(),...,,(
),...,,(),...,,([),(ˆ
TnTmyTnTmynmy
TnTmyTnTmymediannmx
++−+
+−−−=

17. Numerical Example
225 225 225 226 226 226 226 226
225 225 255 226 226 226 225 226
226 226 225 226 0 226 226 255
255 226 225 0 226 226 226 226
225 255 0 225 226 226 226 255
255 225 224 226 226 0 225 226
226 225 225 226 255 226 226 228
226 226 225 226 226 226 226 226
0 225 225 226 226 226 226 226
225 225 226 226 226 226 226 226
225 226 226 226 226 226 226 226
226 226 225 225 226 226 226 226
225 225 225 225 226 226 226 226
225 225 225 226 226 226 226 226
225 225 225 226 226 226 226 226
226 226 226 226 226 226 226 226
Y X
^
Sorted: [0, 0, 0, 225, 225, 225, 226, 226, 226]

18. Image Example
P=0.1
Noisy image Y X^denoised
image
3-by-3 window

19. Image Example (Con’t)
3-by-3 window 5-by-5 window
clean
noisy
(p=0.2)

20. Reflections
 What is good about median operation?
 Since we know impulse noise appears as
black (minimum) or white (maximum) dots,
taking median effectively suppresses the
noise
 What is bad about median operation?
 It affects clean pixels as well
 Noticeable edge blurring after median
filtering

21. Idea of Improving Median Filtering
 Can we get rid of impulse noise without
affecting clean pixels?
 Yes, if we know where the clean pixels are
or equivalently where the noisy pixels are
 How to detect noisy pixels?
 They are black or white dots

22. Median Filtering with Noise Detection
Noisy image Y
x=medfilt2(y,[2*T+1,2*T+1]);
Median filtering
Noise detection
C=(y==0)|(y==255);
xx=c.*x+(1-c).*y;
Obtain filtering results

23. Image Example
clean
noisy
(p=0.2)
w/o
noise
detection
with
noise
detection

24. Image Denoising
 Introduction
 Impulse noise removal
 Median filtering
 Additive white Gaussian noise removal
 2D convolution and DFT
 Periodic noise removal
 Band-rejection and Notch filter

25. Additive White Gaussian Noise
Definition
Each pixel in an image is disturbed by a Gaussian random variable
With zero mean and variance σ2
X: noise-free image, Y: noisy image
Note: unlike impulse noise situation, every pixel in the image contaminated
by AWGN is noisy
WjHiNjiN
jiNjiXjiY
≤≤≤≤
+=
1,1),,0(~),(
),,(),(),(
2
σ

26. Numerical Example
σ2 =1
X Y
128 128 128 128 128 128 128 128
128 128 128 128 128 128 128 128
128 128 128 128 128 128 128 128
128 128 128 128 128 128 128 128
128 128 128 128 128 128 128 128
128 128 128 128 128 128 128 128
128 128 128 128 128 128 128 128
128 128 128 128 128 128 128 128
128 128 129 127 129 126 126 128
126 128 128 129 129 128 128 127
128 128 128 129 129 127 127 128
128 129 127 126 129 129 129 128
127 127 128 127 129 127 129 128
129 130 127 129 127 129 130 128
129 128 129 128 128 128 129 129
128 128 130 129 128 127 127 126

27. MATLAB Command
>Y = IMNOISE(X,’gaussian',m,v)
>Y = X+m+randn(size(X))*v;
or
Note: rand() generates random numbers uniformly distributed over [0,1]
randn() generates random numbers observing Gaussian distribution
N(0,1)

28. Image Denoising
Noisy image Y
filtering
algorithm
Question: Why not use median filtering?
Hint: the noise type has changed.
X^denoised
image

29. f(n) h(n) g(n)
- Linearity
- Time-invariant property
Linear convolution
1D Linear Filtering
See review section
)()()()()()()( nhnfnfnhknfkhng
k
⊗=⊗=−= ∑
∞
−∞=
)()()()( 22112211 nganganfanfa +→+
)()( 00 nngnnf −→−

30. forward
inverse
Note that the input signal is a discrete sequence
while its FT is a continuous function
time-domain convolution frequency-domain multiplication
Fourier Series
∑
∞
∞−
−
= jwn
enfwF )()(
∫−
=
π
ππ
dwewFnf jwn
)(
2
1
)(
)()( nhnf ⊗ )()( wHwF

31. Filter Examples
0 0.5 1 1.5 2 2.5 3 3.5
0
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
1.8
2
LP HP
w
|H(w)|Low-pass (LP)
h(n)=[1,1]
|h(w)|=2cos(w/2)
High-pass (LP)
h(n)=[1,-1]
|h(w)|=2sin(w/2)

32. forward transform inverse transform
• Properties
- periodic
- conjugate symmetric
1D Discrete Fourier Transform
Proof:
Proof:
)()( kyNky =+
)()( *
kykNy =−
∑
−
=
=
1
0
)()(
N
n
kn
NWnxky ∑
−
=
−
=
1
0
)()(
N
n
kn
NWkynx
∑∑
−
=
−
=
+
===+
1
0
1
0
)(
)()(
N
n
kn
Nn
N
n
nNk
Nn kyWxWxNky
∑∑
−
=
−
−
=
−
===−
1
0
*
1
0
)(
)()(
N
n
kn
Nn
N
n
nkN
Nn kyWxWxkNy
}
2
exp{
N
j
WN
π
−=

33. Matrix Representation of 1D DFT
Re
Im
DFT:
FA =












=×
NNN
N
NN
aa
aa
......
............
............
......
1
111
1,
,
1
2
==
=
−
N
N
N
j
N
kl
Nkl
WeW
W
N
a
π
NWkl
N
N
l
lk Wx
N
y ∑=
=
1
1
∑=
=
N
l
lklk xay
1

34. Fast Fourier Transform (FFT)*
 Invented by Tukey and Cooley in 1965
 Basic idea: divide-and-conquer
 Reduce the complexity of N-point DFT from
O(N2) to O(Nlog2N)
N/2-point DFT N/2-point DFT
N-point DFT
∑∑
∑∑∑
−=
−
−
=
−=
−
−
=
−
=
+=
+==
12
2/12
2
2/2
12
)12(
12
2
2
2
1
0
mn
km
Nm
k
mn
km
Nm
mn
mk
Nm
mn
mk
Nm
N
n
kn
Nnk
WxWWx
WxWxWxy
N

35. Filtering in the Frequency Domain
convolution in the time domain is equivalent to
multiplication in the frequency domain
f(n) h(n) g(n) F(k) H(k) G(k)
DFT
)()()( nhnfng ⊗= )()()( kHkFkG =

36. 2D Linear Filtering
f(m,n) h(m,n) g(m,n)
2D convolution
MATLAB function: C = CONV2(A, B)
),(),(),(),(),(
,
nmfnmhlnkmflkhnmg
lk
⊗=−−= ∑
∞
−∞=

37. 2D Filtering=Two Sequential
1D Filtering
 Just as we have observed with 2D
transform, 2D (separable) filtering can
be viewed as two sequential 1D filtering
operations: one along row direction and
the other along column direction
 The order of filtering does not matter
h1 : 1D filter
)()()()(),( 1111
mhnhnhmhnmh ⊗=⊗=

38. Numerical Example
h1(m)=[1,1], h1(n)=[1,-1]1D filter
MATLAB command:
>h1=[1,1];h2=[1,-1];
>conv2(h1,h2)
>conv2(h2,h1)






−−
=
⊗
11
11
)()( 11
nhmh






−−
=
⊗
11
11
)()( 11
mhnh

39. Fourier Series (2D case)
Note that the input signal is discrete
while its FT is a continuous function
spatial-domain convolution frequency-domain multiplication
∑ ∑
∞
−∞=
∞
−∞=
+−
=
m n
nwmwj
enmfwwF )(
21
21
),(),(
),(),( nmhnmf ⊗ ),(),( 2121 wwHwwF

40. Filter Examples
Low-pass (LP)
h1(n)=[1,1]
|h1(w)|=2cos(w/2)
1D
h(n)=[1,1;1,1]
|h(w1,w2)|=4cos(w1/2)cos(w2/2)
2D
w1
w2
|h(w1,w2)|

41. Image DFT Example
Original ray image X choice 1: Y=fft2(X)

42. Image DFT Example (Con’t)
choice 1: Y=fft2(X) choice 2: Y=fftshift(fft2(X))
Low-frequency at the centerLow-frequency at four corners
FFTSHIFT Shift zero-frequency component to center of spectrum.

43. Gaussian Filter
FT
>h=fspecial(‘gaussian’, HSIZE,SIGMA);MATLAB code:
)
2
exp(),( 2
2
2
2
1
21
σ
ww
wwH
+
−=)
2
exp(),( 2
22
σ
nm
nmh
+
−=

44. (σ=1)
PSNR=24.4dB
Image Example
PSNR=20.2dB
noisy
(σ=25)
denoised denoised
(σ=1.5)
PSNR=22.8dB
Matlab functions: imfilter, filter2

45. Gaussian Filter=Heat Diffusion
Linear Heat Flow Equation:
scale A Gaussian filter
with zero mean
and variance of t
Isotropic diffusion:
2
2
2
2
),,(),,(
),,(
),,(
y
tyxI
x
tyxI
tyxI
t
tyxI
∂
∂
+
∂
∂
=∆=
∂
∂
)()0,,(),,( tGyxItyxI ⊗=

46. Basic Idea of Nonlinear
Diffusion*
x
y
I(x,y)
image I
image I viewed as a 3D surface (x,y,I(x,y))
Diffusion should be anisotropic
instead of isotropic

47. Experimental Results
(Gaussian filtering)
PSNR=24.4dBPSNR=20.2dB
noisy
(σ=25)
linear diffusion
(TV filtering)
PSNR=27.5dB
nonlinear diffusion

48. Hammer-Nail Analogy
48
Gaussian filter
median filter
salt-pepper/
impulse noise
Gaussian noise
periodic noise
???

49. Image Denoising
 Introduction
 Impulse noise removal
 Median filtering
 Additive white Gaussian noise removal
 2D convolution and DFT
 Periodic noise removal
 Band-rejection and Notch filter

50. Periodic Noise
 Source: electrical or electromechanical
interference during image acquistion
 Characteristics
 Spatially dependent
 Periodic – easy to observe in frequency domain
 Processing method
 Suppressing noise component in frequency
domain

51. Image Example
spatial
Frequency (note the four pairs of bright dots)

52. Band Rejection Filter
w1
w2




+≤+≤−
=
otherwise
W
Dww
W
D
wwH
1
22
0
),(
2
2
2
1
21

53. Image Example
Before filtering After filtering

54. Advanced Denoising
Techniques*
Basic idea: from linear diffusion (equivalent to Gaussian filtering)
to nonlinear diffusion (with implicit edge-stopping criterion)
IN∇
Is∇
IE∇IW∇
jijiN III ,,1 −=∇ −
jijiS III ,,1 −=∇ +
jijiE III ,1, −=∇ +
jijiW III ,1, −=∇ −
][,
1
, IcIcIcIcII WWEESSNN
t
ji
t
ji ∇+∇+∇+∇+=+
λ
WESNdIgc dd ,,,||),(|| =∇=