Image processing

1. Digital Image Processing
Chapter 5: Image Restoration

2. A Model of the Image
Degradation/Restoration Process

3.  Degradation
 Degradation function H
 Additive noise
 Spatial domain
 Frequency domain
)
,
( y
x

)
,
(
)
,
(
*
)
,
(
)
,
( y
x
y
x
f
y
x
h
y
x
g 


)
,
(
)
,
(
)
,
(
)
,
( v
u
N
v
u
F
v
u
H
v
u
G 


4.  Restoration
)
,
(
ˆ
Filter
n
Restoratio
)
,
( y
x
f
y
x
g 


5. Noise Models
 Sources of noise
 Image acquisition, digitization,
transmission
 White noise
 The Fourier spectrum of noise is
constant
 Assuming
 Noise is independent of spatial
coordinates
 Noise is uncorrelated with respect to
the image itself

6.  Gaussian noise
 The PDF of a Gaussian random variable,
z,
 Mean:
 Standard deviation:
 Variance:
2
2
2
/
)
(
2
1
)
( 





 z
e
z
p


2


7.  70% of its values will be in the range
 95% of its values will be in the range
 
)
(
),
( 


 

 
)
2
(
),
2
( 


 


8.  Rayleigh noise
 The PDF of Rayleigh noise,
 Mean:
 Variance:











a
z
a
z
e
a
z
b
z
p
b
a
z
for
0
for
)
(
2
)
(
/
)
( 2
4
/
b
a 
 

4
)
4
(
2 



b

10.  Erlang (Gamma) noise
 The PDF of Erlang noise, ,
is a positive integer,
 Mean:
 Variance:











0
for
0
0
for
)!
1
(
)
(
1
z
z
e
b
z
a
z
p
z
a
b
b
a
b


2
2
a
b


0

a b

11.  Exponential noise
 The PDF of exponential noise, ,
 Mean:
 Variance:







0
for
0
0
for
)
(
z
z
ae
z
p
z
a
a
1


2
2 1
a


0

a

12.  Uniform noise
 The PDF of uniform noise,
 Mean:
 Variance:









otherwise
0
if
1
)
(
b
z
a
a
b
z
p
2
b
a 


12
)
( 2
2 a
b 



13.  Impulse (salt-and-pepper) noise
 The PDF of (bipolar) impulse noise,
 : gray-level will appear as a
light dot, while level will appear like
a dark dot
 Unipolar: either or is zero








otherwise
0
for
for
)
( b
z
P
a
z
P
z
p b
a
a
b  b
a
a
P b
P

14.  Usually, for an 8-bit image, =0
(black) and =0 (white)
b
a

15.  Modeling
 Gaussian
 Electronic circuit noise, sensor noise due
to poor illumination and/or high
temperature
 Rayleigh
 Range imaging
 Exponential and gamma
 Laser imaging

16.  Impulse
 Quick transients, such as faulty switching
 Uniform
 Least descriptive
 Basis for numerous random number
generators

20.  Periodic noise
 Arises typically from electrical or
electromechanical interference
 Reduced significantly via frequency
domain filtering

22.  Estimation of noise parameters
 Inspection of the Fourier spectrum
 Small patches of reasonably constant
gray level
 For example, 150*20 vertical strips
 Calculate , , , from
  a b



S
z
i
i
i
z
p
z )
(





S
z
i
i
i
z
p
z )
(
)
( 2
2



24. Restoration in the Presence of Noise
Only-Spatial Filtering
 Degradation
 Spatial domain
 Frequency domain
)
,
(
)
,
(
)
,
( y
x
y
x
f
y
x
g 


)
,
(
)
,
(
)
,
( v
u
N
v
u
F
v
u
G 


25.  Mean filters
 Arithmetic mean filter
 Geometric mean filter



xy
S
t
s
t
s
g
mn
y
x
f
)
,
(
)
,
(
1
)
,
(
ˆ
mn
S
t
s xy
t
s
g
y
x
f
1
)
,
(
)
,
(
)
,
(
ˆ








 


26.  Harmonic mean filter
 Works well for salt noise, but fails fpr
pepper noise



xy
S
t
s t
s
g
mn
y
x
f
)
,
( )
,
(
1
)
,
(
ˆ

27.  Contraharmonic mean filter
 : eliminates pepper noise
 : eliminates salt noise






xy
xy
S
t
s
Q
S
t
s
Q
t
s
g
t
s
g
y
x
f
)
,
(
)
,
(
1
)
,
(
)
,
(
)
,
(
ˆ
0

Q
0

Q

30.  Usage
 Arithmetic and geometric mean filters:
suited for Gaussian or uniform noise
 Contraharmonic filters: suited for
impulse noise

32.  Order-statistics filters
 Median filter
 Effective in the presence of both bipolar
and unipolar impulse noise
)}
,
(
{
median
)
,
(
ˆ
)
,
(
t
s
g
y
x
f
xy
S
t
s 


33.  Max and min filters
 max filters reduce pepper noise
 min filters salt noise
)}
,
(
{
max
)
,
(
ˆ
)
,
(
t
s
g
y
x
f
xy
S
t
s 

)}
,
(
{
min
)
,
(
ˆ
)
,
(
t
s
g
y
x
f
xy
S
t
s 


34.  Midpoint filter
 Works best for randomly distributed noise,
like Gaussian or uniform noise





 



)}
,
(
{
min
)}
,
(
{
max
2
1
)
,
(
ˆ
)
,
(
)
,
(
t
s
g
t
s
g
y
x
f
xy
xy S
t
s
S
t
s

35.  Alpha-trimmed mean filter
 Delete the d/2 lowest and the d/2 highest
gray-level values
 Useful in situations involving multiple
types of noise, such as a combination of
salt-and-pepper and Gaussian noise




xy
S
t
s
r t
s
g
d
mn
y
x
f
)
,
(
)
,
(
1
)
,
(
ˆ

39.  Adaptive, local noise reduction filter
 If is zero, return simply the value
of
 If , return a value close to
 If , return the arithmetic
mean value
2


)
,
( y
x
g
2
2
L

 
)
,
( y
x
g
2
2
L

 
L
m
 
L
L
m
y
x
g
y
x
g
y
x
f 

 )
,
(
)
,
(
)
,
(
ˆ
2
2



41.  Adaptive median filter
 = minimum gray level value in
 = maximum gray level value in
 = median of gray levels in
 = gray level at coordinates
 = maximum allowed size of
min
z
max
z
med
z
xy
z
max
S
xy
S
xy
S
xy
S
xy
S
)
,
( y
x

42.  Algorithm:
 Level A: A1=
 A2=
 If A1>0 AND A2<0, Go to
 level B
 Else increase the window size
 If window size
 repeat level A
 Else output
min
z
zmed 
max
z
zmed 
max
S

med
z

43.  Level B: B1=
 B2=
 If B1>0 AND B2<0, output
 Else output
min
z
zxy 
max
z
zxy 
xy
z
med
z

44.  Purposes of the algorithm
 Remove salt-and-pepper (impulse) noise
 Provide smoothing
 Reduce distortion, such as excessive
thinning or thickening of object
boundaries

46. Periodic Noise Reduction by Frequency
Domain Filtering
 Bandreject filters
 Ideal bandreject filter


















2
D
v)
D(u,
if
1
2
D
v)
D(u,
2
D
if
0
2
D
v)
D(u,
if
1
)
,
(
0
0
0
0
W
W
W
W
v
u
H
  2
/
1
2
2
)
2
/
(
)
2
/
(
)
,
( N
v
M
u
v
u
D 




47.  Butterworth bandreject filter of order n
 Gaussian bandreject filter
n
D
v
u
D
W
v
u
D
v
u
H 2
2
0
2
)
,
(
)
,
(
1
1
)
,
(









2
2
0
2
)
,
(
)
,
(
2
1
1
)
,
( 






 



W
v
u
D
D
v
u
D
e
v
u
H

50.  Bandpass filters
)
,
(
1
)
,
( v
u
H
v
u
H br
bp 


52.  Notch filters
 Ideal notch reject filter


 


otherwise
1
D
v)
(u,
D
or
D
v)
(u,
D
if
0
)
,
( 0
2
0
1
v
u
H
  2
/
1
2
0
2
0
1 )
2
/
(
)
2
/
(
)
,
( v
N
v
u
M
u
v
u
D 





  2
/
1
2
0
2
0
2 )
2
/
(
)
2
/
(
)
,
( v
N
v
u
M
u
v
u
D 






53.  Butterworth notch reject filter of
order n
n
v
u
D
v
u
D
D
v
u
H








)
,
(
)
,
(
1
1
)
,
(
2
1
2
0

54.  Gaussian notch reject filter











2
0
2
1 )
,
(
)
,
(
2
1
1
)
,
(
D
v
u
D
v
u
D
e
v
u
H

56.  Notch pass filter
)
,
(
1
)
,
( v
u
H
v
u
H nr
np 


58.  Optimum notch filtering

59.  Interference noise pattern
 Interference noise pattern in the spatial
domain
 Subtract from a weighted
portion of to obtain an
estimate of
)
,
(
)
,
(
)
,
( v
u
G
v
u
H
v
u
N 
)}
,
(
)
,
(
{
)
,
( 1
v
u
G
v
u
H
y
x 



)
,
(
)
,
(
)
,
(
)
,
(
ˆ y
x
y
x
w
y
x
g
y
x
f 


)
,
( y
x
g
)
,
( y
x

)
,
( y
x
f

60.  Minimize the local variance of
 The detailed steps are listed in Page
251
 Result
)
,
(
ˆ y
x
f
)
,
(
)
,
(
)
,
(
)
,
(
)
,
(
)
,
(
)
,
(
2
2
y
x
y
x
y
x
y
x
g
y
x
y
x
g
y
x
w








64. Linear, Position-Invariant Degradations
 Input-output relationship
)
,
(
)]
,
(
[
)
,
( y
x
y
x
f
H
y
x
g 


)]
,
(
[
)
,
( y
x
f
H
y
x
g 
0
)
,
( 
y
x


65.  H is linear if
 Additivity
)]
,
(
[
)]
,
(
[
)]
,
(
)
,
(
[
2
1
2
1
y
x
f
bH
y
x
f
aH
y
x
bf
y
x
af
H



)]
,
(
[
)]
,
(
[
)]
,
(
)
,
(
[
2
1
2
1
y
x
f
H
y
x
f
H
y
x
f
y
x
f
H




66.  Homogeneity
 Position (or space) invariant
)]
,
(
[
)]
,
(
[ 1
1 y
x
f
aH
y
x
af
H 
)]
,
(
)]
,
(
[ 


 



 y
x
g
y
x
f
H

67.  In terms of a continuous impulse
function
 








 





 d
d
y
x
f
y
x
f )
,
(
)
,
(
)
,
(





 



 












 d
d
y
x
f
H
y
x
f
H
y
x
g
)
,
(
)
,
(
)]
,
(
[
)
,
(

68.  
 
 
 
 













































d
d
y
x
h
f
d
d
y
x
H
f
d
d
y
x
f
H
y
x
g
)
,
,
,
(
)
,
(
)
,
(
)
,
(
)
,
(
)
,
(
)
,
(

69.  Impulse response of H
 In optics, the impulse becomes a point
of light
 Point spread function (PSF)
 All physical optical systems blur
(spread) a point of light to some
degree
)]
,
(
[
)
,
,
,
( 



 

 y
x
H
y
x
h
)
,
,
,
( 
 y
x
h

70.  Superposition (or Fredholm) integral of
the first kind
 












 d
d
y
x
h
f
y
x
g
)
,
,
,
(
)
,
(
)
,
(

71.  If H is position invariant
 Convolution integral
)
,
(
)]
,
(
[ 



 



 y
x
h
y
x
H
 














 d
d
y
x
h
f
y
x
g
)
,
(
)
,
(
)
,
(

72.  In the presence of additive noise
 If H is position invariant
)
,
(
)
,
,
,
(
)
,
(
)
,
(
y
x
d
d
y
x
h
f
y
x
g






 

 






)
,
(
)
,
(
)
,
(
)
,
(
y
x
d
d
y
x
h
f
y
x
g






 



 







73.  If H is position invariant
 Restoration approach
 Image deconvolution
 Deconvolution filter
)
,
(
)
,
(
)
,
(
)
,
( y
x
y
x
f
y
x
h
y
x
g 



)
,
(
)
,
(
)
,
(
)
,
( v
u
N
v
u
F
v
u
H
v
u
G 


74. Estimating the Degradation Function
 Estimation by image observation
 In order to reduce the effect of noise in
our observation, we would look for
areas of strong signal content
)
,
(
ˆ
)
,
(
)
,
(
v
u
F
v
u
G
v
u
H
s
s
s 

75.  Estimation by experimentation
 Obtain the impulse response of the
degradation by imaging an impulse
(small dot of light) using the same
system settings
 Observed image
 The strength of the impulse
A
v
u
G
v
u
H
)
,
(
)
,
( 
)
,
( v
u
G
A

77.  Estimation by modeling
 Hufnagel and Stanley
 Physical characteristic of atmospheric
turbulence
6
5
2
2
)
(
)
,
( v
u
k
e
v
u
H 



79.  Image motion
dt
t
y
y
t
x
x
f
y
x
g
T
]
)
(
),
(
[
)
,
(
0
0
0
 



80. dt
dy
dx
e
t
y
y
t
x
x
f
dy
dx
e
dt
t
y
y
t
x
x
f
dy
dx
e
y
x
g
v
u
G
y
v
x
u
j
T
y
v
x
u
j
T
y
v
x
u
j
)]
(
),
(
[
]
)
(
),
(
[
)
,
(
)
,
(
)
(
2
0
0
0
)
(
2
0
0
0
)
(
2
























  
  
 


 







 







81.  Where
)
,
(
)
,
(
)
,
(
)
,
(
)
,
(
0
)]
(
)
(
[
2
0
)]
(
)
(
[
2
0
0
0
0
v
u
H
v
u
F
dt
e
v
u
F
dt
e
v
u
F
v
u
G
T t
y
v
t
x
u
j
T t
y
v
t
x
u
j















T t
y
v
t
x
u
j
dt
e
v
u
H
0
)]
(
)
(
[
2 0
0
)
,
( 

82.  If and
T
at
t
x /
)
(
0  0
)
(
0 
t
y
ua
j
T
T
at
u
j
T t
x
u
j
e
ua
ua
T
dt
e
dt
e
v
u
H
0
]
/
[
2
0
)]
(
[
2
)
sin(
)
,
( 0














83.  If and
T
at
t
x /
)
(
0  T
bt
t
y /
)
(
0 
)
(
)]
(
sin[
)
(
)
,
(
vb
ua
j
e
vb
ua
vb
ua
T
v
u
H




 



85. Inverse Filtering
 Direct inverse filtering
 Limiting the analysis to frequencies
near the origin
)
,
(
)
,
(
)
,
(
)
,
(
)
,
(
)
,
(
ˆ
v
u
H
v
u
N
v
u
F
v
u
H
v
u
G
v
u
F




87. Minimum Mean Square Error (Wiener)
Filtering
 Minimize
 Terms
 = degradation function

 = complex conjugate of

 =

}
)
ˆ
{( 2
2
f
f
E
e 

)
,
( v
u
H
)
,
( v
u
H  )
,
( v
u
H
2
)
,
( v
u
H )
,
(
)
,
( v
u
H
v
u
H 

88.  = power spectrum
of the noise
 = power spectrum
of the undegraded image
2
)
,
(
)
,
( v
u
N
v
u
S 

2
)
,
(
)
,
( v
u
F
v
u
S f 

89.  Wiener filter
)
,
(
)
,
(
/
)
,
(
)
,
(
)
,
(
)
,
(
1
)
,
(
)
,
(
/
)
,
(
)
,
(
)
,
(
*
)
,
(
)
,
(
)
,
(
)
,
(
)
,
(
)
,
(
*
)
,
(
ˆ
2
2
2
2
v
u
G
v
u
S
v
u
S
v
u
H
v
u
H
v
u
H
v
u
G
v
u
S
v
u
S
v
u
H
v
u
H
v
u
G
v
u
S
v
u
H
v
u
S
v
u
S
v
u
H
v
u
F
f
f
f
f


































90.  White noise
)
,
(
)
,
(
)
,
(
)
,
(
1
)
,
(
ˆ
2
2
v
u
G
K
v
u
H
v
u
H
v
u
H
v
u
F











93. Constrained Least Squares Filtering
 Vector-matrix form

 , , :
 :
g
)
,
(
)
,
(
*
)
,
(
)
,
( y
x
y
x
f
y
x
h
y
x
g 


1

MN
MN
MN 
H
η
f
η
Hf
g 


94.  Minimize
 Subject to
 







1
0
1
0
2
2
)
,
(
M
x
N
y
y
x
f
C
2
2
ˆ η
f
H
g 


95.  The solution
 Where is the Fourier transform
of the function
)
,
(
)
,
(
)
,
(
)
,
(
*
)
,
(
ˆ
2
2
v
u
G
v
u
P
v
u
H
v
u
H
v
u
F











)
,
( v
u
P















0
1
0
1
4
1
0
1
0
)
,
( y
x
P

97.  Computing by iteration
 Adjust so that

f
H
g
r ˆ



a


2
2
η
r

98.  Computation






1
0
1
0
2
2
)
,
(
M
x
N
y
y
x
r
r
 
2
1
0
1
0
2
)
,
(
1







M
x
N
y
m
y
x
MN

 







1
0
1
0
)
,
(
1 M
x
N
y
y
x
MN
m 

]
[ 2
2
2


 m
MN 

η

99.  Algorithm
 1: Specify an initial value of
 2: Compute
 3: Stop if is satisfied;
otherwise return to Step 2 after
increasing if or
 decreasing if .
a


2
2
η
r

 a


2
2
η
r
a


2
2
η
r

101. Geometric Mean FIlter
)
,
(
)
,
(
)
,
(
)
,
(
)
,
(
*
)
,
(
)
,
(
*
)
,
(
ˆ
1
2
2
v
u
G
v
u
S
v
u
S
v
u
H
v
u
H
v
u
H
v
u
H
v
u
F
f






































102. Geometric Transformations
 Spatial transformations
 Tiepoints
)
,
(
' y
x
r
x 
)
,
(
' y
x
s
y 

104.  Bilinear equations
4
3
2
1
)
,
(
' c
xy
c
y
c
x
c
y
x
r
x 




8
7
6
5
)
,
(
' c
xy
c
y
c
x
c
y
x
s
y 





105.  Gray-level interpolation
d
y
cx
by
ax
y
x
v 


 '
'
'
'
)
'
,
'
(