CV_2 Image Processing

Computer Vision
Chap.2 Image Processing
SUB CODE: 3171614
SEMESTER: 7TH IT
PREPARED BY:
PROF. KHUSHALI B. KATHIRIYA

Outline
• Pixel transforms
• Color transforms
• Histogram processing & equalization
• Filtering
• Convolution
• Fourier transformation and its applications in sharpening
• Blurring and noise removal
Prepared by: Prof. Khushali B Kathiriya
2

What is Image Processing?
PREPARED BY:

“One picture is worth more than ten thousand words”
Anonymous
4

What is Digital Image Processing?
• Digital image processing focuses on two major tasks
• Improvement of pictorial information for human interpretation
• Processing of image data for storage, transmission and representation for
autonomous machine perception
5

What is DIP? (cont…)
The continuum from image processing to computer vision can be broken up into
low-, mid- and high-level processes
Low Level Process
Input: Image
Output: Image
Examples: Noise
removal, image
sharpening
Mid Level Process
Input: Image
Output: Attributes
Examples: Object
recognition,
segmentation
High Level Process
Input: Attributes
Output: Understanding
Examples: Scene
understanding,
autonomous navigation
6

History of Digital Image Processing
Early 1920s: One of the first applications of digital imaging was in the news-
paper industry
• The Bartlane cable picture transmission service
• Images were transferred by submarine cable between London and New York
• Pictures were coded for cable transfer and reconstructed at the receiving end on a
telegraph printer
Early digital image
7

History of DIP (cont…)
Mid to late 1920s: Improvements to the Bartlane system resulted in higher quality
images
• New reproduction
processes based
on photographic
techniques
• Increased number
of tones in
reproduced images
Improved
digital image Early 15 tone digital
image
8

1960s: Improvements in computing technology and the onset of the space race led
to a surge of work in digital image processing
• 1964: Computers used to
improve the quality of
images of the moon taken
by the Ranger 7 probe
• Such techniques were used
in other space missions
including the Apollo landings
A picture of the moon taken
by the Ranger 7 probe
minutes before landing
9

1970s: Digital image processing begins to be used in medical applications
• 1979: Sir Godfrey N.
Hounsfield & Prof. Allan M.
Cormack share the Nobel
Prize in medicine for the
invention of tomography,
the technology behind
Computerised Axial
Tomography (CAT) scans
Typical head slice CAT
image
10

1980s - Today: The use of digital image processing techniques has exploded and
they are now used for all kinds of tasks in all kinds of areas
• Image enhancement/restoration
• Artistic effects
• Medical visualisation
• Industrial inspection
• Law enforcement
• Human computer interfaces
11

Examples: Image Enhancement
One of the most common uses of DIP techniques: improve quality, remove noise
etc
12

Examples: The Hubble Telescope
•Launched in 1990 the Hubble
telescope can take images of
very distant objects
•However, an incorrect mirror
made many of Hubble’s
images useless
•Image processing
techniques were
used to fix this
13

Examples: Artistic Effects
Artistic effects are used to make
images more visually
appealing, to add special effects
and to make composite images
14

Examples: Medicine
Take slice from MRI scan of canine heart, and find boundaries between types of
tissue
• Image with gray levels representing tissue density
• Use a suitable filter to highlight edges
Original MRI Image of a Dog Heart Edge Detection Image
15

Examples: GIS
Geographic Information Systems
• Digital image processing techniques are used extensively to manipulate satellite
imagery
• Terrain classification
• Meteorology
16

Examples: GIS (cont…)
Night-Time Lights of the World data set
• Global inventory of human
settlement
• Not hard to imagine the kind of
analysis that might be done using
this data
17

Examples: Industrial Inspection
•Human operators are expensive, slow
and unreliable
•Make machines do the job instead
•Industrial vision systems are used in
all kinds of industries
18

Examples: PCB Inspection
Printed Circuit Board (PCB) inspection
• Machine inspection is used to determine that all components are present and that all
solder joints are acceptable
• Both conventional imaging and x-ray imaging are used
19

Examples: Law Enforcement
Image processing techniques are used
extensively by law enforcers
• Number plate recognition for speed
cameras/automated toll systems
• Fingerprint recognition
• Enhancement of CCTV images
20

Examples: HCI
Try to make human computer interfaces more
natural
• Face recognition
• Gesture recognition
21

Key Stages in Digital Image
Processing
PREPARED BY:

Key Stages in Digital Image Processing
Image
Acquisition
Image
Restoration
Morphological
Processing
Segmentation
Representation
& Description
Image
Enhancement
Object
Recognition
Problem Domain
Colour Image
Processing
Image
Compression
23

Key Stages in Digital Image Processing:
Image Aquisition
Image
Acquisition
Image
Restoration
Morphological
Processing
Segmentation
Representation
& Description
Image
Enhancement
Object
Recognition
Problem Domain
Colour Image
Processing
Image
Compression
24

Image Enhancement
Image
Acquisition
Image
Restoration
Morphological
Processing
Segmentation
Representation
& Description
Image
Enhancement
Object
Recognition
Problem Domain
Colour Image
Processing
Image
Compression
25

Image Restoration
Image
Acquisition
Image
Restoration
Morphological
Processing
Segmentation
Representation
& Description
Image
Enhancement
Object
Recognition
Problem Domain
Colour Image
Processing
Image
Compression
26

Morphological Processing
Image
Acquisition
Image
Restoration
Morphological
Processing
Segmentation
Representation
& Description
Image
Enhancement
Object
Recognition
Problem Domain
Colour Image
Processing
Image
Compression
27

Segmentation
Image
Acquisition
Image
Restoration
Morphological
Processing
Segmentation
Representation
& Description
Image
Enhancement
Object
Recognition
Problem Domain
Colour Image
Processing
Image
Compression
28

Object Recognition
Image
Acquisition
Image
Restoration
Morphological
Processing
Segmentation
Representation
& Description
Image
Enhancement
Object
Recognition
Problem Domain
Colour Image
Processing
Image
Compression
29

Representation & Description
Image
Acquisition
Image
Restoration
Morphological
Processing
Segmentation
Representation
& Description
Image
Enhancement
Object
Recognition
Problem Domain
Colour Image
Processing
Image
Compression
30

Image Compression
Image
Acquisition
Image
Restoration
Morphological
Processing
Segmentation
Representation
& Description
Image
Enhancement
Object
Recognition
Problem Domain
Colour Image
Processing
Image
Compression
31

Colour Image Processing
Image
Acquisition
Image
Restoration
Morphological
Processing
Segmentation
Representation
& Description
Image
Enhancement
Object
Recognition
Problem Domain
Colour Image
Processing
Image
Compression
32

Image Enhancement
PREPARED BY:

Image Aquisition
Image
Acquisition
Image
Restoration
Morphological
Processing
Segmentation
Representation
& Description
Image
Enhancement
Object
Recognition
Problem Domain
Colour Image
Processing
Image
Compression
34

Image Enhancement
Image
Acquisition
Image
Restoration
Morphological
Processing
Segmentation
Representation
& Description
Image
Enhancement
Object
Recognition
Problem Domain
Colour Image
Processing
Image
Compression
35

A Note About Grey Levels
• So far when we have spoken about image grey level values we have said
they are in the range [0, 255]
• Where 0 is black and 255 is white
• There is no reason why we have to use this range
• The range [0,255] stems from display technologies
• For many of the image processing operations , grey levels are assumed
to be given in the range [0.0, 1.0]
36

What is a Digital Image? (Cont.)
Common image formats include:
• 1 sample per point (B&W or Grayscale)
• 3 samples per point (Red, Green, and Blue)
• 4 samples per point (Red, Green, Blue, and “Alpha”, a.k.a.
Opacity)
37

What Is Image Enhancement?
Image enhancement is the process of making images more useful
The reasons for doing this include:
• Highlighting interesting detail in images
• Removing noise from images
• Making images more visually appealing
38

Image Enhancement Examples
39

Image Enhancement Examples (cont…)
40

41

42

Spatial & Frequency Domains
There are two broad categories of image enhancement techniques
• Spatial domain techniques
• Direct manipulation of image pixels
• Frequency domain techniques
• Manipulation of Fourier transform or wavelet transform of an image
43

Image Enhancement
Image Histogram
PREPARED BY:

Image Histograms
• The histogram of an image shows us the distribution of grey levels in the
image
• Massively useful in image processing, especially in segmentation
Grey Levels
Frequencies
45

Histogram Examples
46

Histogram Examples (cont…)
47

48

49

50

51

52

53

54

55

• A selection of images and
their histograms
• Notice the relationships
between the images and
their histograms
• Note that the high contrast
image has the most
evenly spaced histogram
56

Histogram Example
• Plot the histogram for following data.
57
Grey Level Number of Pixels
0 40
1 20
2 10
3 15
4 10
5 3
6 2

Histogram Example
• Calculate the probability of occurrence of the grey level (Probability Density
Functions)
58
Grey Level Number of
Pixels(nk)
P(rk) = nk/n
0 40
1 20
2 10
3 15
4 10
5 3
6 2
n=100

Histogram Example
• Calculate the probability of occurrence of the grey level (normalized histogram)
59
Grey Level Number of
Pixels(nk)
P(rk) = nk/n
0 40 0.4
1 20 0.2
2 10 0.1
3 15 0.15
4 10 0.1
5 3 0.03
6 2 0.02
n=100

Image Enhancement
Histogram Stretching
PREPARED BY:

Linear Stretching
• One way to increase the dynamic range is by using a technique
known as histogram stretching
• In this method , we do not alter the basic shape of the histogram ,
but we spread it so as to cover the entire dynamic range
61
• Smax -> Maximum grey level of output image
• Smin -> Minimum grey level of output image
• rmax -> Maximum grey level of input image
• rmin -> Minimum grey level of input image
Rmax - Rmin
Smax – Smin
s = T(r) = (R - Rmin) + Smin

Histogram Stretching : Example
• Perform histogram stretching so that new image has a dynamic range of [0,7]
62
0 0
1 0
2 50
3 60
4 50
5 20
6 10
7 0

63

Histogram
Stretching
:
Example Prepared by: Prof. Khushali B Kathiriya
64

65

• Modified histogram in the dynamic range of [0,7]
66
0 50
1 0
2 60
3 0
4 50
5 20
6 0
7 10

Try out your self…
68
0 100
1 90
2 85
3 70
4 0
5 0
6 0
7 0

69
0 100
1 0
2 90
3 0
4 0
5 85
6 0
7 70

Image Enhancement
Histogram Equalisation
PREPARED BY:

Histogram Equalisation
• Spreading out the frequencies in an image (or equalising the image) is a simple
way to improve dark or washed out images
• The formula for histogram
equalisation is given where
• rk: input intensity
• sk: processed intensity
• k: the intensity range
(e.g 0.0 – 1.0)
• nj: the frequency of intensity j
• n: the sum of all frequencies
71
)
( k
k r
T
s =

=
=
k
j
j
r r
p
1
)
(

=
=
k
j
j
n
n
1

Equalisation Transformation Function
72

Equalisation Examples 1
73

Equalisation Examples
2
74

Equalisation Examples (cont…)
3
4
75

Histogram Equalization
• Histogram--->PDF --- > CDF --- > Equalized
histogram
• CDF – cumulative density function

=
=
r
r
k
k r
p
r
T
S
0
)
(
)
(
76

Equalize the given histogram
Grey
Level
Number
of
Pixels(nk)
PDF
P(rk) =
nk/n
CDF (L-1)*Sk Round
off
0 790
1 1023
2 850
3 656
4 329
5 245
6 122
7 81
4096
77

Grey
Level
Number
of
Pixels(nk)
PDF
P(rk) =
nk/n
CDF
Sk =
(L-1)*Sk Round
off
0 790 0.19
1 1023 0.25
2 850 0.21
3 656 0.16
4 329 0.08
5 245 0.06
6 122 0.03
7 81 0.02
4096

r
r r
p
0
)
(
78

Grey
Level
Number
of
Pixels(nk)
PDF
P(rk) =
nk/n
CDF
Sk =
(L-1)*Sk Round
off
0 790 0.19 0.19
1 1023 0.25 0.44
2 850 0.21 0.65
3 656 0.16 0.81
4 329 0.08 0.89
5 245 0.06 0.95
6 122 0.03 0.98
7 81 0.02 1
4096

r
r r
p
0
)
(
79

Grey
Level
Number
of
Pixels(nk)
PDF
P(rk) =
nk/n
CDF
Sk =
(L-1)*Sk Round
off
0 790 0.19 0.19 1.33 1
1 1023 0.25 0.44 3.08 3
2 850 0.21 0.65 4.55 5
3 656 0.16 0.81 5.67 6
4 329 0.08 0.89 6.23 6
5 245 0.06 0.95 6.65 7
6 122 0.03 0.98 6.86 7
7 81 0.02 1 7 7
4096

r
r r
p
0
)
(
80

Equalized Grey Level
Grey Level Old Number of
Pixels
New Number of
Pixels
0 790 0
1 1023 790
2 850 0
3 656 1023
4 329 0
5 245 850
6 122 656+329 = 985
7 81 245+122+81 = 448
81
Grey
Level
Round
off
0 1
1 3
2 5
3 6
4 6
5 7
6 7
7 7

Histogram Equalization
82

Satelite Origional Image
83

Histogram Equalized Image
84

Try out your self
(Equalize the given histogram)
0 100
1 90
2 50
3 20
4 0
5 0
6 0
7 0
85

Try out your self
(Equalize the given histogram)
86

Noise Removal and Blurring
PREPARED BY:

What is Image Restoration?
• Image restoration attempts to restore images that have been degraded
• Identify the degradation process and attempt to reverse it
• Similar to image enhancement, but more objective
88

Image restoration vs. image enhancement
• Enhancement:
• largely a subjective process
• Priori knowledge about the degradation is not a must (sometimes no
degradation is involved)
• Procedures are heuristic and take advantage of the psychophysical aspects of
human visual system
• Restoration:
• more an objective process
• Images are degraded
• Tries to recover the images by using the knowledge about the degradation
89

Noise and Images
• The sources of noise in digital images arise
during image acquisition (digitization) and
transmission
• Imaging sensors can be affected by ambient
conditions
• Interference can be added
to an image during transmission
90

Noise Model
• We can consider a noisy image to be modelled as follows:
• where f(x, y) is the original image pixel, η(x, y) is the noise term and g(x, y) is
the resulting noisy pixel
• If we can estimate the model the noise in an image is based on this will help us
to figure out how to restore the image
)
,
(
)
,
(
)
,
( y
x
y
x
f
y
x
g 
+
=
91

Noise Models
Gaussian Rayleigh
Erlang Exponential
Uniform
Impulse
There are many different models for the
image
noise term η(x, y):
• Gaussian
• Most common model
• Rayleigh
• Erlang
• Exponential
• Uniform
• Impulse
• Salt and pepper noise
93

Noise Example
• The test pattern to the right is ideal for
demonstrating the addition of noise
• The following slides will show the result of
adding noise based on various models to this
image
Histogram to go here
Image
Histogram
94

Noise Example (cont…)
Gaussian Rayleigh Erlang
95

Noise Example (cont…)
Exponential Uniform Impulse
Histogram to go here
96

Noise Removal
Arithmetic Mean Filter
PREPARED BY:

Filtering to Remove Noise: Arithmetic Mean Filter
• We can use spatial filters of different kinds to remove different kinds of noise
• The arithmetic mean filter is a very simple one and is calculated as follows:
This is implemented as the
simple smoothing filter
Blurs the image to remove
noise


=
xy
S
t
s
t
s
g
mn
y
x
f
)
,
(
)
,
(
1
)
,
(
ˆ
1/9
1/9
1/9
1/9
1/9
1/9
1/9
1/9
1/9
98

Other Means
There are different kinds of mean filters all of which exhibit slightly different
behaviour:
• Geometric Mean
• Harmonic Mean
• Contraharmonic Mean
99

Other Means (cont…)
• There are other variants on the mean which can give different performance
• Geometric Mean:
• Achieves similar smoothing to the arithmetic mean, but tends to lose less image
detail
mn
S
t
s xy
t
s
g
y
x
f
1
)
,
(
)
,
(
)
,
(
ˆ








= 

100

Harmonic Mean:
• Works well for salt noise, but fails for pepper noise
• Also does well for other kinds of noise such as Gaussian noise


=
xy
S
t
s t
s
g
mn
y
x
f
)
,
( )
,
(
1
)
,
(
ˆ
101

• Contraharmonic Mean:
• Q is the order of the filter and adjusting its value changes the filter’s behaviour
• Positive values of Q eliminate pepper noise
• Negative values of Q eliminate salt noise




+
=
xy
xy
S
t
s
Q
S
t
s
Q
t
s
g
t
s
g
y
x
f
)
,
(
)
,
(
1
)
,
(
)
,
(
)
,
(
ˆ
102

Noise Removal Examples
Original
Image
Image
Corrupted
By Gaussian
Noise
After A 3*3
Geometric
Mean Filter
After A 3*3
Arithmetic
Mean Filter
103

Noise Removal Examples (cont…)
Image
Corrupted
By Pepper
Noise
Result of
Filtering Above
With 3*3
Contraharmonic
Q=1.5
104

Image
Corrupted
By Salt
Noise
Result of
Filtering Above
With 3*3
Contraharmonic
Q=-1.5
105

Contraharmonic Filter: Here Be Dragons
Choosing the wrong value for Q when using the contraharmonic filter can have
drastic results
106

Noise Removal
Order Statistics Filters
PREPARED BY:

Order Statistics Filters
• Spatial filters that are based on ordering the pixel values that make up the
neighbourhood operated on by the filter
• Useful spatial filters include
• Median filter
• Max and min filter
• Midpoint filter
• Alpha trimmed mean filter
108

Median Filter
• Median Filter:
• Excellent at noise removal, without the smoothing effects that can occur with
other smoothing filters
• Particularly good when salt and pepper noise is present
)}
,
(
{
)
,
(
ˆ
)
,
(
t
s
g
median
y
x
f
xy
S
t
s 
=
109

Max and Min Filter
Max Filter:
Min Filter:
Max filter is good for pepper noise and min is good for salt noise
)}
,
(
{
max
)
,
(
ˆ
)
,
(
t
s
g
y
x
f
xy
S
t
s 
=
)}
,
(
{
min
)
,
(
ˆ
)
,
(
t
s
g
y
x
f
xy
S
t
s 
=
110

Midpoint Filter
Midpoint Filter:
Good for random Gaussian and uniform noise





 +
=


)}
,
(
{
min
)}
,
(
{
max
2
1
)
,
(
ˆ
)
,
(
)
,
(
t
s
g
t
s
g
y
x
f
xy
xy S
t
s
S
t
s
111

Alpha-Trimmed Mean Filter
• Alpha-Trimmed Mean Filter:
• We can delete the d/2 lowest and d/2 highest grey levels
• So gr(s, t) represents the remaining mn – d pixels


−
=
xy
S
t
s
r t
s
g
d
mn
y
x
f
)
,
(
)
,
(
1
)
,
(
ˆ
112

Noise Removal Examples
Image
Corrupted
By Salt And
Pepper Noise
Result of 1
Pass With A
3*3 Median
Filter
Result of 2
Passes With
A 3*3 Median
Filter
Result of 3
Passes With
A 3*3 Median
Filter
113

Image
Corrupted
By Pepper
Noise
Image
Corrupted
By Salt
Noise
Result Of
Filtering
Above
With A 3*3
Min Filter
Result Of
Filtering
Above
With A 3*3
Max Filter
114

Image
Corrupted
By Uniform
Noise
Image Further
Corrupted
By Salt and
Pepper Noise
Filtered By
5*5 Arithmetic
Mean Filter
Filtered By
5*5 Median
Filter
Filtered By
5*5 Geometric
Mean Filter
Filtered By
5*5 Alpha-Trimmed
Mean Filter
115

Periodic Noise
• Typically arises due to
electrical or electromagnetic
interference
• Gives rise to regular noise
patterns in an image
• Frequency domain techniques
in the Fourier domain are
most effective at removing
periodic noise
116

Band Reject Filters
•Removing periodic noise from an image involves
removing a particular range of frequencies from
that image
•Band reject filters can be used for this purpose
•An ideal band reject filter is given as follows:









+

+


−
−

=
2
)
,
(
1
2
)
,
(
2
0
2
)
,
(
1
)
,
(
0
0
0
0
W
D
v
u
D
if
W
D
v
u
D
W
D
if
W
D
v
u
D
if
v
u
H
117

Band Reject Filters (cont…)
• The ideal band reject filter is shown below, along with Butterworth and Gaussian
versions of the filter
Ideal Band
Reject Filter
Butterworth
Band Reject
Filter (of order 1)
Gaussian
Band Reject
Filter
118

Band Reject Filter Example
Image corrupted
by sinusoidal
noise
Fourier spectrum
of corrupted
image
Butterworth band
reject filter
Filtered image
119

Adaptive Filters
• The filters discussed so far are applied to an entire image without any regard
for how image characteristics vary from one point to another
• The behaviour of adaptive filters changes depending on the characteristics of
the image inside the filter region
• We will take a look at the adaptive median filter
120

Adaptive Median Filtering
• The median filter performs relatively well on impulse noise as long as the
spatial density of the impulse noise is not large
• The adaptive median filter can handle much more spatially dense impulse
noise, and also performs some smoothing for non-impulse noise
• The key insight in the adaptive median filter is that the filter size changes
depending on the characteristics of the image
121

Adaptive Median Filtering (cont…)
Remember that filtering looks at each original pixel image in turn and generates a new filtered
pixel
First examine the following notation:
• zmin = minimum grey level in Sxy
• zmax = maximum grey level in Sxy
• zmed = median of grey levels in Sxy
• zxy = grey level at coordinates (x, y)
• Smax =maximum allowed size of Sxy
122

Level A: A1 = zmed – zmin
A2 = zmed – zmax
If A1 > 0 and A2 < 0, Go to level B
Else increase the window size
If window size ≤ Smax repeat level A
Else output zmed
Level B: B1 = zxy – zmin
B2 = zxy – zmax
If B1 > 0 and B2 < 0, output zxy
Else output zmed
123

The key to understanding the algorithm is to remember that the adaptive median
filter has three purposes:
• Remove impulse noise
• Provide smoothing of other noise
• Reduce distortion
124

Adaptive Filtering Example
Image corrupted by salt
and pepper noise with
probabilities Pa = Pb=0.25
Result of filtering with a
7 * 7 median filter
Result of adaptive
median filtering with i = 7
125

Summary
• Restoration is slightly more objective than enhancement
• Spatial domain techniques are particularly useful for removing random
noise
• Frequency domain techniques are particularly useful for removing
periodic noise
126

Filtering
PREPARED BY:

Contents
In this lecture we will look at spatial filtering techniques:
• Neighbourhood operations
• What is spatial filtering?
• Smoothing operations
• What happens at the edges?
• Correlation and convolution
128

Neighbourhood Operations
• Neighbourhood operations simply operate on a larger neighbourhood of pixels
than point operations
• Neighbourhoods are
mostly a rectangle
around a central pixel
• Any size rectangle
and any shape filter
are possible
Origin x
y Image f (x, y)
(x, y)
Neighbourhood
129

Simple Neighbourhood Operations
Some simple neighbourhood operations include:
• Min: Set the pixel value to the minimum in the neighbourhood
• Max: Set the pixel value to the maximum in the neighbourhood
• Median: The median value of a set of numbers is the midpoint value in that set (e.g.
from the set [1, 7, 15, 18, 24] 15 is the median). Sometimes the median works better
than the average
130

Simple Neighbourhood Operations Example
123 127 128 119 115 130
140 145 148 153 167 172
133 154 183 192 194 191
194 199 207 210 198 195
164 170 175 162 173 151
Original Image x
y
Enhanced Image x
y
131

The Spatial Filtering Process
r s t
u v w
x y z
Origin x
y Image f (x, y)
eprocessed = v*e +
r*a + s*b + t*c +
u*d + w*f +
x*g + y*h + z*i
Filter
Simple 3*3
Neighbourhood
e 3*3 Filter
a b c
d e f
g h i
Original Image
Pixels
*
The above is repeated for every pixel in the original image to generate the
filtered image
132

Spatial Filtering: Equation Form

−
= −
=
+
+
=
a
a
s
b
b
t
t
y
s
x
f
t
s
w
y
x
g )
,
(
)
,
(
)
,
(
Filtering can be given in
equation form as shown
above
Notations are based on the
image shown to the left
133

Smoothing Spatial Filters
One of the simplest spatial filtering operations we can perform is a smoothing
operation
• Simply average all of the pixels in a neighbourhood around a central value
• Especially useful
in removing noise
from images
• Also useful for
highlighting gross
detail
1/9
1/9
1/9
1/9
1/9
1/9
1/9
1/9
1/9
Simple
averaging
filter
(Box filter)
134

Smoothing Spatial Filtering
1/9
1/9
1/9
1/9
1/9
1/9
1/9
1/9
1/9
Origin x
y Image f (x, y)
e = 1/9*106 +
1/9*104 + 1/9*100 + 1/9*108 +
1/9*99 + 1/9*98 +
1/9*95 + 1/9*90 + 1/9*85
= 98.3333
Filter
Simple 3*3
Neighbourhood
106
104
99
95
100 108
98
90 85
1/9
1/9
1/9
1/9
1/9
1/9
1/9
1/9
1/9
3*3 Smoothing
Filter
104 100 108
99 106 98
95 90 85
Original Image
Pixels
*
The above is repeated for every pixel in the original image to generate the
smoothed image
135

Image Smoothing Example
• The image at the top left
is an original image of
size 500*500 pixels
• The subsequent images
show the image after
filtering with an averaging
filter of increasing sizes
• 3, 5, 9, 15 and 35
• Notice how detail begins
to disappear
136

Correlation
• The equation of correlation is defined as follows :
• It performs dot product between weights and function
137
a b
s a t b
g( x,y ) ω( s,t ) f ( x s,y t )
g ω f
=− =−
= + +
=
 

3
2
1
2
2
1
3
2
3
2
2
1
2
2
3
2 5
3
2
1
2
2
1
3
2
3
2
2
1
2
2
3
2
1
2
1
-2
4
1
-1
-1
1
1
1
1
-1
2
1
-1
-1
1
Input Image, f
output
Image, g
Correlation
138

3
2
1
2
2
1
3
2
3
2
2
1
2
2
3
2 10
5
3
2
1
2
2
1
3
2
3
2
2
1
2
2
3
2
3
1
2
-2
4
2
-1
-1
1
1
1
1
-1
2
1
-1
-1
1
Input Image, f
output
Image, g
Correlation
139

3
2
1
2
2
1
3
2
3
2
2
1
2
2
3
2 10 10
5
3
2
1
2
2
1
3
2
3
2
2
1
2
2
3
2
3
3
1
-3
4
2
-1
-1
1
1
1
1
-1
2
1
-1
-1
1
Input Image, f
output
Image, g
Correlation
140

3
2
1
2
2
1
3
2
3
2
2
1
2
2
3
2 10 10 15
5
3
2
1
2
2
1
3
2
3
2
2
1
2
2
3
2
1
3
3
-1
6
2
-1
-1
1
1
1
1
-1
2
1
-1
-1
1
Input Image, f
output
Image, g
Correlation
141

3
2
1
2
2
1
3
2
3
2
2
1
2
2
3
2 10 10
3
15
5
3
2
1
2
2
1
3
2
3
2
2
1
2
2
3
2
2
2
1
-1
4
1
-2
-2
1
1
1
1
-1
2
1
-1
-1
1
Input Image, f
output
Image, g
Correlation
142

3
2
1
2
2
1
3
2
3
2
2
1
2
2
3
2
4
10 10
3
15
5
3
2
1
2
2
1
3
2
3
2
2
1
2
2
3
2
1
2
2
-3
2
2
-2
-2
2
1
1
1
-1
2
1
-1
-1
1
Input Image, f
output
Image, g
Correlation
143

4
11
4
10
4
4
6
10
11
3
5
-5
9
7
15
5
Final output Image, g
Correlation
144

Weighted Smoothing Filters
• More effective smoothing filters can be generated by allowing different pixels in
the neighbourhood different weights in the averaging function
• Pixels closer to the
central pixel are more
important
• Often referred to as a
weighted averaging
1/16
2/16
1/16
2/16
4/16
2/16
1/16
2/16
1/16
Weighted
averaging filter
145

Another Smoothing Example
By smoothing the original image we get rid of lots of the finer detail
which leaves only the gross features for thresholding
Original Image Smoothed Image Thresholded Image
146

Averaging Filter Vs. Median Filter Example
• Filtering is often used to remove noise from images
• Sometimes a median filter works better than an averaging filter
Original Image
With Noise
Image After
Averaging Filter
Image After
Median Filter
147

148

149

150

123 127 128 119 115 130
140 145 148 153 167 172
133 154 183 192 194 191
194 199 207 210 198 195
164 170 175 162 173 151
x
y
151

Strange Things Happen At The Edges!
Origin x
y Image f (x, y)
e
e
e
e
• At the edges of an image we are missing pixels to form a neighbourhood
e e
e
152

Strange Things Happen At The Edges! (cont…)
There are a few approaches to dealing with missing edge pixels:
• Omit missing pixels
• Only works with some filters
• Can add extra code and slow down processing
• Pad the image
• Typically with either all white or all black pixels
• Replicate border pixels
• Truncate the image
• Allow pixels wrap around the image
• Can cause some strange image artefacts
153

123 127 128 119 115 130
140 145 148 153 167 172
133 154 183 192 194 191
194 199 207 210 198 195
164 170 175 162 173 151
x
y
154

Strange Things Happen At The Edges! (cont…)
Original
Image
Filtered Image:
Zero Padding
Filtered Image:
Replicate Edge Pixels
Filtered Image:
Wrap Around Edge Pixels

Correlation & Convolution
• The filtering we have been talking about so far is referred to as
correlation with the filter itself referred to as the correlation kernel
• Convolution is a similar operation, with just one subtle difference
• For symmetric filters it makes no difference
eprocessed = v*e +
z*a + y*b + x*c +
w*d + u*e +
t*f + s*g + r*h
r s t
u v w
x y z
Filter
a b c
d e e
f g h
Original Image
Pixels
*
156

Convolution
• The equation of correlation is defined as follows :
• It performs cross product between weights and function
157
a b
s a t b
g( x,y ) ω( s,t ) f ( x s,y t )
g ω f
=− =−
= − −
= 
 

Convolution
158
1 -1 -1
1 2 -1
1 1 1
2 2 2 3
2 1 3 3
2 2 1 2
1 3 2 2
Rotate 180o
1
-1
-1
1
2
-1
1
1
1
Convolution kernel, ω

Convolution
Input Image, f
3
2
1
2
2
1
3
2
3
2
2
1
2
2
3
2 5
3
2
1
2
2
1
3
2
3
2
2
1
2
2
3
2
1
-2
-1
2
4
-1
1
1
1
1
-1
-1
1
2
-1
1
1
1
Output
Image, g
159

3
2
1
2
2
1
3
2
3
2
2
1
2
2
3
2 4
5
3
2
1
2
2
1
3
2
3
2
2
1
2
2
3
2
3
-1
-2
2
4
-2
1
1
1
1
-1
-1
1
2
-1
1
1
1
Input Image, f
Output
Image, g
Convolution
160

3
2
1
2
2
1
3
2
3
2
2
1
2
2
3
2 4 4
5
3
2
1
2
2
1
3
2
3
2
2
1
2
2
3
2
3
-3
-1
3
4
-2
1
1
1
1
-1
-1
1
2
-1
1
1
1
Input Image, f
Output
Image, g
Convolution
161

3
2
1
2
2
1
3
2
3
2
2
1
2
2
3
2 4 4 -2
5
3
2
1
2
2
1
3
2
3
2
2
1
2
2
3
2
1
-3
-3
1
6
-2
1
1
1
1
-1
-1
1
2
-1
1
1
1
Input Image, f
Output
Image, g
Convolution
162

3
2
1
2
2
1
3
2
3
2
2
1
2
2
3
2 4 4
9
-2
5
3
2
1
2
2
1
3
2
3
2
2
1
2
2
3
2
2
-2
-1
1
4
-1
2
2
1
1
-1
-1
1
2
-1
1
1
1
Input Image, f
Output
Image, g
Convolution
163

3
2
1
2
2
1
3
2
3
2
2
1
2
2
3
2
6
4 4
9
-2
5
3
2
1
2
2
1
3
2
3
2
2
1
2
2
3
2
1
-2
-2
3
2
-2
2
2
2
1
-1
-1
1
2
-1
1
1
1
Input Image, f
Output
Image, g
Convolution
164

12
7
6
4
8
6
14
4
5
9
5
9
5
11
-2
5
Final output Image, g
Convolution
165

Image Enhancement
Fourier transformation
PREPARED BY:

Filter and Frequency
• Frequency – The number of times that a periodic function repeats the same
sequence of values during a unit variation of the independent variable.
• Filter – A device or material for suppressing or minimizing waves or oscillations
of certain frequencies.
167

The Big Idea
=
• Any function that periodically repeats itself can be
expressed as a sum of sines and cosines of
different frequencies each multiplied by a different
coefficient – a Fourier series
168

The Big Idea (cont…)
Notice how we get closer and closer to the original function as we add more
and more frequencies
169

The Big Idea (cont…) Frequency
domain signal
processing
example in Excel
170

Complex Numbers
• A complex number, C, is defined as
C = R + jI,
where R is real number & I is Imaginary number equal to the square root of
-1.
ie. j = 1
−
171

The Discrete Fourier Transform (DFT)
• The Discrete Fourier Transform of f(x, y), for x = 0, 1, 2…M-1 and y = 0,1,2…N-1,
denoted by F(u, v), is given by the equation:
for u = 0, 1, 2…M-1 and v = 0, 1, 2…N-1.

−
=
−
=
+
−
=
1
0
1
0
)
/
/
(
2
)
,
(
)
,
(
M
x
N
y
N
vy
M
ux
j
e
y
x
f
v
u
F 
172

DFT & Images
• The DFT of a two dimensional image can be visualised by showing the
spectrum of the image component frequencies
DFT
173

DFT & Images
174

DFT & Images
175

DFT & Images (cont…)
DFT
Scanning electron microscope
image of an integrated circuit
magnified ~2500 times
Fourier spectrum of the image
176

177

178

The Inverse DFT
• It is really important to note that the Fourier transform is completely reversible
• The inverse DFT is given by:
for x = 0, 1, 2…M-1 and y = 0, 1, 2…N-1

−
=
−
=
+
=
1
0
1
0
)
/
/
(
2
)
,
(
1
)
,
(
M
u
N
v
N
vy
M
ux
j
e
v
u
F
MN
y
x
f 
179

The DFT and Image Processing
To filter an image in the frequency domain:
1. Compute F(u,v) the DFT of the image
2. Multiply F(u,v) by a filter function H(u,v)
3. Compute the inverse DFT of the result
180

Some Basic Frequency Domain Filters
Low Pass Filter
High Pass Filter
181

Frequency Domain Methods
Spatial Domain Frequency Domain
182

Major filter categories
• Typically, filters are classified by examining their
properties in the frequency domain:
(1) Low-pass
(2) High-pass
(3) Band-pass
(4) Band-stop
183

Example
Original signal
Low-pass filtered
High-pass filtered
Band-pass filtered
Band-stop filtered
184

Low-pass filters
(i.e., smoothing filters)
• Preserve low frequencies - useful for noise suppression
frequency domain time domain
Example:
185

High-pass filters
(i.e., sharpening filters)
• Preserves high frequencies - useful for edge detection
frequency
domain
time
domain
Example:
186

Band-pass filters
• Preserves frequencies within a certain band
frequency
domain
time
domain
Example:
187

Band-stop filters
• How do they look like?
Band-pass Band-stop
188

Frequency Domain Methods
Case 1: H(u,v) is specified in
the frequency domain.
Case 2: h(x,y) is specified in
the spatial domain.
(real)
189

Frequency domain filtering: steps
F(u,v) = R(u,v) + jI(u,v)
190

Frequency domain filtering: steps (cont’d)
G(u,v)= F(u,v)H(u,v) = H(u,v) R(u,v) + jH(u,v)I(u,v)
(Case 1)
191

Example
f(x,y) fp(x,y) fp(x,y)(-1)x+y
F(u,v)
H(u,v) - centered G(u,v)=F(u,v)H(u,v)
g(x,y)
gp(x,y)
192

CV_2 Image Processing

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to CV_2 Image Processing

Similar to CV_2 Image Processing (20)

More from Khushali Kathiriya

More from Khushali Kathiriya (20)

Recently uploaded

Recently uploaded (20)

CV_2 Image Processing