Filtering In images(Basics)

1. • Role of Artificial Intelligence and Image
Processing in Computer Vision
• Industrial Machine Vision Applications
• System Architecture
• State of Art
CHAPTER 1

2. 2
Introduction
• What is computer vision?
The analysis of digital images by a computer

3. 3
The Three Stages of Computer Vision

7. 7
Low-Level
blurring
sharpening

8. 8
Low-Level
Canny
ORT
Mid-Level
original image edge image
edge image circular arcs and line segments
data
structure

9. 9
Mid-level
K-means
clustering
original color image regions of homogeneous color
(followed by
connected
component
analysis)
data
structure

10. 10
edge image
consistent
line clusters
low-level
mid-level
high-level
Low- to High-Level
Building Recognition

11. System Architecture
Typical Computer Vision System. The figure shows the basic components of a
computer vision system. The left most component is the scene or object of study. The
next required component shown in the figure is the sensing device used to collect
data from the scene. The third component is the computation device. The device
computes information such as visual cues and reasons on this information to generate
interpretations of the scene such as objects present or actions being performed.

12. System Architecture
Typical computer vision system involves three
components,
1. Scene under study
2. Sensing device that can be used to analyze
the scene (used to collect data from the
scene)

13. System Architecture
3. Computational device that can perform the analysis of the scene
based on the data from the sensor. The computation devices
generates two possible forms of data, information such as visual
cues, and interpretations of information such as actions being
performed or the presence of objects. The two forms of data can
each be used to refine the other until the output of the vision
system is computed with a predefined amount of certainty. The
result can be the 3D location for every pixel within and image or the
certainty that a person is performing jumping jacks. With proper
selection of the information and the interpretation algorithm,
computer vision systems can be applied in a large number of
applications.

14. 14
Digital Image Terminology:
0 0 0 0 1 0 0
0 0 1 1 1 0 0
0 1 95 96 94 93 92
0 0 92 93 93 92 92
0 0 93 93 94 92 93
0 1 92 93 93 93 93
0 0 94 95 95 96 95
pixel (with value 94)
its 3x3 neighborhood
• binary image
• gray-scale (or gray-tone) image
• color image
• multi-spectral image
• range image
• labeled image
region of medium
intensity
resolution (7x7)

15. Spatial Filtering

20. Image filtering
Hybrid Images, Oliva et al., http://cvcl.mit.edu/hybridimage.htm
Slides from Steve Seitz and Rick Szeliski

21. Image filtering
Hybrid Images, Oliva et al., http://cvcl.mit.edu/hybridimage.htm

22. What is an image?

23. Images as functions
•We can think of an image as a function, f, from R2
to R:
– f( x, y ) gives the intensity at position ( x, y )
– Realistically, we expect the image only to be defined
over a rectangle, with a finite range:
• f: [a,b]x[c,d]  [0,1]

24. A color image is just three functions pasted together. We
can write this as a “vector-valued” function:
( , )
( , ) ( , )
( , )
r x y
f x y g x y
b x y
 
 
 
  

25. Images as functions

26. What is a digital image?
•We usually work with digital (discrete) images:
– Sample the 2D space on a regular grid
– Quantize each sample (round to nearest integer)
•If our samples are D apart, we can write this as:
f[i ,j] = Quantize{ f(i D, j D) }

29. Spatial Filtering (cont’d)
• The word “filtering” has been borrowed from
the frequency domain.
• Filters are classified as:
– Low-pass (i.e., preserve low frequencies)
– High-pass (i.e., preserve high frequencies)
– Band-pass (i.e., preserve frequencies within a band)
– Band-reject (i.e., reject frequencies within a band)

30. Spatial Filtering – Neighborhood (or
Mask)
• Typically, the neighborhood is rectangular and its size
is much smaller than that of f(x,y)
- e.g., 3x3 or 5x5

31. Filtering noise
•How can we “smooth” away noise in an
image?
0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
0 0 0 100 130 110 120 110 0 0
0 0 0 110 90 100 90 100 0 0
0 0 0 130 100 90 130 110 0 0
0 0 0 120 100 130 110 120 0 0
0 0 0 90 110 80 120 100 0 0
0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0

32. Mean filtering

33. Mean filtering
0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 0 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 0 0 0 0 0 0 0
0 0 90 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
0 10 20 30 30 30 20 10
0 20 40 60 60 60 40 20
0 30 60 90 90 90 60 30
0 30 50 80 80 90 60 30
0 30 50 80 80 90 60 30
0 20 30 50 50 60 40 20
10 20 30 30 30 30 20 10
10 10 10 0 0 0 0 0

34. Cross-correlation filtering
•Let’s write this down as an equation. Assume
the averaging window is (2k+1)x(2k+1):
We can generalize this idea by allowing different weights for
different neighboring pixels:

35. This is called a cross-correlation operation and
written:
H is called the “filter,” “kernel,” or “mask.”
The above allows negative filter indices. When you implement
need to use: H[u+k,v+k] instead of H[u,v]

36. Mean kernel
•What’s the kernel for a 3x3 mean filter?
0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 0 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 0 0 0 0 0 0 0
0 0 90 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0

37. Mean of neighborhood

38. Effect of filter size

39. Mean vs. Gaussian filtering

40. Gaussian filtering
• A Gaussian kernel gives less weight to pixels
further from the center of the window

42. Linear vs Non-Linear
Spatial Filtering Methods
• A filtering method is linear when the output is a weighted sum of
the input pixels.
• Methods that do not satisfy the above property are called non-
linear.
– e.g.,

43. Linear Spatial Filtering Methods
• Main linear spatial filtering methods:
– Correlation
– Convolution

45. Correlation (cont’d)
Often used in applications where
we need to measure the similarity
between images or parts of images
(e.g., template matching).

46. Convolution
• Similar to correlation except that the mask is
first flipped both horizontally and vertically.
Note: if w(i, j) is symmetric, that is w(i, j)=w(-i,-j),
then convolution is equivalent to correlation!
/2 /2
/2 /2
( , ) ( , ) ( , ) ( , ) ( , )
K K
s K t K
g i j w i j f i j w s t f i s j t
 
     

47. Image gradient
•How can we differentiate a digital image F[x,y]?
– Option 1: reconstruct a continuous image, f, then
take gradient
– Option 2: take discrete derivative (finite
difference)

48. Image gradient
It points in the direction of most rapid change in intensity

49. It points in the direction of most rapid change in intensity

52. smoothed – original
(scaled by 4, offset +128)
original smoothed (5x5 Gaussian)
Why does
this work?

53. Laplacian of Gaussian
(2nd derivative operator)
Gaussian delta function

54. More on filters…
•Cross-correlation/convolution is useful for, e.g.,
– Blurring
– Sharpening
– Edge Detection
– Interpolation
•Convolution has a number of nice properties
– Commutative, associative
– Convolution corresponds to product in the Fourier domain
•More sophisticated filtering techniques can often yield superior results for
these and other tasks:
– Polynomial (e.g., bicubic) filters
– Steerable filters
– Median filters
– Bilateral Filters

55. Filters
• We will mainly focus on two types of filters:
– Smoothing (low-pass)
– Sharpening (high-pass)

56. Smoothing Filters (low-pass)
• Useful for reducing noise and eliminating small details.
• The elements of the mask must be positive.
• Sum of mask elements is 1 (after normalization).
Gaussian

57. Smoothing filters – Example
smoothed imageinput image
• Useful for reducing noise and eliminating
small details.

58. Sharpening Filters (high-pass)
• Useful for highlighting fine details.
• The elements of the mask contain both
positive and negative weights.
• Sum of mask elements is 0.
1st derivative
of Gaussian
2nd derivative
of Gaussian

59. Sharpening Filters (cont’d)
• Useful for highlighting fine details.
– e.g., emphasize edges

60. Sharpening Filters - Example
• Note that the response of sharpening might be negative.
• Values must be re-mapped to [0, 255]
sharpened imageinput image

61. Smoothing Filters
• Averaging
• Gaussian
• Median filtering (non-linear)

62. Smoothing Filters: Averaging

63. Smoothing Filters: Averaging (cont’d)
• Mask size determines the degree of smoothing (loss of detail).
3x3 5x5 7x7
15x15 25x25
original

64. Smoothing Filters: Averaging (cont’d)
15 x 15 averaging image thresholding
Example: extract largest, brightest objects

65. Smoothing filters: Gaussian
• The weights are Gaussian samples:
mask size is
a function of σ:
σ = 1.4

66. Smoothing filters: Gaussian (cont’d)
• σ controls the amount of smoothing
• As σ increases, more samples must be obtained to represent
the Gaussian function accurately.
σ = 3

67. Smoothing filters: Gaussian (cont’d)

68. Averaging vs Gaussian Smoothing
Averaging
Gaussian

69. Smoothing Filters: Median Filtering
(non-linear)
• Very effective for removing “salt and pepper” noise (i.e.,
random occurrences of black and white pixels).
averaging
median
filtering

70. Smoothing Filters: Median Filtering (cont’d)
• Replace each pixel by the median in a
neighborhood around the pixel.

71. Sharpening Filters
• Unsharp masking
• High Boost filter
• Gradient (1st derivative)
• Laplacian (2nd derivative)

72. Sharpening Filters: Unsharp Masking
• Obtain a sharp image by subtracting a
lowpass filtered (i.e., smoothed) image
from the original image:
- =
(after contrast
enhancement)

73. Sharpening Filters: High Boost
• Image sharpening emphasizes edges but low frequency
components are lost.
• High boost filter: amplify input image, then subtract a
lowpass image.
(A-1) + =

74. Sharpening Filters: High Boost (cont’d)
• If A=1, we get unsharp masking.
• If A>1, part of the original image is added back to the high
pass filtered image.
• One way to implement high boost filtering is using the
masks below:

75. Sharpening Filters: High Boost (cont’d)
A=1.4 A=1.9

76. Sharpening Filters: Derivatives
• Taking the derivative of an image results in
sharpening the image.
• The derivative of an image (i.e., 2D signal) can
be computed using the gradient.

77. Laplacian
The Laplacian (2nd derivative) is defined as:
(dot product)
Approximate
2nd derivatives:

78. Laplacian (cont’d)
Laplacian Mask
Edges can be found
by detect the zero-crossings

79. Example: Laplacian vs Gradient
Laplacian Sobel

81. FOV (Field of View)

82. FOV (Field of View)

83. FOV (Field of View)