Image restoration and enhancement

1. Selected Topics in
Computer Science
Image Restoration and Enhancement #2
Gera
2020
11

2. Fundamental Steps in Digital Image Processing
2
Image
Acquisition
Image
Restoration
Morphological
Processing
Segmentation
Object
Recognition
Image
Enhancement
Representation
& Description
Problem Domain
Colour Image
Processing
Image
Compression

3. Fundamental Steps in Digital Image Processing
• The diagram does not imply that every process is applied to
an image.
• Rather, the intention is to convey an idea of all the
methodologies that can be applied to images for different
purposes and possibly with different objectives.
3

4. Fundamental Steps in Digital Image Processing
4
Image
Acquisition
Image
Restoration
Morphological
Processing
Segmentation
Object
Recognition
Image
Enhancement
Representation
& Description
Problem Domain
Colour Image
Processing
Image
Compression
Example: Take a picture

5. Fundamental Steps in Digital Image Processing
• Image acquisition is the first process in the above figure.
• Note that acquisition could be as simple as being given an
image that is already in digital form.
• Generally, the image acquisition stage involves
preprocessing, such as scaling.
5

6. Fundamental Steps in Digital Image Processing
6
Image
Acquisition
Image
Restoration
Morphological
Processing
Segmentation
Object
Recognition
Image
Enhancement
Representation
& Description
Problem Domain
Colour Image
Processing
Image
Compression
Example: Change contrast

7. Fundamental Steps in Digital Image Processing
• Image enhancement is the process of manipulating an image
so that the result is more suitable than the original for a
specific application.
• The word specific is important here, because it establishes at
the outset that enhancement techniques are problem
oriented.
• Thus, for example, a method that is quite useful for
enhancing X-ray images may not be the best approach for
enhancing satellite images taken in the infrared band of the
electromagnetic spectrum.
7

8. Fundamental Steps in Digital Image Processing
8
Image
Acquisition
Image
Restoration
Morphological
Processing
Segmentation
Object
Recognition
Image
Enhancement
Representation
& Description
Problem Domain
Colour Image
Processing
Image
Compression
Example: Remove
Noise

9. Fundamental Steps in Digital Image Processing
• Image restoration is an area that also deals with improving
the appearance of an image.
• However, unlike enhancement, which is subjective, image
restoration is objective, in the sense that restoration
techniques tend to be based on mathematical or
probabilistic models of image degradation.
• Enhancement, on the other hand, is based on human
subjective preferences regarding what constitutes a “good”
enhancement result
9

10. Fundamental Steps in Digital Image Processing
10
Image
Acquisition
Image
Restoration
Morphological
Processing
Segmentation
Object
Recognition
Image
Enhancement
Representation
& Description
Problem Domain
Colour Image
Processing
Image
Compression
Extract attributes
useful for
describing image

11. Fundamental Steps in Digital Image Processing
• Morphological processing deals with tools for extracting
image components that are useful in the representation and
description of shape.
• The begins a transition from processes that output images to
processes that output image attributes.
11

12. Fundamental Steps in Digital Image Processing
12
Image
Acquisition
Image
Restoration
Morphological
Processing
Segmentation
Object
Recognition
Image
Enhancement
Representation
& Description
Problem Domain
Colour Image
Processing
Image
Compression
Divide image into
constituent parts

13. Fundamental Steps in Digital Image Processing
• Segmentation procedures partition an image into its constituent
parts or objects.
• In general, autonomous segmentation is one of the most difficult
tasks in digital image processing.
• A rugged segmentation procedure brings the process a long way
toward successful solution of imaging problems that require
objects to be identified individually.
• On the other hand, weak or erratic segmentation algorithms
almost always guarantee eventual failure.
• In general, the more accurate the segmentation, the more likely
recognition is to succeed.
13

14. Fundamental Steps in Digital Image Processing
14
Image
Acquisition
Image
Restoration
Morphological
Processing
Segmentation
Object
Recognition
Image
Enhancement
Representation
& Description
Problem Domain
Colour Image
Processing
Image
Compression
Image regions
transformed
suitable for
computer
processing

15. Fundamental Steps in Digital Image Processing
• Representation and description almost always follow the output of a segmentation stage, which
usually is raw pixel data, constituting either the boundary of a region (i.e., the set of pixels separating
one image region from another) or all the points in the region itself.
• In either case, converting the data to a form suitable for computer processing is necessary.
• The first decision that must be made is whether the data should be represented as a boundary or as a
complete region.
• Boundary representation is appropriate when the focus is on external shape characteristics, such as
corners and inflections.
• Regional representation is appropriate when the focus is on internal properties, such as texture or
skeletal shape.
• In some applications, these representations complement each other.
• Choosing a representation is only part of the solution for transforming raw data into a form suitable for
subsequent computer processing.
• A method must also be specified for describing the data so that features of interest are highlighted.
• Description, also called feature selection, deals with extracting attributes that result in some
quantitative information of interest or are basic for differ entiating one class of objects from another.
15

16. Fundamental Steps in Digital Image Processing
16
Image
Acquisition
Image
Restoration
Morphological
Processing
Segmentation
Object
Recognition
Image
Enhancement
Representation
& Description
Problem Domain
Colour Image
Processing
Image
Compression
Finds & Labels
objects in scene
(e.g. motorbike)

17. Fundamental Steps in Digital Image Processing
• Recognition is the process that assigns a label (e.g., “vehicle”) to an object based on its
descriptors.
• So far we have said nothing about the need for prior knowledge or about the interaction between
the knowledge base and the processing modules.
• Knowledge about a problem domain is coded into an image processing system in the form of a
knowledge database.
• This knowledge may be as simple as detailing regions of an image where the information of
interest is known to be located, thus limiting the search that has to be conducted in seeking that
information.
• The knowledge base also can be quite complex, such as an interrelated list of all major possible
defects in a materials inspection problem or an image database containing high-resolution
satellite images of a region in connection with change-detection applications.
• In addition to guiding the operation of each processing module, the knowledge base also controls
the interaction between modules.
17

18. Fundamental Steps in Digital Image Processing
18
Image
Acquisition
Image
Restoration
Morphological
Processing
Segmentation
Object
Recognition
Image
Enhancement
Representation
& Description
Problem Domain
Colour Image
Processing
Image
Compression
Reduce image size
(e.g. JPEG)

19. Fundamental Steps in Digital Image Processing
• Compression, as the name implies, deals with techniques for reducing
the storage required to save an image, or the bandwidth required to
transmit it.
• Although storage technology has improved significantly over the past
decade, the same cannot be said for transmission capacity.
• This is true particularly in uses of the Internet, which are
characterized by significant pictorial content.
• Image compression is familiar (perhaps inadvertently) to most users
of computers in the form of image file extensions, such as the jpg file
extension used in the JPEG (Joint Photographic Experts Group) image
compression standard.
19

20. Fundamental Steps in Digital Image Processing
20
Image
Acquisition
Image
Restoration
Morphological
Processing
Segmentation
Object
Recognition
Image
Enhancement
Representation
& Description
Problem Domain
Colour Image
Processing
Image
Compression
Consider color
images (color
models, etc)

21. Fundamental Steps in Digital Image Processing
•Color image processing is an area that has been
gaining in importance because of the significant
increase in the use of digital images over the
Internet.
21

22. Image Restoration
• As in image enhancement, the principal goal of restoration
techniques is to improve an image in some predefined sense.
• Although there are areas of overlap, image enhancement is
largely a subjective process, while image restoration is for the
most part an objective process.
• Restoration attempts to recover an image that has been
degraded by using a priori knowledge of the degradation
phenomenon.
• Thus, restoration techniques are oriented toward modeling the
degradation and applying the inverse process in order to
recover the original image.
22

23. A Model of the Image Degradation
•A model of the image degradation/restoration
process.
23

24. A Model of the Image Degradation
• The degradation process is modeled as a degradation function that,
together with an additive noise term, operates on an input image
𝑓(𝑥, 𝑦) to produce a degraded image 𝑔(𝑥, 𝑦).
• Given 𝑔(𝑥, 𝑦), some knowledge about the degradation function H,
and some knowledge about the additive noise term 𝑛(𝑥, 𝑦), the
objective of restoration is to obtain an estimate መ𝑓(𝑥, 𝑦)of the original
image.
• We want the estimate to be as close as possible to the original input
image and, in general, the more we know about 𝐻 and 𝑛, the closer
መ𝑓(𝑥, 𝑦) will be to 𝑓(𝑥, 𝑦).
24

25. Noise Models
• The principal sources of noise in digital images arise during image acquisition and/or
transmission.
• The performance of imaging sensors is affected by a variety of factors, such as
environmental conditions during image acquisition, and by the quality of the
sensing elements themselves.
• For instance, in acquiring images with a CCD camera, light levels and sensor
temperature are major factors affecting the amount of noise in the resulting image.
• Images are corrupted during transmission principally due to interference in the
channel used for transmission.
• For example, an image transmitted using a wireless network might be corrupted as a
result of lightning or other atmospheric disturbance.
25

26. Types of Image Noise
• A low-pass filter is designed to emphasize larger,
homogeneous areas of similar tone and reduce
the smaller detail in an image.
• Thus, low-pass filters generally serve to
smooth the appearance of an image.
• Average and median filters are examples of
low-pass filters.
• High-pass filters do the opposite and serve to
sharpen the appearance of fine detail in an image.
• One implementation of a high-pass filter first
applies a low-pass filter to an image and then
subtracts the result from the original, leaving
behind only the high spatial frequency
information.
26

27. Types of Image Noise…
• Speckle noise
• It is a granular 'noise' that inherently exists in and
degrades the quality of the image
• It is caused by coherent processing of
backscattered signals from multiple distributed
targets.
• Speckle noise can be modeled by random values
multiplied by pixel values of an image.
• Periodic noise
• Periodic noise is appearance when signal is subject
to a periodic, rather than a random disturbance.
• Periodic noise in an image arises typically from
electrical or electromechanical interference during
image acquisition.
• Periodic noise can be reduced significantly via
frequency domain filtering
27

28. Spatial Filtering
• A common filtering procedure involves moving a 'window' of a few pixels
in dimension (e.g. 3x3, 5x5, etc.) over each pixel in the image, applying a
mathematical calculation using the pixel values under that window, and
replacing the central pixel with the new value.
• The window is moved along in both the row and column dimensions
one pixel at a time and the calculation is repeated until the entire image
has been filtered and a "new" image has been generated.
• By varying the calculation performed and the weightings of the
individual pixels in the filter window, filters can be designed to enhance
or suppress different types of features.
28

29. Spatial Filtering
• The term spatial domain refers to the image plane itself, and
image processing methods in this category are based on direct
manipulation of pixels in an image.
• Filtering is a technique for modifying or enhancing an image.
• Spatial filtering encompasses a set of digital processing
functions which are used to enhance the appearance of an
image.
• Spatial filters are designed to highlight or suppress specific
features in an image based on their spatial frequency.
29

30. Spatial Filtering…
• A low-pass filter is designed to emphasize larger, homogeneous
areas of similar tone and reduce the smaller detail in an image.
• Thus, low-pass filters generally serve to smooth the
appearance of an image.
• Average and median filters are examples of low-pass filters.
• High-pass filters do the opposite and serve to sharpen the
appearance of fine detail in an image.
• One implementation of a high-pass filter first applies a low-
pass filter to an image and then subtracts the result from the
original, leaving behind only the high spatial frequency
information.
30

31. Noise Filtering Techniques
• Minimum filtering: current pixel will be replaced by minimum pixel
value of its neighboring pixels
• Maximum filtering: current pixel will be replaced by maximum pixel
value of its neighboring pixels
• Mean filtering: current pixel will be replaced by average of its
neighboring pixel values
• Median filtering: current pixel will be replaced by median or middle
pixel value of its neighboring pixels
• New generated filtering: current pixel will be replaced by arithmetic
mean of mid-1, mid, mid+1 of its neighboring pixels
31

32. Median Filtering
• In signal processing, it is often desirable to be able to perform some kind of noise
reduction on an image or signal. Median filtering is very widely used in digital
image processing because it preserves edges while removing noise.
• The median filter is a sliding-window spatial filter.
• It replaces the value of the center pixel with the median of the intensity values in the
neighborhood of that pixel.
• Median filtering is a nonlinear operation often used in image processing to reduce
"salt and pepper" noise because of its appearance as white and black dots
superimposed on an image.
• A median filter is more effective than convolution when the goal is to simultaneously
reduce noise and preserve edges.
• For every pixel, a 3x3 neighborhood with the pixel as center is considered. In median
filtering, the value of the pixel is replaced by the median of the pixel values in the 3x3
neighborhood.
32

33. Median Filtering…
•Example: The resulting image after applying median
filtering
33

34. Median Filtering…
Exercise 1
1.What is the value of the yellow box after
median filtering?
Answers: a) 7 b) 1 c) 3 d) 9
34

35. Median Filtering…
Exercise 2
• What are the values of yellow boxes after median filtering?
Answers:
a) b)
35
c)

36. Median Filtering…
• What’s the kernel for a 3x3 mean filter?
36
0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 0 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 0 0 0 0 0 0 0
0 0 90 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0

37. Image Enhancement
• Image enhancement techniques can be divided into two broad
categories:
• Spatial domain methods, which operate directly on pixels
• Frequency domain methods, which operate on the Fourier
transform of an image.
• Unfortunately, there is no general theory for determining what is
`good' image enhancement when it comes to human perception. If it
looks good, it is good!
• However, when image enhancement techniques are used as pre-
processing tools for other image processing techniques, then
quantitative measures can determine which techniques are most
appropriate.
37

38. Image Enhancement
38
Neighbourhood operations
Sharpening / blurring
Edge detection
-1 -1 -1
-1 9 -1
-1 -1 -1
Result
1 1 1
1 1 1
1 1 1
1 0 -1
1 0 -1
1 0 -1
111
000
-1-1-1
Image
• Contract enhancement
• Thresholding
• Histogram equalisation

39. Morphological Processing
• Morphology is a broad set of image processing operations that
process images based on shapes. Morphological operations apply a
structuring element to an input image, creating an output image of
the same size.
• In a morphological operation, the value of each pixel in the output
image is based on a comparison of the corresponding pixel in the
input image with its neighbors. By choosing the size and shape of
the neighborhood, you can construct a morphological operation
that is sensitive to specific shapes in the input image.
• The most basic morphological operations are dilation and erosion.
• Dilation adds pixels to the boundaries of objects in an image, while
erosion removes pixels on object boundaries.
• The number of pixels added or removed from the objects in an
image depends on the size and shape of the structuring
element used to process the image.
39

40. Morphological Processing
• Morphological operators often take a binary image and a structuring
element as input and combine them using a set operator (intersection,
union, inclusion, complement).
• The structuring element is shifted over the image and at each pixel of the
image its elements are compared with the set of the underlying pixels.
• If the two sets of elements match the condition defined by the set
operator (e.g. if set of pixels in the structuring element is a subset of
the underlying image pixels), the pixel underneath the origin of the
structuring element is set to a pre-defined value (0 or 1 for binary
images).
• A morphological operator is therefore defined by its structuring
element and the applied set operator.
40

41. Morphological Processing (Application)
• Image pre-processing: noise filtering, shape simplification
• Enhancing object structures: skeletonization, thinning, convex hull, object
marking
• Segmentation of the object from background
• Morphological operations are used as a step in image processing and
analysis.
• It is used to modify the shape of objects in an image, by using local
operations.
• It can be used to remove unwanted effects in segmentation post-
processing
• Remove small objects (that is assumed to be noise)
• Smooth the edges of larger objects
• Fill holes in objects
• Link objects together
41

42. Morphological Processing
• In the morphological dilation and erosion operations, the state of any given pixel
in the output image is determined by applying a rule to the corresponding pixel
and its neighbors in the input image.
• The rule used to process the pixels defines the operation as a dilation or an
erosion. This table lists the rules for both dilation and erosion.
42
Dilation The value of the output pixel is the maximum value
of all the pixels in the input pixel's neighborhood. In a
binary image, if any of the pixels is set to the value1,
the output pixel is set to 1.
Erosion The value of the output pixel is the minimum value of
all the pixels in the input pixel's neighborhood. In a
binary image, if any of the pixels is set to 0, the
output pixel is set to 0.

43. References
➢T. S. Huang and K. Aizawa, “Image processing: Some challenging
problems,” in Proceedings of the National Academy of Sciences of
the United States of America, 1993, vol. 90, no. 21, pp. 9766–9769.
➢E. Miranda, M. Aryuni, and E. Irwansyah, “A survey of medical image
classification techniques,” Proc. 2016 Int. Conf. Inf. Manag. Technol.
ICIMTech 2016, November, pp. 56–61, 2017.
➢R. C. Gonzalez and R. E. Woods, Digital Image Processing, 3rd ed., no.
September. Upper Saddle River, New Jersey: Pearson Education, Inc,
2015.
➢Emmanuel Agu, “Digital Image Processing,” Worcester Polytechnic
Institiute.
43
Gerabirhan Paulos
ToCourseInfo@gmail.com