The document discusses various feature types and selection methods used in pattern recognition, detailing template matching, structural decomposition, and series expansion as techniques for feature extraction. It also covers different classifier types, including statistical methods, neural networks, and rule-based methods, highlighting their applications in recognizing and classifying patterns. Key processes in a pattern recognition system include data acquisition, pre-processing, feature extraction, model learning, classification, and optional post-processing to enhance performance.

1. ERI SUMMER TRAINING
COMPUTERS & SYSTEMS DEPT.
Dr. Randa ElanwarLecture 4

2. Content
Feature types
Feature selection methods
Classifier types
Template Matching
Statistical methods
2
ERI Summer training (C&S) Dr. Randa Elanwar
Statistical methods
Stochastic Processes
Structural Matching
Neural Networks
Rule-based methods
The real pattern recognition system

3. Feature types
3
After having the big picture about how classification
is done, we need to discuss some points in more
details to have deeper understanding
Features Types
Three different groups of feature extraction techniques
exist: templates, structural decomposition and series
expansion.
ERI Summer training (C&S) Dr. Randa Elanwar

4. Feature types
1. Template Matching, i.e. using a template (as a mask) to
compute certain features. For example:
4
ERI Summer training (C&S) Dr. Randa Elanwar
using a grid to compute the ratio of foreground black
pixels to the back ground white pixels, or the count of
black pixels per column, or the number of connected
pixels. The template can be circular mask with sectors and
the foreground black pixels area could be the feature,
and so on.

5. Feature types
5
Template Matching is very sensitive to pattern/sample
size, rotation and translation variations as well as to
noise. I.e., two typical patterns may have different
feature vectors if one of them is bigger in size, rotated,
or translated a little. Thus, it has limited application.
2. Structural Decomposition i.e., describing the geometry
of the pattern shape. For example
(1) knowledge about the contour of the object like height
contour and chain code features,
(2) knowledge about what sort of components make up that
object like end points, T-joints and X-joints, loops, curves, etc.
ERI Summer training (C&S) Dr. Randa Elanwar

6. Feature types
3. Series Expansion i.e., representing a signal by a linear
combination of a series of simpler well-defined functions,
where the coefficients of the linear combination are
considered as features.
If these functions are orthogonal to each other, then the
6
ERI Summer training (C&S) Dr. Randa Elanwar
If these functions are orthogonal to each other, then the
coefficients provide non-redundant (distinct) information
(features) like: Moments, Fourier transform, Gabor transform
and wavelets.
There is an infinite number of potential features that one can
extract from a finite 2D pattern. However, best features are
those relevant to the problem domain.

7. Feature types
Usually, features used are selected and tested in groups till
the best group of features leading to the best classification
results are selected eventually.
As we mentioned before, increasing the number of features
to over fit the data classification in the feature space is a
7
ERI Summer training (C&S) Dr. Randa Elanwar
to over fit the data classification in the feature space is a
wrong practice because once we have a new set of data the
decision boundary will not be able to classify them correctly.
Moreover, as we illustrated, increasing features more than
needed will lead to introducing noise to the system and
cause samples overlap in the feature space.

8. Feature selection methods
8
At this stage, we are not going to go into deep
details describing the features selection techniques
but for knowledge these are the techniques names
only:
Suboptimal solution sets , Sequential forward selection
(SFS), Generalized SFS, Sequential backward selection
(SBS), Generalized SBS, etc.
ERI Summer training (C&S) Dr. Randa Elanwar

9. Feature selection methods
9
All of them go around the same idea either
initialized by a small set of features and adding to
it gradually or initializing by all features in one set
and taking away from it gradually according to the
calculation of probability of correct classification.calculation of probability of correct classification.
Now it's time to have a closer look to other
classifiers than NN for broader vision of machine
learning.
ERI Summer training (C&S) Dr. Randa Elanwar

10. Classifier types
As we mentioned before, recognition involves classifying
each unknown object into one of a finite number of
categories (or classes). There are different methods of
classification to implement the above recognition
approaches:
Template Matching.
10
ERI Summer training (C&S) Dr. Randa Elanwar
Template Matching.
Statistical methods.
Stochastic Processes (Markov Chains).
Structural Matching (Trees, Chains, etc.).
Neural Networks.
Rule-based methods.
Many recent methods mix several techniques together in
order to obtain improved reliability.

11. Template Matching
11
determine the degree of similarity between two
vectors (Groups of pixels, shapes, curvatures, etc) in
the feature space using some minimum distance
approach (e.g. Euclidean distance).
Examples: Direct Template Matching, String
Matching, Elastic Template Matching.
ERI Summer training (C&S) Dr. Randa Elanwar

12. Statistical methods
12
concerned with statistical decision functions and a
set of optimal criteria, which determine the
probability of the observed pattern belonging to a
certain class.
Examples: k-Nearest-Neighbor (k-NN), and
Bayesian Classifier (Maximum likelihood).
ERI Summer training (C&S) Dr. Randa Elanwar

13. Stochastic Processes
is a doubly stochastic process (probabilistic models)
offered for modeling patterns. One of the most
important properties is the existence of efficient
algorithms to automatically train the models without
any need of segmenting patterns.
13
ERI Summer training (C&S) Dr. Randa Elanwar
any need of segmenting patterns.
Example: Hidden Markov Model (HMM)

14. Structural Matching
14
patterns are represented as unions of smaller sub-
patterns (primitives). It can find the relationship
among them.
Examples: grammatical methods and graphical
methods.
ERI Summer training (C&S) Dr. Randa Elanwar

15. Rule-based methods
Sometimes called "Ad-hoc rules", they use abstract descriptions of
patterns to recognize them.
For example, a rule of recognizing the letter ‘x’ might be ‘two lines
that cross over and are at approximately 45 and 135 degrees’.
15
ERI Summer training (C&S) Dr. Randa Elanwar
The problem with these methods is that it is not possible to design an
exhaustive set of rules that model all possible ways of forming a
letter. They are still not as good as the performance of statistical
methods.
They are however very useful in disambiguating between certain
class pairs using very few parameters. One example is the
disambiguation between a ‘v’ and ‘u’ based on the presence or
absence of
a hook at the end

16. The real pattern recognition system
16
ERI Summer training (C&S) Dr. Randa Elanwar

17. The real pattern recognition system
1. Data acquisition and sensing:
Measurements of physical variables (capturing image,
recording speech or video etc.)
Important factors (parameters) in this step that affects the
patterns due to data acquisition: bandwidth, resolution,
17
ERI Summer training (C&S) Dr. Randa Elanwar
patterns due to data acquisition: bandwidth, resolution,
sensitivity, distortion, SNR, latency, etc.
Data collected is divided to 3 sections (train, test,
validation). The training data is that used for learning
models. From time to time you need to make a test to adjust
the system parameters (thresholds) or rules. In such case,
validation data is used. When the system performs quite
well (without over fitting), test section is used for final
evaluation.

18. The real pattern recognition system
18
2. Pre-processing:
an essential task that precedes the tasks of image representation
and recognition. Its importance is derived from the fact that the
discrimination power is directly proportional to the digital image
quality, in the sense that, the higher the image quality the less
confusions we have and thus more powerful classification we can
make.make.
Some common operations performed as preprocessing are:
binarization, the task of converting gray-scale image into a
binary black-white image; noise removal, the extraction of the
foreground textual matter by removing textured background, salt
and pepper noise or interfering strokes; image enhancement and
restoration, the task of converting the image to be more suitable
than the original image for a specific application ; morphological
image processing, the task of extracting image components that
are useful in the representation and description of region shape.
ERI Summer training (C&S) Dr. Randa Elanwar

19. The real pattern recognition system
The basic morphological algorithms are: boundary
extraction; region filling; extraction of connected components;
thinning and thickening.
And the most common morphological operations used are
19
ERI Summer training (C&S) Dr. Randa Elanwar
And the most common morphological operations used are
dilation, where the value of the output pixel is the maximum
value of all the pixels in the input pixel's neighborhood;
erosion, where the value of the output pixel is the minimum
value of all the pixels in the input pixel's neighborhood;
opening, which smoothes contours and eliminates small
islands and sharp peaks; and closing, which smoothes
contours, fuses narrow breaks and eliminates small holes.

20. The real pattern recognition system
20
ERI Summer training (C&S) Dr. Randa Elanwar

21. The real pattern recognition system
Segmentation of pattern
image is sometimes
considered as
preprocessing operation
and sometimes is a
separate stage.
21
ERI Summer training (C&S) Dr. Randa Elanwar
separate stage.
3. Feature extraction:
Finding a new
representation in terms of
features

22. The real pattern recognition system
22
4. Model learning and estimation:
Learning a mapping between features and pattern groups
and categories.
5. Classification:
Using features and learned models to assign a pattern to a
class.
6. Post-processing (extra optional stage to improve
performance):
Evaluation of confidence in decisions.
Exploitation of context to improve performance.
Combination of multiple classifiers decisions.
ERI Summer training (C&S) Dr. Randa Elanwar