3. Pattern Classification……By Ranjan Ganguli
3
What is a Pattern Recognition?
• The study of how machines can observe the
environment,
• learn to distinguish patterns of interest from
their background, and
• make sound and reasonable decisions about
the categories of the patterns.
• What is a pattern?
• What kinds of category we have?
4. What is a Pattern?
• “A pattern is essentially an arrangement” ………..(Definition 1)
• “It can also be defined by the common denominator among the
multiple instances of an entity”.
e.g., commonality in all fingerprint images defines the
fingerprint pattern; thus, a pattern could be a fingerprint
image, a handwritten cursive word, a human face, a
speech signal ……………………………...(Definition II)
•For example, a pattern could be
• A fingerprint images
• A handwritten cursive word
• A human face
A speech signal …..etc Pattern Classification……By Ranjan Ganguli
5. What is Pattern Category?
• It is a collection of similar, not necessarily
identical objects. Often, individual patterns may
be grouped into a category based on their
common properties; the resultant is also a
pattern and is often called a pattern category.
Pattern Classification……By Ranjan Ganguli
6. Pattern Recognition System
•The design model of a pattern recognition
system essentially involves the following three
steps:
I. Data acquisition and pre-processing
II. Feature extraction
III. Decision making
Pattern Classification……By Ranjan Ganguli
7. Block diagram of a pattern recognition system:
7
Pattern Classification……By Ranjan Ganguli
8. Pattern Recognition Models
•The four best known approaches
• template matching
• statistical classification
• syntactic or structural matching
• neural networks
Pattern Classification……By Ranjan Ganguli
10. PATTERN RECOGNITION ALGORITHMS
•The design pattern of algorithms consists of three
basic elements, i.e., data perception, feature
extraction and classification.
•Algorithms for pattern recognition depend on the type of
label output, on whether learning is supervised or
unsupervised
Pattern Classification……By Ranjan Ganguli
11. What is a Supervised Learning?
•In supervised learning, there is a teacher
who provides a category label or cost for
each pattern in the training set which is used
as a classifier.
• So basically a supervised learning method
is used for classification purpose.
Pattern Classification……By Ranjan Ganguli
12. • In the given figure, the input image consist of
mixture of two alphabets, i.e., A and B. Then the
classification algorithm classifies the input to two
different categories
Here a set of combined input is classified using
supervised learning approach.
Pattern Classification……By Ranjan Ganguli
13. What is Unsupervised learning?
The system forms clusters or “natural
groupings” of the input patterns
Pattern Classification……By Ranjan Ganguli
14. •Here the input consists of some unlabeled values
whose distinguishing feature is initially not known.
The following input consists of such a combination
with all values technically same but still its clusters
are formed using some metric which is different for
each algorithm
Pattern Classification……By Ranjan Ganguli
15. 15
An Example
• “Sorting incoming Fish on a conveyor according to
species using optical sensing”
Sea bass
Species
Salmon
Pattern Classification……By Ranjan Ganguli
16. 16
• Problem Analysis
• Set up a camera and take some sample images to extract
features
• Length
• Lightness
• Width
• Number and shape of fins
• Position of the mouth, etc…
• This is the set of all suggested features to explore for use in our
classifier!
Pattern Classification……By Ranjan Ganguli
17. 17
• Preprocessing
• Use a segmentation operation to isolate fishes from one
another and from the background
• Information from a single fish is sent to a feature
extractor whose purpose is to reduce the data by
measuring certain features
• The features are passed to a classifier
Pattern Classification……By Ranjan Ganguli
23. 23
• Threshold decision boundary and cost relationship
• Move our decision boundary toward smaller values of
lightness in order to minimize the cost (reduce the number
of sea bass that are classified salmon!)
Task of decision theory
Pattern Classification……By Ranjan Ganguli
24. 24
• Adopt the lightness and add the width of the fish
Fish xT
= [x1, x2]
Lightness Width
Pattern Classification……By Ranjan Ganguli
26. 26
• We might add other features that are not correlated
with the ones we already have. A precaution should
be taken not to reduce the performance by adding
such “noisy features”
• Ideally, the best decision boundary should be the one
which provides an optimal performance such as in the
following figure:
Pattern Classification……By Ranjan Ganguli
28. 28
• However, our satisfaction is premature because
the central aim of designing a classifier is to
correctly classify novel input
Issue of generalization!
Pattern Classification……By Ranjan Ganguli
32. Decision Trees
“Splitting datasets one feature at a time”
The decision tree is one of the most commonly used
classification techniques; recent surveys claim that it’s
the most commonly used technique.
36. 1. To build a decision tree, we need to make a
first decision on the dataset to dictate which feature is
used to split the data.
2. To determine this, we try every feature and measure
which split will give you the best results.
3. After that, we’ll split the dataset into subsets.
4. The subsets will then traverse down the branches of
the first decision node. If the data on the branches is
the same class, then you’ve properly classified it and
Steps:
37. 5. If the data isn’t the same, then we need
to repeat the splitting process on this
subset. The decision on how to split this
subset is done the same way as
the original dataset, and we repeat this
process until we’ve classified all the data.
38. Information gain
•We choose to split our dataset in a way
that makes our unorganized data more
organized. One way to organize this is to
measure the information.
•Using information theory, we can measure
the information before and after the split
•The change in information before and after
the split is known as the information
gain.
39. Note:
Highest information gain helps to
split the data set
The attribute with the highest
information gain is chosen as the
splitting
40. Information gain = Entropy
What is entropy?
Entropy is defined as the expected value
of the information.
(Here, it is measured on each attribute)
41. For entropy to calculate, we need the
expected value of all the information
of all possible values of our class.
This is given by:
