Slides of pattern recognition Course of Professor Zohreh Azimifar at Shiraz University. اسلاید های درس شناسایی آماری الگو استاد زهره عظیمی فر در دانشگاه شیراز.

1. Introduction
Definition of Pattern Recognition
Pattern Recognition Applications
Definition of Object, Feature, and Pattern
Definition of Feature Space
Data Preprocessing and Normalization
Feature Extraction and Selection
Train and Test
Different Learning Techniques
Supervised Learning
Lecture Summary
Statistical Pattern Recognition
Lecture1
Dr Zohreh Azimifar
School of Electrical and Computer Engineering
Shiraz University
September 20, 2014
Dr Zohreh Azimifar, 2014 Statistical Pattern Recognition Lecture1 1 / 18

2. Introduction
Definition of Pattern Recognition
Pattern Recognition Applications
Definition of Object, Feature, and Pattern
Definition of Feature Space
Data Preprocessing and Normalization
Feature Extraction and Selection
Train and Test
Different Learning Techniques
Supervised Learning
Lecture Summary
Table of contents
1 Introduction
2 Definition of Pattern Recognition
3 Pattern Recognition Applications
4 Definition of Object, Feature, and Pattern
5 Definition of Feature Space
6 Data Preprocessing and Normalization
7 Feature Extraction and Selection
8 Train and Test
9 Different Learning Techniques
10 Supervised Learning
11 Lecture Summary
Dr Zohreh Azimifar, 2014 Statistical Pattern Recognition Lecture1 2 / 18

3. Introduction
Definition of Pattern Recognition
Pattern Recognition Applications
Definition of Object, Feature, and Pattern
Definition of Feature Space
Data Preprocessing and Normalization
Feature Extraction and Selection
Train and Test
Different Learning Techniques
Supervised Learning
Lecture Summary
Introduction
Dr Zohreh Azimifar, 2014 Statistical Pattern Recognition Lecture1 3 / 18

4. Introduction
Definition of Pattern Recognition
Pattern Recognition Applications
Definition of Object, Feature, and Pattern
Definition of Feature Space
Data Preprocessing and Normalization
Feature Extraction and Selection
Train and Test
Different Learning Techniques
Supervised Learning
Lecture Summary
Definition of Pattern Recognition
Pattern Recognition is a subset of artificial intelligence, in which
intelligent softwares and hardwares are designed to mimic human
thinking and behaviour. Such systems are capable of recognizing,
describing, classifying and predicting unseen datum.
Pattern Recognition is the best possible venue to make an automatic
decision by using available sensors, processors and knowledge.
Well-known concepts from statistical decision theory are unitized to
compute decision boundaries between pattern classes.
Dr Zohreh Azimifar, 2014 Statistical Pattern Recognition Lecture1 4 / 18

5. Introduction
Definition of Pattern Recognition
Pattern Recognition Applications
Definition of Object, Feature, and Pattern
Definition of Feature Space
Data Preprocessing and Normalization
Feature Extraction and Selection
Train and Test
Different Learning Techniques
Supervised Learning
Lecture Summary
Pattern Recognition Applications
Table: Pattern Recognition Applications
Problem Domain Application Input Pattern Pattern Class
Bioinformatics Sequence Analysis DNA/Protein Known types of
sequence genes/patterns
Searching for Points in Compact and
Data Mining meaningful patterns multidimensional well-separated
space clusters
Document Internest Search Text document Semantic categories
Classification (e.g., sport)
Industrial Printed circuit Intensity or Defective /
Automation board inspection range image non-defective nature
of product
Biometric Personal identification Video clip Action, posture
Recognition
Speech Telephone dictionary Speech waveform Spoken words
Recognition
Dr Zohreh Azimifar, 2014 Statistical Pattern Recognition Lecture1 5 / 18

6. Introduction
Definition of Pattern Recognition
Pattern Recognition Applications
Definition of Object, Feature, and Pattern
Definition of Feature Space
Data Preprocessing and Normalization
Feature Extraction and Selection
Train and Test
Different Learning Techniques
Supervised Learning
Lecture Summary
Definition of Object, Feature, and Pattern
Object is defined by a set of characters and measures.
Feature is defined as a measurement (numeric or nominal) obtained
from the object and is viewed in a d-dimensional space.
Observation is an example of feature vector obtained from a given
object.
Pattern is defined as opposite of chaos; an identity the can be given
a name (Watanabe:1985).
Dr Zohreh Azimifar, 2014 Statistical Pattern Recognition Lecture1 6 / 18

7. Introduction
Definition of Pattern Recognition
Pattern Recognition Applications
Definition of Object, Feature, and Pattern
Definition of Feature Space
Data Preprocessing and Normalization
Feature Extraction and Selection
Train and Test
Different Learning Techniques
Supervised Learning
Lecture Summary
Definition of Object, Feature, and Pattern
Figure: Each fruit is an object, and every object is characterized by a feature
vector, including color and weight. Feature vector is called a pattern, which can
be labelled. Here labels are Banana and Orange.
Dr Zohreh Azimifar, 2014 Statistical Pattern Recognition Lecture1 7 / 18

8. Introduction
Definition of Pattern Recognition
Pattern Recognition Applications
Definition of Object, Feature, and Pattern
Definition of Feature Space
Data Preprocessing and Normalization
Feature Extraction and Selection
Train and Test
Different Learning Techniques
Supervised Learning
Lecture Summary
Definition of Feature Space
Patterns are represented by feature vectors.
Each feature element is considered as a dimension is a space.
Feature space is a multidimensional space whose size is defined by
the number of feature elements in the pattern’s feature vector.
Dr Zohreh Azimifar, 2014 Statistical Pattern Recognition Lecture1 8 / 18

9. Introduction
Definition of Pattern Recognition
Pattern Recognition Applications
Definition of Object, Feature, and Pattern
Definition of Feature Space
Data Preprocessing and Normalization
Feature Extraction and Selection
Train and Test
Different Learning Techniques
Supervised Learning
Lecture Summary
Definition of Feature Space
Table: Features and Labels associated with object and feature vector definition
Color Weight Label
170 230 Banana
110 190 Orange
160 220 Banana
210 180 Banana
125 200 Orange
115 185 Orange
180 240 Banana
130 190 Orange
Dr Zohreh Azimifar, 2014 Statistical Pattern Recognition Lecture1 9 / 18

10. Introduction
Definition of Pattern Recognition
Pattern Recognition Applications
Definition of Object, Feature, and Pattern
Definition of Feature Space
Data Preprocessing and Normalization
Feature Extraction and Selection
Train and Test
Different Learning Techniques
Supervised Learning
Lecture Summary
Definition of Feature Space
Figure: 2D feature space wherein each point represents an individual pattern.
Dr Zohreh Azimifar, 2014 Statistical Pattern Recognition Lecture1 10 / 18

11. Introduction
Definition of Pattern Recognition
Pattern Recognition Applications
Definition of Object, Feature, and Pattern
Definition of Feature Space
Data Preprocessing and Normalization
Feature Extraction and Selection
Train and Test
Different Learning Techniques
Supervised Learning
Lecture Summary
Data Preprocessing and Normalization
The preprocessing module segments the pattern of interest from
background, removes noise, and normalizes the pattern.
A feature vector includes various features characterizing the pattern
from different views (e.g., weight, color, height).
Feature entities appear in different measurement ranges (e.g.,
[−1, 1], [−1000, 1000], [0.500, 0.505]), with different impacts.
Normalization rescales all values of a feature vector to a predefined
range (e.g., [-1,1]). This process leads to an even and fair impact of
all features in the decision stage.
Dr Zohreh Azimifar, 2014 Statistical Pattern Recognition Lecture1 11 / 18

12. Introduction
Definition of Pattern Recognition
Pattern Recognition Applications
Definition of Object, Feature, and Pattern
Definition of Feature Space
Data Preprocessing and Normalization
Feature Extraction and Selection
Train and Test
Different Learning Techniques
Supervised Learning
Lecture Summary
Feature Extraction and Selection
Object is medium which is sensed.
Observation is the measurement gathered by our sensory elements.
Feature is a preprocessed observation ready for modelling the
medium.
Dr Zohreh Azimifar, 2014 Statistical Pattern Recognition Lecture1 12 / 18

13. Introduction
Definition of Pattern Recognition
Pattern Recognition Applications
Definition of Object, Feature, and Pattern
Definition of Feature Space
Data Preprocessing and Normalization
Feature Extraction and Selection
Train and Test
Different Learning Techniques
Supervised Learning
Lecture Summary
Train and Test
The recognition system is operated in two modes: training (learning)
and testing
Training includes: feature extraction/selection for representing input
patterns and a classifier to partition the feature space.
Testing assigns/predict a category to an unseen pattern based on a
vector of feature extracted from it.
Dr Zohreh Azimifar, 2014 Statistical Pattern Recognition Lecture1 13 / 18

14. Introduction
Definition of Pattern Recognition
Pattern Recognition Applications
Definition of Object, Feature, and Pattern
Definition of Feature Space
Data Preprocessing and Normalization
Feature Extraction and Selection
Train and Test
Different Learning Techniques
Supervised Learning
Lecture Summary
Train and Test
Figure: Train: system learns fruits label. Test: system uses its learned
parameters to predict label for unseen patterns.
Dr Zohreh Azimifar, 2014 Statistical Pattern Recognition Lecture1 14 / 18

15. Introduction
Definition of Pattern Recognition
Pattern Recognition Applications
Definition of Object, Feature, and Pattern
Definition of Feature Space
Data Preprocessing and Normalization
Feature Extraction and Selection
Train and Test
Different Learning Techniques
Supervised Learning
Lecture Summary
Train and Test
Figure: Train and Test
Dr Zohreh Azimifar, 2014 Statistical Pattern Recognition Lecture1 15 / 18

16. Introduction
Definition of Pattern Recognition
Pattern Recognition Applications
Definition of Object, Feature, and Pattern
Definition of Feature Space
Data Preprocessing and Normalization
Feature Extraction and Selection
Train and Test
Different Learning Techniques
Supervised Learning
Lecture Summary
Different Learning Techniques
Supervised Learning
Classifier
Regression
Unsupervised Learning
clustering
Density Estimation
Dr Zohreh Azimifar, 2014 Statistical Pattern Recognition Lecture1 16 / 18

17. Introduction
Definition of Pattern Recognition
Pattern Recognition Applications
Definition of Object, Feature, and Pattern
Definition of Feature Space
Data Preprocessing and Normalization
Feature Extraction and Selection
Train and Test
Different Learning Techniques
Supervised Learning
Lecture Summary
Supervised Learning
Figure: General Diagram of Supervised Learning
Dr Zohreh Azimifar, 2014 Statistical Pattern Recognition Lecture1 17 / 18

18. Introduction
Definition of Pattern Recognition
Pattern Recognition Applications
Definition of Object, Feature, and Pattern
Definition of Feature Space
Data Preprocessing and Normalization
Feature Extraction and Selection
Train and Test
Different Learning Techniques
Supervised Learning
Lecture Summary
Lecture Summary
1
2
3
4
5
Dr Zohreh Azimifar, 2014 Statistical Pattern Recognition Lecture1 18 / 18