SlideShare a Scribd company logo
Fishers Linear Discriminant for
Dimensionality Reduction
Department of Computer Science and Engineering
National Institute of Technology Silchar, Assam, INDIA
21-11-2021 1
By-
Nurul Amin Choudhury
NIT Silchar - PhD Scholar
Introduction
• Fishers Linear Discriminant (FLD), is a statistical dimensionality reduction principal which is widely popular
for Machine Learning as well as Pattern Recognition techniques.
• This principal is incorporated by the techniques like – Linear Discriminant Analysis (LDA), Normal
Discriminant Analysis (NDA) and many more.
• FLD work better than PCA as it –
• Maximizes the distance between different classes.
• Minimizes the distance within the class.
• It preserves the projection line direction which is very useful for separating different classes.
Goal: To project the dataset with feature set (N) into feature set (M), where [ M <= (N-1) ]
21-11-2021 2
Example Problem
Consider the data of 2 classes as-
Class 1 has 5 Samples (C1): [(4,1),(2,4),(2,3),(3,6),(4,4)]
Class 2 has 5 Samples (C2): [(9,10),(6,8),(9,5),(8,7),(10,8)]
Step 1: Compute the Mean (u) of each class-
ui = (x1 + x2 +….. + xn)/n
Therefore,
u1 = [3 , 3.6]
u2 = [8.4 , 7.6]
Step 2: Compute within-class Scatter matrix (Sw)
Sw= S1+S2
where S1 and S2 are the covariance matrix of class C1 and C2 respectively.
S1 =
1
ℕ 𝒊=𝟏
𝒏
𝒙𝒊 − 𝒖𝟏 𝒙𝒊 − 𝒖𝟏
𝑻 and similarly for S2 as well.
21-11-2021 3
S. No. X Y Class
1 4 1 C1
2 2 4 C1
3 2 3 C1
4 3 6 C1
5 4 4 C1
6 9 10 C2
7 6 8 C2
8 9 5 C2
9 8 7 C2
10 10 8 C2
Table 1. Sample Dataset
S1 = (1/5)* 𝛴 1 -1 -1 0 1
-2.6 0.4 -0.6 2.4 0.4
Cntd…
21-11-2021 4
*
T
(xi - ui)
(xi - ui)T
Going One by one will leads us to-
1
-2.6 *[1 -2.6] =
1 −2.6
−2.6 6.76
1. 2.
-1
0.4 * [-1 0.4] =
1 −0.4
−0.4 0.16
3.
-1
-0.6 * [-1 -0.6] =
1 0.6
0.6 0.36
4.
0
2.4 * [0 2.4] =
0 0
0 5.76
5.
1
0.4 * [1 -0.4] =
1 0.4
0.4 0.16
Adding Equation 1, 2, 3,4, 5 well get
4 −2
−2 13.2
and then dividing each element by (1/5) will lead to S1=
𝟎. 𝟖 − 𝟎. 𝟒
−𝟎. 𝟒 𝟐. 𝟔
Note: We can also do direct matrix
multiplication of the above two
(using Scientific Calc.) matrix. The
result will be same.
21-11-2021 5
Similarly,
S2= [𝟏. 𝟖𝟏 − 𝟎. 𝟎𝟒
-0.04 2.64]
Therefore,
Sw = [2.64 -0.44
-0.44 5.28]
S1=
𝟎. 𝟖 − 𝟎. 𝟒
−𝟎. 𝟒 𝟐. 𝟔
S1=
𝟏
𝟓
∗
𝟒 − 𝟐
−𝟖 𝟏𝟑. 𝟐
So,
Therefore,
Step 3: Now we need to compute between class Scatter matrix (SB).
SB = (u1 – u2) * (u1 – u2)T
SB= [(- 5.4, -4) * (- 5.4, -4)T]
SB= [29.16 21.6
21.6 16.0]
Step 4: Finding the best Projection Vector using Sw and Mean.
𝑣1
𝑣2
= Sw
-1 *(u1 – u2)
Sw
-1 =
𝑎 𝑏
𝑐 𝑑
-1 =
1
𝑎𝑑−𝑏𝑐
𝑑 −𝑏
−𝑐 𝑎
Sw
-1 =
𝟎. 𝟑𝟖𝟒 𝟎. 𝟎𝟑𝟐
𝟎. 𝟎𝟑𝟐 𝟎. 𝟏𝟗𝟐
Cntd…
21-11-2021 6
Therefore our optimal Line of Direction V (optimal projection) is –
V = Sw
-1 * (- 5.4, -4)
V=
𝟎. 𝟑𝟖𝟒 𝟎. 𝟎𝟑𝟐
𝟎. 𝟎𝟑𝟐 𝟎. 𝟏𝟗𝟐
* (- 5.4, -4)
V=
−𝟐. 𝟐𝟎
−𝟎. 𝟗𝟒
Step 5: Finally, we can calculate actual 1D data with the help of –
y1 = VT * C1
T
y2 = VT * C2
T
Cntd…
21-11-2021 7
21-11-2021 8
Cntd…
We know, (C1): [(4,1),(2,4),(2,3),(3,6),(4,4)]
& (C2): [(9,10),(6,8),(9,5),(8,7),(10,8)]
Also, V=
−𝟐. 𝟐𝟎
−𝟎. 𝟗𝟒
Therefore Y1 and Y2 are-
Y1= [-2.20-0.94] *
4 2 2 3 4
1 4 3 6 4
= [-9.74 -8.6 -7.22 -12.24 -12.56]
Y2= [-2.20-0.94] *
9 6 9 8 10
10 8 5 7 8
= [-29.2 -20.72 -24.5 -24.18 -29.52]
Finally, the optimal line of Projection is V=
−𝟐. 𝟐𝟎
−𝟎. 𝟗𝟒
and 1D data of Class 1 and 2 are shown above.
21-11-2021 9
Advantages of FLD
• Minimizes the variance and maximize the class distance between two variables.
• Works better for multi class problem too.
• Can be use for classification task too.
21-11-2021 10
Disadvantages of FLD
• When the discriminative information are not in the means of classes.
• Small sample size problem.
• Not suitable for non linear models
• Works with Labelled Data only (Supervised in nature).
21-11-2021 11
References
[1] Silva, T. S. (2019, January 3). An illustrative introduction to Fisher’s Linear Discriminant. Thalles’ Blog.
https://sthalles.github.io/fisher-linear-discriminant/
[2] Wikipedia contributors. (2021, October 13). Linear discriminant analysis. Wikipedia.
https://en.wikipedia.org/wiki/Linear_discriminant_analysis
[3] Bishop. (2006). Pattern Recognition and Machine Learning | Christopher M. Bishop | Springer. Pattern
Recognition. Retrieved 2021, from https://www.springer.com/gp/book/9780387310732
[4] https://www.csd.uwo.ca/~oveksler/Courses/CS434a_541a/Lecture8.pdf
21-11-2021 12
Thank You

More Related Content

What's hot

Multilayer perceptron
Multilayer perceptronMultilayer perceptron
Multilayer perceptron
omaraldabash
 
Rule based system
Rule based systemRule based system
Rule based system
Dr. C.V. Suresh Babu
 
Neural Networks: Multilayer Perceptron
Neural Networks: Multilayer PerceptronNeural Networks: Multilayer Perceptron
Neural Networks: Multilayer Perceptron
Mostafa G. M. Mostafa
 
Artificial Neural Networks Lect3: Neural Network Learning rules
Artificial Neural Networks Lect3: Neural Network Learning rulesArtificial Neural Networks Lect3: Neural Network Learning rules
Artificial Neural Networks Lect3: Neural Network Learning rules
Mohammed Bennamoun
 
Bayseian decision theory
Bayseian decision theoryBayseian decision theory
Bayseian decision theory
sia16
 
Inductive bias
Inductive biasInductive bias
Inductive bias
swapnac12
 
DBSCAN : A Clustering Algorithm
DBSCAN : A Clustering AlgorithmDBSCAN : A Clustering Algorithm
DBSCAN : A Clustering Algorithm
Pınar Yahşi
 
Kernels and Support Vector Machines
Kernels and Support Vector  MachinesKernels and Support Vector  Machines
Kernels and Support Vector Machines
Edgar Marca
 
Types of clustering and different types of clustering algorithms
Types of clustering and different types of clustering algorithmsTypes of clustering and different types of clustering algorithms
Types of clustering and different types of clustering algorithms
Prashanth Guntal
 
Concept learning
Concept learningConcept learning
Concept learning
Musa Hawamdah
 
Artificial Neural Network
Artificial Neural NetworkArtificial Neural Network
Artificial Neural Network
Atul Krishna
 
Naive Bayes
Naive BayesNaive Bayes
Naive Bayes
CloudxLab
 
Clustering in Data Mining
Clustering in Data MiningClustering in Data Mining
Clustering in Data Mining
Archana Swaminathan
 
Neural networks.ppt
Neural networks.pptNeural networks.ppt
Neural networks.ppt
SrinivashR3
 
Data cleaning-outlier-detection
Data cleaning-outlier-detectionData cleaning-outlier-detection
Data cleaning-outlier-detection
Chathurangi Shyalika
 
2.3 bayesian classification
2.3 bayesian classification2.3 bayesian classification
2.3 bayesian classification
Krish_ver2
 
Logistic regression
Logistic regressionLogistic regression
Logistic regression
YashwantGahlot1
 
Decision tree
Decision treeDecision tree
Decision tree
R A Akerkar
 
Soft computing
Soft computingSoft computing
Soft computing
Dr Sandeep Kumar Poonia
 
Perceptron
PerceptronPerceptron
Perceptron
Nagarajan
 

What's hot (20)

Multilayer perceptron
Multilayer perceptronMultilayer perceptron
Multilayer perceptron
 
Rule based system
Rule based systemRule based system
Rule based system
 
Neural Networks: Multilayer Perceptron
Neural Networks: Multilayer PerceptronNeural Networks: Multilayer Perceptron
Neural Networks: Multilayer Perceptron
 
Artificial Neural Networks Lect3: Neural Network Learning rules
Artificial Neural Networks Lect3: Neural Network Learning rulesArtificial Neural Networks Lect3: Neural Network Learning rules
Artificial Neural Networks Lect3: Neural Network Learning rules
 
Bayseian decision theory
Bayseian decision theoryBayseian decision theory
Bayseian decision theory
 
Inductive bias
Inductive biasInductive bias
Inductive bias
 
DBSCAN : A Clustering Algorithm
DBSCAN : A Clustering AlgorithmDBSCAN : A Clustering Algorithm
DBSCAN : A Clustering Algorithm
 
Kernels and Support Vector Machines
Kernels and Support Vector  MachinesKernels and Support Vector  Machines
Kernels and Support Vector Machines
 
Types of clustering and different types of clustering algorithms
Types of clustering and different types of clustering algorithmsTypes of clustering and different types of clustering algorithms
Types of clustering and different types of clustering algorithms
 
Concept learning
Concept learningConcept learning
Concept learning
 
Artificial Neural Network
Artificial Neural NetworkArtificial Neural Network
Artificial Neural Network
 
Naive Bayes
Naive BayesNaive Bayes
Naive Bayes
 
Clustering in Data Mining
Clustering in Data MiningClustering in Data Mining
Clustering in Data Mining
 
Neural networks.ppt
Neural networks.pptNeural networks.ppt
Neural networks.ppt
 
Data cleaning-outlier-detection
Data cleaning-outlier-detectionData cleaning-outlier-detection
Data cleaning-outlier-detection
 
2.3 bayesian classification
2.3 bayesian classification2.3 bayesian classification
2.3 bayesian classification
 
Logistic regression
Logistic regressionLogistic regression
Logistic regression
 
Decision tree
Decision treeDecision tree
Decision tree
 
Soft computing
Soft computingSoft computing
Soft computing
 
Perceptron
PerceptronPerceptron
Perceptron
 

Similar to Fishers linear discriminant for dimensionality reduction.

coefficient variation
coefficient variationcoefficient variation
coefficient variation
MARIA KATRINA MACAPAZ
 
self learning ppt of EM-1 swami group ppt
self learning ppt of EM-1 swami group pptself learning ppt of EM-1 swami group ppt
self learning ppt of EM-1 swami group ppt
AmolAher20
 
Algebra Lineal con geogebra.pdf
Algebra Lineal con geogebra.pdfAlgebra Lineal con geogebra.pdf
Algebra Lineal con geogebra.pdf
Jesus Fuentes
 
Measures of dispersion unitedworld school of business
Measures of dispersion   unitedworld school of businessMeasures of dispersion   unitedworld school of business
Measures of dispersion unitedworld school of business
Arnab Roy Chowdhury
 
Dispersion
DispersionDispersion
ANOVA - Dr. Manu Melwin Joy - School of Management Studies, Cochin University...
ANOVA - Dr. Manu Melwin Joy - School of Management Studies, Cochin University...ANOVA - Dr. Manu Melwin Joy - School of Management Studies, Cochin University...
ANOVA - Dr. Manu Melwin Joy - School of Management Studies, Cochin University...
manumelwin
 
B.Ed.104 unit4.2-statistics
B.Ed.104 unit4.2-statisticsB.Ed.104 unit4.2-statistics
B.Ed.104 unit4.2-statistics
GANGOTRIROKADE1
 
Measures of dispersion - united world school of business
Measures of dispersion -  united world school of businessMeasures of dispersion -  united world school of business
Measures of dispersion - united world school of business
Unitedworld School Of Business
 
Minor 1 11th paper.pdf
Minor 1 11th paper.pdfMinor 1 11th paper.pdf
Minor 1 11th paper.pdf
ssuser33d0b9
 
K means clustering
K means clusteringK means clustering
K means clustering
Ahmedasbasb
 
Tarea 1 vectores, matrices y determinantes laura montes
Tarea 1   vectores, matrices y determinantes laura montesTarea 1   vectores, matrices y determinantes laura montes
Tarea 1 vectores, matrices y determinantes laura montes
LAURAXIMENAMONTESEST
 
Tugas Calculus : Limit (Hal.8-14) Kelompok 2
Tugas Calculus : Limit (Hal.8-14) Kelompok 2Tugas Calculus : Limit (Hal.8-14) Kelompok 2
Tugas Calculus : Limit (Hal.8-14) Kelompok 2
Polman Negeri Bangka Belitung
 
Tugas Calculus : Limit (Hal. 8-14)
Tugas Calculus : Limit (Hal. 8-14)Tugas Calculus : Limit (Hal. 8-14)
Tugas Calculus : Limit (Hal. 8-14)
cinjy
 
Manual calcuculadora casio fx-991ES PLUS
Manual calcuculadora casio fx-991ES PLUSManual calcuculadora casio fx-991ES PLUS
Manual calcuculadora casio fx-991ES PLUS
Raul Ibañez
 
summary statistics
summary statisticssummary statistics
summary statistics
ClarissaCambosaReyes
 
Quantitative techniques for MBA (QDM)
Quantitative techniques for MBA (QDM)Quantitative techniques for MBA (QDM)
Quantitative techniques for MBA (QDM)
npazare
 
DSM.pptx
DSM.pptxDSM.pptx
DSM.pptx
shahzaib601980
 
Central Tendancy.pdf
Central Tendancy.pdfCentral Tendancy.pdf
Central Tendancy.pdf
MuhammadFaizan389
 
第2回 基本演算,データ型の基礎,ベクトルの操作方法
第2回 基本演算,データ型の基礎,ベクトルの操作方法第2回 基本演算,データ型の基礎,ベクトルの操作方法
第2回 基本演算,データ型の基礎,ベクトルの操作方法
Wataru Shito
 
Dispersion uwsb
Dispersion   uwsbDispersion   uwsb
Dispersion uwsb
Arnab Roy Chowdhury
 

Similar to Fishers linear discriminant for dimensionality reduction. (20)

coefficient variation
coefficient variationcoefficient variation
coefficient variation
 
self learning ppt of EM-1 swami group ppt
self learning ppt of EM-1 swami group pptself learning ppt of EM-1 swami group ppt
self learning ppt of EM-1 swami group ppt
 
Algebra Lineal con geogebra.pdf
Algebra Lineal con geogebra.pdfAlgebra Lineal con geogebra.pdf
Algebra Lineal con geogebra.pdf
 
Measures of dispersion unitedworld school of business
Measures of dispersion   unitedworld school of businessMeasures of dispersion   unitedworld school of business
Measures of dispersion unitedworld school of business
 
Dispersion
DispersionDispersion
Dispersion
 
ANOVA - Dr. Manu Melwin Joy - School of Management Studies, Cochin University...
ANOVA - Dr. Manu Melwin Joy - School of Management Studies, Cochin University...ANOVA - Dr. Manu Melwin Joy - School of Management Studies, Cochin University...
ANOVA - Dr. Manu Melwin Joy - School of Management Studies, Cochin University...
 
B.Ed.104 unit4.2-statistics
B.Ed.104 unit4.2-statisticsB.Ed.104 unit4.2-statistics
B.Ed.104 unit4.2-statistics
 
Measures of dispersion - united world school of business
Measures of dispersion -  united world school of businessMeasures of dispersion -  united world school of business
Measures of dispersion - united world school of business
 
Minor 1 11th paper.pdf
Minor 1 11th paper.pdfMinor 1 11th paper.pdf
Minor 1 11th paper.pdf
 
K means clustering
K means clusteringK means clustering
K means clustering
 
Tarea 1 vectores, matrices y determinantes laura montes
Tarea 1   vectores, matrices y determinantes laura montesTarea 1   vectores, matrices y determinantes laura montes
Tarea 1 vectores, matrices y determinantes laura montes
 
Tugas Calculus : Limit (Hal.8-14) Kelompok 2
Tugas Calculus : Limit (Hal.8-14) Kelompok 2Tugas Calculus : Limit (Hal.8-14) Kelompok 2
Tugas Calculus : Limit (Hal.8-14) Kelompok 2
 
Tugas Calculus : Limit (Hal. 8-14)
Tugas Calculus : Limit (Hal. 8-14)Tugas Calculus : Limit (Hal. 8-14)
Tugas Calculus : Limit (Hal. 8-14)
 
Manual calcuculadora casio fx-991ES PLUS
Manual calcuculadora casio fx-991ES PLUSManual calcuculadora casio fx-991ES PLUS
Manual calcuculadora casio fx-991ES PLUS
 
summary statistics
summary statisticssummary statistics
summary statistics
 
Quantitative techniques for MBA (QDM)
Quantitative techniques for MBA (QDM)Quantitative techniques for MBA (QDM)
Quantitative techniques for MBA (QDM)
 
DSM.pptx
DSM.pptxDSM.pptx
DSM.pptx
 
Central Tendancy.pdf
Central Tendancy.pdfCentral Tendancy.pdf
Central Tendancy.pdf
 
第2回 基本演算,データ型の基礎,ベクトルの操作方法
第2回 基本演算,データ型の基礎,ベクトルの操作方法第2回 基本演算,データ型の基礎,ベクトルの操作方法
第2回 基本演算,データ型の基礎,ベクトルの操作方法
 
Dispersion uwsb
Dispersion   uwsbDispersion   uwsb
Dispersion uwsb
 

Recently uploaded

Predictably Improve Your B2B Tech Company's Performance by Leveraging Data
Predictably Improve Your B2B Tech Company's Performance by Leveraging DataPredictably Improve Your B2B Tech Company's Performance by Leveraging Data
Predictably Improve Your B2B Tech Company's Performance by Leveraging Data
Kiwi Creative
 
End-to-end pipeline agility - Berlin Buzzwords 2024
End-to-end pipeline agility - Berlin Buzzwords 2024End-to-end pipeline agility - Berlin Buzzwords 2024
End-to-end pipeline agility - Berlin Buzzwords 2024
Lars Albertsson
 
一比一原版(Bradford毕业证书)布拉德福德大学毕业证如何办理
一比一原版(Bradford毕业证书)布拉德福德大学毕业证如何办理一比一原版(Bradford毕业证书)布拉德福德大学毕业证如何办理
一比一原版(Bradford毕业证书)布拉德福德大学毕业证如何办理
mbawufebxi
 
一比一原版(UIUC毕业证)伊利诺伊大学|厄巴纳-香槟分校毕业证如何办理
一比一原版(UIUC毕业证)伊利诺伊大学|厄巴纳-香槟分校毕业证如何办理一比一原版(UIUC毕业证)伊利诺伊大学|厄巴纳-香槟分校毕业证如何办理
一比一原版(UIUC毕业证)伊利诺伊大学|厄巴纳-香槟分校毕业证如何办理
ahzuo
 
Global Situational Awareness of A.I. and where its headed
Global Situational Awareness of A.I. and where its headedGlobal Situational Awareness of A.I. and where its headed
Global Situational Awareness of A.I. and where its headed
vikram sood
 
办(uts毕业证书)悉尼科技大学毕业证学历证书原版一模一样
办(uts毕业证书)悉尼科技大学毕业证学历证书原版一模一样办(uts毕业证书)悉尼科技大学毕业证学历证书原版一模一样
办(uts毕业证书)悉尼科技大学毕业证学历证书原版一模一样
apvysm8
 
STATATHON: Unleashing the Power of Statistics in a 48-Hour Knowledge Extravag...
STATATHON: Unleashing the Power of Statistics in a 48-Hour Knowledge Extravag...STATATHON: Unleashing the Power of Statistics in a 48-Hour Knowledge Extravag...
STATATHON: Unleashing the Power of Statistics in a 48-Hour Knowledge Extravag...
sameer shah
 
一比一原版(UCSB文凭证书)圣芭芭拉分校毕业证如何办理
一比一原版(UCSB文凭证书)圣芭芭拉分校毕业证如何办理一比一原版(UCSB文凭证书)圣芭芭拉分校毕业证如何办理
一比一原版(UCSB文凭证书)圣芭芭拉分校毕业证如何办理
nuttdpt
 
Natural Language Processing (NLP), RAG and its applications .pptx
Natural Language Processing (NLP), RAG and its applications .pptxNatural Language Processing (NLP), RAG and its applications .pptx
Natural Language Processing (NLP), RAG and its applications .pptx
fkyes25
 
一比一原版(Glasgow毕业证书)格拉斯哥大学毕业证如何办理
一比一原版(Glasgow毕业证书)格拉斯哥大学毕业证如何办理一比一原版(Glasgow毕业证书)格拉斯哥大学毕业证如何办理
一比一原版(Glasgow毕业证书)格拉斯哥大学毕业证如何办理
g4dpvqap0
 
一比一原版(UofS毕业证书)萨省大学毕业证如何办理
一比一原版(UofS毕业证书)萨省大学毕业证如何办理一比一原版(UofS毕业证书)萨省大学毕业证如何办理
一比一原版(UofS毕业证书)萨省大学毕业证如何办理
v3tuleee
 
Everything you wanted to know about LIHTC
Everything you wanted to know about LIHTCEverything you wanted to know about LIHTC
Everything you wanted to know about LIHTC
Roger Valdez
 
University of New South Wales degree offer diploma Transcript
University of New South Wales degree offer diploma TranscriptUniversity of New South Wales degree offer diploma Transcript
University of New South Wales degree offer diploma Transcript
soxrziqu
 
一比一原版(BCU毕业证书)伯明翰城市大学毕业证如何办理
一比一原版(BCU毕业证书)伯明翰城市大学毕业证如何办理一比一原版(BCU毕业证书)伯明翰城市大学毕业证如何办理
一比一原版(BCU毕业证书)伯明翰城市大学毕业证如何办理
dwreak4tg
 
Intelligence supported media monitoring in veterinary medicine
Intelligence supported media monitoring in veterinary medicineIntelligence supported media monitoring in veterinary medicine
Intelligence supported media monitoring in veterinary medicine
AndrzejJarynowski
 
Learn SQL from basic queries to Advance queries
Learn SQL from basic queries to Advance queriesLearn SQL from basic queries to Advance queries
Learn SQL from basic queries to Advance queries
manishkhaire30
 
一比一原版(Dalhousie毕业证书)达尔豪斯大学毕业证如何办理
一比一原版(Dalhousie毕业证书)达尔豪斯大学毕业证如何办理一比一原版(Dalhousie毕业证书)达尔豪斯大学毕业证如何办理
一比一原版(Dalhousie毕业证书)达尔豪斯大学毕业证如何办理
mzpolocfi
 
Challenges of Nation Building-1.pptx with more important
Challenges of Nation Building-1.pptx with more importantChallenges of Nation Building-1.pptx with more important
Challenges of Nation Building-1.pptx with more important
Sm321
 
State of Artificial intelligence Report 2023
State of Artificial intelligence Report 2023State of Artificial intelligence Report 2023
State of Artificial intelligence Report 2023
kuntobimo2016
 
一比一原版(Chester毕业证书)切斯特大学毕业证如何办理
一比一原版(Chester毕业证书)切斯特大学毕业证如何办理一比一原版(Chester毕业证书)切斯特大学毕业证如何办理
一比一原版(Chester毕业证书)切斯特大学毕业证如何办理
74nqk8xf
 

Recently uploaded (20)

Predictably Improve Your B2B Tech Company's Performance by Leveraging Data
Predictably Improve Your B2B Tech Company's Performance by Leveraging DataPredictably Improve Your B2B Tech Company's Performance by Leveraging Data
Predictably Improve Your B2B Tech Company's Performance by Leveraging Data
 
End-to-end pipeline agility - Berlin Buzzwords 2024
End-to-end pipeline agility - Berlin Buzzwords 2024End-to-end pipeline agility - Berlin Buzzwords 2024
End-to-end pipeline agility - Berlin Buzzwords 2024
 
一比一原版(Bradford毕业证书)布拉德福德大学毕业证如何办理
一比一原版(Bradford毕业证书)布拉德福德大学毕业证如何办理一比一原版(Bradford毕业证书)布拉德福德大学毕业证如何办理
一比一原版(Bradford毕业证书)布拉德福德大学毕业证如何办理
 
一比一原版(UIUC毕业证)伊利诺伊大学|厄巴纳-香槟分校毕业证如何办理
一比一原版(UIUC毕业证)伊利诺伊大学|厄巴纳-香槟分校毕业证如何办理一比一原版(UIUC毕业证)伊利诺伊大学|厄巴纳-香槟分校毕业证如何办理
一比一原版(UIUC毕业证)伊利诺伊大学|厄巴纳-香槟分校毕业证如何办理
 
Global Situational Awareness of A.I. and where its headed
Global Situational Awareness of A.I. and where its headedGlobal Situational Awareness of A.I. and where its headed
Global Situational Awareness of A.I. and where its headed
 
办(uts毕业证书)悉尼科技大学毕业证学历证书原版一模一样
办(uts毕业证书)悉尼科技大学毕业证学历证书原版一模一样办(uts毕业证书)悉尼科技大学毕业证学历证书原版一模一样
办(uts毕业证书)悉尼科技大学毕业证学历证书原版一模一样
 
STATATHON: Unleashing the Power of Statistics in a 48-Hour Knowledge Extravag...
STATATHON: Unleashing the Power of Statistics in a 48-Hour Knowledge Extravag...STATATHON: Unleashing the Power of Statistics in a 48-Hour Knowledge Extravag...
STATATHON: Unleashing the Power of Statistics in a 48-Hour Knowledge Extravag...
 
一比一原版(UCSB文凭证书)圣芭芭拉分校毕业证如何办理
一比一原版(UCSB文凭证书)圣芭芭拉分校毕业证如何办理一比一原版(UCSB文凭证书)圣芭芭拉分校毕业证如何办理
一比一原版(UCSB文凭证书)圣芭芭拉分校毕业证如何办理
 
Natural Language Processing (NLP), RAG and its applications .pptx
Natural Language Processing (NLP), RAG and its applications .pptxNatural Language Processing (NLP), RAG and its applications .pptx
Natural Language Processing (NLP), RAG and its applications .pptx
 
一比一原版(Glasgow毕业证书)格拉斯哥大学毕业证如何办理
一比一原版(Glasgow毕业证书)格拉斯哥大学毕业证如何办理一比一原版(Glasgow毕业证书)格拉斯哥大学毕业证如何办理
一比一原版(Glasgow毕业证书)格拉斯哥大学毕业证如何办理
 
一比一原版(UofS毕业证书)萨省大学毕业证如何办理
一比一原版(UofS毕业证书)萨省大学毕业证如何办理一比一原版(UofS毕业证书)萨省大学毕业证如何办理
一比一原版(UofS毕业证书)萨省大学毕业证如何办理
 
Everything you wanted to know about LIHTC
Everything you wanted to know about LIHTCEverything you wanted to know about LIHTC
Everything you wanted to know about LIHTC
 
University of New South Wales degree offer diploma Transcript
University of New South Wales degree offer diploma TranscriptUniversity of New South Wales degree offer diploma Transcript
University of New South Wales degree offer diploma Transcript
 
一比一原版(BCU毕业证书)伯明翰城市大学毕业证如何办理
一比一原版(BCU毕业证书)伯明翰城市大学毕业证如何办理一比一原版(BCU毕业证书)伯明翰城市大学毕业证如何办理
一比一原版(BCU毕业证书)伯明翰城市大学毕业证如何办理
 
Intelligence supported media monitoring in veterinary medicine
Intelligence supported media monitoring in veterinary medicineIntelligence supported media monitoring in veterinary medicine
Intelligence supported media monitoring in veterinary medicine
 
Learn SQL from basic queries to Advance queries
Learn SQL from basic queries to Advance queriesLearn SQL from basic queries to Advance queries
Learn SQL from basic queries to Advance queries
 
一比一原版(Dalhousie毕业证书)达尔豪斯大学毕业证如何办理
一比一原版(Dalhousie毕业证书)达尔豪斯大学毕业证如何办理一比一原版(Dalhousie毕业证书)达尔豪斯大学毕业证如何办理
一比一原版(Dalhousie毕业证书)达尔豪斯大学毕业证如何办理
 
Challenges of Nation Building-1.pptx with more important
Challenges of Nation Building-1.pptx with more importantChallenges of Nation Building-1.pptx with more important
Challenges of Nation Building-1.pptx with more important
 
State of Artificial intelligence Report 2023
State of Artificial intelligence Report 2023State of Artificial intelligence Report 2023
State of Artificial intelligence Report 2023
 
一比一原版(Chester毕业证书)切斯特大学毕业证如何办理
一比一原版(Chester毕业证书)切斯特大学毕业证如何办理一比一原版(Chester毕业证书)切斯特大学毕业证如何办理
一比一原版(Chester毕业证书)切斯特大学毕业证如何办理
 

Fishers linear discriminant for dimensionality reduction.

  • 1. Fishers Linear Discriminant for Dimensionality Reduction Department of Computer Science and Engineering National Institute of Technology Silchar, Assam, INDIA 21-11-2021 1 By- Nurul Amin Choudhury NIT Silchar - PhD Scholar
  • 2. Introduction • Fishers Linear Discriminant (FLD), is a statistical dimensionality reduction principal which is widely popular for Machine Learning as well as Pattern Recognition techniques. • This principal is incorporated by the techniques like – Linear Discriminant Analysis (LDA), Normal Discriminant Analysis (NDA) and many more. • FLD work better than PCA as it – • Maximizes the distance between different classes. • Minimizes the distance within the class. • It preserves the projection line direction which is very useful for separating different classes. Goal: To project the dataset with feature set (N) into feature set (M), where [ M <= (N-1) ] 21-11-2021 2
  • 3. Example Problem Consider the data of 2 classes as- Class 1 has 5 Samples (C1): [(4,1),(2,4),(2,3),(3,6),(4,4)] Class 2 has 5 Samples (C2): [(9,10),(6,8),(9,5),(8,7),(10,8)] Step 1: Compute the Mean (u) of each class- ui = (x1 + x2 +….. + xn)/n Therefore, u1 = [3 , 3.6] u2 = [8.4 , 7.6] Step 2: Compute within-class Scatter matrix (Sw) Sw= S1+S2 where S1 and S2 are the covariance matrix of class C1 and C2 respectively. S1 = 1 ℕ 𝒊=𝟏 𝒏 𝒙𝒊 − 𝒖𝟏 𝒙𝒊 − 𝒖𝟏 𝑻 and similarly for S2 as well. 21-11-2021 3 S. No. X Y Class 1 4 1 C1 2 2 4 C1 3 2 3 C1 4 3 6 C1 5 4 4 C1 6 9 10 C2 7 6 8 C2 8 9 5 C2 9 8 7 C2 10 10 8 C2 Table 1. Sample Dataset
  • 4. S1 = (1/5)* 𝛴 1 -1 -1 0 1 -2.6 0.4 -0.6 2.4 0.4 Cntd… 21-11-2021 4 * T (xi - ui) (xi - ui)T Going One by one will leads us to- 1 -2.6 *[1 -2.6] = 1 −2.6 −2.6 6.76 1. 2. -1 0.4 * [-1 0.4] = 1 −0.4 −0.4 0.16 3. -1 -0.6 * [-1 -0.6] = 1 0.6 0.6 0.36 4. 0 2.4 * [0 2.4] = 0 0 0 5.76 5. 1 0.4 * [1 -0.4] = 1 0.4 0.4 0.16 Adding Equation 1, 2, 3,4, 5 well get 4 −2 −2 13.2 and then dividing each element by (1/5) will lead to S1= 𝟎. 𝟖 − 𝟎. 𝟒 −𝟎. 𝟒 𝟐. 𝟔 Note: We can also do direct matrix multiplication of the above two (using Scientific Calc.) matrix. The result will be same.
  • 5. 21-11-2021 5 Similarly, S2= [𝟏. 𝟖𝟏 − 𝟎. 𝟎𝟒 -0.04 2.64] Therefore, Sw = [2.64 -0.44 -0.44 5.28] S1= 𝟎. 𝟖 − 𝟎. 𝟒 −𝟎. 𝟒 𝟐. 𝟔 S1= 𝟏 𝟓 ∗ 𝟒 − 𝟐 −𝟖 𝟏𝟑. 𝟐 So, Therefore,
  • 6. Step 3: Now we need to compute between class Scatter matrix (SB). SB = (u1 – u2) * (u1 – u2)T SB= [(- 5.4, -4) * (- 5.4, -4)T] SB= [29.16 21.6 21.6 16.0] Step 4: Finding the best Projection Vector using Sw and Mean. 𝑣1 𝑣2 = Sw -1 *(u1 – u2) Sw -1 = 𝑎 𝑏 𝑐 𝑑 -1 = 1 𝑎𝑑−𝑏𝑐 𝑑 −𝑏 −𝑐 𝑎 Sw -1 = 𝟎. 𝟑𝟖𝟒 𝟎. 𝟎𝟑𝟐 𝟎. 𝟎𝟑𝟐 𝟎. 𝟏𝟗𝟐 Cntd… 21-11-2021 6
  • 7. Therefore our optimal Line of Direction V (optimal projection) is – V = Sw -1 * (- 5.4, -4) V= 𝟎. 𝟑𝟖𝟒 𝟎. 𝟎𝟑𝟐 𝟎. 𝟎𝟑𝟐 𝟎. 𝟏𝟗𝟐 * (- 5.4, -4) V= −𝟐. 𝟐𝟎 −𝟎. 𝟗𝟒 Step 5: Finally, we can calculate actual 1D data with the help of – y1 = VT * C1 T y2 = VT * C2 T Cntd… 21-11-2021 7
  • 8. 21-11-2021 8 Cntd… We know, (C1): [(4,1),(2,4),(2,3),(3,6),(4,4)] & (C2): [(9,10),(6,8),(9,5),(8,7),(10,8)] Also, V= −𝟐. 𝟐𝟎 −𝟎. 𝟗𝟒 Therefore Y1 and Y2 are- Y1= [-2.20-0.94] * 4 2 2 3 4 1 4 3 6 4 = [-9.74 -8.6 -7.22 -12.24 -12.56] Y2= [-2.20-0.94] * 9 6 9 8 10 10 8 5 7 8 = [-29.2 -20.72 -24.5 -24.18 -29.52] Finally, the optimal line of Projection is V= −𝟐. 𝟐𝟎 −𝟎. 𝟗𝟒 and 1D data of Class 1 and 2 are shown above.
  • 9. 21-11-2021 9 Advantages of FLD • Minimizes the variance and maximize the class distance between two variables. • Works better for multi class problem too. • Can be use for classification task too.
  • 10. 21-11-2021 10 Disadvantages of FLD • When the discriminative information are not in the means of classes. • Small sample size problem. • Not suitable for non linear models • Works with Labelled Data only (Supervised in nature).
  • 11. 21-11-2021 11 References [1] Silva, T. S. (2019, January 3). An illustrative introduction to Fisher’s Linear Discriminant. Thalles’ Blog. https://sthalles.github.io/fisher-linear-discriminant/ [2] Wikipedia contributors. (2021, October 13). Linear discriminant analysis. Wikipedia. https://en.wikipedia.org/wiki/Linear_discriminant_analysis [3] Bishop. (2006). Pattern Recognition and Machine Learning | Christopher M. Bishop | Springer. Pattern Recognition. Retrieved 2021, from https://www.springer.com/gp/book/9780387310732 [4] https://www.csd.uwo.ca/~oveksler/Courses/CS434a_541a/Lecture8.pdf