This document summarizes Eric Xing's lecture on dimensionality reduction and machine learning. It discusses several techniques for dimensionality reduction including principal component analysis (PCA), locality preserving projections (LLE), and isomap. PCA finds orthogonal directions of maximum variance in high-dimensional data to project it to a lower-dimensional space. LLE and isomap are nonlinear dimensionality reduction techniques that can discover low-dimensional manifold structures. Applications discussed include text retrieval, image analysis, and super resolution image reconstruction. Dimensionality reduction is useful for pattern recognition, information retrieval, and exploring high-dimensional datasets.
The variational Gaussian process (VGP), a Bayesian nonparametric model which adapts its shape to match com- plex posterior distributions. The VGP generates approximate posterior samples by generating latent inputs and warping them through random non-linear mappings; the distribution over random mappings is learned during inference, enabling the transformed outputs to adapt to varying complexity.
The variational Gaussian process (VGP), a Bayesian nonparametric model which adapts its shape to match com- plex posterior distributions. The VGP generates approximate posterior samples by generating latent inputs and warping them through random non-linear mappings; the distribution over random mappings is learned during inference, enabling the transformed outputs to adapt to varying complexity.
Universal Approximation Property via Quantum Feature Maps
----
The quantum Hilbert space can be used as a quantum-enhanced feature space in machine learning (ML) via the quantum feature map to encode classical data into quantum states. We prove the ability to approximate any continuous function with optimal approximation rate via quantum ML models in typical quantum feature maps.
---
Contributed talk at Quantum Techniques in Machine Learning 2021, Tokyo, November 8-12 2021.
By Quoc Hoan Tran, Takahiro Goto and Kohei Nakajima
Bayes Nets Meetup Sept 29th 2016 - Bayesian Network Modelling by Marco ScutariBayes Nets meetup London
A talk given at the Bayes Nets meetup on Sept 29th 2016 by Dr Marco Scutari from the University of Oxford. Title of the talk was Bayesian Network Modelling with examples in Genetics and Systems Biology, with case studies.
Summary of survey papers on deep learning method to 3D dataArithmer Inc.
Slide for study session given by Dr. Takashi Nakano (Arithmer inc.) at Arithmer inc.
It is a summary of recent survey papers on deep learning method to 3D data.
Arithmer株式会社は東京大学大学院数理科学研究科発の数学の会社です。私達は現代数学を応用して、様々な分野のソリューションに、新しい高度AIシステムを導入しています。AIをいかに上手に使って仕事を効率化するか、そして人々の役に立つ結果を生み出すのか、それを考えるのが私たちの仕事です。
Arithmer began at the University of Tokyo Graduate School of Mathematical Sciences. Today, our research of modern mathematics and AI systems has the capability of providing solutions when dealing with tough complex issues. At Arithmer we believe it is our job to realize the functions of AI through improving work efficiency and producing more useful results for society.
Textural Feature Extraction of Natural Objects for Image ClassificationCSCJournals
The field of digital image processing has been growing in scope in the recent years. A digital image is represented as a two-dimensional array of pixels, where each pixel has the intensity and location information. Analysis of digital images involves extraction of meaningful information from them, based on certain requirements. Digital Image Analysis requires the extraction of features, transforms the data in the high-dimensional space to a space of fewer dimensions. Feature vectors are n-dimensional vectors of numerical features used to represent an object. We have used Haralick features to classify various images using different classification algorithms like Support Vector Machines (SVM), Logistic Classifier, Random Forests Multi Layer Perception and Naïve Bayes Classifier. Then we used cross validation to assess how well a classifier works for a generalized data set, as compared to the classifications obtained during training.
Content Based Image Retrieval Using Gray Level Co-Occurance Matrix with SVD a...ijcisjournal
In this paper, gray level co-occurrence matrix, gray level co-occurrence matrix with singular value
decomposition and local binary pattern are presented for content based image retrieval. Based upon the
feature vector parameters of energy, contrast, entropy and distance metrics such as Euclidean distance,
Canberra distance, Manhattan distance the retrieval efficiency, precision, and recall of the images are
calculated. The retrieval results of the proposed method are tested on Corel-1k database. The results after
being investigated shows a significant improvement in terms of average retrieval rate, average retrieval
precision and recall of different algorithms such as GLCM, GLCM & SVD, LBP with radius one and LBP
with radius two based on different distance metrics.
How Cyberflow Analytics have used KeyLines’ network visualization functionality to develop the next generation of cyber security analytics platform – built for the scope and scale of the Internet of Things.
Plotcon 2016 Visualization Talk by Alexandra JohnsonSigOpt
Machine learning is full of ideas that are far abstracted away from the underlying data and difficult to understand. Luckily, this represents an amazing opportunity for visualization! These slides dive into the machine learning meta-problem of hyperparameter optimization. We'll show 4 opportunities for visualization in helping people understand, implement, and evaluate hyperparameter optimization strategies.
Universal Approximation Property via Quantum Feature Maps
----
The quantum Hilbert space can be used as a quantum-enhanced feature space in machine learning (ML) via the quantum feature map to encode classical data into quantum states. We prove the ability to approximate any continuous function with optimal approximation rate via quantum ML models in typical quantum feature maps.
---
Contributed talk at Quantum Techniques in Machine Learning 2021, Tokyo, November 8-12 2021.
By Quoc Hoan Tran, Takahiro Goto and Kohei Nakajima
Bayes Nets Meetup Sept 29th 2016 - Bayesian Network Modelling by Marco ScutariBayes Nets meetup London
A talk given at the Bayes Nets meetup on Sept 29th 2016 by Dr Marco Scutari from the University of Oxford. Title of the talk was Bayesian Network Modelling with examples in Genetics and Systems Biology, with case studies.
Summary of survey papers on deep learning method to 3D dataArithmer Inc.
Slide for study session given by Dr. Takashi Nakano (Arithmer inc.) at Arithmer inc.
It is a summary of recent survey papers on deep learning method to 3D data.
Arithmer株式会社は東京大学大学院数理科学研究科発の数学の会社です。私達は現代数学を応用して、様々な分野のソリューションに、新しい高度AIシステムを導入しています。AIをいかに上手に使って仕事を効率化するか、そして人々の役に立つ結果を生み出すのか、それを考えるのが私たちの仕事です。
Arithmer began at the University of Tokyo Graduate School of Mathematical Sciences. Today, our research of modern mathematics and AI systems has the capability of providing solutions when dealing with tough complex issues. At Arithmer we believe it is our job to realize the functions of AI through improving work efficiency and producing more useful results for society.
Textural Feature Extraction of Natural Objects for Image ClassificationCSCJournals
The field of digital image processing has been growing in scope in the recent years. A digital image is represented as a two-dimensional array of pixels, where each pixel has the intensity and location information. Analysis of digital images involves extraction of meaningful information from them, based on certain requirements. Digital Image Analysis requires the extraction of features, transforms the data in the high-dimensional space to a space of fewer dimensions. Feature vectors are n-dimensional vectors of numerical features used to represent an object. We have used Haralick features to classify various images using different classification algorithms like Support Vector Machines (SVM), Logistic Classifier, Random Forests Multi Layer Perception and Naïve Bayes Classifier. Then we used cross validation to assess how well a classifier works for a generalized data set, as compared to the classifications obtained during training.
Content Based Image Retrieval Using Gray Level Co-Occurance Matrix with SVD a...ijcisjournal
In this paper, gray level co-occurrence matrix, gray level co-occurrence matrix with singular value
decomposition and local binary pattern are presented for content based image retrieval. Based upon the
feature vector parameters of energy, contrast, entropy and distance metrics such as Euclidean distance,
Canberra distance, Manhattan distance the retrieval efficiency, precision, and recall of the images are
calculated. The retrieval results of the proposed method are tested on Corel-1k database. The results after
being investigated shows a significant improvement in terms of average retrieval rate, average retrieval
precision and recall of different algorithms such as GLCM, GLCM & SVD, LBP with radius one and LBP
with radius two based on different distance metrics.
How Cyberflow Analytics have used KeyLines’ network visualization functionality to develop the next generation of cyber security analytics platform – built for the scope and scale of the Internet of Things.
Plotcon 2016 Visualization Talk by Alexandra JohnsonSigOpt
Machine learning is full of ideas that are far abstracted away from the underlying data and difficult to understand. Luckily, this represents an amazing opportunity for visualization! These slides dive into the machine learning meta-problem of hyperparameter optimization. We'll show 4 opportunities for visualization in helping people understand, implement, and evaluate hyperparameter optimization strategies.
AI & ML in Cyber Security - Welcome Back to 1999 - Security Hasn't ChangedRaffael Marty
We are writing the year 2017. Cyber security has been a discipline for many years and thousands of security companies are offering solutions to deter and block malicious actors in order to keep our businesses operating and our data confidential. But fundamentally, cyber security has not changed during the last two decades. We are still running Snort and Bro. Firewalls are fundamentally still the same. People get hacked for their poor passwords and we collect logs that we don't know what to do with. In this talk I will paint a slightly provocative and dark picture of security. Fundamentally, nothing has really changed. We'll have a look at machine learning and artificial intelligence and see how those techniques are used today. Do they have the potential to change anything? How will the future look with those technologies? I will show some practical examples of machine learning and motivate that simpler approaches generally win. Maybe we find some hope in visualization? Or maybe Augmented reality? We still have a ways to go.
Machine learning is the hacker art of describing the features of instances that we want to make predictions about, then fitting the data that describes those instances to a model form. Applied machine learning has come a long way from it's beginnings in academia, and with tools like Scikit-Learn, it's easier than ever to generate operational models for a wide variety of applications. Thanks to the ease and variety of the tools in Scikit-Learn, the primary job of the data scientist is model selection. Model selection involves performing feature engineering, hyperparameter tuning, and algorithm selection. These dimensions of machine learning often lead computer scientists towards automatic model selection via optimization (maximization) of a model's evaluation metric. However, the search space is large, and grid search approaches to machine learning can easily lead to failure and frustration. Human intuition is still essential to machine learning, and visual analysis in concert with automatic methods can allow data scientists to steer model selection towards better fitted models, faster. In this talk, we will discuss interactive visual methods for better understanding, steering, and tuning machine learning models.
Network analyses are powerful methods for both visual analytics and machine learning but can suffer as their complexity increases. By embedding time as a structural element rather than a property, we will explore how time series and interactive analysis can be improved on Graph structures. Primarily we will look at decomposition in NLP-extracted concept graphs using NetworkX and Graph Tool.
The dynamics of networks enables the function of a variety of systems we rely on every day, from gene regulation and metabolism in the cell to the distribution of electric power and communication of information. Understanding, steering and predicting the function of interacting nonlinear dynamical systems, in particular if they are externally driven out of equilibrium, relies on obtaining and evaluating suitable models, posing at least two major challenges. First, how can we extract key structural system features of networks if only time series data provide information about the dynamics of (some) units? Second, how can we characterize nonlinear responses of nonlinear multi-dimensional systems externally driven by fluctuations, and consequently, predict tipping points at which normal operational states may be lost? Here we report recent progress on nonlinear response theory extended to predict tipping points and on model-free inference of network structural features from observed dynamics.
Research Inventy : International Journal of Engineering and Science is published by the group of young academic and industrial researchers with 12 Issues per year. It is an online as well as print version open access journal that provides rapid publication (monthly) of articles in all areas of the subject such as: civil, mechanical, chemical, electronic and computer engineering as well as production and information technology. The Journal welcomes the submission of manuscripts that meet the general criteria of significance and scientific excellence. Papers will be published by rapid process within 20 days after acceptance and peer review process takes only 7 days. All articles published in Research Inventy will be peer-reviewed.
BPSO&1-NN algorithm-based variable selection for power system stability ident...IJAEMSJORNAL
Due to the very high nonlinearity of the power system, traditional analytical methods take a lot of time to solve, causing delay in decision-making. Therefore, quickly detecting power system instability helps the control system to make timely decisions become the key factor to ensure stable operation of the power system. Power system stability identification encounters large data set size problem. The need is to select representative variables as input variables for the identifier. This paper proposes to apply wrapper method to select variables. In which, Binary Particle Swarm Optimization (BPSO) algorithm combines with K-NN (K=1) identifier to search for good set of variables. It is named BPSO&1-NN. Test results on IEEE 39-bus diagram show that the proposed method achieves the goal of reducing variables with high accuracy.
Multiscale Granger Causality and Information Decomposition danielemarinazzo
Slides for the C3S workshop in Cologne, in which I presented an extension to multiple temporal scale of Granger causality and Partial Information Decomposition, using the robust state space formulation
Reduct generation for the incremental data using rough set theorycsandit
n today’s changing world huge amount of data is ge
nerated and transferred frequently.
Although the data is sometimes static but most comm
only it is dynamic and transactional. New
data that is being generated is getting constantly
added to the old/existing data. To discover the
knowledge from this incremental data, one approach
is to run the algorithm repeatedly for the
modified data sets which is time consuming. The pap
er proposes a dimension reduction
algorithm that can be applied in dynamic environmen
t for generation of reduced attribute set as
dynamic reduct.
The method analyzes the new dataset, when it become
s available, and modifies
the reduct accordingly to fit the entire dataset. T
he concepts of discernibility relation, attribute
dependency and attribute significance of Rough Set
Theory are integrated for the generation of
dynamic reduct set, which not only reduces the comp
lexity but also helps to achieve higher
accuracy of the decision system. The proposed metho
d has been applied on few benchmark
dataset collected from the UCI repository and a dyn
amic reduct is computed. Experimental
result shows the efficiency of the proposed method
Fault diagnosis using genetic algorithms and principal curveseSAT Journals
Abstract Several applications of nonlinear principal component analysis (NPCA) have appeared recently in process monitoring and fault diagnosis. In this paper a new approach is proposed for fault detection based on principal curves and genetic algorithms. The principal curve is a generation of linear principal component (PCA) introduced by Hastie as a parametric curve passes satisfactorily through the middle of data. The existing principal curves algorithms employ the first component of the data as an initial estimation of principal curve. However the dependence on initial line leads to a lack of flexibility and the final curve is only satisfactory for specific problems. In this paper we extend this work in two ways. First, we propose a new method based on genetic algorithms to find the principal curve. Here, lines are fitted and connected to form polygonal lines (PL). Second, potential application of principal curves is discussed. An example is used to illustrate fault diagnosis of nonlinear process using the proposed approach. Index Terms: Principal curve, Genetic Algorithm, Nonlinear principal component analysis, Fault detection.
A Predictive Stock Data Analysis with SVM-PCA Model .......................................................................1
Divya Joseph and Vinai George Biju
HOV-kNN: A New Algorithm to Nearest Neighbor Search in Dynamic Space.......................................... 12
Mohammad Reza Abbasifard, Hassan Naderi and Mohadese Mirjalili
A Survey on Mobile Malware: A War without End................................................................................... 23
Sonal Mohite and Prof. R. S. Sonar
An Efficient Design Tool to Detect Inconsistencies in UML Design Models............................................. 36
Mythili Thirugnanam and Sumathy Subramaniam
An Integrated Procedure for Resolving Portfolio Optimization Problems using Data Envelopment
Analysis, Ant Colony Optimization and Gene Expression Programming ................................................. 45
Chih-Ming Hsu
Emerging Technologies: LTE vs. WiMAX ................................................................................................... 66
Mohammad Arifin Rahman Khan and Md. Sadiq Iqbal
Introducing E-Maintenance 2.0 ................................................................................................................. 80
Abdessamad Mouzoune and Saoudi Taibi
Detection of Clones in Digital Images........................................................................................................ 91
Minati Mishra and Flt. Lt. Dr. M. C. Adhikary
The Significance of Genetic Algorithms in Search, Evolution, Optimization and Hybridization: A Short
Review ...................................................................................................................................................... 103
"Trans Failsafe Prog" on your BMW X5 indicates potential transmission issues requiring immediate action. This safety feature activates in response to abnormalities like low fluid levels, leaks, faulty sensors, electrical or mechanical failures, and overheating.
What Does the Active Steering Malfunction Warning Mean for Your BMWTanner Motors
Discover the reasons why your BMW’s Active Steering malfunction warning might come on. From electrical glitches to mechanical failures and software anomalies, addressing these promptly with professional inspection and maintenance ensures continued safety and performance on the road, maintaining the integrity of your driving experience.
What Exactly Is The Common Rail Direct Injection System & How Does It WorkMotor Cars International
Learn about Common Rail Direct Injection (CRDi) - the revolutionary technology that has made diesel engines more efficient. Explore its workings, advantages like enhanced fuel efficiency and increased power output, along with drawbacks such as complexity and higher initial cost. Compare CRDi with traditional diesel engines and discover why it's the preferred choice for modern engines.
Why Is Your BMW X3 Hood Not Responding To Release CommandsDart Auto
Experiencing difficulty opening your BMW X3's hood? This guide explores potential issues like mechanical obstruction, hood release mechanism failure, electrical problems, and emergency release malfunctions. Troubleshooting tips include basic checks, clearing obstructions, applying pressure, and using the emergency release.
In this presentation, we have discussed a very important feature of BMW X5 cars… the Comfort Access. Things that can significantly limit its functionality. And things that you can try to restore the functionality of such a convenient feature of your vehicle.
5 Warning Signs Your BMW's Intelligent Battery Sensor Needs AttentionBertini's German Motors
IBS monitors and manages your BMW’s battery performance. If it malfunctions, you will have to deal with an array of electrical issues in your vehicle. Recognize warning signs like dimming headlights, frequent battery replacements, and electrical malfunctions to address potential IBS issues promptly.
What Does the PARKTRONIC Inoperative, See Owner's Manual Message Mean for You...Autohaus Service and Sales
Learn what "PARKTRONIC Inoperative, See Owner's Manual" means for your Mercedes-Benz. This message indicates a malfunction in the parking assistance system, potentially due to sensor issues or electrical faults. Prompt attention is crucial to ensure safety and functionality. Follow steps outlined for diagnosis and repair in the owner's manual.
𝘼𝙣𝙩𝙞𝙦𝙪𝙚 𝙋𝙡𝙖𝙨𝙩𝙞𝙘 𝙏𝙧𝙖𝙙𝙚𝙧𝙨 𝙞𝙨 𝙫𝙚𝙧𝙮 𝙛𝙖𝙢𝙤𝙪𝙨 𝙛𝙤𝙧 𝙢𝙖𝙣𝙪𝙛𝙖𝙘𝙩𝙪𝙧𝙞𝙣𝙜 𝙩𝙝𝙚𝙞𝙧 𝙥𝙧𝙤𝙙𝙪𝙘𝙩𝙨. 𝙒𝙚 𝙝𝙖𝙫𝙚 𝙖𝙡𝙡 𝙩𝙝𝙚 𝙥𝙡𝙖𝙨𝙩𝙞𝙘 𝙜𝙧𝙖𝙣𝙪𝙡𝙚𝙨 𝙪𝙨𝙚𝙙 𝙞𝙣 𝙖𝙪𝙩𝙤𝙢𝙤𝙩𝙞𝙫𝙚 𝙖𝙣𝙙 𝙖𝙪𝙩𝙤 𝙥𝙖𝙧𝙩𝙨 𝙖𝙣𝙙 𝙖𝙡𝙡 𝙩𝙝𝙚 𝙛𝙖𝙢𝙤𝙪𝙨 𝙘𝙤𝙢𝙥𝙖𝙣𝙞𝙚𝙨 𝙗𝙪𝙮 𝙩𝙝𝙚 𝙜𝙧𝙖𝙣𝙪𝙡𝙚𝙨 𝙛𝙧𝙤𝙢 𝙪𝙨.
Over the 10 years, we have gained a strong foothold in the market due to our range's high quality, competitive prices, and time-lined delivery schedules.
Comprehensive program for Agricultural Finance, the Automotive Sector, and Empowerment . We will define the full scope and provide a detailed two-week plan for identifying strategic partners in each area within Limpopo, including target areas.:
1. Agricultural : Supporting Primary and Secondary Agriculture
• Scope: Provide support solutions to enhance agricultural productivity and sustainability.
• Target Areas: Polokwane, Tzaneen, Thohoyandou, Makhado, and Giyani.
2. Automotive Sector: Partnerships with Mechanics and Panel Beater Shops
• Scope: Develop collaborations with automotive service providers to improve service quality and business operations.
• Target Areas: Polokwane, Lephalale, Mokopane, Phalaborwa, and Bela-Bela.
3. Empowerment : Focusing on Women Empowerment
• Scope: Provide business support support and training to women-owned businesses, promoting economic inclusion.
• Target Areas: Polokwane, Thohoyandou, Musina, Burgersfort, and Louis Trichardt.
We will also prioritize Industrial Economic Zone areas and their priorities.
Sign up on https://profilesmes.online/welcome/
To be eligible:
1. You must have a registered business and operate in Limpopo
2. Generate revenue
3. Sectors : Agriculture ( primary and secondary) and Automative
Women and Youth are encouraged to apply even if you don't fall in those sectors.
Core technology of Hyundai Motor Group's EV platform 'E-GMP'Hyundai Motor Group
What’s the force behind Hyundai Motor Group's EV performance and quality?
Maximized driving performance and quick charging time through high-density battery pack and fast charging technology and applicable to various vehicle types!
Discover more about Hyundai Motor Group’s EV platform ‘E-GMP’!
Symptoms like intermittent starting and key recognition errors signal potential problems with your Mercedes’ EIS. Use diagnostic steps like error code checks and spare key tests. Professional diagnosis and solutions like EIS replacement ensure safe driving. Consult a qualified technician for accurate diagnosis and repair.
Things to remember while upgrading the brakes of your carjennifermiller8137
Upgrading the brakes of your car? Keep these things in mind before doing so. Additionally, start using an OBD 2 GPS tracker so that you never miss a vehicle maintenance appointment. On top of this, a car GPS tracker will also let you master good driving habits that will let you increase the operational life of your car’s brakes.
37. Applying PCA and LDA:
Eigen-faces and Fisher-faces
L. Fei-Fei
Computer Science Dept.
Stanford University
38. Machine learning in computer vision
• Aug 13, Lecture 7: Dimensionality reduction,
Manifold learning
– Eigen- and Fisher- faces
– Applications to object representation
8/8/2010 38L. Fei-Fei, Dragon Star 2010, Stanford
39. The Space of Faces
• An image is a point in a high dimensional space
– An N x M image is a point in RNM
– We can define vectors in this space as we did in the 2D case
+=
[Thanks to Chuck Dyer, Steve Seitz, Nishino]
40. Key Idea
}ˆ{ P
RLx=χ• Images in the possible set are highly correlated.
• So, compress them to a low-dimensional subspace that
captures key appearance characteristics of the visual DOFs.
• EIGENFACES: [Turk and Pentland]
USE PCA!
41. Principal Component Analysis (PCA)
• PCA is used to determine the most representing
features among data points.
– It computes the p-dimensional subspace such that the
projection of the data points onto the subspace has the
largest variance among all p-dimensional subspaces.
44. Mathematical Formulation
Find a transformation, W,
m-dimensional n-dimensionalOrthonormal
Total scatter matrix:
Wopt corresponds to m eigen-
vectors of ST
45. Eigenfaces
• PCA extracts the eigenvectors of A
– Gives a set of vectors v1, v2, v3, ...
– Each one of these vectors is a direction in face space
• what do these look like?
46. Projecting onto the Eigenfaces
• The eigenfaces v1, ..., vK span the space of faces
– A face is converted to eigenface coordinates by
47. Algorithm
1. Align training images x1, x2, …, xN
2. Compute average face u = 1/N Σ xi
3. Compute the difference image φi = xi – u
Training
Note that each image is formulated into a long vector!
48. Algorithm
Testing
1. Projection in Eigenface
Projection ωi = W(X – u), W = {eigenfaces}
2. Compare projections
ST = 1/NΣ φi φi
T = BBT, B=[φ1, φ2 … φN]
4. Compute the covariance matrix (total scatter matrix)
5. Compute the eigenvectors of the covariance
matrix , W
49. Illustration of Eigenfaces
These are the first 4 eigenvectors from a training set of 400
images (ORL Face Database). They look like faces, hence
called Eigenface.
The visualization of eigenvectors:
52. Only selecting the top P eigenfaces reduces the dimensionality.
Fewer eigenfaces result in more information loss, and hence less
discrimination between faces.
Reconstruction and Errors
P = 4
P = 200
P = 400
53. Summary for PCA and Eigenface
• Non-iterative, globally optimal solution
• PCA projection is optimal for reconstruction
from a low dimensional basis, but may NOT be
optimal for discrimination…
54. Linear Discriminant Analysis (LDA)
• Using Linear Discriminant Analysis (LDA) or
Fisher’s Linear Discriminant (FLD)
• Eigenfaces attempt to maximise the scatter of the
training images in face space, while Fisherfaces
attempt to maximise the between class scatter,
while minimising the within class scatter.
55. Illustration of the Projection
Poor Projection Good Projection
x1
x2
x1
x2
Using two classes as example:
57. Variables
• N Sample images:
• c classes:
• Average of each class:
• Total average:
{ }Nxx ,,1
{ }cχχ ,,1
∑=
∈ ikx
k
i
i x
N χ
µ
1
∑=
=
N
k
kx
N 1
1
µ
58. Scatters
• Scatter of class i: ( )( )T
ik
x
iki xxS
ik
µµ
χ
−∑ −=
∈
∑=
=
c
i
iW SS
1
( )( )∑ −−=
=
c
i
T
iiiBS
1
µµµµχ
BWT SSS +=
• Within class scatter:
• Between class scatter:
• Total scatter:
60. Mathematical Formulation (1)
After projection:
Between class scatter (of y’s):
Within class scatter (of y’s):
k
T
k xWy =
WSWS B
T
B =
~
WSWS W
T
W =
~
61. Mathematical Formulation (2)
• The desired projection:
WSW
WSW
S
S
W
W
T
B
T
W
B
opt
WW
maxarg~
~
maxarg ==
miwSwS iWiiB ,,1 == λ
• How is it found ? Generalized Eigenvectors
Data dimension is much larger than the
number of samples
The matrix is singular:
Nn >>
( ) cNSRank W −≤WS
62. Fisherface (PCA+FLD)
• Project with FLD to space
• Project with PCA to space
1−c k
T
fldk zWy =
WWSWW
WWSWW
W
pcaW
T
pca
T
pcaB
T
pca
T
fld
W
maxarg=
cN − k
T
pcak xWz =
WSWW T
T
pca
W
maxarg=
66. discussion
• Removing the first three principal
components results in better performance
under variable lighting conditions
• The Firsherface methods had error rates
lower than the Eigenface method for the
small datasets tested.
72. Find a low-D basis for
describing high-D data.
X ~= X' S.T.
dim(X') << dim(X)
uncovers the intrinsic
dimensionality
manifold learning
73. If we knew all pairwise distances…
Chicago Raleigh Boston Seattle S.F. Austin Orlando
Chicago 0
Raleigh 641 0
Boston 851 608 0
Seattle 1733 2363 2488 0
S.F. 1855 2406 2696 684 0
Austin 972 1167 1691 1764 1495 0
Orlando 994 520 1105 2565 2458 1015 0
Distances calculated with geobytes.com/CityDistanceTool
74. Multidimensional Scaling (MDS)
For n data points, and a distance matrix D,
Dij =
...we can construct a m-dimensional space to preserve
inter-point distances by using the top eigenvectors of D
scaled by their eigenvalues
j
i
83. Isomap for images
Build a data graph G.
Vertices: images
(u,v) is an edge iff SSD(u,v) is small
For any two images, we approximate the
distance between them with the “shortest path”
on G
84. Isomap
1. Build a sparse graph with K-nearest neighbors
Dg =
(distance matrix is
sparse)
85. Isomap
2. Infer other interpoint distances by finding shortest
paths on the graph (Dijkstra's
algorithm).
Dg =
88. - preserves global structure
- few free parameters
- sensitive to noise, noise edges
- computationally expensive (dense matrix
eigen-reduction)
Isomap: pro and con