The document provides an overview of machine learning, including its definitions, categories, and training methods, along with critical considerations for improving algorithms. Key topics discussed include supervised and unsupervised learning, pitfalls like overfitting and underfitting, and various classification techniques such as k-means clustering and support vector machines. It emphasizes the importance of feature selection and data preprocessing in the training process.

1. Machine Learning Training
in Mohali
E2Matrix Training and Research Institute Jalandhar
Contact : +91 9041262727
Email : support@e2matrix.com

2. What is machine learning?
Data Model
training
Output
Predictions
Classifications
Clusters
Ordinals
examples
Why: Face Recognition?

3. Categories of problems
Classification Ordinal Reg.
Regression Prediction
By output:
Clustering
By input:
Vector, X Time Series, x(t)

4. One size never fits all…
• Improving an algorithm:
– First option: better features
• Visualize classes
• Trends
• Histograms
– Next: make the algorithm smarter (more complicated)
• Interaction of features
• Better objective and training criteria
WEKA or GGOBI

5. -4 -2 0 2 4 6
-20
-10
0
10
20
30
40
y=1 + 0.5t + 4t2 - t3
-4 -2 0 2 4 6
-20
-10
0
10
20
30
40
input
output
Categories of ML algorithms
By training:
Supervised (labeled) Unsupervised (unlabeled)
By model:
Non-parametric
Raw data only
Parametric
Model parameters only
-4 -2 0 2 4 6
-20
-10
0
10
20
30
40
input
output
Kernel
methods

6. -4 -2 0 2 4 6
-20
-10
0
10
20
30
40
input
output
50 100 150 200 250
0
0
0.05
0.1
0.15
0.2
-2 0 2 4 6
-20
-10
0
10
20
30
40
input
tupt
u
o
-2 0 2 4 6 -4
-20
-10
0
10
20
30
40
input
tupt
u
o
-4 -2 0 2 4 6 -4
-20
-10
0
10
20
30
40
input
output

7. Training a ML algorithm
• Choose data
• Optimize model parameters according to:
– Objective function
-4 -2 0 2 4 6
-20
-10
0
10
20
30
40
Regression Classification
-2 0 2 4 6 8
-2
0
2
4
6
8
10
1
2
Mean Square Error
Max Margin

8. Pitfalls of ML algorithms
• Clean your features:
– Training volume: more is better
– Outliers: remove them!
– Dynamic range: normalize it!
• Generalization
– Over fitting
– Under fitting
• Speed: parametric vs. non
• What are you learning? …features, features, features…

9. outliers
-4 -2 0 2 4 6
-20
-10
0
10
20
30
40
input
output
-4 -2 0 2 4 6
-20
-10
0
10
20
30
40
input
output
-4 -2 0 2 4 6
-10
-20
0
10
20
50
40
30
input
output
Keep a “good” percentile range!
5-95, 1-99: depends on your data

10. Dynamic range
0 0.2 0.4 0.6 0.8 1
-0.2
0
0.2
0.4
0.6
0.8
1
1.2
f1
f2
1
2
0 200 400 600 800 1000
-1
0
1
2
3
4
5
6
f1
f2
1
2
0 200 400 600 800 1000
0
50
100
150
200
250
300
350
400
f1
f2
1
2
-2 0 2 4 6 8
-1
0
1
2
3
4
5
6
f1
f2
1
2

11. Over fitting and comparing
algorithms
• Early stop
• Regularization
• Validation Sets

12. Under
fitting
Curse of dimensionality

13. Under
fitting
Curse of dimensionality

14. K-Means clustering
•Planar decision boundaries,
depending on space you are in…
•Highly Efficient
•Not always great (but usually
pretty good)
•Needs good starting criteria

15. K-Nearest Neighbor
•Arbitrary decision boundaries
•Not so efficient…
•With enough data in each class…
optimal
•Easy to train, known as a lazy classifier

16. Mixture of Gaussians
•Arbitrary decision boundaries
with enough boundaries
•Efficient, depending on number
of models and Gaussians
•Can represent more than just
Gaussian distributions
•Generative, sometimes tough to
train up
•Spurious singularities
•Can get a distribution for a
specific class and feature(s)… and
get a Bayesian classifier

17. Components
Analysis (principal or
independent)•Reduces dimensionality
•All other classifiers work in a
rotated space
•Remember Eigen-values and
Vectors?

18. Trees Classifiers
•Arbitrary Decision boundaries
•Can be quite efficient (or not!)
•Needs good criteria for splitting
•Easy to visualize

19. Multi-Layer Perceptron
•Arbitrary (but linear) Decision
boundaries
•Can be quite efficient (or not!)
•What did it learn?

20. Support Vector Machines
•Arbitrary Decision boundaries
•Efficiency depends on support
vector size and feature size

21. Hidden Markov Models
•Arbitrary Decision boundaries
•Efficiency depends on state
space and number of models
•Generalizes to incorporate
features that change over time

22. More sophisticated approaches
• Graphical models (like an HMM)
– Bayesian network
– Markov random fields
• Boosting
– Adaboost
• Voting
• Cascading
• Stacking…