Introduction of two machine learning practice using R

1. MACHINE LEARNING
PROJECTS WITH R
Yiou (Leo) Li

2. Outline
 Classification of glass data
 Clustering of glass data

3. Classification by ridge regression
3

4. Plotting the three classes by four features
4
Simple Scatterplot Matrix
11 12 13 14 15 0.5 1.0 1.5 2.0
1.525
V2
1.515
15
14
13
V3
12
11
4
3
V4
2
1
0
2.0
1.5
V5
1.0
0.5
1.515 1.525 0 1 2 3 4

5. Performance looks good when consider only the
classification error rate
5

6. Performance is poor when consider ROC
6

7. Using high order polynomial helps improve ROC
7
Decision point

8. Using high order polynomial helps improve TPR
and FPR!
8
Y ~ [V2, V3, …, V10, V2*V3, V2*V4, …]
Training Test
True Positive Rate 0.6820833 0.55
False Positive Rate 0.008368031 0.0804762
Error rate 0.03953965 0.1270588
Y ~ [V2, V3 … , V10]
Training Test
True Positive Rate 0 0
False Positive Rate 0.00685288 0.007142857
Error rate 0.1104277 0.1102941

9. Notes on ridge regression
9
1. The ridge solutions are not invariant under scaling of the inputs --- usually
standardize the input --- so that the solution is invariant to scaling of inputs
2. Intercept β0 should be left out of the penalty term! --- so that the solution is
invariant to the choice of origin of inputs and outputs

10. Outline
 Classification of glass data
 Clustering of glass data

11. Multi-Dimensional Scaling of glass
data (Labeled as: 1,2,3,5,6,7)
Metric MDS
6
1
2
3
5
6
4
7
Coordinate 2
2
0
-2
-4 -2 0 2 4 6
Coordinate 1

12. Kmeans of glass
K-means cluster
1.0
0.8
0.6
Correct rate
0.4
0.2
0.0
Original labels

13. Hierarchical of glass
Hierachical cluster
1.0
0.8
0.6
Correct rate
0.4
0.2
0.0
Original labels

14. Correct rate
0.0 0.2 0.4 0.6 0.8 1.0
EM of glass
EM
Original labels