This document discusses machine learning projects for classifying and clustering glass data using R. It first describes classifying glass data using ridge regression and plotting the results. It shows the classification performance is good based on error rate but poor based on ROC. Using higher order polynomials improves ROC, TPR and FPR. It also notes how to properly implement ridge regression. The document then demonstrates clustering glass data using multi-dimensional scaling, k-means clustering, and hierarchical clustering and compares the clustering to the original labels.

1. MACHINE LEARNING
PROJECTS WITH R
Yiou (Leo) Li

2. Outline
 Classification of glass data
 Clustering of glass data

3. Classification by ridge regression
3

4. Plotting the three classes by four features
4
Simple Scatterplot Matrix
11 12 13 14 15 0.5 1.0 1.5 2.0
1.525
V2
1.515
15
14
13
V3
12
11
4
3
V4
2
1
0
2.0
1.5
V5
1.0
0.5
1.515 1.525 0 1 2 3 4

5. Performance looks good when consider only the
classification error rate
5

6. Performance is poor when consider ROC
6

7. Using high order polynomial helps improve ROC
7
Decision point

8. Using high order polynomial helps improve TPR
and FPR!
8
Y ~ [V2, V3, …, V10, V2*V3, V2*V4, …]
Training Test
True Positive Rate 0.6820833 0.55
False Positive Rate 0.008368031 0.0804762
Error rate 0.03953965 0.1270588
Y ~ [V2, V3 … , V10]
Training Test
True Positive Rate 0 0
False Positive Rate 0.00685288 0.007142857
Error rate 0.1104277 0.1102941

9. Notes on ridge regression
9
1. The ridge solutions are not invariant under scaling of the inputs --- usually
standardize the input --- so that the solution is invariant to scaling of inputs
2. Intercept β0 should be left out of the penalty term! --- so that the solution is
invariant to the choice of origin of inputs and outputs

10. Outline
 Classification of glass data
 Clustering of glass data

11. Multi-Dimensional Scaling of glass
data (Labeled as: 1,2,3,5,6,7)
Metric MDS
6
1
2
3
5
6
4
7
Coordinate 2
2
0
-2
-4 -2 0 2 4 6
Coordinate 1

12. Kmeans of glass
K-means cluster
1.0
0.8
0.6
Correct rate
0.4
0.2
0.0
Original labels

13. Hierarchical of glass
Hierachical cluster
1.0
0.8
0.6
Correct rate
0.4
0.2
0.0
Original labels

14. Correct rate
0.0 0.2 0.4 0.6 0.8 1.0
EM of glass
EM
Original labels