Machine Learning of SECOM dataset (https://archive.ics.uci.edu/ml/datasets/SECOM) Following CRISP-DM Part 4: Building statistical model

2. ●
●
●

3. Build the classification model
1. Select features 2. Split dataset 3. Build models 4. Assess models
- Remove highly correlated
features (>0.75)
- Features reduced from 436
to 208
- 3 feature subsets
● LVQ #20
● RFE #15
● Boruta #11
- Model Set: first 80%
● Train 70%
● Test 30 %
- Validation: last 20% data
- Linear methods: Linear
Discriminant Analysis and Logistic
Regression.
- Non-Linear methods: Neural
Network, SVM, kNN
- Trees and Rules: CART
- Ensembles of Trees:
Bagging CART, Random Forest
and Stochastic Gradient Boosting
- Features selected using
RFE gave the best results
with the minimum error rate
and the highest precision
- Bagging CART selected
based on Cohen’s Kappa
(Kursa and Rudnicki 2010),
(Guyon and Elisseeff 203)
(Holte 1993) (Lee, Lessler, and Stuart 2010),
(Cutler and Zhao 2001), (Mohanbir
1996), (Kohavi 1995)
(Wilson 1927), (Cohen 1960)

4. Features BACKUP

5. BACKUP

6. BACKUP

9. ●
●
●
BACKUP

12. ●
●
●
LVQ RFE Boruta
Accuracy 92% 94% 92%
Sensitivity 0% 15% 12%
Precision 0% 67% 33%
*Wilson score interval

13. BACKUP

14. LVQ RFE Boruta All
Testing Validation Testing Validation Testing Validation Testing Validation
Accuracy 92% 95% 94% 88% 92% 91% 91% 92%
Sensitivity 0% 6% 15% 0% 12% 0% 0% 6%
Precision 0% 50% 67% 0% 33% 0% 0% 10%
Dataset:
Testing 375
Validation 314
Prevalence 7%

15. Gradient
Boosting
CART
Big Data
Random
Forest
model
Deep
neural
network
Accuracy 77% 85% 84%
Sensitivity 60% 48% 29%
Precision 17% 21% 15%
BACKUP

17. 1.
2.