The document discusses various machine learning models, both linear and Bayesian, focusing on logistic regression for failure detection, showcasing metrics such as Matthews correlation coefficient and AUC values. It presents the analysis of feature sets, model parameters, and the performance of combined models, including the use of the Weibull distribution for reliability studies. The author acknowledges support from Bosch for attending a conference related to the topic.

1. Machine Learning, Linear and
Bayesian Models for Logistic
Regression in Failure Detection
Problems
B. Pavlyshenko (Ph.D.)
SoftServe, Inc.,
Ivan Franko National University of Lviv,
Lviv,Ukraine

2. MACHINE LEARNING MODEL
The most important features and their gain values:
Matthews correlation coefficient (MCC) :

3. MACHINE LEARNING MODEL
ROC curve for
classification results
AUC=0.753
Matthews correlation
coefficient for logistic regression
for different values of probability
threshold.

4. Matthews correlation coefficient for different samples sets
MACHINE LEARNING MODEL

5. ROC curve and Matthews correlation coefficient for different sets of features
MACHINE LEARNING MODEL
Features set 1:
AUC=0.75
Features set 2:
AUC=0.91

6. MULTILEVEL MODEL

7. GENERALIZED LINEAR MODEL
Dependence of total within-clusters sum of
squares from number of clusters.

8. Dependence of Lambda from AUC
value.
Coefficients of the generalized linear
model for logistic regression
(Lambda=0.03 )
GENERALIZED LINEAR MODEL

9. GENERALIZED LINEAR MODEL
Histograms, correlation coefficients, pairs scatterplots for features.

10. BAYESIAN MODEL
model{
for (i in 1:n) {
y[i] ~ dbern(p[i])
logit(p[i]) <- b0+inprod(b[ ],x[i,])
}
b0 ~ dnorm(0,0.0001)
for (j in 1:nfeat) {
b[j] ~ dnorm(0,0.0001)
}
}
Probabilistic model for logistic regression using BUGS syntax

11. BAYESIAN MODEL
Trace plot for Intercept parameter. Probability density function for
Intercept parameter.

12. BAYESIAN MODEL
Box plots for logistic regression coefficients.

13. Combining Machine Learning with
Linear and Bayesian Models

14. Combining Machine Learning with Linear Model
Parameters set 1:
max.depth = 15,
colsample_bytree = 0.7
Parameters set 2:
max.depth = 5,
colsample_bytree = 0.7
Parameters set 3:
max.depth = 15,
colsample_bytree = 0.3
Matthews correlation coefficient for different
XGBoost parameter sets (features set 2):
Matthews correlation coefficient for different
XGBoost parameter sets (features set 1):

15. Combining Machine Learning with Bayesian Model

16. Study of Reliability of Parts
Weibull distribution

17. Thank you for your attention !
Special thanks to Bosch company
for awarding me the travel grant for
attending the IEEE BigData 2016
conference !