Lviv Data Science Club 25.01.2018 Богдан Павлишенко (Bohdan Pavlyshenko) - "Linear, Machine Learning and Probabilistic Approaches for Predictive Analytics"

1. Linear, Machine Learning and
Probabilistic Approaches for
Predictive Analytics
B.M.Pavlyshenko
(Ph.D.)
SoftServe,
Ivan Franko National University of Lviv
e-mail: b.pavlyshenko@gmail.com

2. Linear Regression
𝑦 𝑥, 𝒘 = 𝑤0 +
𝑗=1
𝑀−1
𝑤𝑗 ∅ 𝑗(𝑥)
𝑡 = 𝑦 𝑥, 𝒘 + 𝜀
𝑝 𝑡 𝑥, 𝑤, 𝛽 = 𝑁 𝑡 𝑦 𝑥, 𝒘 , 𝛽−1
𝒘 = ( )Φ 𝑇Φ −1Φ 𝑇 𝑡
Φ =
)∅0(𝑥1 )∅1(𝑥1 ⋯ )∅ 𝑀−1(𝑥1
)∅0(𝑥2 )∅1(𝑥2 ⋯ )∅ 𝑀−1(𝑥2
⋮ ⋮ ⋱ ⋮
)∅0(𝑥 𝑁 )∅1(𝑥 𝑁 ⋯ )∅ 𝑀−1(𝑥 𝑁
𝐸 =
1
2
𝑛=1
𝑁
𝑡 𝑛 − 𝒘 𝑻∅(𝒙
2
+
𝜆
2
𝒘 𝑻 𝒘
𝒘 = ( )λ𝐈 + Φ 𝑇
Φ −1
Φ 𝑇
𝑡
𝐸 =
1
2
𝑛=1
𝑁
𝑡 𝑛 − 𝒘 𝑻∅(𝒙
2
+
𝜆
2
𝑗=1
𝑀
𝑤𝑗

3. Sampling
𝑝 𝒘 𝐷 =
𝑝 𝐷 𝒘 𝑝(𝒘)
𝑝(𝐷)
𝑝 𝐷 = 𝑝 𝐷 𝒘 𝑝 𝒘 𝑑𝒘
𝐸(𝑓 𝑧 ) = 𝑓 𝑧 𝑝 𝑧 𝑑𝑧
𝑧𝑖|𝑖 = 1, … , 𝐿
𝑓 =
1
𝐿
𝑖=1
𝐿
𝑓(𝑧𝑖)
Bayesian Inference
𝑡 = 1,2, … , 𝑇
𝒛 = 𝑧𝑖|𝑖 = 1, … , 𝑀
𝑧1
𝑡+1
~𝑝 𝑧1|𝑧2
𝑡
, 𝑧3
𝑡
, … , 𝑧 𝑀
𝑡
𝑧2
𝑡+1
~𝑝 𝑧2|𝑧1
𝑡+1
, 𝑧3
𝑡
, … , 𝑧 𝑀
𝑡
𝑧𝑗
𝑡+1
~𝑝 𝑧𝑗|𝑧1
𝑡+1
, … , 𝑧𝑗−1
𝑡+1
, 𝑧𝑗+1
𝑡
, … , 𝑧 𝑀
𝑡
𝑧 𝑀
𝑡+1
~𝑝 𝑧 𝑀|𝑧1
𝑡+1
, 𝑧2
𝑡+1
, … , 𝑧 𝑀−1
𝑡+1

4. XGBoost
http://xgboost.readthedocs.io/

5. XGBoost

6. {
Typical time series of store sales
LINEAR MODELS AND MACHINE
LEARNING APPROACHES

7. Time series forecastings by different methods

9. Time series forecastings by different methods

11. LSTM NN

14. BAYESIAN INFERENCE
Density of distributions
of regression coefficients.
Box plots for regression
coefficients

17. Winner Solution for Grupo Bimbo
Inventory Demand Kaggle Competition

20. most important features

21. Multilevel predictive model

26. Analysis of Perishable Products
Sales Using Bayesian Inference

30. Profit = SoldAmount*(Price – MarginalPrice) – UnsoldAmount* MarginalPrice

32. Machine Learning, Linear and
Bayesian Models for Logistic
Regression in Failure Detection
Problems

33. MACHINE LEARNING MODEL
The most important features and their gain values:
Matthews correlation coefficient (MCC) :

34. MACHINE LEARNING MODEL
ROC curve for
classification results
AUC=0.753
Matthews correlation
coefficient for logistic regression
for different values of probability
threshold.

35. ROC curve and Matthews correlation coefficient for different sets of features
MACHINE LEARNING MODEL
Features set 1:
AUC=0.75
Features set 2:
AUC=0.91

36. Dependence of Lambda from AUC
value.
Coefficients of the generalized linear
model for logistic regression
(Lambda=0.03 )
GENERALIZED LINEAR MODEL

37. BAYESIAN MODEL
model{
for (i in 1:n) {
y[i] ~ dbern(p[i])
logit(p[i]) <- b0+inprod(b[ ],x[i,])
}
b0 ~ dnorm(0,0.0001)
for (j in 1:nfeat) {
b[j] ~ dnorm(0,0.0001)
}
}
Probabilistic model for logistic regression using BUGS syntax

38. BAYESIAN MODEL
Trace plot for Intercept parameter. Probability density function for
Intercept parameter.

39. BAYESIAN MODEL
Box plots for logistic regression coefficients.

40. Combining Machine Learning with
Linear and Bayesian Models

41. Forecasting of Social, Market
and Financial Trends

42. The formation of keyword frequent sets with the biggest support value
The analysis of financial tweets

43. The analysis of financial tweets
The analysis of causal relationship between the frequent sets in tweets and
Apple stock prices.
The results obtained show that it is possible to predict stock prices on the
basis of data mining of informational streams in social networks.

44. The examples of test studies of semantic concepts in Twitter messages
Royal baby’s name forecasting
The name George was
dominating in the spectrum of
names before the official
announcement.

45. The examples of test studies of semantic concepts in Twitter messages
Royal baby’s name forecasting
10 first frequent sets were
created by five names, the
three of which are the
components of Prince’s
full name George
Alexander Louis.

46. The examples of test studies of semantic concepts in Twitter
messages
The Royal baby’s name forecasting
Users’ societies, which formed the discussion trends.

47. Bitcoin Price Forecasting Using
Models with Experts Opinions

49. Thank you !