Machine Learning based predictive analytics

Outline
Introduction
Artiﬁcial neural network
Penalised Linear Regression
Machine Learning based predictive analytics
Dr. Sanjay Shitole
Associate Professor in Information Technology
Secretary-IEEE-GRSS Bombay Chapter
HOD-Department of Information Technology
Usha Mittal Institute of Technology
SNDT Women’s University, Mumbai.
3rd Biennial International Conference on ‘‘Recent Trends in Image Processing and Pattern Recognition
Jan 03, 2020
Dr. Sanjay Shitole, Secretary (IEEE-GRSS Bombay Chapter) www.sanjayshitole.com

Outline
Introduction
Outline of Topics
Outline
Introduction
Useful web links and Book
What is Machine Learning
Choosing Algorithm: Linear or Nonlinear
Performance measures
Data set

Outline
Introduction
Need

Outline
Introduction
Predictive Analytics using Machine Learning with R Link
Predictive analytics and machine learning Link
Penalized linear regression video youtube Link
Book Machine Learning in Python by Michael Bowles

Outline
Introduction
What is Machine Learning ( source: Andrew Ng’s Coursera online course)
Arthur Samual -1959 Field of study that gives computers the
ability to learn without being explicitly programmed.

Outline
Introduction
What is Machine Learning ( source: Andrew Ng’s Coursera online course)
Arthur Samual -1959 Field of study that gives computers the
ability to learn without being explicitly programmed.
Tom Mitchell -1998 A computer program is said to learn from
experience E with respect to some task T and some
performance measure P, if its performance on T, as
measured by P, improves with experience E.

Outline
Introduction
Machine Learning algorithms
Boosted decision trees
Random forest
Bagged decision tress
logistic regression
Support vector machine
K nearest neighbours
Artiﬁcial neural networks
. . .

Outline
Introduction
Types of Learning
Learning rules
Implementation

Outline
Introduction
Linear Regression : Function approximation
( source: Jack M. Zurada: Intro. to ANN, here Equation 1.2 is h(x) = 0.8sinπx)

Outline
Introduction
Data set
Regression
( source: Andrew Ng’s Coursera online course)

Outline
Introduction
Data set
Classiﬁcation

Outline
Introduction
Data set
Linear Regression

Outline
Introduction
Data set
Determining what attributes to use for making predictions is
called feature engineering
Data cleaning and feature engineering take 80% to 90% of a
data scientist’s time

Outline
Introduction
Data set
Linear Nonlinear
Preferred when data set
has more columns than
rows
Preferred for complex
problemswith many
rows data
when underlying prob-
lem is simple
Training time Much faster Many models generated
during training, takes
time

Outline
Introduction
Data set
Performance measures of regression algorithm
In regression problem both the target and the prediction are
real numbers
Error is naturally deﬁned as the diﬀerence between the target
and the prediction
It is useful to generate statistical summaries of the errors for
comparison and for diagnostic
The most frequently used summarise are
Mean Square Error (MSE)
Mean Absolute Error (MAE)
Root Mean Square Error (RMSE)

Outline
Introduction
Data set

Outline
Introduction
Data set
Training data set : Testing data set
Test data size Normally 25 to 35 Percent
Traing data size Normally 75 to 65 Percent
Figure: N fold cross validation: Source: Machine Learning by Michale Bowles

Outline
Introduction
Data set
Training data set : Testing data set
Test data size Normally 25 to 35 Percent
Traing data size Normally 75 to 65 Percent
Note One has to keep in mind that the performance of the
trained model deteriorates as the size of the training
data set shrinks
Figure: N fold cross validation: Source: Machine Learning by Michale Bowles

Outline
Introduction
Data set
After training “After a model has been trained and tested it is
good practice to recombine the training and test data
into a single set and retrain the model on the largest
data set”
Source: Machine Learning by Michale Bowles
Overfit If there is significant difference between errors on the
training data and testing data

Outline
Introduction
Data set
Ridge regression
Over ﬁtting is inherent in Ordinary least square (OLS)

Outline
Introduction
Data set
Ridge regression
Ridge regression is speciﬁc example of a penalized linear
regression algorithm

Outline
Introduction
Data set
Ridge regression
Ridge regression is specific example of a penalized linear
regression algorithm
Ridge regression regulates over fitting by penalizing the sum
of the regression coefficients squared.

Outline
Introduction
Data set
Why Penalised Linear Regression
Extremely fast coeﬃcient estimation

Outline
Introduction
Data set
Variable importance information

Outline
Introduction
Data set
Extremely fast evaluation when deployed

Outline
Introduction
Data set
Reliable performance

Outline
Introduction
Data set
Spares solutions

Outline
Introduction
Data set
Spares solutions
Problem may require linear model

Outline
Introduction
Data set
Easy to use, they do not have many tunable parameters

Outline
Introduction
Data set
They have well deﬁned and well structured input types

Outline
Introduction
Data set
They solve several types of problems in regression and
classiﬁcation

Outline
Introduction
Data set
They solve several types of problems in regression and
classiﬁcation
One of their most important features is that they indicate
which of their input variables is most important features for
producing variables. This turn out to be an invaluable feature
in machine learning algorithm.

Outline
Introduction
Data set
Example Training Set
OUTCOMES FEATURES 1 FEATURE2 FEATURE3
$ spent 2013 Gender $ spent 2012 Age
100 M 0.0 25
225 F 250 32
75 F 12 17
Outcomes (target, label, endpoint), feature2 and feature 3 are
real valued [‘FEATURES’ are attributes ]
The gender attribute (Feature 1) is two values, making it
categorical (factor) attribute

Outline
Introduction
Data set
Goal with a function approximation problem
build a function relating the attributes to the outcome
to minimize the error in the same sense

Outline
Introduction
Data set
Dataset shown in Table can be shown in Matrix from
Y =





y1
y2
...
yn





X =





x11 x12 . . . x1m
x21 x22 . . . x2m
...
...
...
...
xn1 xn2 . . . xnm





Vector of outcomes Matrix of attributes
β =





β1
β2
...
βm





Vector of model coeﬃcients

Outline
Introduction
Data set
Prediction of yi = xi ∗ β + β0
= xi1 ∗ β1 + xi2 ∗ β2 + . . . + xim ∗ βm + β0
To make a prediction, take each attribute, multiply it by
corresponding beta, sum these products, and add a constant.
Training a model means Finding the numbers that make
vector β and constant β0.
Error means diﬀerence between the actaul value of yi and the
prediction of yi .
Minimzation of above equation yields ordianry least square values
βbest
0 βbest = argminβ0β(1
n
n
i=1(yi − (xi ∗ β + β0))2)

Outline
Introduction
Data set
Adding a coefficient penalty to the OLS formulation
Following coefficient penalty equation is used. This will be useful
to balance the conflicting goals of minimizing the squared
prediction error and the squared values of the coefficients.
λβT β
2
=
λ(β2
1 + β2
2 + . . . + β2
n)
2
The parameter λ can range between 0 and plus infinity.
Solve penalized minimization problem for a verity of different
values of λ and test each of these solution on out of sample data,
and the solution that minimizes the out of sample error is used for
making real word predictions.

Outline
Introduction
Data set
Thank you

Machine Learning based predictive analytics

Recommended

Recommended

More Related Content

What's hot

What's hot (11)

Similar to Machine Learning based predictive analytics

Similar to Machine Learning based predictive analytics (20)

More from Dr. Sanjay Shitole

More from Dr. Sanjay Shitole (11)

Recently uploaded

Recently uploaded (20)

Machine Learning based predictive analytics