Machine Learning: Decision Trees

Copyright © SAS Institute Inc. All rights reserved.
Welcome to Webinar
Machine Learning : Decision Trees
NGSACE - SAS

Company Confidential - For Internal Use Only
Copyright © 2016, SAS Institute Inc. All rights reserved.
Global Sales Support & Enablement
AGENDA
2
Machine learning Overview
Introduction to Decision Trees
Industrial Applications of Neural Nets
A walk through demo of Neural Network Applications
SAS – NGASCE

WHAT IS MACHINE LEARNING?
FUN FACT: More than 30 years ago, SAS CEO, Jim Goodnight wrote a procedure for "k-nearest neighbor discriminant
analysis," which is a machine learning method! And growing since….
SAS Data Mining Primer course 1998
Machine
Learning
Machine learning is a branch of
artificial intelligence that
automates the building of systems
that learn iteratively from data,
identify patterns, and predict future
results – with minimal human
intervention.
It shares many approaches with other
related field, but it focuses on predictive
accuracy rather than interpretability of the
model

Automate
• Provide automation to the model building
process by minimizing human
intervention
Customize
• Build powerful models using SAS’s state-
of-the-art algorithms in conjunction with
open source tools
Speed
• Fast response time for sophisticated
analytics applied to data of any size or
complexity
SAS ANALYTICS IN ACTION
Machine Learning

DataMining Machine Learning
TRANSDUCTION
REINFORCEMENT
LEARNING
DEVELOPMENTA
LEARNING
*In semi-supervised learning, supervised prediction and classification algorithms are often combined with clustering.
SEMI-
SUPERVISED
LEARNING
Prediction and
classification*
Clustering*
EM
TSVM
Manifold
regularization
Autoencoders
Multilayer perceptron
Restricted Boltzmann
machines
SUPERVISED
LEARNING
Regression
LASSO regression
Logistic regression
Ridge regression
Decision tree
Gradient boosting
Random forests
Decision Trees
SVM
Naïve Bayes
Neighbors
Gaussian
processes
UNSUPERVISED
LEARNING
A priori rules
Clustering
k-means clustering
Mean shift clustering
Spectral clustering
Kernel density
estimation
Nonnegative
matrix
factorization
PCA
Kernel PCA
Sparse PCA
Singular value
decomposition
SOM

MACHINE LEARNING : WHY IS IT SO IMPORTANT NOW?
Data Computing
Power
Algorithms

Data is the key to driving efficient machine learning modeling: while machine learning relies on algorithms that
learn from data and require less constraints than traditional algorithms, you can use ML tools to efficiently
prepare data for further modeling
Structured Data
Online / Digital Data
Machine Data
Social Media Data

Discovery is the core of machine learning, as machine learning fully exploits the idea of creativity by allowing to
use powerful algorithms to identify patterns and trends in data
Visualization
Prediction
Machine Learning
Optimization

Deployment is a sometimes underestimated capability of the machine learning space, as while the models are
efficiently and powerfully developed, they also need to be deployed for execution in other environments and be
managed for enterprise purposes
Data Warehouse
CRM / Call Center
Mobile Channel
Devices

MACHINE LEARNING PROCESS FLOW
Data
Preprocessing
Feature
Engineering
Learning
Algorithm
Model
Evaluation
Classify/
Predict
Trained Model
Post
Processing
Raw
Data
New Incoming
Data
%
Accuracy

APPLICATIONS OF MACHINE LEARNING
Predictive Asset
Maintenance
Fraud
Credit Scoring
Next Best Offers Customer Segmentation
Targeted Acquisition /
Retention / Attrition
Real-time Ad
placements
Natural Language
Processing
Network Intrusion
Detection
Online
Recommendations
Customer Lifetime
Value

Enterprise Miner/Text Miner
• GUI interface
• Analytical data preparation
• Feature Engineering
• Sophisticated learning models
• Natural language processing
• Model life-cycle management
• Production scoring
• Integration with open source
• Industry specific modules
CAPABILITIES AND BENEFITS: TECHNOLOGY

DECISION TREES
INTRODUCTION AND INDUSTRIAL APPLICATIONS

WHAT IS DECISION TREE ?
14
A tree has many analogies in real life,
and turns out that it has influenced a
wide area of machine learning,
covering both classification and
regression. In decision analysis, a
decision tree can be used to visually
and explicitly represent decisions and
decision making. As the name goes, it
uses a tree-like model of decisions.

WHAT IS A DECISION TREE ?
15

16

• Decision tree refers to the tree structure of rules (often association
rules).
• The decision tree modeling process involves collecting those variables
that the analyst thinks might bear on the decision at issue, and analyzing
these variables for their ability to predict the outcome.
• The algorithm automatically determines which variables are most
important, based on their ability to sort the data into the correct output
category.
• Decision tree has a relative advantage over ANN or SVM in that a
explainatory set of rules are provided, this explaining model conclusions.

DECISION TREE APPLICATIONS
18

ADVANTAGES OF DECISION TREES
19
• Simple to understand and interpret. People are able to understand decision tree
models after a brief explanation. Trees can also be displayed graphically in a way
that is easy for non-experts to interpret.
• Able to handle both numerical and categorical data
• Requires little data preparation. Other techniques often require data normalization.
• Uses a white box model. It Possible to validate a model using statistical tests. That
makes it possible to account for the reliability of the model.
• Non-statistical approach that makes no assumptions of the training data or
prediction residuals; e.g., no distributional, independence, or constant variance
assumptions
• Performs well with large datasets. Large amounts of data can be analyzed using
standard computing resources in reasonable time.

20
Model Essentials: Decision Trees
...
Pruning
Split search
Predict new cases.
Select useful inputs
Optimize complexity
Select useful inputs.
Optimize complexity.
Prediction rules
Split search
Pruning

21
Simple Prediction Illustration
...
0.0 0.50.1 0.2 0.3 0.4 0.6 0.7 0.8 0.9 1.0
0.0
0.5
0.1
0.2
0.3
0.4
0.6
0.7
0.8
0.9
1.0
x1
x2
Predict dot color
for each x1 and x2.
Training Data

22
Simple Prediction Illustration
...
0.0 0.50.1 0.2 0.3 0.4 0.6 0.7 0.8 0.9 1.0
0.0
0.5
0.1
0.2
0.3
0.4
0.6
0.7
0.8
0.9
1.0
x1
x2
Predict dot color
for each x1 and x2.
Training Data

23
Decision Tree Prediction Rules
...
0.0 0.50.1 0.2 0.3 0.4 0.6 0.7 0.8 0.9 1.0
0.0
0.5
0.1
0.2
0.3
0.4
0.6
0.7
0.8
0.9
1.0
x1
x2
40%
60%
55%
70%
x1
<0.52 ≥0.52 <0.51 ≥0.51
x1
x2
<0.63 ≥0.63
root node
interior node
leaf node

24
...
0.0 0.50.1 0.2 0.3 0.4 0.6 0.7 0.8 0.9 1.0
0.0
0.5
0.1
0.2
0.3
0.4
0.6
0.7
0.8
0.9
1.0
x1
x2
40%
60%
55%
x1
<0.52 ≥0.52
<0.63
70%
<0.51 ≥0.51
x1
x2
≥0.63
root node
interior node
leaf node
Predict:

25
...
≥0.51
40%
60%
55%
x1
<0.52 ≥0.52
<0.63
40%
60%
55%
x1
<0.52 ≥0.52 ≥0.51
<0.63
0.0 0.50.1 0.2 0.3 0.4 0.6 0.7 0.8 0.9 1.0
0.0
0.5
0.1
0.2
0.3
0.4
0.6
0.7
0.8
0.9
1.0
x1
x2
Decision =
Estimate = 0.70
70%
<0.51
x1
x2
≥0.63
Predict:

26
...
40%
60%
55%
x1
<0.52 ≥0.52 ≥0.51
<0.63
0.0 0.50.1 0.2 0.3 0.4 0.6 0.7 0.8 0.9 1.0
0.0
0.5
0.1
0.2
0.3
0.4
0.6
0.7
0.8
0.9
1.0
x1
x2
Decision =
Estimate = 0.70
70%
<0.51
x1
x2
≥0.63
Predict:

27
...
Pruning
Split search
Predict new cases.
Select useful inputs.
Prediction rules

28
...
Prediction rulesPrediction rules
Split searchSplit search
Predict new cases.Predict new cases.
Select useful inputs.Select useful inputs.
PruningOptimize complexity.

29
Split search
Prediction rulesPredict new cases.
Pruning
Select useful inputs
Select useful inputs. Split search

30
Decision Tree Split Search
0.0 0.50.1 0.2 0.3 0.4 0.6 0.7 0.8 0.9 1.0
x1
0.0
0.5
0.1
0.2
0.3
0.4
0.6
0.7
0.8
0.9
1.0
x2
Calculate the logworth
of every partition on
input x1.
left right
Classification Matrix

31
0.0 0.50.1 0.2 0.3 0.4 0.6 0.7 0.8 0.9 1.0
x1
0.0
0.5
0.1
0.2
0.3
0.4
0.6
0.7
0.8
0.9
1.0
x2
Calculate the logworth
of every partition on
input x1.
left right
Classification Matrix

32
0.0 0.50.1 0.2 0.3 0.4 0.6 0.7 0.8 0.9 1.0
0.0
0.5
0.1
0.2
0.3
0.4
0.6
0.7
0.8
0.9
1.0
x1
x2
max
logworth(x1)
0.95
0.52
left right
Select the partition with
the maximum logworth.
53%
47%
42%
58%

33
0.0 0.50.1 0.2 0.3 0.4 0.6 0.7 0.8 0.9 1.0
0.0
0.5
0.1
0.2
0.3
0.4
0.6
0.7
0.8
0.9
1.0
x1
x2
max
logworth(x1)
0.95
left right
53% 42%
47% 58%
Repeat for input x2.

34
max
logworth(x1)
0.95
left right
53% 42%
47% 58%
0.0 0.50.1 0.2 0.3 0.4 0.6 0.7 0.8 0.9 1.0
0.0
0.5
0.1
0.2
0.3
0.4
0.6
0.7
0.8
0.9
1.0
x1
x2
0.63
max
logworth(x2)
4.92
bottom top
54%
46%
35%
65%

35
0.0 0.50.1 0.2 0.3 0.4 0.6 0.7 0.8 0.9 1.0
0.0
0.5
0.1
0.2
0.3
0.4
0.6
0.7
0.8
0.9
1.0
x1
x2
max
logworth(x2)
4.92
bottom top
max
logworth(x1)
0.95
left right
Compare partition
logworth ratings.
54%
46%
35%
65%
53%
47%
42%
58%

36
0.0 0.50.1 0.2 0.3 0.4 0.6 0.7 0.8 0.9 1.0
0.0
0.5
0.1
0.2
0.3
0.4
0.6
0.7
0.8
0.9
1.0
x1
x2
0.63
x2
<0.63 ≥0.63
Create a partition rule
from the best partition
across all inputs.

37
0.0 0.50.1 0.2 0.3 0.4 0.6 0.7 0.8 0.9 1.0
0.0
0.5
0.1
0.2
0.3
0.4
0.6
0.7
0.8
0.9
1.0
x1
x2
x2
<0.63 ≥0.63
Repeat the process
in each subset.

38
0.0 0.50.1 0.2 0.3 0.4 0.6 0.7 0.8 0.9 1.0
0.0
0.5
0.1
0.2
0.3
0.4
0.6
0.7
0.8
0.9
1.0
x1
x2
0.52
max
logworth(x1)
5.72
left right
61%
39%
55%
45%

39
0.0 0.50.1 0.2 0.3 0.4 0.6 0.7 0.8 0.9 1.0
0.0
0.5
0.1
0.2
0.3
0.4
0.6
0.7
0.8
0.9
1.0
x1
x2
max
logworth(x1)
5.72
left right
61% 55%
39% 45%
0.02
max
logworth(x2)
-2.01
bottom top
38%
62%
55%
45%

40
0.0 0.50.1 0.2 0.3 0.4 0.6 0.7 0.8 0.9 1.0
0.0
0.5
0.1
0.2
0.3
0.4
0.6
0.7
0.8
0.9
1.0
x1
x2
max
logworth(x2)
-2.01
bottom top
38%
62%
55%
45%
max
logworth(x1)
5.72
left right
61%
39%
55%
45%

41
0.0 0.50.1 0.2 0.3 0.4 0.6 0.7 0.8 0.9 1.0
0.0
0.5
0.1
0.2
0.3
0.4
0.6
0.7
0.8
0.9
1.0
x1
x2
0.52
max
logworth(x2)
-2.01
bottom top
38% 55%
62% 45%
max
logworth(x1)
5.72
left right
61%
39%
55%
45%

42
0.0 0.50.1 0.2 0.3 0.4 0.6 0.7 0.8 0.9 1.0
0.0
0.5
0.1
0.2
0.3
0.4
0.6
0.7
0.8
0.9
1.0
x1
x2
x2
x1
<0.63 ≥0.63
<0.52 ≥0.52
Create a second
partition rule.

43
Repeat to form a maximal tree.
0.0 0.50.1 0.2 0.3 0.4 0.6 0.7 0.8 0.9 1.0
0.0
0.5
0.1
0.2
0.3
0.4
0.6
0.7
0.8
0.9
1.0
x1
x2

44
Pruning
Split searchSelect useful inputs
Select useful inputs. Split search

45
Optimize complexityOptimize complexity.
Split searchSelect useful inputs.
Pruning

46
Predictive Model Sequence
6
5
4
3
2
Training Data Validation Data
Model
Complexity
1
Create a sequence
of models with
increasing
complexity from the
training data.
inputs target inputs target

47
Maximal Tree
6
5
4
3
2
Model
Complexity
1
Maximal
Tree
Create a sequence
of models with
increasing
complexity from the
training data.
A maximal tree is the
most complex model
in the sequence.

48
Maximal Tree
5
4
3
2
1
A maximal tree is
the most complex
model in the
sequence.
Model
Complexity

49
Pruning One Split
4
3
2
1
Model
Complexity
The next model in
the sequence is
formed by pruning
one split from the
maximal tree.

50
Pruning One Split
4
3
2
1
Model
Complexity
Each subtree’s
predictive
performance is rated
on validation data.

51
Pruning One Split
4
3
2
1
Model
Complexity
The subtree with
the highest
validation
assessment is
selected.

52
Pruning Two Splits
4
3
2
1
Model
Complexity
Similarly, this is
done for
subsequent
models.

53
Pruning Two Splits
3
2
1
Model
Complexity
Prune two splits from the
maximal tree,…

54
Pruning Two Splits
3
2
1
Model
Complexity
…rate each subtree using validation
assessment, and…

55
Pruning Two Splits
3
2
1
Model
Complexity
…select the subtree with
the best assessment rating.

56
Subsequent Pruning
Model
Complexity
Continue pruning until all
subtrees are considered.

57
Selecting the Best Tree
Model
Complexity
Validation
Assessment
Compare validation assessment
between tree complexities.

58
Validation Assessment
Choose the simplest model with
the highest validation assessment.
Model
Complexity
Validation
Assessment

59
Validation Assessment
What are appropriate
validation assessment
ratings?

60
Assessment Statistics
inputs target
Validation Data
target measurement
(binary, continuous,
and so on)
prediction type (decisions,
rankings, estimates)
Ratings depend on…

61
Binary Targets
inputs
primary outcome
secondary outcome
target
1
0
1
1
0

62
Binary Target Predictions
inputs target
1
0
1
1
0
prediction
primary
secondary
0.249
720
520 rankings
estimates
decisions

63
Decision Optimization
inputs target
1
0
1
1
0
prediction
0.249
720
520
primary
secondary
decisions

64
Decision Optimization: Misclassification
inputs target
1
0
1
1
0
prediction
0.249
720
520
secondary
primarysecondary
primary
false negative
false positive
Minimizemisclassification:
disagreementbetween
outcomeandprediction

65
Ranking Optimization
inputs target
1
0
1
1
0
prediction
0.249
720
520
secondary
primary
1
0
720
520 rankings
estimates
decisions

66
Ranking Optimization: Concordance
inputs target
1
0
1
1
0
prediction
0.249
720
520
secondary
primary
1
0
720
520
Maximize concordance:
proper ordering of
primary and secondary
outcomes
target=0→low score
target=1→high score

67
Estimate Optimization
inputs target
1
0
1
1
0
prediction
0.249
secondary
primary
720
520
1 0.249
rankings
estimates
decisions

68
Estimate Optimization: Squared Error
inputs target
1
0
1
1
0
prediction
0.249
secondary
primary
720
520
1 0.249 (target – estimate)2
Minimize squared error:
squared difference between
target and prediction

69
Complexity Optimization: Summary
inputs target
1
0
1
1
0
prediction
0.249
secondary
primary
720
520
concordance / discordance
squared error
accuracy / misclassification
rankings
estimates
decisions

ORGANIC PRODUCTS
What are my potential customers ??
Can I find pointed segments for buyers??
Can I predict who will buy organic products ??
What advertising strategy I should adopt to
sell products ??
OR

PREDICTING ORGANIC PRODUCT BUYING BEHAVIOR
71
• DEMONSTRATION

UNIVERSITY & AIRLINE BOMBER – UNABOMBER
Mathematical
Genius
Youngest PhD
Holder & Professor
In jail Now !

PREDICTING AUTHORSHIP: UNSTRUCTURED DATA ML
73
• DEMONSTRATION

Global Sales Support & Enablement74
NGASCE and SAS

ABOUT NMIMS AND SAS
In 1981, Shri Vile Parle Kelavani Mandal (SVKM) established Narsee Monjee Institute of Management Studies (NMIMS)
to meet the growing demand for management education.
In 2003, NMIMS was declared a deemed-to- be university under section 3 of the UGC Act 1956. With the legacy of
35+ years, NMIMS has grown to being not only one of the top 10 B-schools in India but has also emerged as a multi-
disciplinary University.
SAS tops all predictive and advanced analytics suppliers and data integration suppliers, according to the IDC report,
IDC Worldwide Business Intelligence and Analytics Tools Software Market Shares, 2016: Here Comes the Cloud. SAS
held a 30.5% market share for 2016 in the advanced and predictive analytics category, well over twice the market
share of the next-closest competitor. SAS has led in this category since IDC started tracking the market in 1997. SAS
has demonstrated continued growth every year in the category, with 2016 showing a 5 percent revenue growth.
"SAS has been able to retain authority in the advanced and predictive analytics market and continue to grow year
over year,"" said Dan Vesset, Group Vice President of Analytics and Information Management at IDC.
In addition, IDC ranked SAS as the 2016 market share leader for analytic data integration software with 21.9 percent
market share.

About NMIMS and SAS partnership
2011- 1ST SAS EAS
License procured for 1
Campus of SVKM’S
NMIMS.
2012 – Upgraded to
campus license across 4
Campus of SVKM’S
NMIMS.
2013 – SAS
Education Services
got incorporated for 3
SVKM’S NMIMS
Schools.
2014 – Upgraded to
SAS E MINER License
in 4 Campus of SVKM’S
NMIMS.
2016 | 2018 – SVKM’S Trust +
NMIMS University Upgraded
across the SVKM’S Institutions
for SAS VA License.
50+ Faculty will be SAS Trained
+ SAS Globally Certified & SAS
Accredited 2016-2017 and
another 20+ Faculties to be in
2017.
Minimum 1000 Students will be
SAS Globally certified each
year across 33 Institutions
under SVKM for the next 5
Years i.e. 2016 - 2020.
SVKM’S NMIMS Center of
Excellence – Business
Analytics & Data Sciences.
SVKM’S NMIMS DISTANCE
LEARNING ONLINE COURSES
For the first time in India, NMIMS and SAS
have joined hands to provide a platform to
working professionals and students, hard pressed for
time, to learn the most important SAS tools through
the Online Learning mode.

PROGRAMS DETAILS:
Management Program in Data Visualization
Duration: 6 months include: [Lectures, Practice sessions, Exams and Global Certification]
 Business Statistics [24 hrs/2 hrs per session/1 month]
Doubt clearing session post all session are over
 Visual Analytics [32 Hours/2 hours per session /4 months]
Post every 4 sessions doubt clearing session with the faculty
 VA Exams and Global Certification [1 month]
2 attempts to clear the subject and prep session before re-sit attempt
Prep session before Global Certification
SAS VA tool: 20 hrs during lectures and 30 hrs post lectures for practice
CERTIFICATION:
SAS Certified Visual Business Analyst: Exploration and Design Using SAS Visual Analytics
* Additional 15 hrs @ 5020 + 18% GST (minimum requirement)

PROGRAMS DETAILS:
Executive Program in Big Data and Machine Learning [Predictive Model]
Duration: 9 months include: [Lectures, Practice sessions, Exams and Global Certification]
 Business Statistics [24 hours/2 hours per session/1 month]
Doubt clearing session post all session are over
 Analytics Bridge Course [24 hours/ 2 hrs per session/ 1.5 months]
 Predictive Model-E-guide [24 hours/ 2 hours per session/2 months]
 E-Miner [24 hrs / 2 hours per session/2.5 months]
 Domain specific case studies “Financial risk Management”, “Marketing analytics”, “Operations and
Supply chain analytics” [1 month]
 ML Exams and Global Certification [1 month]
2 attempts to clear the subject and prep session before re-sit attempt
Prep session before Global Certification
SAS EMINER tool: 30 hrs during lectures and 20 hrs post lectures for practice
CERTIFICATION:
SAS Certified Predictive Modeler Using SAS Enterprise Miner 14
* Additional 15 hrs @ 5020 + 18% GST (minimum requirement)

WHY THIS PROGRAM WITH NMIMS
 Latest course material from SAS
 SAS Faculties
 E-Badge [2 levels]
 Sessions on “Business Statistics”, to strengthen your journey on Visual Analytics
 Session on “Analytics Bridge Course” benefiting your learning on Predictive Model
 Mentored sessions by SAS faculties
 Practicing data
 Fixing opportunities identified during exams
 Defined batch size for Quality interaction during lectures
 Sessions conducted through Amazon Web Services, exclusively used for NMIMS
 Global Certification from SAS [Preparation session conducted by SAS]
 Weekend classes and Online interface
 Joint Certification NMIMS and SAS
 Flexible Exam pattern
 Shorter Duration
 Huge depth of Analytical content
 24/7 access to SAS Software

Questions !!
Thanks !!
For Queries
Email: ngasce@nmims.edu
Toll Free: 1-800-1025-136 Mon-Sat (10am – 6pm)

Machine Learning: Decision Trees

Recommended

Recommended

More Related Content

Similar to Machine Learning: Decision Trees

Similar to Machine Learning: Decision Trees (20)

More from NMIMS Global Access School of Continuing Education (NGA-SCE)

More from NMIMS Global Access School of Continuing Education (NGA-SCE) (6)

Recently uploaded

Recently uploaded (20)

Machine Learning: Decision Trees