This presentation about Decision Tree Tutorial will help you understand what is decision tree, what problems can be solved using decision trees, how does a decision tree work and you will also see a use case implementation in which we do survival prediction using R. Decision tree is one of the most popular Machine Learning algorithms in use today, this is a supervised learning algorithm that is used for classifying problems. It works well classifying for both categorical and continuous dependent variables. In this algorithm, we split the population into two or more homogeneous sets based on the most significant attributes/ independent variables. In simple words, a decision tree is a tree shaped algorithm used to determine a course of action. Each branch of the tree represents a possible decision, occurrence or reaction. Now let us get started and understand how does Decision tree work. Below topics are explained in this Decision tree in R presentation : 1. What is Decision tree? 2. What problems can be solved using Decision Trees? 3. How does a Decision Tree work? 4. Use case: Survival prediction in R About Simplilearn Machine Learning course: A form of artificial intelligence, Machine Learning is revolutionizing the world of computing as well as all people’s digital interactions. Machine Learning powers such innovative automated technologies as recommendation engines, facial recognition, fraud protection and even self-driving cars. This Machine Learning course prepares engineers, data scientists and other professionals with knowledge and hands-on skills required for certification and job competency in Machine Learning. Why learn Machine Learning? Machine Learning is taking over the world- and with that, there is a growing need among companies for professionals to know the ins and outs of Machine Learning The Machine Learning market size is expected to grow from USD 1.03 Billion in 2016 to USD 8.81 Billion by 2022, at a Compound Annual Growth Rate (CAGR) of 44.1% during the forecast period. By the end of this Machine Learning course, you will be able to: 1. Master the concepts of supervised, unsupervised and reinforcement learning concepts and modelling. 2. Gain practical mastery over principles, algorithms, and applications of Machine Learning through a hands-on approach which includes working on 28 projects and one capstone project. 3. Acquire thorough knowledge of the mathematical and heuristic aspects of Machine Learning. 4. Understand the concepts and operation of support vector machines, kernel SVM, naive Bayes, decision tree classifier, random forest classifier, logistic regression, K-nearest neighbours, K-means clustering and more. 5. Be able to model a wide variety of robust Machine Learning algorithms including deep learning, clustering, and recommendation systems Learn more at: https://www.simplilearn.com/big-data-and-analytics/machine-learning-certification-training-course

2. What’s in it for you?
What is a Decision Tree?
What problems can be solved using Decision Trees?
How does a Decision Tree work?
Use Case: Survival prediction in R

3. What is a Decision Tree?

4. Left or right?
What is a Decision Tree?

5. Wait or go? Left or right?
What is a Decision Tree?

6. Decision Tree is a tree shaped algorithm used to determine a course of action.
Each branch of the tree represents a possible decision, occurrence or reaction
Wait or go? Left or right?
What is a Decision Tree?

7. What is a Decision Tree?
Shopkeeper

8. What is a Decision Tree?
I must organize
my stall..
Shopkeeper

9. What is a Decision Tree?
Shopkeeper

10. What is a Decision Tree?
Is it coloured orange?
Is it round?
No Yes
YesNo
Shopkeeper

11. What problems can be solved using Decision Tree?
Classification:
Identifying to which set an object belongs
Example: Carrot is orange while broccoli is
green

12. What problems can be solved using Decision Tree?
Classification:
Identifying to which set an object belongs
Example: Carrot is orange while broccoli is
green
Regression:
Regression problems have continuous or
numerical valued output variables
Example: Predicting the profits of a company

13. How does a Decision Tree work?
Terms you must know
first…..

14. How does a Decision Tree work?
NODES
Each internal node in a decision tree is a test which splits the objects into
different categories
Is it coloured orange?
Is it round?
No Yes
YesNo
Terms you must know
first…..

15. How does a Decision Tree work?
Is it coloured orange?
Is it round?
No Yes
YesNoThis is a node
Terms you must know
first…..
NODES
Each internal node in a decision tree is a test which splits the objects into
different categories

16. How does a Decision Tree work?
Is it coloured orange?
Is it round?
No Yes
YesNo
ROOT NODE
The node at the top of the decision tree is called the Root node
Terms you must know
first…..

17. How does a Decision Tree work?
ROOT NODE
The node at the top of the decision tree is called the Root node
Is it coloured orange?
Is it round?
No Yes
YesNo
This is a root node
Terms you must know
first…..

18. How does a Decision Tree work?
LEAF NODE
Each external node in a decision tree is called the leaf node. The leaf
node is the output
Is it coloured orange?
Is it round?
No Yes
YesNo
Terms you must know
first…..

19. How does a Decision Tree work?
Is it coloured orange?
Is it round?
No Yes
YesNo
This is a leaf node
LEAF NODE
Each external node in a decision tree is called the leaf node. The leaf
node is the output
Terms you must know
first…..

20. How does a Decision Tree work?
ENTROPY
Entropy is a measure of the messiness of your data collection. The
messier or more random your data, higher will be the entropy
Terms you must know
first…..

21. How does a Decision Tree work?
This collection has high entropy
Terms you must know
first…..
ENTROPY
Entropy is a measure of the messiness of your data collection. The
messier or more random your data, higher will be the entropy

22. How does a Decision Tree work?
This collection has high entropy
This collection has low entropy
Terms you must know
first…..
ENTROPY
Entropy is a measure of the messiness of your data collection. The
messier or more random your data, higher will be the entropy

23. How does a Decision Tree work?
INFORMATION GAIN
Information gain is the decrease obtained in entropy by splitting the data
set based on some condition
Is it coloured orange?
Is it round?
No Yes
YesNo
E1
E2
Terms you must know
first…..

24. How does a Decision Tree work?
INFORMATION GAIN
Information gain is the decrease obtained in entropy by splitting the data
set based on some condition
Is it coloured orange?
Is it round?
No Yes
YesNo
E1
E2
E1>E2
INFORMATION GAIN=E1-E2
Terms you must know
first…..

25. How does a Decision Tree work?
Hi, my cupboard is
a mess. I must
organize my stuff..

26. How does a Decision Tree work?
Classify the objects based
on their attribute set using
decision trees

27. How does a Decision Tree work?
Shape Size Label Number
round 10 ball 2
rectangle 6 book 2
rectangle 4 card 1
rectangle 2 blocks 4
total 9
Let’s look at the attributes
for each object

28. How does a Decision Tree work?
• We split at each level based on certain
conditions on the attributes
Shape Size Label Number
round 10 ball 2
rectangle 6 book 2
rectangle 4 card 1
rectangle 2 blocks 4
total 9

29. How does a Decision Tree work?
• We split at each level based on certain
conditions on the attributes
• Splitting aims at reducing the entropy
Shape Size Label Number
round 10 ball 2
rectangle 6 book 2
rectangle 4 card 1
rectangle 2 blocks 4
total 9

30. How does a Decision Tree work?
• We split at each level based on certain
conditions on the attributes
• Splitting aims at reducing the entropy
-∑ⁱₓ₌₁p(valueₓ).log₂(p(valueₓ))
Shape Size Label Number
round 10 ball 2
rectangle 6 book 2
rectangle 4 card 1
rectangle 2 blocks 4
total 9

31. How does a Decision Tree work?
-[2/9.log₂(2/9) + 2/9.log₂(2/9) + 1/9.log₂(1/9) + 4/9.log₂(4/9)]=
-∑ⁱₓ₌₁p(valueₓ).log₂(p(valueₓ))
Shape Size Label Number
round 10 ball 2
rectangle 6 book 2
rectangle 4 card 1
rectangle 2 blocks 4
total 9

32. How does a Decision Tree work?
-[2/9.log₂(2/9) + 2/9.log₂(2/9) + 1/9.log₂(1/9) + 4/9.log₂(4/9)]= 1.8282
-∑ⁱₓ₌₁p(valueₓ).log₂(p(valueₓ))
Shape Size Label Number
round 10 ball 2
rectangle 6 book 2
rectangle 4 card 1
rectangle 2 blocks 4
total 9

33. How does a Decision Tree work?
Now we must find the conditions
for our split. Every split must give
us the maximum achievable
information gain Shape Size Label Number
round 10 ball 2
rectangle 6 book 2
rectangle 4 card 1
rectangle 2 blocks 4
total 9

34. How does a Decision Tree work?
Our first split will be on
shape as that will
directly segregate the
balls
Shape Size Label Number
round 10 ball 2
rectangle 6 book 2
rectangle 4 card 1
rectangle 2 blocks 4
total 9
E1 = 1.8282

35. How does a Decision Tree work?
Shape Size Label Number
round 10 ball 2
rectangle 6 book 2
rectangle 4 card 1
rectangle 2 blocks 4
total 9
Shape == Rectangle?
E2 = 1.3784
E1 = 1.8282

36. How does a Decision Tree work?
Our second split will be
on size as that will
directly segregate the
books
Shape Size Label Number
round 10 ball 2
rectangle 6 book 2
rectangle 4 card 1
rectangle 2 blocks 4
total 9
Shape == Rectangle?
E2 = 1.3784
E1 = 1.8282

37. How does a Decision Tree work?
Shape == Rectangle?
Size>5?
Shape Size Label Number
round 10 ball 2
rectangle 6 book 2
rectangle 4 card 1
rectangle 2 blocks 4
total 9
E3 = 0.716
E2 = 1.3784
E1 = 1.8282

38. How does a Decision Tree work?
Our third split will
once again be on
size
Size>5?
Shape Size Label Number
round 10 ball 2
rectangle 6 book 2
rectangle 4 card 1
rectangle 2 blocks 4
total 9
Shape == Rectangle?
E3 = 0.716
E2 = 1.3784
E1 = 1.8282

39. How does a Decision Tree work?
Size>3?
Size>5?
Shape Size Label Number
round 10 ball 2
rectangle 6 book 2
rectangle 4 card 1
rectangle 2 blocks 4
total 9
Shape == Rectangle?
E3 = 0.716
E2 = 1.3784
E1 = 1.8282
E4 = 0

40. How does a Decision Tree work?
Size>3?
Size>5?
Shape Size Label Number
round 10 ball 2
rectangle 6 book 2
rectangle 4 card 1
rectangle 2 blocks 4
total 9
Shape == Rectangle?
E2 = 1.3784
E3 = 0.716
E1 = 1.8282
E4 = 0

41. How does a Decision Tree work?
All our objects are now
classified with 100%
accuracy
Size>3?
Size>5?
Shape Size Label Number
round 10 ball 2
rectangle 6 book 2
rectangle 4 card 1
rectangle 2 blocks 4
total 9
Shape == Rectangle?

42. How does a Decision Tree work?
All our objects are now
classified with 100%
accuracy
Size>3?
Size>5?
Shape Size Label Number
round 10 ball 2
rectangle 6 book 2
rectangle 4 card 1
rectangle 2 blocks 4
total 9
Shape == Rectangle?

43. Use Case: Survival prediction in R
Let’s implement classification of a data set based on
Information Gain

44. Let’s implement classification of a data set based on
Information Gain
This is the ID3 algorithm
Use Case: Survival prediction in R

45. Let’s implement classification of a data set based on
Information Gain
This is the ID3 algorithm
We will be using the RStudio IDE
Use Case: Survival prediction in R

46. • A ship had 20 lifeboats
Use Case: Survival prediction in R

47. • A ship had 20 lifeboats
• The lifeboats were
distributed based on the
class, gender and age of
the passengers
Use Case: Survival prediction in R

48. • A ship had 20 lifeboats
• The lifeboats were
distributed based on the
class, gender and age of
the passengers
• We will develop a model
that recognises the
relationship between these
factors and predicts the
survival of a passenger
accordingly
Use Case: Survival prediction in R

49. Use Case: Survival prediction in R
We will be using a data set which
specifies if a passenger on a ship
survived it’s wreck or not

50. 1 indicates the
person survived the
wreck
We will be using a data set which
specifies if a passenger on a ship
survived it’s wreck or not
Use Case: Survival prediction in R

51. The luxury class
of the cabin
Use Case: Survival prediction in R
We will be using a data set which
specifies if a passenger on a ship
survived it’s wreck or not

52. Numbers of siblings
on board
Use Case: Survival prediction in R
We will be using a data set which
specifies if a passenger on a ship
survived it’s wreck or not

53. Numbers of parents
on board
Use Case: Survival prediction in R
We will be using a data set which
specifies if a passenger on a ship
survived it’s wreck or not

54. Disembark location
Use Case: Survival prediction in R
We will be using a data set which
specifies if a passenger on a ship
survived it’s wreck or not

55. What is DecisionTree? Problems solved using DecisionTrees
How does a decision tree work?
Key Takeaways
Predicting survivors using R Determining accuracy of prediction
Terms to know

56. So what’s
your next step?