The document describes decision trees, which are graphical representations of possible solutions to a decision based on certain conditions. Decision trees start with a single root node and branch into multiple solutions. The strengths are that decision trees can generate understandable rules, perform classification with little computation, and handle both continuous and categorical variables. Weaknesses include being less suitable for estimating continuous attributes and being prone to errors with many classes and small training examples. An example decision tree is provided to classify profit based on age, vehicles owned, and vehicle type. Information gain is used to determine the best attributes to branch on at each node.

1. Decision Tree

2. Created By:
Muhammad Umer Khan 2017-cs-027
Basit Hussain 2017-cs-050

3. Introduction
A decision tree is a decision support tool that
uses a tree like graph or model of decisions and
their possible consequences, including chance
event outcomes, resource cost, and profitable.
A decision tree is a graphical representation of possible
solutions to a decision based on certain conditions. It's
called a decision tree because it starts with a single box
(or root), which then branches off into a number of
solutions, just like a tree.

4. Strengths and Weakness
The strengths of decision tree methods are:
▪ Decision trees are able to generate understandable
rules.
▪ Decision trees perform classification without requiring
much computation.
▪ Decision trees are able to handle both continuous and
categorical variables.
▪ Decision trees provide a clear indication of which
fields are most important for prediction or
classification.

5. Strengths and Weakness
The weaknesses of decision tree methods :
▪ Decision trees are less appropriate for estimation
tasks where the goal is to predict the value of a
continuous attribute.
▪ Decision trees are prone to errors in classification
problems with many class and relatively small
number of training examples.

6. Formula
▪ Information Gain:
I (P , n) = -p/p+n log2 ( p/p+n) – n/p+n log2 (n/p+n) ………..(i)
▪ Entropy:
E(A) = ∑ [(pi + ni) / (P + n)] [I (p,n)] ………..(ii)
▪ Gain:
(A) = I(p,n) – E(A) ………..(iii)
V
i = 1

7. Decision Tree Example
Person
20
2%
20-50
50
5%U M
10% 20% 80%
Uk k

8. Dataset
Age Vehicles Type Profit
Old Yes Car Down
Old No Car Down
Old No Bike Down
Mid Yes Car Down
Mid Yes Bike Down
Mid No Bike Up
Mid No Car Up
New Yes Car Up
New No Bike Up
New No Car Up
Class attribute

9. Entropy of Class Attribute
P N
5 5
I (Pi , Ni) = -p/p+n log2 ( p/p+n) – n/p+n log2 (n/p+n)
= -5/10 log2 (5/10) – 5/10 log2 (5/10)
= 1

10. Dataset
Age Vehicles Type Profit
Old Yes Car Down
Old No Car Down
Old No Bike Down
Mid Yes Car Down
Mid Yes Bike Down
Mid No Bike Up
Mid No Car Up
New Yes Car Up
New No Bike Up
New No Car Up

11. Pi Ni I(Pi, Ni)
Old 0 3 0
Mid 2 2 1
New 3 0 0
I (P , n) = -p/p+n log2 ( p/p+n) – n/p+n log2 (n/p+n)
Entropy (Age):
E(A) = ∑ [(pi + ni) / (P + n)] [I (p,n)]
= (0+3/5+5)(0) + (2+2/ 5+5)(1)+(3+0/5+5)(0)
= 4 / 10
= 0.4
Gain = Class Entropy – Entropy(Age)
= 1 – 0.4
= 0.6
Age

12. Pi Ni I(Pi, Ni)
Yes 1 3 0.8112
No 4 2 0.9182
I (P , n) = -p/p+n log2 ( p/p+n) – n/p+n log2 (n/p+n)
Entropy (Vehicles):
= (1+3/5+5)(0.8112) + (4+2/ 5+5)(0.9182)
= 0.8754
Gain = Class Entropy – Entropy(vehicles)
= 1 – 0.8754
= 0.1245
Vehicles

13. Pi Ni I(Pi, Ni)
Car 3 3 1
Bike 2 2 1
Entropy (Type):
= (3+3/5+5)(1) + (2+2/ 5+5)(1)
= 6/10+ 4/10
=10/10
= 1
Gain = Class Entropy – Entropy(type)
= 1 – 1
= 0
Type

14. Information Gain
Age 0.6
Vheicles 0.1245
Type 0
Age
Root Node
Down
?
Up
Age Profit
Old Down
Old Down
Old Down
Age Profit
New Up
New Up
New Up
Mid

15. Information Gain(Contd…)
Age Vehicles Type Profit
Mid Yes Car Down
Mid Yes Bike Down
Mid No Bike Up
Mid No Car Up

16. Vehicles
Pi Ni I(Pi, Ni)
Yes 0 2 0
No 2 0 0
Entropy (vehicles):
= (2/4)(0) + (2/ 4)(0)
= 0
Gain = Class Entropy – Entropy(vehicles)
= 1 – 0
= 1

17. Type
Pi Ni I(Pi, Ni)
Car 1 1 1
Bike 1 1 1
Entropy (competition):
= (2/4)(1) + (2/ 4)(1)
= 1
Gain = Class Entropy – Entropy(competition)
= 1 – 1
= 0

18. Gain
Vehicles 1 node
Type 0
Age
Down Up
Vehicles
Down Up
Mid

19. Thank You 