The ID3 algorithm builds decision trees using a top-down, greedy search. It uses entropy and information gain to determine the attribute that best splits the data. The algorithm is demonstrated on a weather dataset to predict whether it is suitable for playing ball. Entropy is calculated for each attribute to determine which has the highest information gain to become the root node, which is found to be the outlook attribute. The tree is then grown further by calculating entropy for the subsets based on outlook.

1. Decision Trees
Premalatha M

2. Decision Trees
 The core algorithm for building decision trees called ID3
by J. R. Quinlan which employs a top-down, greedy search
through the space of possible branches with no
backtracking. ID3 uses Entropy and Information Gain to
construct a decision tree.

3. Entropy
 A decision tree is built top-down from a root node and
involves partitioning the data into subsets that contain
instances with similar values (homogenous). ID3 algorithm
uses entropy to calculate the homogeneity of a sample. If
the sample is completely homogeneous the entropy is
zero and if the sample is an equally divided it has entropy
of one.

4. Entropy
To build a decision tree, we need to calculate two types of entropy using
frequency tables as follows:
a) Entropy using the frequency table of one attribute:

5. Data
Day Outlook Temperature Humidity Wind Class: Play ball
D1 Sunny Hot High False No
D2 Sunny Hot High True No
D3 Overcast Hot High False Yes
D4 Rain Mild High False Yes
D5 Rain Cool Normal False Yes
D6 Rain Cool Normal True No
D7 Overcast Cool Normal True Yes
D8 Sunny Mild High False No
D9 Sunny Cool Normal False Yes
D10 Rain Mild Normal False Yes
D11 Sunny Mild Normal True Yes
D12 Overcast Mild High True Yes
D13 Overcast Hot Normal False Yes
D14 Rain Mild High True No

7. D=14
 Yes = 9
 No=5
 In Outlook,
 Sunny=5 (Y=2, N=3)
 Overcast=4 (Y=4, N=0)
 Rainy=5 (Y=3, N=2)
 InTemp,
 Hot=4 (Y=2, N=2)
 Mild=6 (Y=4, N=2)
 Cool=4 (Y=3, N=1)
 In Humidity,
 High=7 (Y=3, N=4)
 Normal=7 (Y=6, N=1)
 InWind,
 False=8 (Y=6, N=2)
 True=6 (Y=3 N=3)
outlook temperature humidity windy play
yes no yes no yes no yes no yes no
sunny 2 3 hot 2 2 high 3 4 false 6 2 9 5
overcast 4 0 mild 4 2 normal 6 1 true 3 3
rainy 3 2 cool 3 1
sunny 2/9 3/5 hot 2/9 2/5 high 3/9 4/5 false 6/9 2/5 9/14 5/14
overcast 4/9 0/5 mild 4/9 2/5 normal 6/9 1/5 true 3/9 3/5
rainy 3/9 2/5 cool 3/9 1/5

8. Entropy -

9. Gain Outlook
 Gainout(d)=Info(D)-Infooutlook(D)
In Outlook,
Sunny=5 (Y=2, N=3)
Overcast=4 (Y=4, N=0)
Rainy=5 (Y=3, N=2)

10. Gain Temp
 Gaintemp(d)=Info(D)-Infotemp(D)
In Temp,
Hot=4 (Y=2, N=2)
Mild=6 (Y=4, N=2)
Cool=4 (Y=3, N=1)

11. Gain Hum
 Gainhum(d)=Info(D)-Infohum(D)
In Humidity,
High=7 (Y=3, N=4)
Normal=7 (Y=6, N=1)

12. Gain Windy
 Gainwind(d)=Info(D)-Infowind(D)
 Maximum Gain -> Outlook = 0.246
In Windy,
False=8 (Y=6, N=2)
True=6 (Y=3 N=3)

13. Tree with root = Outlook

14. D=5 [Sunny]
 Yes = 2
 No = 3
 In Temp,
 Hot=2 (Y=0, N=2)
 Mild=2 (Y=1, N=1)
 Cool=1 (Y=1, N=0)
 In Humidity,
 High=3 (Y=0, N=3)
 Normal=2 (Y=2, N=0)
 In Windy,
 False=3 (Y=1, N=2)
 True=2 (Y=1 N=1)

15. Info(D) - Sunny

16. Gain Temp - Sunny
 Gaintemp(d)=Info(D)-Infotemp(D)
In Temp,
Hot=2 (Y=0, N=2)
Mild=2 (Y=1, N=1)
Cool=1 (Y=1, N=0)

17. Gain Hum - Sunny
 Gainhum(d)=Info(D)-Infohum(D)
In Humidity,
High=3 (Y=0, N=3)
Normal=2 (Y=2, N=0)

18. Gain Windy - Sunny
 Gainwind(d)=Info(D)-Infowind(D)
 Maximum Gain -> Outlook -> Sunny = Hum = 0.971
InWindy,
False=3 (Y=1, N=2)
True=2 (Y=1 N=1)

20. D=5 [Rainy]
 Yes = 3
 No = 2
 In Temp,
 Hot=0 (Y=0, N=0)
 Mild=3 (Y=2, N=1)
 Cool=2 (Y=1, N=1)
 In Humidity,
 High=2 (Y=1, N=1)
 Normal=3 (Y=2, N=1)
 In Windy,
 False=3 (Y=3, N=0)
 True=2 (Y=0 N=2)

21. Info(D) - Rainy

22. Gain Temp - Rainy
 Gaintemp(d)=Info(D)-Infotemp(D)
In Temp,
Hot=0 (Y=0, N=0)
Mild=3 (Y=2, N=1)
Cool=2 (Y=1, N=1)

23. Gain Hum - Rainy
 Gainhum(d)=Info(D)-Infohum(D)
In Humidity,
High=2 (Y=1, N=1)
Normal=3 (Y=2, N=1)

24. Gain Windy - Rainy
 Gainwind(d)=Info(D)-Infowind(D)
 Maximum Gain -> Outlook -> Rainy = Windy = 0.971
InWindy,
False=3 (Y=3, N=0)
True=2 (Y=0 N=2)

25. Predict:
 Outlook=Overcast,Temperature=Mild, Humidity=Normal,Windy=False
 Outlook=rainy, temp=cold, humidiy=normal, windy=true = No

27. To do