Types of Machine Learning. & Decision Tree.

1. Types of Learning
Decision Tree
OMega TechEd

2. Introduction
An agent is learning if it improves its performance on future tasks
after making observations about the world.
• Designers cannot anticipate all possible situations.
• The designers cannot anticipate all changes over time.
• Human programmers have no idea how to program a solution
themselves.
2
OMega TechEd

3. Types of Learning
3
Unsupervised learning
The agent learns patterns in
the input even though no
explicit feedback is supplied.
Supervised learning
The agent observes some
example input–output pairs
and learns a function that
maps from input to output.
Classification
When the output y is one of a finite set
of values (such as sunny, cloudy or
rainy), the learning problem is called
classification.
Regression
When y is a number (such as tomorrow’s
temperature), the learning problem is called
regression. Regression algorithms are used if
there is a relationship between the input
variable and the output variable.
Reinforcement
learning
The agent learns from a
series of reinforcement
rewards or punishments.
OMega TechEd

4. Decision Tree
 Decision Tree is a Supervised learning technique that can be used for both
classification and Regression problems, but mostly it is preferred for solving
Classification problems.
 A decision tree represents a function that takes as input a vector of attribute
values and returns a “decision”—a single output value. The input and output
values can be discrete or continuous.
 In a Decision tree, there are two nodes, which are the Decision Node and Leaf
Node. Decision nodes are used to make any decision and have multiple branches,
whereas Leaf nodes are the output of those decisions and do not contain any
further branches.
 The decisions or the test are performed based on features of the given dataset.
 A decision tree simply asks a question and based on the answer (Yes/No), it
further split the tree into subtrees.
4
OMega TechEd

5. Example
As an example, we will build a decision tree to decide whether to wait for a
table at a restaurant. Target WillWait can be True or False.
5
OMega TechEd

6. The Training Set
6
An example for a Boolean decision tree consists of an (x, y) pair, where x
is a vector of values for the input attributes, and y is a single Boolean
output value. A training set of 12 examples is shown in Figure

7. Decision Trees
Learning Decision Trees from Observations
Type? Patrons?
1 3 4 6 8 1
2
2 5 7 9 1
0
11
1 3 4 6 8 1
2
2 5 7 9 1
0
11
French Some
Thai Burger
1
5
6
10
4 8
2 11
3 1
2
7 9
None
Italian Full
7 11 1 3 6 8 4 12
2 5 9 10
Hungry?
Yes
No
5 9 4 1
2
2 10
An important (or good) attribute splits samples into groups that are
(ideally) all positive or negative. Therefore Patrons is more
important than Type. Testing good attributes first allows us to
minimize the tree depth.

8. Decision Tree Output
8
The tree is clearly simpler than the original tree .The learning
algorithm has no reason to include tests for Raining and
Reservation, because it can classify all the examples without them.
OMega TechEd

9. Decision Tree Algorithm
Four basic underlying ideas of the algorithm:
1. If there are some positive and negative samples, then choose the
best attribute to split them.
2. If all the remaining samples are all positive or all negative, we have
reached a leaf node. Assign label as positive (or negative).
3. If there are no samples left, it means that no such sample has been
observed. Return a default value calculated from the majority
classification at the node’s parent.
4. If there are no attributes left, but both positive and negative samples,
it means that these samples have the same feature values but
different classifications.
9
OMega TechEd

10. Decision Tree Algorithm (Cont.)
This may happen because some of the data could be incorrect, or the
attributes do not give enough information to describe the situation fully
(i.e., we lack other useful attributes), or the problem is truly non-
deterministic, i.e., given two samples describing the same conditions,
we may make different decisions.
Solution: Call it a leaf node and assign the majority vote as the label
10
OMega TechEd

11. Thank you
Reference:
Artificial Intelligence: A Modern Approach, 3rd ed.
Stuart Russell and Peter Norvig
OMega TechEd