these slide is contained basic pruning method in decision tree learning .

1. Post pruning methods
Mojtaba amiri
M.S of artificial intelligence

2. Approaches
• top-down approach :
proceed from the root towards the leaves of tree
• bottom-up approach :
starting the analysis from the leaves and finishing with the root node

3. Based methods
1. Reduced Error Pruning
2. Pessimistic error pruning
3. Error complexity pruning
4. Minimum Error pruning
5. Cost based pruning
6. Iterative growing and pruning algorithm

4. Reduced Error Pruning
• This method was proposed by Quinlan .
• This method considers each of the decision nodes in the tree to be
candidates for pruning, consist of removing the subtree rooted at that
node, making it a leaf node .
• The available data is divided into three parts:
training examples , validation examples for pruning , a set of test
examples for estimate of accuracy over future unseen examples
• Next step calculating new tree accuracy

5. Reduced Error Pruning
• The problems related to the method of reduced error pruning are
basically two:
1. The use of a pruning set distinct from the training set is inadequate
when a small number of observations are available
2. the parts of the original tree that correspond to special cases outside
the test set may be lost after pruning. Therefore trees pruned via that
pruning method may fail in correctly classifying exceptional cases.

6. Reduced Error Pruning
• Pros :
linear computational complexity
• Cons :
over pruning , if the test set is much smaller than the training set

7. Pessimistic error pruning
• This pruning method, proposed by Quinlan
• it avoids using an independent pruning set
• The misclassification rate, estimated by means of the training set
• If results to be optimistic , thus it always happens that pruning
operations based on the same data set produce trees that are larger
than it is needed .

8. Pessimistic error pruning
• Pros :
This method has a linear complexity in the number of leaves
• Cons:
the worst case is that in which the tree has not to be pruned at all

9. Error complexity pruning
• It finds error complexity at each node .
1. Calculating the error cost of the node
2. r(t) is error rate of a node
3. p(t) is probability of occurrence of a node
4. If node t was not pruned then error
cost of subtree T , rooted at t:
5. Then , The error complexity of the node

10. Error complexity pruning
• The method consists of following steps:
1. a ( error complexity ) is computed for each node.
2. the minimum a node is pruned.
3. the above is repeated and a forest of pruned tree is formed.
4. the tree with best accuracy is selected.

11. Minimum Error pruning
• This method was developed by Niblett and Brotko
• bottom-up approach
• seeks a single tree that minimizes the expected error rate on an
independent data set
• If it is predicted that all future examples will be in class c, the
following equation is used to predict the expected error rate of
pruning at node t :
1. K : # of class
2. nt : # of examples in node t
3. nt,c : # of examples assigned to class c in node t

12. Minimum Error pruning
• The method consists of following steps
1. At each non leaf node in the tree, calculate expected error rate if
that subtree is pruned.
2. Calculate the expected error rate for that node if subtree is not
pruned.
3. If pruning the node leads to greater expected error rate, then keep
the subtree; otherwise, prune it .

13. Cost based pruning
• In this method not only an error rate is considered at each node but
also a cost is considered .
• That is for pruning decision tree error rate and cost of deciding
selection of one or more class-label attribute is considered .

14. Iterative growing and pruning algorithm
• Gelfand et al
• These goals are reached by splitting the data set into two subsets
• then by repeatedly growing and pruning a tree on different subsets
• a tree is grown by using the first subset
• then it is pruned by using the second subset

15. Tanks for reading