The document summarizes key concepts related to decision trees for classification and regression problems. It defines common terminology like entropy, information gain, and gini impurity used in decision trees for classification. It also discusses the process of how decision trees handle classification and regression problems. The advantages of decision trees are provided as being simple to understand, needing little data preprocessing, and able to handle both regression and classification. Overfitting and instability when new data is added are identified as disadvantages that can be addressed through techniques like tree pruning and using random forests.

1. INSTRUCTOR NAME: MD. SHOHEL ARMAN
DECISION TREE

8. What are the
terminologies
used in Decision
Tree in the case
of
classification?

9. The only
difference is that
entropy lies
between 0 to 1
and gini impurity
lies between 0 to
0.5.
Graph displacing Gini Impurity and Entropy

10. Information
Gain:

11. What happens in
the Decision
Tree in case of
classification
problems?

12. What are the
terminologies
used in Decision
Tree in case of
regression?

13. What happens in
the Decision
Tree in case of
regression
problems?
•The Decision Tree tries to identify a cluster of
points to draw a decision boundary.
•The decision tree considers every point and draws
a boundary by calculating the error of the point.
•This process goes on for every point then from all
the errors the lowest error point is considered to
draw the boundary.
•This process is very expensive in terms of
computational time.
So, the Decision Tree opts for the greedy approach in
which nodes are divided into two parts in a given
condition.

14. The Advantages
of Decision Tree
•The decision tree is one of the simplest algorithms to
understand and interpret. Also, we can visualize the
tree.
•The decision tree needs fewer data preprocessing
times as compared to other algorithms.
•The cost of using a decision tree is logarithmic
•It can be used for both regression and classification
problems.
•It is able to handle multi-output problems.

15. The
Disadvantages of
Decision Tree
•The main disadvantage of the Decision Tree is the
problem of Overfitting.
•The Decision Tree can be unstable by introducing new
data points resulting in a completely new tree generated.
•Predictions of decision trees are neither smooth nor
continuous, but piecewise constant approximations as
seen in the above figure. Therefore, they are not good at
extrapolation.

16. What is the
solution to the
disadvantages of
the Decision
Tree?
1.For the overfitting problem, we can limit the height,
nodes, or leaves of the decision tree by hyper-tunning the
model. This process is called Tree Pruning.
2.To handle the unstable decision tree, we can use an
ensemble technique like the most famous “Random
Forest”.

17. Required
modules from
python for
importing
Decision Tree

18. Thank You