# Classification Algorithms

S
1 of 14

## Similar to Classification Algorithms

Decision treeVarun Jain
256 views39 slides
Decision treeEstiak Khan
852 views22 slides

### Similar to Classification Algorithms(20)

Decision tree
Varun Jain256 views
Induction of Decision Trees
nep_test_account508 views
Decision tree
Estiak Khan852 views
decisiontrees.ppt
LvlShivaNagendra19 views
decisiontrees (3).ppt
LvlShivaNagendra6 views
decisiontrees.ppt
unit 1.pptx
sirishaYerraboina152 views
Textmining Predictive Models
Datamining Tools231 views
Textmining Predictive Models
guest0edcaf1.3K views
Textmining Predictive Models
DataminingTools Inc471 views
Lecture4.ppt
Minakshee Patil3 views
83 learningdecisiontree
tahseen shaikh290 views
Decision tree presentation
Primer on major data mining algorithms
Vikram Sankhala IIT, IIM, Ex IRS, FRM, Fin.Engr572 views
Hx3115011506
IJERA Editor364 views

How Leaders See Data? (Level 1)
Narendra Narendra9 views
Project Summary M_Covricova.pdf
MARIACOVRICOVA16 views
2022-Scripting_Hacks.pdf
Roland Schock9 views
Journey of Generative AI
thomasjvarghese4915 views
Timothy Spann40 views
Introduction to Microsoft Fabric.pdf
ishaniuudeshika15 views
Microsoft Fabric.pptx
Shruti Chaurasia12 views
krishnashamuktikendra.pdf
gagankrish8 views
RuleBookForTheFairDataEconomy.pptx
noraelstela162 views
PROGRAMME.pdf
HiNedHaJar6 views
PTicketInput.pdf
stuartmcphersonflipm284 views
Personal brand exploration
ML in rare diseases
anooshaqaisar8 views
RIO GRANDE SUPPLY COMPANY INC, JAYSON.docx
JaysonGarabilesEspej5 views

### Classification Algorithms

• 1. Classification Algorithms Decision Tree Induction Bayesian Classification
• 2. Decision Tree Induction • A decision tree is a flow-chart like structure, where each internal node(non-leaf node) denotes a test on an attribute. • Each branch represents an outcome of the test • And each leaf node(terminal node) holds a class label. • The topmost node in a tree is the root node.
• 4. Why are decision tree classifiers so popular? • It does not require any domain knowledge. • Decision trees can handle multi-dimensional data. • It is easy to comprehend. • The learning and classification steps of a decision tree are simple and fast. Applications: Applications of decision tree induction include astronomy, financial analysis, medical diagnosis, manufacturing and production, molecular biology.
• 5. Decision Tree Algorithms • CART (Classification And Regression Trees) • ID3 (Iterative Dichotomiser) In the late 1970s and early 1980s, J.Ross Quinlan, a researcher in machine learning developed a decision tree algorithm for machine learning. Later, he presented C4.5, which was the successor of ID3. ID3 and C4.5 and CART adopt a greedy(non-backtracking) approach in which decision trees are constructed in a top- down recursive divide-and-conquer manner.
• 6. Decision Tree Algorithm The strategy for the algorithm is as follows: (1) The algorithm is called with three parameters: attribute list, attribute selection method and data partition. (2) Initially, data partition is the complete set of training tuples and their associated class labels. The attribute list describes the attributes of the training set tuples. RID Age Student Credit_rati ng Buys 1 Youth Yes Fair Yes 2 Youth Yes Fair Yes 3 Youth Yes Fair No 4 Youth no Fair No 5 Middle No Excellent Yes 6 Senior Yes Fair No Class label
• 7. Decision Tree Algorithm (3) The attribute selection method describes the method for selecting the best attribute for discrimination among tuples. The methods used for attribute selection can either be Information Gain or Gini Index. The structure of the tree (binary or non-binary) is decided by the attribute selection method. (4) The tree starts as a single node representing the training tuples in data partition. Age youth middle senior RID class 1 Yes 2 Yes 3 No 4 no RID class 5 yes RID class 6 No
• 8. Decision Tree Induction (5) If the tuples in the Data Partition are all of the same class, then node becomes a leaf and is labeled with that class. (terminating condition) (6) otherwise, the attribute selection method is called to determine the splitting criterion. (7) The algorithm uses the same process recursively to form a decision tree for the tuples at each resulting partition. (8) The recursive partitioning stops only when any one of the following terminating conditions is true:
• 9. Decision Tree Induction (i) all the tuples in partition belong to the same class. (ii) There are no remaining attributes on which the tuples may be further partitioned. In this case, majority voting is employed. This involves converting node into a leaf and labeling it with the most common class in partition. (iii) There are no tuples for a given branch, in this case also, a leaf is created with the majority class in partition. (9) The resulting decision tree is returned.
• 11. Tree Pruning • An attempt to improve accuracy. • Tree pruning is performed in order to remove anomalies the method to reduce the unwanted branches of the tree. This will reduce the complexity of the tree and help in effective predictive analysis. It reduces the overfitting as it removes the unimportant branches from the trees.
• 12. Bayesian Classification • Bayesian classifiers are statistical classifiers. • They can predict class membership probabilities such as the probability that a given tuple belongs to a particular class. • Bayesian classification is based on Bayes’ Theorem. • Bayesian classifiers have also exhibited high accuracy and speed when applied to large databases.
• 13. Bayes’ Theorem • Bayes theorem is named after Thomas Bales who did early work in probability and decision theory during 18th century. • Let X be a data tuple. In bayesian terms X is considered as “evidence”. Let H be hypothesis such that the data tuple belong to a specified class C. • P(H|X) is the posterior probability that the hypothesis H holds the evidence or data tuple X. Or, the probability that X belongs to a specified class C. e.g. data tuples comprise of attributes, age and income. X is of 35 years with an income of \$40,000. H is hypothesis that X will buy computer or not. P(H|X) is the probability that X will buy computer given his age and income. • P(H) is the prior probability. e.g. probability that X will buy computer or not, regardless of age and income. i.e. , P(H) is independent of X.
• 14. Bayes’ Theorem • P(X|H) is the posterior probability (likelihood) that the customer X is of 35 years and earns \$40,000 given that we know that X will buy computer. • P(H) is the prior probability (marginal). e.g. probability that X is of 35years and earns \$40,000, regardless he will buy computer or not. Bayes’ Theorem is given by P(H|X) = e.g. P(Queen|Face) = P(face|queen) P(queen) / P(face) = (1 * 4/52 ) / (12/52) = 1/3 = 33.33%
Current LanguageEnglish
Español
Portugues
Français
Deutsche