This document provides an overview of the Naive Bayes algorithm. It introduces naive Bayes classification and conditional probability. The document then explains Bayes' theorem and how the naive Bayes algorithm uses conditional probabilities to classify data. Several examples are provided. The advantages of naive Bayes including its ability to handle multiple classes and require less training data are discussed. Limitations around its independence assumptions are also outlined.

1. Naive Bayes Algorithm
1
A. Ramya Bhargavi

2. Contents
● Machine learning Introduction & Naive
Bayes Classifier
● Conditional probability
● Bayes theorem
● Naive Bayes Algorithm
● Applications
● Advantages and Disadvantages
2

3. “
3 3

4. Probability , Conditional probability
Prob ( Yellow squares ) = number of Yellow squares = 9
Total number of squares 16
Prob ( Yellow / Red ) = number of Yellow squares with red boxes = 2
Total number of red boxes 5
4

5. Bayes Theorem
5
Naive Bayes Classifier

6. http://shatterline.com/blog/2013/09/12/not-so-naive-classification-with-the-naive-bayes-classifier/
prob(class=yes / feature ) = p ( feature / class= yes )
p(feature)
prob(class=no / feature ) = p ( feature / class= no )
p(feature)
If A< B then that feature belongs to class NO,
Else if belongs to class YES
What is Naive Bayes Classifier ?
https://ieeexplore.ieee.org/document/4403192
A
B
6

7. Application
Implementation of Naïve Bayes Classification
Method In Predicting Purchases
- The 6th International Conference on Cyber and IT Service Management (CITSM 2018)
- Fitriana Harahap
7

8. P(yes) = 11/20 = 0.55
P(no) = 9/20 = 0.45
Based on the Price, Spare Part
cost, Cylinder Volume and
Resale Price our model has to
predict whether a customer
should buy a car or not.
8

9. The conditional probabilities are
9

10. Example:
Suppose that there is a X model car and it has criteria as follows,
- Price = Normal
- Spare part = medium
- Cylinder Volume = medium
- Resale price = Normal
A customer wants to know if he has to buy or not ?
Here kε(yes, no)
When k = yes
⇒ P(yes) * P(Normal/yes) * P(Medium/yes) * P(Medium/yes) * P(Normal/yes)
⇒ 0.55 * 0.09 * 0.27 * 0.81 * 0.09
⇒ 0.00097
When k = no
⇒ P(no) * P(Normal/no) * P(Medium/no) * P(Medium/no) * P(Normal/no)
⇒ 0.45 * 0.11 * 0.44 * 0.33 * 0.11
⇒ 0.00079
Since the model choose the maximum value it output the result as yes.
We know that,
10

11. 11
Advantages of Naive Bayes Algorithm :
● This algorithm works quickly and can save a lot of time.
● Naive Bayes is suitable for solving multi-class prediction problems.
● If its assumption of the independence of features holds true, it can perform better than
other models and requires much less training data.
● Naive Bayes is better suited for categorical input variables than numerical variables.

12. 12
Disadvantages of Naive Bayes Algorithm :
● Naive Bayes assumes that all predictors (or features) are independent, rarely
happening in real life. This limits the applicability of this algorithm in real-world use
cases.
● This algorithm faces the ‘zero-frequency problem’ where it assigns zero probability to a
categorical variable whose category in the test data set wasn’t available in the training
dataset. It would be best if you used a smoothing technique to overcome this issue.

13. References :
1. H. Zhang and D. Li, "Naïve Bayes Text Classifier," 2007 IEEE International Conference
on Granular Computing (GRC 2007), 2007, pp. 708-708, doi: 10.1109/GrC.2007.40.
https://ieeexplore.ieee.org/document/4403192
2. F. Harahap, A. Y. N. Harahap, E. Ekadiansyah, R. N. Sari, R. Adawiyah and C. B. Harahap,
"Implementation of Naïve Bayes Classification Method for Predicting Purchase,"
2018 6th International Conference on Cyber and IT Service Management (CITSM), 2018,
pp. 1-5, doi:
10.1109/CITSM.2018.8674324.https://ieeexplore.ieee.org/document/8674324
3. http://www.hep.upenn.edu/~johnda/Papers/Bayes.pdf
4. https://www.geeksforgeeks.org/naive-bayes-classifiers/
13