Introduction Naive Bayes is a family of probabilistic algorithms based on Bayes' Theorem. It’s particularly suited for classification tasks and is widely used in natural language processing (NLP), spam detection, sentiment analysis, and recommendation systems. The term “naive” comes from the assumption that features are conditionally independent given the class label, which simplifies the computation but is rarely true in real-world data. Despite this simplification, Naive Bayes often performs surprisingly well. Bayes’ Theorem At the core of Naive Bayes lies Bayes' Theorem: � ( � ∣ � ) = � ( � ∣ � ) � ( � ) � ( � ) P(A∣B)= P(B) P(B∣A)P(A) ​ In classification terms: � ( � ∣ � ) P(A∣B): Posterior probability of class A given data B. � ( � ∣ � ) P(B∣A): Likelihood of data B given class A. � ( � ) P(A): Prior probability of class A. � ( � ) P(B): Probability of data B. Naive Bayes Classifier: Principle Given a set of features � = { � 1 , � 2 , . . . , � � } X={x 1 ​ ,x 2 ​ ,...,x n ​ } and a class variable � C, Naive Bayes calculates the probability of each class � � C i ​ given the features and selects the class with the highest probability: � ^ = arg ⁡ max ⁡ � � � ( � � ) ∏ � = 1 � � ( � � ∣ � � ) C ^ =arg C i ​ max ​ P(C i ​ ) j=1 ∏ n ​ P(x j ​ ∣C i ​ ) This formula assumes that all features � � x j ​ are independent given the class � � C i ​ . Types of Naive Bayes Classifiers 1. Gaussian Naive Bayes Used when features are continuous and follow a Gaussian distribution. � ( � � ∣ � ) = 1 2 � � � 2 exp ⁡ ( − ( � � − � � ) 2 2 � � 2 ) P(x i ​ ∣y)= 2πσ y 2 ​ ​ 1 ​ exp(− 2σ y 2 ​ (x i ​ −μ y ​ ) 2 ​ ) 2. Multinomial Naive Bayes Used for discrete features such as word counts in a document (common in text classification). 3. Bernoulli Naive Bayes Used when features are binary (presence/absence of a feature, e.g., word in a document). Applications of Naive Bayes Text Classification: Spam filtering, sentiment analysis. Medical Diagnosis: Predicting diseases based on symptoms. Document Categorization: Grouping articles into categories. Recommendation Systems: Predicting user preferences. Credit Scoring: Classifying customers as risky or not. Advantages of Naive Bayes Simple and easy to implement. Requires less training data. Handles both continuous and discrete data. Works well with high-dimensional data. Robust to irrelevant features. Disadvantages Assumes feature independence. Performance may degrade if features are highly correlated. Poor estimates for zero probabilities (handled using Laplace smoothing). Laplace Smoothing To avoid assigning zero probability to unseen features: � ( � � ∣ � ) = � � � � � ( � � , � ) + 1 � � � � � ( � ) + ∣ � ∣ P(x i ​ ∣y)= count(y)+∣X∣ count(x i ​ ,y)+1 ​ Where |X| is the total number of unique features. Model Training and Prediction Training Phase: Compute prior probabilities � ( � ) P(C) Compute conditional probabilities � (

1. Naive Bayes Classifier

3. Naive Bayes Classifier:
• Naive Bayes classifiers are a collection of classification algorithms based on Bayes’
Theorem.
• It is not a single algorithm but a family of algorithms where all of them share a common
principle, i.e. every pair of features being classified is independent of each other.

4. Bayes Theorem:

6. Bayes Theorem

7. Dataset: iris

8. # Load the dataset
D = iris
# View dataset
View(D)
# Check dataset structure
names(D)
dim(D)
# Split the data into training and testing sets
set.seed(123) # Set seed for reproducibility
Split_D = sample.split(D$Species, SplitRatio = 0.7)
# Create training and testing subsets
train_d = subset(D, Split_D == TRUE)
test_d = subset(D, Split_D == FALSE)
# Fit the Naïve Bayes model
NB = naiveBayes(Species ~ ., data = train_d)
print(NB)
# Predict on the test datapred_
d = predict(NB, newdata = test_d)
# Create confusion matrix
cm = table(test_d$Species, pred_d)
print(cm)
# Use confusionMatrix from caret
conf_matrix = confusionMatrix(pred_d, test_d$Species)
print(conf_matrix)
R-CODE:

9. >NB = naiveBayes(Species ~ ., data = train_d)
>print(NB)

12. Types of Naive Bayes Classifier:
1) Multinomial Naive Bayes:
– This is mostly used for document classification
problem, i.e whether a document belongs to the
category of sports, politics, technology etc.
– The features/predictors used by the classifier are
the frequency of the words present in the
document.
2)Bernoulli Naive Bayes:
– This is similar to the multinomial naive bayes but
the predictors are boolean variables.
– The parameters that we use to predict the class
variable take up only values yes or no, for example
if a word occurs in the text or not.

13. 3) Gaussian Naive Bayes:
– When the predictors take up a continuous value
and are not discrete, we assume that these values
are sampled from a Gaussian distribution.

14. Advantages of a Naive Bayes Classifier:
•It doesn’t require larger amounts of training data.
•It is straightforward to implement.
•Convergence is quicker than other models, which are discriminative.
•It is highly scalable with several data points and predictors.
•It can handle both continuous and categorical data.
•It is not sensitive to irrelevant data and doesn’t follow the assumptions it
holds.
•It is used in real-time predictions.

15. Disadvantages of a Naive Bayes Classifier:
•The Naive Bayes Algorithm has trouble with the ‘zero-frequency problem’. It
happens when you assign zero probability for categorical variables in the training
dataset that is not available. When you use a smooth method for overcoming this
problem, you can make it work the best.
•It will assume that all the attributes are independent, which rarely happens in real
life. It will limit the application of this algorithm in real-world situations.
•It will estimate things wrong sometimes, so you shouldn’t take its probability
outputs seriously.