Frequency Based Algorithms.pdf

1. Frequency Based Classification

2. Types
• K Nearest Neighborhood (KNN)
• Naïve Bayes

3. K-Nearest Neighbour
• Simplest: Count Based
• Not sensitive to variance-covariance
• Measure ‘distances’
• Distances ‘type’
• Very useful and simple
• But bit ad-hoc
• Other applications
12-Dec-17 3

4. Naïve Bayes
• Bayes Theorem
• 𝑃 𝑦𝑖 = 1|𝑋 =
𝑃 𝑦𝑖=1 ∗𝑃 𝑋|𝑦𝑖=1
𝑃 𝑦𝑖=1 ∗𝑃 𝑋|𝑦𝑖=1 +𝑃 𝑦𝑖=0 ∗𝑃 𝑋|𝑦𝑖=0
• Re visit Bayes Theorem with an Example:
• P(G)=0.95, P(a=g|G)=0.9, P(a=g|B)=0.15
• So what is P(G|a=g)?

5. Naïve Bayes
• How does it Work
• No a priori specification is needed
• Can be used to do multi class classification as
well