k-Nearest Neighbors (k-NN) is a simple machine learning algorithm that classifies new data points based on their similarity to existing data points. It stores all available data and classifies new data based on a distance function measurement to find the k nearest neighbors. k-NN is a non-parametric lazy learning algorithm that is widely used for classification and pattern recognition problems. It performs well when there is a large amount of sample data but can be slow and the choice of k can impact performance.

1. Algorithms: k Nearest
Neighbours (k-NN)
-Rishav K Sharma
2020/SPD/11

2. Analogy..
Tell me about your friends(neighbours),
and I will tell you who you are.

3. Instance-Based
Learning

4. k-NN Different Names
● K-Nearest Neighbors
● Memory-Based Reasoning
● Example-Based Learning
● Instance-Based Learning
● Lazy Learning

5. What is k-NN?
● A powerful classification algorithm used in pattern
recognition.
● K- nearest neighbors stores all available cases and
classify new cases based on similarity measure(eg.
Distance function)
● It is one of the top data mining algorithm used today.
● A non-parametric lazy learning algorithm (instance based
learning method)

6. k-NN: Classification Approach
● An object (a new instance) is
classified by majority votes
from its neighboring classes.
● The object is assigned to the
most common class amongst
its K nearest
neighbors.(measured by a
distant function)

7. Distance Measure for Continuous
Variables

8. How to choose K?
● If K is too small, efficiency is
increased but becomes
susceptible to noise.
● Larger K works well. But too
large K may include majority
points from other classes, but
risk of over-smoothing
classification results

9. Strengths of k-NN
● Very simple and intuitive.
● Can be applied to the data from any distribution.
● Good classification if the number of samples is large enough.
Weaknesses of k-NN
● Takes more time to classify a new example.
○ need to calculate and compare distance from new example to all other examples.
● Choosing k may be tricky.
● Need large number of samples for accuracy.

10. k-NN: A Simple Classifier

11. Implementing k-NN in Image Classification
● Step 1 – Gather Dataset: Dataset consists of several images(~1000) of each class. Each image is
represented in RGB color space. They are pre-processed by resizing it to 32x32 pixels.Therefore the
resized image with RGB channels has 32x32x3=3072 integers.
● Step 2 – Split the Dataset: Split half of the dataset for training and other half for testing.
● Step 3 – Train the Classifier: k-NN classifier will be trained on the raw pixel intensities of the images in
the training set and feature matrix is formed.(nearly~9Mb)
● Step 4 – Evaluate:Once our k-NN classifier is trained, performance can be evaluated.

12. Conclusion
k-NN algorithm is simple, easy to understand and gives a
baseline to implement neural networks and convolutional
neural networks.