1. KSI Microsoft AEP mentorrbuddy.co
m
K-MEANS
CLUSTERING

2. KSI Microsoft AEP mentorrbuddy.co
m
CLUSTERING
• Clustering falls into the category of un-
supervised learning.
• Clustering is the task of dividing the
population or data points into a number
of groups such that data points in the
same groups are more similar to other
data points in the same group and
dissimilar to the data points in other
groups.

3. KSI Microsoft AEP mentorrbuddy.co
m
WHAT K-MEANS DOES FOR YOU ?
𝑥1
𝑥2
Before K-
Means
K-
Mean
s
𝑥1
𝑥2
After K-Means

4. KSI Microsoft AEP mentorrbuddy.co
m
HOW DOES IT WORKS?
STEP 1: Choose the number K clusters.
STEP 2: Select at random K points, the centroids(not necessarily from your dataset).
STEP 3: Assign each data point to the closest centroid – That forms K clusters.
STEP 4: Compute and place the new centroid of each cluster.
STEP 5: Reassign each data point to the new closet centroid.
If reassignment took place, go to step 4, otherwise go to FIN.(Model Created)

5. KSI Microsoft AEP mentorrbuddy.co
m
STEP 1: CHOOSE THE NUMBER K OF CLUSTERS: K=2

6. KSI Microsoft AEP mentorrbuddy.co
m
STEP 2: SELECT AT RANDOM K POINTS, THE CENTROID
(NOT NECESSARILY FROM YOUR DATASET)

7. KSI Microsoft AEP mentorrbuddy.co
m
STEP 3: ASSIGN EACH DATA POINT TO THE CLOSEST
CENTROID – THAT FORMS K CLUSTERS

8. KSI Microsoft AEP mentorrbuddy.co
m
STEP 4: COMPUTE AND PLACE THE NEW CENTROID
OF EACH CLUSTER

9. KSI Microsoft AEP mentorrbuddy.co
m
STEP 5: REASSIGN EACH DATA POINT TO THE NEW
CLOSEST CENTROID. IF AN REASSIGNMENT TOOK PLACE
GO TO STEP 4, OTHERWISE GO TO FIN.

10. KSI Microsoft AEP mentorrbuddy.co
m
FINISHED MODEL

11. KSI Microsoft AEP mentorrbuddy.co
m
RANDOM INITIALIZATION TRAP
𝑥1
𝑥2
𝑥1

12. KSI Microsoft AEP mentorrbuddy.co
m
RANDOM INITIALIZATION TRAP
But what would happen if we choose a bad random initialization?

13. KSI Microsoft AEP mentorrbuddy.co
m
RANDOM INITIALIZATION TRAP
𝑥1
𝑥2
𝑥1

14. KSI Microsoft AEP mentorrbuddy.co
m
AVOIDING THE TRAP
Solution K-
Means++

15. KSI Microsoft AEP mentorrbuddy.co
m
WITHIN CLUSTERS SUM OF SQUARES
WCSS= 𝛴 𝑑𝑖𝑠𝑡𝑎𝑛𝑐𝑒(𝑃𝑖, 𝐶1)2 + 𝛴 𝑑𝑖𝑠𝑡𝑎𝑛𝑐𝑒(𝑃𝑖, 𝐶2)2 + 𝛴 𝑑𝑖𝑠𝑡𝑎𝑛𝑐𝑒(𝑃𝑖, 𝐶3)2
𝑃𝑖 𝑖𝑛 𝑐𝑙𝑢𝑠𝑡𝑒𝑟 1 𝑃𝑖 𝑖𝑛 𝑐𝑙𝑢𝑠𝑡𝑒𝑟 2 𝑃𝑖 𝑖𝑛 𝑐𝑙𝑢𝑠𝑡𝑒𝑟 3

16. KSI Microsoft AEP mentorrbuddy.co
m
CONTD..
𝑥1
𝑥2
𝑥1
WCSS= 𝛴 𝑑𝑖𝑠𝑡𝑎𝑛𝑐𝑒(𝑃𝑖, 𝐶1)2 + 𝛴 𝑑𝑖𝑠𝑡𝑎𝑛𝑐𝑒(𝑃𝑖, 𝐶2)2 + 𝛴 𝑑𝑖𝑠𝑡𝑎𝑛𝑐𝑒(𝑃𝑖, 𝐶3)2
𝑃𝑖 𝑖𝑛 𝑐𝑙𝑢𝑠𝑡𝑒𝑟 1 𝑃𝑖 𝑖𝑛 𝑐𝑙𝑢𝑠𝑡𝑒𝑟 2 𝑃𝑖 𝑖𝑛 𝑐𝑙𝑢𝑠𝑡𝑒𝑟 3

17. KSI Microsoft AEP mentorrbuddy.co
m
𝑥1
𝑥2
𝑥1
WCSS= 𝛴 𝑑𝑖𝑠𝑡𝑎𝑛𝑐𝑒(𝑃𝑖, 𝐶1)2
𝑃𝑖 𝑖𝑛 𝑐𝑙𝑢𝑠𝑡𝑒𝑟 1

18. KSI Microsoft AEP mentorrbuddy.co
m
𝑥1
𝑥2
𝑥1
WCSS= 𝛴 𝑑𝑖𝑠𝑡𝑎𝑛𝑐𝑒(𝑃𝑖, 𝐶1)2 + 𝛴 𝑑𝑖𝑠𝑡𝑎𝑛𝑐𝑒(𝑃𝑖, 𝐶2)2
𝑃𝑖 𝑖𝑛 𝑐𝑙𝑢𝑠𝑡𝑒𝑟 1 𝑃𝑖 𝑖𝑛 𝑐𝑙𝑢𝑠𝑡𝑒𝑟 2

19. KSI Microsoft AEP mentorrbuddy.co
m
𝑥1
𝑥2
𝑥1
WCSS= 𝛴 𝑑𝑖𝑠𝑡𝑎𝑛𝑐𝑒(𝑃𝑖, 𝐶1)2 + 𝛴 𝑑𝑖𝑠𝑡𝑎𝑛𝑐𝑒(𝑃𝑖, 𝐶2)2 + 𝛴 𝑑𝑖𝑠𝑡𝑎𝑛𝑐𝑒(𝑃𝑖, 𝐶3)2
𝑃𝑖 𝑖𝑛 𝑐𝑙𝑢𝑠𝑡𝑒𝑟 1 𝑃𝑖 𝑖𝑛 𝑐𝑙𝑢𝑠𝑡𝑒𝑟 2 𝑃𝑖 𝑖𝑛 𝑐𝑙𝑢𝑠𝑡𝑒𝑟 3

20. KSI Microsoft AEP mentorrbuddy.co
m
CHOOSING THE RIGHT NUMBER OF CLUSTERS
𝑥1
𝑥2
𝑥1
𝑥2
𝑥1

21. KSI Microsoft AEP mentorrbuddy.co
m
THE ELBOW METHOD (1/2)
• It is a graph of distortion score (WCSS) vs k
• Distortion can be measured by WCSS as discussed in the
previous slides.
• Depending on the distortion appropriate number of
clusters k is chosen.

22. KSI Microsoft AEP mentorrbuddy.co
m
THE ELBOW METHOD (2/2)
0
1000
2000
3000
4000
5000
6000
7000
8000
9000
1 2 3 4 5 6 7 8 9 10
Series2

23. KSI Microsoft AEP mentorrbuddy.co
m
ASSUMPTIONS
• If any one of these 3 assumptions is violated, then k-means will fail:
• k-means assume the variance of the distribution of each attribute (variable) is
spherical;
• All variables have the same variance;
• the prior probability for all k clusters are the same, i.e. each cluster has roughly equal
number of observations
• Clusters in K-means are defined by taking the mean of all the data points in the
cluster.

24. KSI Microsoft AEP mentorrbuddy.co
m
ADVANTAGES
• Relatively simple to implement.
• Scales to large data sets.
• Guarantees convergence.
• Can warm-start the positions of centroids.
• Easily adapts to new examples.
• Generalizes to clusters of different shapes and sizes, such as elliptical clusters.

25. KSI Microsoft AEP mentorrbuddy.co
m
DISADVANTAGES
• The user has to specify 𝑘 (the number of clusters) in the beginning.
• k-means can only handle numerical data.
• Being dependent on initial values.
• Clustering data of varying sizes and density.
• Clustering outliers.
• Scaling with number of dimensions.

26. KSI Microsoft AEP mentorrbuddy.co
m
APPLICATIONS
• Marketing : It can be used to characterize &
discover customer segments for marketing
purposes.
• Biology : It can be used for classification among
different species of plants and animals.
• Libraries : It is used in clustering different books
on the basis of topics and information.
• Insurance : It is used to acknowledge the
customers, their policies and identifying the
frauds.

27. KSI Microsoft AEP mentorrbuddy.co
m
THANK YOU