This document summarizes a paper presentation on selecting the optimal number of clusters (K) for k-means clustering. The paper proposes a new evaluation measure to automatically select K without human intuition. It reviews existing methods, analyzes factors influencing K selection, describes the proposed measure, and applies it to real datasets. The method was validated on artificial and benchmark datasets. It aims to suggest multiple K values depending on the required detail level for clustering. However, it is computationally expensive for large datasets and the data used may not reflect real complexity.

1. 2013 KSE Seminar
2013/10/11
Jung hoon Kim

2. TOPIC

3. Selection of K in
K-means clustering

4. Why I choose this paper
• There is always an assumption in k-means
algorithm, but I really want to execute without
human’s intuition or insight.
• This paper is first review existing automatical
method for selecting the number of clusters for
k-means algorithm

5. Paper Format
1)
2)
3)
4)
5)
Introduction
review the main known method for selecting K
analyses the factors influencing the selection of K
describes the proposed evaluation measure
presents the results of applying the proposed
measure to select K for different data sets
6) concludes the paper

6. Small introduction

7. K-means Algorithm
• k-means algorithm is a method of clustering
algorithm originally from signal processing, that is
popular for machine learning and data mining.
• k-means clustering aims to partition n
observations into k clusters in which each
observation belongs to the cluster with the
nearest mean until move distance is smaller than
threshold

8. K-means Algorithm
1) Pick a number (k) of point randomly
2) Assign every node to its nearest cluster center
3) Move each cluster center to the mean of its
assigned nodes
4) Repeat 2-3 until convergence

9. Clustering: Example 2, Step 1
Algorithm: k-means, Distance Metric: Euclidean Distance
expression in condition 2
5
4
k1
3
k2
2
1
k3
0
0
1
2
3
4
expression in condition 1
5

10. Clustering: Example 2, Step 2
Algorithm: k-means, Distance Metric: Euclidean Distance
expression in condition 2
5
4
k1
3
k2
2
1
k3
0
0
1
2
3
4
expression in condition 1
5

11. Clustering: Example 2, Step 3
Algorithm: k-means, Distance Metric: Euclidean Distance
expression in condition 2
5
4
k1
3
2
k3
k2
1
0
0
1
2
3
4
expression in condition 1
5

12. Clustering: Example 2, Step 4
Algorithm: k-means, Distance Metric: Euclidean Distance
expression in condition 2
5
4
k1
3
2
k3
k2
1
0
0
1
2
3
4
expression in condition 1
5

13. Clustering: Example 2, Step 5
Algorithm: k-means, Distance Metric: Euclidean Distance
expression in condition 2
5
4
k1
3
2
k2
k3
1
0
0
1
2
3
4
expression in condition 1
5

14. Comments on the K-Means Metho
d
• Strength
• Relatively efficient: O(tkn), where n is # instances, c is # clusters
, and t is # iterations. Normally, k, t << n.
• Often terminates at a local optimum. The global optimum may
be found using techniques such as: simulated annealing or ge
netic algorithms
• Weakness
• Need to specify c, the number of clusters, in advance
• Initialization Problem
• Not suitable to discover clusters with non-convex shapes

15. What’s the problem?

16. What’s the problem?
• Initialization problem
• it's a problem which is caused when much point is assigned to the part
of high density and less point is assigned to the part of low density

17. What’s the problem?
• hard to find cluster in non-convex shape

18. What’s the problem?
• Selection of K

19. Existing Method
• Values of K determined through human’s viewpoint
• Using probabilistic theory
• Akeike’s information criterion
• if data sets are constructed by a set of Gaussian dist
• Hardy method
• if data sets are constructed by a set of Possion dist
• Monte Carlo techniques(associated null hypothesis)

20. Paper proposed

21. Formula

22. Research Method
• The method has been validated on
15 artificial and 12 benchmark data sets.
• Also there are 12 benchmark data sets from the
UCI Repository Machine Learning Databases
• These fifteen artificial data sets show effective
sample of lots of distribution which can be usually
generated.

23. Sample

24. Sample

25. Sample

26. Sample

27. Recommendation Example
f(X) < 0.85, K = X
else K=1

28. Conclusion
• The new method is closely related to the approach
of K-means clustering because it takes into account
information reflecting the performance of the
algorithm
• The proposed method can suggest multiple values
of K to users for cases when different clustering
results could be obtained with various required
levels of detail
• this method is computationally expensive if used
with large data sets

29. improvement
• This paper did not mentioned how can we calculate
threshold(e.g, f(x) < 0.85), if we have lots of data
sets, we can apply learning algorithm to determine
threshold
• Experiment data sets are almost biased. This means,
having set of data is too ideal. It doesn't consider
the complexity in reality at all. It can be a way to
evaluate data randomly.
• It is an important issue that we know the range, or
maximum value of K.

30. Do you
have any question?

31. thank you