- Hierarchical clustering produces nested clusters organized as a hierarchical tree called a dendrogram. It can be either agglomerative, where each point starts in its own cluster and clusters are merged, or divisive, where all points start in one cluster which is recursively split. - Common hierarchical clustering algorithms include single linkage (minimum distance), complete linkage (maximum distance), group average, and Ward's method. They differ in how they calculate distance between clusters during merging. - K-means is a partitional clustering algorithm that divides data into k non-overlapping clusters based on minimizing distance between points and cluster centroids. It is fast but sensitive to initialization and assumes spherical clusters of similar size and density.

1. Data Mining Cluster Analysis: Basic Concepts and Algorithms Lecture Notes for Chapter 8 Introduction to Data Mining by Tan, Steinbach, Kumar © Tan,Steinbach, Kumar Introduction to Data Mining 4/18/2004

5. Notion of a Cluster can be Ambiguous How many clusters? Four Clusters Two Clusters Six Clusters

7. Partitional Clustering Original Points A Partitional Clustering

8. Hierarchical Clustering Traditional Hierarchical Clustering Non-traditional Hierarchical Clustering Non-traditional Dendrogram Traditional Dendrogram

22. Two different K-means Clusterings Original Points Sub-optimal Clustering Optimal Clustering

23. Importance of Choosing Initial Centroids

24. Importance of Choosing Initial Centroids

26. Importance of Choosing Initial Centroids …

27. Importance of Choosing Initial Centroids …

29. 10 Clusters Example Starting with two initial centroids in one cluster of each pair of clusters

30. 10 Clusters Example Starting with two initial centroids in one cluster of each pair of clusters

31. 10 Clusters Example Starting with some pairs of clusters having three initial centroids, while other have only one.

32. 10 Clusters Example Starting with some pairs of clusters having three initial centroids, while other have only one.

38. Bisecting K-means Example

40. Limitations of K-means: Differing Sizes Original Points K-means (3 Clusters)

41. Limitations of K-means: Differing Density Original Points K-means (3 Clusters)

42. Limitations of K-means: Non-globular Shapes Original Points K-means (2 Clusters)

44. Overcoming K-means Limitations Original Points K-means Clusters

45. Overcoming K-means Limitations Original Points K-means Clusters

60. Hierarchical Clustering: MIN Nested Clusters Dendrogram 1 2 3 4 5 6 1 2 3 4 5

64. Hierarchical Clustering: MAX Nested Clusters Dendrogram 1 2 3 4 5 6 1 2 5 3 4

68. Hierarchical Clustering: Group Average Nested Clusters Dendrogram 1 2 3 4 5 6 1 2 5 3 4

71. Hierarchical Clustering: Comparison Group Average Ward’s Method MIN MAX 1 2 3 4 5 6 1 2 5 3 4 1 2 3 4 5 6 1 2 5 3 4 1 2 3 4 5 6 1 2 5 3 4 1 2 3 4 5 6 1 2 3 4 5

77. DBSCAN: Core, Border, and Noise Points

79. DBSCAN: Core, Border and Noise Points Original Points Point types: core , border and noise Eps = 10, MinPts = 4

84. Clusters found in Random Data Random Points K-means DBSCAN Complete Link

93. Using Similarity Matrix for Cluster Validation DBSCAN

103. External Measures of Cluster Validity: Entropy and Purity