Hierarchical clustering

1. Hierarchical
Clustering
4/30/17
1

2. Two type of hierarchical clustering
• Divisive hierarchical clustering
• Top – down approach
• Agglomerative hierarchical
clustering
• Bottom – up approach
4/30/17
2

3. Algorithm:
agglomerative
hierarchical clustering
4/30/17
3

4. Cluster similarity or dissimilarity
• Distance metric
• Euclidean distance
• Manhattan distance
• Jaccard index*
• Linkage criteria
• Single linkage
• Complete linkage
• Average linkage
* https://en.wikipedia.org/wiki/Jaccard_index
4/30/17
4

5. Single linkage
4/30/17
5

6. Complete linkage
4/30/17
6

7. Average linkage
4/30/17
7

8. Example: Complete linkage
Final set of clusters is (1,3,5),(2),(4)
4/30/17
8

9. Impact of metrics
Distance metric
• In a 2-dimensional space, the distance
between the point (1,1) and the origin
(0,0) can be 2 under Manhattan
distance, √2 under Euclidean distance.
01
Linkage criteria
• Distance between to clusters can be
different based on linkage criteria used
02
4/30/17
9

10. Linkage criteria
• Complete linkage is the most popular metric used for hierarchical clustering. It
is less sensitive to outliers.
• Single linkage can handle non-elliptical shapes. But, single linkage can lead to
clusters that are quite heterogeneous internally and it more sensitive to outliers
and noise.
4/30/17
10

11. Pros and cons: Hierarchical Clustering
• Pros
• No assumption of particular number of clusters
• Cons
• Too slow for large data sets, O(n2 log(n) )
• Once a decision is made to combine two clusters, it can’t be undone
4/30/17
11

12. References
• http://www.saedsayad.com/clustering_hierarchical.htm
• https://onlinecourses.science.psu.edu/stat555/node/86
• https://www.slideshare.net/xuyangela?utm_campaign=profiletracking&utm_me
dium=sssite&utm_source=ssslideview
4/30/17
12