There are two types of hierarchical clustering: divisive, which is a top-down approach, and agglomerative, which is a bottom-up approach. Agglomerative hierarchical clustering starts by treating each observation as a separate cluster and merges them into successively larger clusters. There are different methods to measure the distance between clusters including Euclidean distance, Manhattan distance, and Jaccard index. Different linkage criteria can also be used such as single, complete, or average linkage to determine the distance between clusters when merging them. Complete linkage is often used as it is less sensitive to outliers than other methods.

1. Hierarchical
Clustering
4/30/17
1

2. Two type of hierarchical clustering
• Divisive hierarchical clustering
• Top – down approach
• Agglomerative hierarchical
clustering
• Bottom – up approach
4/30/17
2

3. Algorithm:
agglomerative
hierarchical clustering
4/30/17
3

4. Cluster similarity or dissimilarity
• Distance metric
• Euclidean distance
• Manhattan distance
• Jaccard index*
• Linkage criteria
• Single linkage
• Complete linkage
• Average linkage
* https://en.wikipedia.org/wiki/Jaccard_index
4/30/17
4

5. Single linkage
4/30/17
5

6. Complete linkage
4/30/17
6

7. Average linkage
4/30/17
7

8. Example: Complete linkage
Final set of clusters is (1,3,5),(2),(4)
4/30/17
8

9. Impact of metrics
Distance metric
• In a 2-dimensional space, the distance
between the point (1,1) and the origin
(0,0) can be 2 under Manhattan
distance, √2 under Euclidean distance.
01
Linkage criteria
• Distance between to clusters can be
different based on linkage criteria used
02
4/30/17
9

10. Linkage criteria
• Complete linkage is the most popular metric used for hierarchical clustering. It
is less sensitive to outliers.
• Single linkage can handle non-elliptical shapes. But, single linkage can lead to
clusters that are quite heterogeneous internally and it more sensitive to outliers
and noise.
4/30/17
10

11. Pros and cons: Hierarchical Clustering
• Pros
• No assumption of particular number of clusters
• Cons
• Too slow for large data sets, O(n2 log(n) )
• Once a decision is made to combine two clusters, it can’t be undone
4/30/17
11

12. References
• http://www.saedsayad.com/clustering_hierarchical.htm
• https://onlinecourses.science.psu.edu/stat555/node/86
• https://www.slideshare.net/xuyangela?utm_campaign=profiletracking&utm_me
dium=sssite&utm_source=ssslideview
4/30/17
12