This document discusses hierarchical clustering, an unsupervised machine learning technique that produces nested clusters organized as a hierarchical tree. There are two main types of hierarchical clustering: agglomerative, which starts with each point as an individual cluster and merges them; and divisive, which starts with everything in one cluster and splits them. Different linkage methods like single, complete, average, and Ward's linkage define how the distance between clusters is calculated during the merging or splitting process. Hierarchical clustering has strengths like not requiring pre-specifying the number of clusters but has weaknesses like high computational complexity of O(n3) time for most algorithms.
Lecture 7: Hierarchical clustering, DBSCAN, Mixture models and the EM algorithm (ppt,pdf)
Chapter 8,9 from the book “Introduction to Data Mining” by Tan, Steinbach, Kumar.
Lecture 11 - KNN and Clustering, a lecture in subject module Statistical & Ma...Maninda Edirisooriya
Supervised ML technique, K-Nearest Neighbor and Unsupervised Clustering techniques are learnt in this lesson. This was one of the lectures of a full course I taught in University of Moratuwa, Sri Lanka on 2023 second half of the year.
Lecture 7: Hierarchical clustering, DBSCAN, Mixture models and the EM algorithm (ppt,pdf)
Chapter 8,9 from the book “Introduction to Data Mining” by Tan, Steinbach, Kumar.
Lecture 11 - KNN and Clustering, a lecture in subject module Statistical & Ma...Maninda Edirisooriya
Supervised ML technique, K-Nearest Neighbor and Unsupervised Clustering techniques are learnt in this lesson. This was one of the lectures of a full course I taught in University of Moratuwa, Sri Lanka on 2023 second half of the year.
A brief introduction to clustering with Scikit learn. In this presentation, we provide an overview with real examples of how to make use and optimize within k-means clustering.
A network is said to have community structure if the nodes of the network can be easily grouped into (potentially overlapping)a sets of nodes such that each set of nodes is densely connected internally.
Classification of common clustering algorithm and techniques, e.g., hierarchical clustering, distance measures, K-means, Squared error, SOFM, Clustering large databases.
A brief introduction to clustering with Scikit learn. In this presentation, we provide an overview with real examples of how to make use and optimize within k-means clustering.
A network is said to have community structure if the nodes of the network can be easily grouped into (potentially overlapping)a sets of nodes such that each set of nodes is densely connected internally.
Classification of common clustering algorithm and techniques, e.g., hierarchical clustering, distance measures, K-means, Squared error, SOFM, Clustering large databases.
Introduction to AI for Nonprofits with Tapp NetworkTechSoup
Dive into the world of AI! Experts Jon Hill and Tareq Monaur will guide you through AI's role in enhancing nonprofit websites and basic marketing strategies, making it easy to understand and apply.
A Strategic Approach: GenAI in EducationPeter Windle
Artificial Intelligence (AI) technologies such as Generative AI, Image Generators and Large Language Models have had a dramatic impact on teaching, learning and assessment over the past 18 months. The most immediate threat AI posed was to Academic Integrity with Higher Education Institutes (HEIs) focusing their efforts on combating the use of GenAI in assessment. Guidelines were developed for staff and students, policies put in place too. Innovative educators have forged paths in the use of Generative AI for teaching, learning and assessments leading to pockets of transformation springing up across HEIs, often with little or no top-down guidance, support or direction.
This Gasta posits a strategic approach to integrating AI into HEIs to prepare staff, students and the curriculum for an evolving world and workplace. We will highlight the advantages of working with these technologies beyond the realm of teaching, learning and assessment by considering prompt engineering skills, industry impact, curriculum changes, and the need for staff upskilling. In contrast, not engaging strategically with Generative AI poses risks, including falling behind peers, missed opportunities and failing to ensure our graduates remain employable. The rapid evolution of AI technologies necessitates a proactive and strategic approach if we are to remain relevant.
Normal Labour/ Stages of Labour/ Mechanism of LabourWasim Ak
Normal labor is also termed spontaneous labor, defined as the natural physiological process through which the fetus, placenta, and membranes are expelled from the uterus through the birth canal at term (37 to 42 weeks
A workshop hosted by the South African Journal of Science aimed at postgraduate students and early career researchers with little or no experience in writing and publishing journal articles.
Acetabularia Information For Class 9 .docxvaibhavrinwa19
Acetabularia acetabulum is a single-celled green alga that in its vegetative state is morphologically differentiated into a basal rhizoid and an axially elongated stalk, which bears whorls of branching hairs. The single diploid nucleus resides in the rhizoid.
Francesca Gottschalk - How can education support child empowerment.pptxEduSkills OECD
Francesca Gottschalk from the OECD’s Centre for Educational Research and Innovation presents at the Ask an Expert Webinar: How can education support child empowerment?
A review of the growth of the Israel Genealogy Research Association Database Collection for the last 12 months. Our collection is now passed the 3 million mark and still growing. See which archives have contributed the most. See the different types of records we have, and which years have had records added. You can also see what we have for the future.
Safalta Digital marketing institute in Noida, provide complete applications that encompass a huge range of virtual advertising and marketing additives, which includes search engine optimization, virtual communication advertising, pay-per-click on marketing, content material advertising, internet analytics, and greater. These university courses are designed for students who possess a comprehensive understanding of virtual marketing strategies and attributes.Safalta Digital Marketing Institute in Noida is a first choice for young individuals or students who are looking to start their careers in the field of digital advertising. The institute gives specialized courses designed and certification.
for beginners, providing thorough training in areas such as SEO, digital communication marketing, and PPC training in Noida. After finishing the program, students receive the certifications recognised by top different universitie, setting a strong foundation for a successful career in digital marketing.
This slide is special for master students (MIBS & MIFB) in UUM. Also useful for readers who are interested in the topic of contemporary Islamic banking.
1. 1
Hierarchical Clustering
Class
Program
University
Semester
Lecturer
Algorithmic Methods of Data Mining
M. Sc. Data Science
Sapienza University of Rome
Fall 2015
Carlos Castillo http://chato.cl/
Sources:
●
● Mohammed J. Zaki, Wagner Meira, Jr., Data Mining and Analysis:
Fundamental Concepts and Algorithms, Cambridge University
Press, May 2014. Chapter 14. [download]
Evimaria Terzi: Data Mining course at Boston University
http://www.cs.bu.edu/~evimaria/cs565-13.html
4. Hierarchical Clustering
• Produces a set of nested clusters organized as
a hierarchical tree
• Can be visualized as a dendrogram
– A tree-like diagram that records the sequences of
merges or splits
5. Strengths of Hierarchical
Clustering
• No assumptions on the number of clusters
– Any desired number of clusters can be obtained by
‘cutting’ the dendogram at the proper level
• Hierarchical clusterings may correspond to
meaningful taxonomies
– Example in biological sciences (e.g., phylogeny
reconstruction, etc), web (e.g., product catalogs) etc
6. Hierarchical Clustering
Algorithms
• Two main types of hierarchical clustering
– Agglomerative:
• Start with the points as individual clusters
•At each step, merge the closest pair of clusters until only one cluster
(or k clusters) left
– Divisive:
• Start with one, all-inclusive cluster
•At each step, split a cluster until each cluster contains a point (or
there are k clusters)
• Traditional hierarchical algorithms use a similarity or
distance matrix
– Merge or split one cluster at a time
7. Complexity of hierarchical
clustering
• Distance matrix is used for deciding which
clusters to merge/split
• At least quadratic in the number of data points
• Not usable for large datasets
8. Agglomerative clustering algorithm
•
•
Most popular hierarchical clustering technique
Basic algorithm:
Compute the distance matrix between the input data points
Let each data point be a cluster
Repeat
Merge the two closest clusters
Update the distance matrix
Until only a single cluster remains
Key operation is the computation of the distance
between two clusters
Different definitions of the distance between clusters lead
to different algorithms
9. Input/ Initial setting
• Start with clusters of individual points
and a distance/proximity matrix
. .
p1 p2 p3 p4 p5 .
p1
p2
p3
p4
p5
.
.
.
Distance/Proximity Matrix
10. Intermediate State
• After some merging steps, we have some clusters
C1
C4
C2 C5
C3
C1 C2 C3 C4 C5
C1
C2
C3
C4
C5
Distance/Proximity Matrix
11. Intermediate State
• Merge the two closest clusters (C2 and C5) and update the
distance matrix.
C1
C4
C2 C5
C3
C1 C2 C3 C4 C5
C1
C2
C3
C4
C5
Distance/Proximity Matrix
12. After Merging
• “How do we update the distance matrix?”
C2 U
C1
C4
C2 U C5
C3
C5
C2 U C
C1 C3 C4
C
1
?
5 ? ? ? ?
C
3
?
C
4
?
13. Distance between two
clusters
• Each cluster is a set of points
• How do we define distance between
two sets of points
– Lots of alternatives
– Not an easy task
14. Distance between two
clusters
• Single-link distance between clusters Ci and
Cj is the minimum distance between any
object in Ci and any object in Cj
• The distance is defined by the two most
similar objects
17. 17
Exercise: 1-dimensional clustering
5 11 13 16 25 36 38 39 42 60 62 64 67
Exercise:
Create a hierarchical agglomerative clustering for this data.
To make this deterministic, if there are ties, pick the left-most link.
Verify: clustering with 4 clusters has 25 as singleton.
http://chato.cl/2015/data-analysis/exercise-answers/hierarchical-clustering_exercise_01_answer.txt
18. Strengths of single-link clustering
Original Points Two Clusters
• Can handle non-elliptical shapes
19. Limitations of single-link clustering
Original Points Two Clusters
• Sensitive to noise and outliers
• It produces long, elongated clusters
20. Distance between two
clusters
• Complete-link distance between clusters Ci
and Cj is the maximum distance between any
object in Ci and any object in Cj
• The distance is defined by the two most
dissimilar objects
21. Complete-link clustering: example
• Distance between clusters is determined by the
two most distant points in the different clusters
1 2 3 4 5
25. Distance between two
clusters
• Group average distance between clusters Ci
and Cj is the average distance between any
object in Ci and any object in Cj
29. Distance between two
clusters
• Centroid distance between clusters Ci and Cj is
the distance between the centroid ri of Ci and the
centroid rj of Cj
30. Distance between two
clusters
• Ward’s distance between clusters Ci and Cj is the
difference between the total within cluster sum of
squares for the two clusters separately, and the
within cluster sum of squares resulting from
merging the two clusters in cluster Cij
• ri: centroid of Ci
• rj: centroid of Cj
• rij: centroid of Cij
31. Ward’s distance for
clusters
• Similar to group average and centroid distance
• Less susceptible to noise and outliers
• Biased towards globular clusters
• Hierarchical analogue of k-means
– Can be used to initialize k-means
33. Hierarchical Clustering: Time and Space
requirements
• For a dataset X consisting of n points
• O(n2 ) space; it requires storing the distance matrix
• O(n3 ) time in most of the cases
– There are n steps and at each step the size n2 distance
matrix must be updated and searched
– Complexity can be reduced to O(n2 log(n) ) time for
some approaches by using appropriate data structures