SSA KPI, profile picture
Uploaded bySSA KPI
PDF, PPTX6,694 views

Cluster Analysis

The document provides information about cluster analysis and hierarchical agglomerative clustering. It discusses how hierarchical agglomerative clustering works by successively merging the most similar clusters together based on a distance measure between clusters. It begins with each object as its own cluster and merges them into larger clusters, forming a dendrogram, until all objects are in a single cluster. Different linkage criteria like single, complete, and average linkage can be used to define the distance between clusters.

Embed presentation

Download as PDF, PPTX
Summer School “Achievements and Applications of Contemporary Informatics, Mathematics and Physics” (AACIMP 2011) August 8-20, 2011, Kiev, Ukraine Cluster Analysis Erik Kropat University of the Bundeswehr Munich Institute for Theoretical Computer Science, Mathematics and Operations Research Neubiberg, Germany
The Knowledge Discovery Process
PATTERN EVALUATION Knowledge DATA MINING Patterns Strategic planning PRE- PROCESSING Preprocessed Data Patterns, clusters, correlations automated classification Raw outlier / anomaly detection Standardizing association rule learning… Data Missing values / outliers
Clustering
Clustering … is a tool for data analysis, which solves classification problems. Problem Given n observations, split them into K similar groups. Question How can we define “similarity” ?
Similarity A cluster is a set of entities which are alike, and entities from different clusters are not alike.
Distance A cluster is an aggregation of points such that the distance between any two points in the cluster is less than the distance between any point in the cluster and any point not in it.
Density Clusters may be described as connected regions of a multidimensional space containing a relatively high density of points, separated from other such regions by a region containing a relatively low density of points.
Min Max-Problem Homogeneity: Objects within the same cluster should be similar to each other. Separation: Objects in different clusters should be dissimilar from each other. Distance between clusters Distance between objects similarity ⇔ distance
Types of Clustering Clustering Hierarchical Partitional Clustering Clustering agglomerative divisive
Similarity and Distance
Distance Measures A metric on a set G is a function d: G x G → R+ that satisfies the following conditions: (D1) d(x, y) = 0 ⇔ x=y (identity) (D2) d(x, y) = d(y, x) ≥ 0 for all x, y ∈ G (symmetry & non-negativity) (D3) d(x, y) ≤ d(x, z) + d(z, y) for all x, y, z ∈ G (triangle inequality) z y x
Examples Minkowski-Distance 1 _ n r r d r (x, y) = Σ | xi − yi | i=1 , r ∈ [1, ∞) , x, y ∈ Rn. o r = 1: Manhatten distance o r = 2: Euklidian distance
Euclidean Distance 1 _ n Σ 2 2 d2 (x, y) = ( xi − yi ) , x, y ∈ Rn i=1 y x = (1, 1) y = (4, 3) _ 1 ____ 2 2 2 d2 (x, y) = (1 - 4) + (1 - 3) = √13 x
Manhatten Distance n d1 (x, y) = Σ | xi − yi | , x, y ∈ Rn i=1 y x = (1, 1) y = (4, 3) d1 (x, y) = | 1 - 4 | + | 1 - 3 | = 3 + 2 = 5 x
Maximum Distance d∞ (x, y) = max | xi − yi | , x, y ∈ Rn 1≤i≤n y x = (1, 1) y = (4, 3) d∞ (x, y) = max (3, 2) = 3 x
Similarity Measures A similarity function on a set G is a function S: G x G → R that satisfies the following conditions: (S1) S (x, y) ≥ 0 for all x, y ∈ G (positive defined) (S2) S (x, y) ≤ S (x, x) for all x, y ∈ G (auto-similarity) (S3) S (x, y) = S (x, x) ⇔ x=y for all x, y ∈ G (identity) The value of the similarity function is greater when two points are closer.
Similarity Measures • There are many different definitions of similarity. • Often used (S4) S (x, y) = S (y, x) for all x, y ∈ G (symmetry)
Hierachical Clustering
Dendrogram Cluster Dendrogram Euclidean distance (complete linkage) Euclidean distance (complete linkage) Gross national product of EU countries – agriculture (1993) www.isa.uni-stuttgart.de/lehre/SAHBD
Hierarchical Clustering Hierarchical clustering creates a hierarchy of clusters of the set G. Hierarchical Clustering agglomerative divisive Agglomerative clustering: Clusters are successively merged together Divisive clustering: Clusters are recursively split
Agglomerative Clustering Merge clusters with smallest distance between the two clusters Step 3 e1, e2 , e3, e4 1 cluster Step 2 e1, e2, e3 e4 2 clusters Step 1 e1, e2 e3 e4 3 clusters Step 0 e1 e2 e3 e4 4 clusters
Divisive Clustering Chose a cluster, that optimally splits in two particular clusters according to a given criterion. Step 0 e1, e2 , e3, e4 1 cluster Step 1 e1, e2 e3, e4 2 clusters Step 2 e1, e2 e3 e4 3 clusters Step 3 e1 e2 e3 e4 4 clusters
Agglomerative Clustering
INPUT Given n objects G = { e1,...,en } represented by p-dimensional feature vectors x1,...,xn ∈ Rp Feature p Feature 1 Feature 2 Feature 3 Object x1 = ( x11 x12 x13 ... x1p ) x2 = ( x21 x22 x23 ... x2p ) ⁞ ⁞ ⁞ ⁞ ⁞ xn = ( xn1 xn2 xn3 ... xnp )
Example I An online shop collects data from its customers. For each of the n customers it exists a p-dimensional feature vector Object
Example II In a clinical trial laboratory values of a large number of patients are gathered. For each of the n patients it exists a p-dimensional feature vector
Agglomerative Algorithms • Begin with disjoint clustering C1 = { {e1}, {e2}, ... , {en} } • Terminate when all objects are in one cluster Cn = { {e1, e2, ... , en} } e1 e2 e3 e4 • Iterate find the most similar pair of clusters and merge them into a single cluster. Sequence of clusterings (Ci )i=1,...n of G with Ci ̶ 1 ⊂ Ci for i = 2,...,n.
What is the distance between two clusters? A d (A,B) B ⇒ Various hierarchical clustering algorithms
Agglomerative Hierarchical Clustering There exist many metrics to measure the distance between clusters. They lead to particular agglomerative clustering methods: • Single-Linkage Clustering • Complete-Linkage Clustering • Average Linkage Clustering • Centroid Method • ...
Single-Linkage Clustering Nearest-Neighbor-Method The distance between the clusters A und B is the minimum distance between the elements of each cluster: d(A,B) = min { d (a, b) | a ∈ A, b ∈ B } a d(A,B) b
Single-Linkage Clustering • Advantage: Can detect very long and even curved clusters. Can be used to detect outliers. • Drawback: Chaining phenomen Clusters that are very distant to each other may be forced together due to single elements being close to each other. B C A
Complete-Linkage Clustering Furthest-Neighbor-Method The distance between the clusters A and B is the maximum distance between the elements of each cluster: d(A,B) = max { d(a,b) | a ∈ A, b ∈ B } a d (A, B) b
Complete-Linkage Clustering • … tends to find compact clusters of approximately equal diameters. • … avoids the chaining phenomen. • … cannot be used for outlier detection.
Average-Linkage Clustering The distance between the clusters A and B is the mean distance between the elements of each cluster: 1 d (A, B) = ⋅ Σ d (a, b) |A| ⋅ |B| a ∈ A, b∈B a b A B d(A,B)
Centroid Method The distance between the clusters A and B is the (squared) Euclidean distance of the cluster centroids. d (A, B) x x
Agglomerative Hierarchical Clustering d (A, B) d (A, B) a d (A, B) d (A, B) b d (A, B) x (A, B) x
Bioinformatics Alizadeh et al., Nature 403 (2000): pp.503–511
Exercise Berlin Kiev Paris Odessa
Exercise The following table shows the distances between 4 cities: Kiev Odessa Berlin Paris Kiev ̶ 440 1200 2000 Odessa 440 ̶ 1400 2100 Berlin 1200 1400 ̶ 900 Paris 2000 2100 900 ̶ Determine a hierarchical clustering with the single linkage method.
Solution - Single Linkage Step 0: Clustering {Kiev}, {Odessa}, {Berlin}, {Paris} Distances between clusters Kiev Odessa Berlin Paris Kiev ̶ 440 1200 2000 Odessa 440 ̶ 1400 2100 Berlin 1200 1400 ̶ 900 Paris 2000 2100 900 ̶
Solution - Single Linkage Step 0: Clustering {Kiev}, {Odessa}, {Berlin}, {Paris} Distances between clusters minimal distance Kiev Odessa Berlin Paris Kiev ̶ 440 1200 2000 Odessa 440 ̶ 1400 2100 Berlin 1200 1400 ̶ 900 Paris 2000 2100 900 ̶ ⇒ Merge clusters { Kiev } and { Odessa } Distance value: 440
Solution - Single Linkage Step 1: Clustering {Kiev, Odessa}, {Berlin}, {Paris} Distances between clusters Kiev, Odessa Berlin Paris Kiev, Odessa ̶ 1200 2000 Berlin 1200 ̶ 900 Paris 2000 900 ̶
Solution - Single Linkage Step 1: Clustering {Kiev, Odessa}, {Berlin}, {Paris} Distances between clusters minimal distance Kiev, Odessa Berlin Paris Kiev, Odessa ̶ 1200 2000 Berlin 1200 ̶ 900 Paris 2000 900 ̶ ⇒ Merge clusters { Berlin } and { Paris } Distance value: 900
Solution - Single Linkage Step 2: Clustering {Kiev, Odessa}, {Berlin, Paris} Distances between clusters minimal distance Kiev, Odessa Berlin, Paris Kiev, Odessa ̶ 1200 Berlin, Paris 1200 ̶ ⇒ Merge clusters { Kiev, Odessa } and { Berlin, Paris } Distance value: 1200
Solution - Single Linkage Step 3: Clustering {Kiev, Odessa, Berlin, Paris}
Solution - Single Linkage Hierarchy Distance values 2540 1 cluster 1200 1340 2 clusters 900 440 3 clusters 440 0 4 clusters Kiev Odessa Berlin Paris
Divisive Clustering
Divisive Algorithms • Begin with one cluster C1 = { {e1, e2, ... , en} } e1 e2 e3 e4 • Terminate when all objects are in disjoint clusters Cn = { {e1}, {e2}, ... , {en} } • Iterate Chose a cluster Cf , that optimally splits two particular clusters Ci and Cj according to a given criterion. Sequence of clusterings (Ci )i=1,...n of G with C i ⊃ C i + 1 for i = 1,...,n-1.
Partitional Clustering ̶ Minimal Distance Methods ̶
Partitional Clustering K=2 • Aims to partition n observations into K clusters. • The number of clusters and initial partition an initial partition are given. • The initial partition is considered as “not optimal“ and should be K=2 iteratively repartitioned. The number of clusters is given !!! final partition
Partitional Clustering Difference to hierarchical clustering • number of clusters is fixed. • an object can change the cluster. Initial partition is obtained by • random or • the application of an hierarchical clustering algorithm in advance. Estimation of the number of clusters • specialized methods (e.g., Silhouette) or • the application of an hierarchical clustering algorithm in advance.
Partitional Clustering - Methods In this course we will introduce the minimal distance methods . . . • K-Means and • Fuzzy-C-Means
K-Means
K-Means Aims to partition n observations into K clusters in which each observation belongs to the cluster with the nearest mean. G C3 Find K cluster centroids µ1 ,..., µK that minimize the objective function K C1 Σ Σ 2 J = dist ( µi, x ) i =1 x ∈ Ci C2
K-Means Aims to partition n observations into K clusters in which each observation belongs to the cluster with the nearest mean. G C3 Find K cluster centroids µ1 ,..., µK that minimize the objective function x x K C1 Σ Σ 2 J = dist ( µi, x ) x i =1 x ∈ Ci C2
K-Means - Minimal Distance Method Given: n objects, K clusters 1. Determine initial partition. 2. Calculate cluster centroids. x x 3. For each object, calculate the distances to all cluster centroids. repartition 4. If the distance to the centroid of another cluster is smaller than the distance to the actual cluster centroid, then assign the object to the other cluster. 5. If clusters are repartitioned: GOTO 2. ELSE: STOP.
Example ₓ ₓ ₓ ₓ Initial Partition Final Partition
Exercise ₓ ₓ ₓ ₓ Initial Partition Final Partition
K-Means • K-Means does not determine the global optimal partition. • The final partition obtained by K-Means depends on the initial partition.
Hard Clustering / Soft Clustering Clustering Hard Clustering Soft Clustering Each object is a member Each object has a fractional of exactly one cluster membership in all clusters K-Means Fuzzy-c-Means
Fuzzy-c-Means
Fuzzy Clustering vs. Hard Clustering • When clusters are well separated, hard clustering (K-Means) makes sense. • In many cases, clusters are not well separated. In hard clustering, borderline objects are assigned to a cluster in an arbitrary manner.
Fuzzy Set Theory • Fuzzy Theory was introduced by Lofti Zadeh in 1965. • An object can belong to a set with a degree of membership between 0 and 1. • Classical set theory is a special case of fuzzy theory that restricts membership values to be either 0 or 1.
Fuzzy Clustering • Is based on fuzzy logic and fuzzy set theory. • Objects can belong to more than one cluster. • Each object belongs to all clusters with some weight (degree of membership) 1 Cluster 1 Cluster 2 Cluster 3 0
Hard Clustering • K-Means − The number K of clusters is given. − Each object is assigned to exactly one cluster. Partition Object C3 Cluster e1 e2 e3 e4 e3 e4 C1 0 1 0 0 C2 1 0 0 0 e2 e1 C1 C2 C3 0 0 1 1
Fuzzy Clustering • Fuzzy-c-means − The number c of clusters is given. − Each object has a fractional membership in all clusters Object Cluster e1 e2 e3 e4 C1 0.8 0.2 0.1 0.0 Fuzzy-Clustering C2 0.2 0.2 0.2 0.0 There is no strict sub-division of clusters. C3 0.0 0.6 0.7 1.0 Σ 1 1 1 1
Fuzzy-c-Means • Membership Matrix U = ( u i k ) ∈ [0, 1]c x n The entry u i k denotes the degree of membership of object k in cluster i . Object 1 Object 2 … Object n Cluster 1 u11 u12 … u1n Cluster 2 u21 u22 … u2n … … … … Cluster c uc1 uc2 … ucn
Restrictions (Membership Matrix) 1. All weights for a given object, ek, must add up to 1. c Σ u ik = 1 (k = 1,...,n) i =1 2. Each cluster contains – with non-zero weight – at least one object, but does not contain – with a weight of one – all the objects. n 0< Σ u ik < n (i = 1,...,c) k =1
Fuzzy-c-Means • Vector of prototypes (cluster centroids) T V = ( v1,...,vc ) ∈ Rc Remark The cluster centroids and the membership matrix are initialized randomly. Afterwards they are iteratively optimized.
Fuzzy-c-Means ALGORITHM 1. Select an initial fuzzy partition U = (u i k ) ⇒ assign values to all u i k 2. Repeat 3. Compute the centroid of each cluster using the fuzzy partition 4. Update the fuzzy partition U = (u i k ) 5. Until the centroids do not change. Other stopping criterion: “change in the u i k is below a given threshold”.
v3 u3k Fuzzy-c-Means xk • K-Means and Fuzzy-c-Means attempt to minimize v1 u1k u2k the sum of the squared errors (SSE). v2 • In K-Means: K Σ Σ 2 SSE = dist ( vi, x ) i =1 x ∈ Ci • In Fuzzy-c-Means: c n m SSE = Σ Σ u ik . 2 dist ( vi, xk ) i =1 k =1 m ∈ [1, ∞] is a parameter (fuzzifier) that determines the influence of the weights.
v3 u3k Computing Cluster Centroids xk • For each cluster i = 1,...,c the centroid is defined by v1 u1k u2k n v2 Σ u m xk k = 1 ik _________________ ( i = 1,...,c ) vi = n m Σ u ik (V) k=1 • This is an extension of the definition of centroids of k-means. • All points are considered and the contribution of each point to the centroid is weighted by its membership degrees.
Update of the Fuzzy Partition (Membership Matrix) • Minimization of SSE subject to the constraints leads to the following update formula: 1 ______________________________________ u ik = 1 _____ c 2 (U) dist ( v i , xk ) m – 1 Σ __________ 2 s=1 dist ( vs , xk )
Fuzzy-c-Means Initialization Determine (randomly) • Matrix U of membership grades • Matrix V of cluster centroids. Iteration Calculate updates of • Matrix U of membership grades with (U) • Matrix V of cluster centroids with (V) until cluster centroids are stable or the maximum number of iterations is reached.
Fuzzy-c-means • Fuzzy-c-means depends on the Euclidian metric ⇒ spherical clusters. • Other metrics can be applied to obtain different cluster shapes. • Fuzzy covariance matrix (Gustafson/Kessel 1979) ⇒ ellipsoidal clusters.
Cluster Validity Indizes
Cluster Validity Indexes Fuzzy-c-means requires the number of clusters as input. Question: How can we determine the “optimal” number of clusters? Idea: Determine the cluster partition for a given number of clusters. Then, evaluate the cluster partition by a cluster validity index. Method: For all possible number of clusters calculate the cluster validity index. Then, determine the optimal number of clusters. Note: CVIs usually do not depend on the clustering algorithm.
Cluster Validity Indexes • Partition Coefficient (Bezdek 1981) c n 1 2 PC (c) = __ Σ Σ uik , 2 ≤ c ≤ n-1 n i=1 k=1 • Optimal number of clusters c∗ : PC (c∗) = max PC (c) 2 ≤ c ≤ n-1
Cluster Validity Indexes • Partition Entropy (Bezdek 1974) c n 1 PC (c) = _ __ Σ Σ u i k log2 u i k , 2 ≤ c ≤ n-1 n i=1 k=1 • Optimal number of clusters c∗ : PC (c∗) = min PC (c) 2 ≤ c ≤ n-1 • Drawback of PC and PE: Only degrees of memberships are considered. The geometry of the data set is neglected.
Cluster Validity Indexes • Fukuyama-Sugeno Index (Fukuyama/Sugeno 1989) c n m Σ Σ 2 Compactness FS (c) = uik dist ( vi , xk ) of clusters i=1 k=1 c n _ m _ Σ Σ Separation 2 uik dist ( vi , v ) of clusters i=1 k=1 • Optimal number of clusters c∗ : _ 1 c PC (c∗) = max PC (c) v = __ Σ vi 2 ≤ c ≤ n-1 c i =1
Application
Data Mining and Decision Support Systems ̶ Landslide Events (UniBw, Geoinformatics Group: W. Reinhardt, E. Nuhn) → Spatial Data Mining / Early Warning Systems for Landslide Events → Fuzzy clustering approaches (feature weighting) • Measurements (pressure values, tension, deformation vectors) • Simulations (finite-element model)
Hard Clustering Data Partition Problem: Uncertain data from measurements and simulations
Fuzzy Clustering Data Fuzzy-Cluster Fuzzy-Partition
Fuzzy Clustering
Feature Weighting Nuhn/Kropat/Reinhardt/Pickl: Preparation of complex landslide simulation results with clustering approaches for decision support and early warning. Submitted to Hawaii International Conference on System Sciences (HICCS 45), Grand Wailea, Maui, 2012.
Thank you very much!

More Related Content

PDF
Exploratory data analysis data visualization
byDr. Hamdan Al-Sabri
 
PPTX
Data Preparation.pptx
byDrAbhishekKumarSingh3
 
PPTX
Cluster Analysis
byDataminingTools Inc
 
PDF
Data Mining Techniques
bySanzid Kawsar
 
PPT
Web data mining
byInstitute of Technology Telkom
 
PDF
Outlier detection method introduction
byDaeJin Kim
 
PPT
Data preprocessing in Data Mining
byDHIVYADEVAKI
 
ODP
Web Content Mining
byDaminda Herath
 
Exploratory data analysis data visualization
byDr. Hamdan Al-Sabri
 
Data Preparation.pptx
byDrAbhishekKumarSingh3
 
Cluster Analysis
byDataminingTools Inc
 
Data Mining Techniques
bySanzid Kawsar
 
Web data mining
byInstitute of Technology Telkom
 
Outlier detection method introduction
byDaeJin Kim
 
Data preprocessing in Data Mining
byDHIVYADEVAKI
 
Web Content Mining
byDaminda Herath
 

What's hot

PDF
4 Dimensionality reduction (PCA & t-SNE)
byDmytro Fishman
 
PDF
Data Science - Part VII - Cluster Analysis
byDerek Kane
 
PPTX
Data Mining: clustering and analysis
byDataminingTools Inc
 
PPTX
Data Reduction
byRajan Shah
 
PPTX
Cluster analysis
byJewel Refran
 
PPTX
Pattern Recognition.pptx
byhafeez504942
 
PPTX
Naive bayesian classification
byDr-Dipali Meher
 
PPTX
Machine learning clustering
byCosmoAIMS Bassett
 
PPT
Cluster analysis
byKamalakshi Deshmukh-Samag
 
PPTX
CART – Classification & Regression Trees
byHemant Chetwani
 
PPTX
Principal Component Analysis PCA
byAbdullah al Mamun
 
PPT
Clustering
byM Rizwan Aqeel
 
PPTX
Data clustring
bySalman Memon
 
PDF
Cluster analysis
byVenkata Reddy Konasani
 
PDF
PCA (Principal component analysis)
byLearnbay Datascience
 
PPT
Data cleaning-outlier-detection
byChathurangi Shyalika
 
PPTX
Association rule mining.pptx
bymaha797959
 
PPTX
Decision tree induction \ Decision Tree Algorithm with Example| Data science
byMaryamRehman6
 
PPT
Data Mining: Concepts and Techniques_ Chapter 6: Mining Frequent Patterns, ...
bySalah Amean
 
PPTX
Statistics for data science
byzekeLabs Technologies
 
4 Dimensionality reduction (PCA & t-SNE)
byDmytro Fishman
 
Data Science - Part VII - Cluster Analysis
byDerek Kane
 
Data Mining: clustering and analysis
byDataminingTools Inc
 
Data Reduction
byRajan Shah
 
Cluster analysis
byJewel Refran
 
Pattern Recognition.pptx
byhafeez504942
 
Naive bayesian classification
byDr-Dipali Meher
 
Machine learning clustering
byCosmoAIMS Bassett
 
Cluster analysis
byKamalakshi Deshmukh-Samag
 
CART – Classification & Regression Trees
byHemant Chetwani
 
Principal Component Analysis PCA
byAbdullah al Mamun
 
Clustering
byM Rizwan Aqeel
 
Data clustring
bySalman Memon
 
Cluster analysis
byVenkata Reddy Konasani
 
PCA (Principal component analysis)
byLearnbay Datascience
 
Data cleaning-outlier-detection
byChathurangi Shyalika
 
Association rule mining.pptx
bymaha797959
 
Decision tree induction \ Decision Tree Algorithm with Example| Data science
byMaryamRehman6
 
Data Mining: Concepts and Techniques_ Chapter 6: Mining Frequent Patterns, ...
bySalah Amean
 
Statistics for data science
byzekeLabs Technologies
 

Similar to Cluster Analysis

PDF
Cluster analysis
byHohai university
 
PPT
Clustering
byShubra Singh
 
PDF
Improved Performance of Unsupervised Method by Renovated K-Means
byIJASCSE
 
PPT
Cs345 cl
bymattriley
 
PDF
A COMPARATIVE STUDY ON DISTANCE MEASURING APPROACHES FOR CLUSTERING
byIJORCS
 
PDF
Cluster Analysis: Measuring Similarity & Dissimilarity
byShivarkarSandip
 
PDF
Clustering Algorithms - Kmeans,Min ALgorithm
bySharmila Chidaravalli
 
PDF
Standardizing inter-element distances in repertory grids
byMark Heckmann
 
PDF
Dbm630 lecture09
byTokyo Institute of Technology
 
PDF
Clustering: A Survey
byRaffaele Capaldo
 
PDF
Google BigQuery is a very popular enterprise warehouse that’s built with a co...
byAbebe Admasu
 
PPT
Lect4
bysumit621
 
PDF
Methods from Mathematical Data Mining (Supported by Optimization)
bySSA KPI
 
PPT
[PPT]
bybutest
 
PDF
Unit III Unsupervised learning-clustering
byShivarkarSandip
 
PDF
Connectivity-Based Clustering for Mixed Discrete and Continuous Data
byIJCI JOURNAL
 
PPT
Lecture slides week14-15
byShani729
 
PDF
Clustering
byRashmi Bhat
 
PPT
Chapter - 7 Data Mining Concepts and Techniques 2nd Ed slides Han &amp; Kamber
byerror007
 
PPT
Digital Distance Geometry
byppd1961
 
Cluster analysis
byHohai university
 
Clustering
byShubra Singh
 
Improved Performance of Unsupervised Method by Renovated K-Means
byIJASCSE
 
Cs345 cl
bymattriley
 
A COMPARATIVE STUDY ON DISTANCE MEASURING APPROACHES FOR CLUSTERING
byIJORCS
 
Cluster Analysis: Measuring Similarity & Dissimilarity
byShivarkarSandip
 
Clustering Algorithms - Kmeans,Min ALgorithm
bySharmila Chidaravalli
 
Standardizing inter-element distances in repertory grids
byMark Heckmann
 
Dbm630 lecture09
byTokyo Institute of Technology
 
Clustering: A Survey
byRaffaele Capaldo
 
Google BigQuery is a very popular enterprise warehouse that’s built with a co...
byAbebe Admasu
 
Lect4
bysumit621
 
Methods from Mathematical Data Mining (Supported by Optimization)
bySSA KPI
 
[PPT]
bybutest
 
Unit III Unsupervised learning-clustering
byShivarkarSandip
 
Connectivity-Based Clustering for Mixed Discrete and Continuous Data
byIJCI JOURNAL
 
Lecture slides week14-15
byShani729
 
Clustering
byRashmi Bhat
 
Chapter - 7 Data Mining Concepts and Techniques 2nd Ed slides Han &amp; Kamber
byerror007
 
Digital Distance Geometry
byppd1961
 

More from SSA KPI

PDF
Germany presentation
bySSA KPI
 
PDF
Grand challenges in energy
bySSA KPI
 
PDF
Engineering role in sustainability
bySSA KPI
 
PDF
Consensus and interaction on a long term strategy for sustainable development
bySSA KPI
 
PDF
Competences in sustainability in engineering education
bySSA KPI
 
PDF
Introducatio SD for enginers
bySSA KPI
 
PPT
DAAD-10.11.2011
bySSA KPI
 
PDF
Talking with money
bySSA KPI
 
PDF
'Green' startup investment
bySSA KPI
 
PDF
From Huygens odd sympathy to the energy Huygens' extraction from the sea waves
bySSA KPI
 
PDF
Dynamics of dice games
bySSA KPI
 
PPT
Energy Security Costs
bySSA KPI
 
PPT
Naturally Occurring Radioactivity (NOR) in natural and anthropic environments
bySSA KPI
 
PDF
Advanced energy technology for sustainable development. Part 5
bySSA KPI
 
PDF
Advanced energy technology for sustainable development. Part 4
bySSA KPI
 
PDF
Advanced energy technology for sustainable development. Part 3
bySSA KPI
 
PDF
Advanced energy technology for sustainable development. Part 2
bySSA KPI
 
PDF
Advanced energy technology for sustainable development. Part 1
bySSA KPI
 
PPT
Fluorescent proteins in current biology
bySSA KPI
 
PPTX
Neurotransmitter systems of the brain and their functions
bySSA KPI
 
Germany presentation
bySSA KPI
 
Grand challenges in energy
bySSA KPI
 
Engineering role in sustainability
bySSA KPI
 
Consensus and interaction on a long term strategy for sustainable development
bySSA KPI
 
Competences in sustainability in engineering education
bySSA KPI
 
Introducatio SD for enginers
bySSA KPI
 
DAAD-10.11.2011
bySSA KPI
 
Talking with money
bySSA KPI
 
'Green' startup investment
bySSA KPI
 
From Huygens odd sympathy to the energy Huygens' extraction from the sea waves
bySSA KPI
 
Dynamics of dice games
bySSA KPI
 
Energy Security Costs
bySSA KPI
 
Naturally Occurring Radioactivity (NOR) in natural and anthropic environments
bySSA KPI
 
Advanced energy technology for sustainable development. Part 5
bySSA KPI
 
Advanced energy technology for sustainable development. Part 4
bySSA KPI
 
Advanced energy technology for sustainable development. Part 3
bySSA KPI
 
Advanced energy technology for sustainable development. Part 2
bySSA KPI
 
Advanced energy technology for sustainable development. Part 1
bySSA KPI
 
Fluorescent proteins in current biology
bySSA KPI
 
Neurotransmitter systems of the brain and their functions
bySSA KPI
 

Recently uploaded

PPTX
28 January 2026 Rebecca Frankum Are high-stakes exams and assessments still r...
byEduSkills OECD
 
PPTX
Drugs modulating serotonergic system.pptx
byNorman Arockiaraj G
 
PDF
ĐỀ MINH HỌA KỲ THI TỐT NGHIỆP TRUNG HỌC PHỔ THÔNG NĂM 2026 MÔN TIẾNG ANH 2026...
byNguyen Thanh Tu Collection
 
PPTX
Poster Based Ethical Reflection - Dharma and Values Poster.pptx
byRajshreeBeneymadoo
 
PPTX
Statistical Data Analysis using R Programming.pptx
byVETRISELVIP1
 
PPTX
COMMUNICATION ITS PROCESS ELEMENTS & BARRIER .pptx
byPoojaSen20
 
PPTX
Appreciations - Jan 26 01.pptxkjkjkkjkjkjkj
bypreetheshparmar
 
PDF
U.S. Departments of Education and Treasury fact sheet
byMebane Rash
 
PPTX
bundle of care.pptx Priti Bala @ BRD med
bypritibala13
 
PPTX
Maja Videnovik - Transforming Cybersecurity Learning into an Adventure
bybvtricks
 
PDF
Problem Solving Agents in Artificial Intelligence with Examples and Water Jug...
byARTHIDEVARANIP
 
PPTX
Toba Tek Singh - Visualising Partition through Manto's Lens
byMiral Joshi
 
PDF
Infrared Spectroscopy (IR spectroscopy).ppt
byLife sciences
 
PPTX
NMR Spectroscopy: Principles and Applications
byVanitha Rajakumar
 
PPTX
Metabolism ( BIOCHEMISTRY & CLINICAL PATHOLOGY )
byBushraDeshpande
 
PDF
Risks and opportunities of artificial intelligence in education: A critical view
byYannis
 
PDF
VIVA or SHORT ANSWER QUESTIONS or 2 MARKS QUESTIONS CHILD HEALTH NURSING.pdf
byPRADEEP ABOTHU
 
PPTX
Powerpoint for testing in embed test with Sway
byjeanpierrefort3
 
PPTX
Central Line Associated Bloodstream Infection
bynabinabhaila40
 
PDF
Oscillations /Revision notes prepared by K sandeep swamy (Msc,BEd)/For online...
bySandeep Swamy
 
28 January 2026 Rebecca Frankum Are high-stakes exams and assessments still r...
byEduSkills OECD
 
Drugs modulating serotonergic system.pptx
byNorman Arockiaraj G
 
ĐỀ MINH HỌA KỲ THI TỐT NGHIỆP TRUNG HỌC PHỔ THÔNG NĂM 2026 MÔN TIẾNG ANH 2026...
byNguyen Thanh Tu Collection
 
Poster Based Ethical Reflection - Dharma and Values Poster.pptx
byRajshreeBeneymadoo
 
Statistical Data Analysis using R Programming.pptx
byVETRISELVIP1
 
COMMUNICATION ITS PROCESS ELEMENTS & BARRIER .pptx
byPoojaSen20
 
Appreciations - Jan 26 01.pptxkjkjkkjkjkjkj
bypreetheshparmar
 
U.S. Departments of Education and Treasury fact sheet
byMebane Rash
 
bundle of care.pptx Priti Bala @ BRD med
bypritibala13
 
Maja Videnovik - Transforming Cybersecurity Learning into an Adventure
bybvtricks
 
Problem Solving Agents in Artificial Intelligence with Examples and Water Jug...
byARTHIDEVARANIP
 
Toba Tek Singh - Visualising Partition through Manto's Lens
byMiral Joshi
 
Infrared Spectroscopy (IR spectroscopy).ppt
byLife sciences
 
NMR Spectroscopy: Principles and Applications
byVanitha Rajakumar
 
Metabolism ( BIOCHEMISTRY & CLINICAL PATHOLOGY )
byBushraDeshpande
 
Risks and opportunities of artificial intelligence in education: A critical view
byYannis
 
VIVA or SHORT ANSWER QUESTIONS or 2 MARKS QUESTIONS CHILD HEALTH NURSING.pdf
byPRADEEP ABOTHU
 
Powerpoint for testing in embed test with Sway
byjeanpierrefort3
 
Central Line Associated Bloodstream Infection
bynabinabhaila40
 
Oscillations /Revision notes prepared by K sandeep swamy (Msc,BEd)/For online...
bySandeep Swamy
 

Cluster Analysis

  • 1.
    Summer School “Achievements andApplications of Contemporary Informatics, Mathematics and Physics” (AACIMP 2011) August 8-20, 2011, Kiev, Ukraine Cluster Analysis Erik Kropat University of the Bundeswehr Munich Institute for Theoretical Computer Science, Mathematics and Operations Research Neubiberg, Germany
  • 2.
  • 3.
    PATTERN EVALUATION Knowledge DATA MINING Patterns Strategic planning PRE- PROCESSING Preprocessed Data Patterns, clusters, correlations automated classification Raw outlier / anomaly detection Standardizing association rule learning… Data Missing values / outliers
  • 4.
  • 5.
    Clustering … is atool for data analysis, which solves classification problems. Problem Given n observations, split them into K similar groups. Question How can we define “similarity” ?
  • 6.
    Similarity A cluster isa set of entities which are alike, and entities from different clusters are not alike.
  • 7.
    Distance A cluster isan aggregation of points such that the distance between any two points in the cluster is less than the distance between any point in the cluster and any point not in it.
  • 8.
    Density Clusters may bedescribed as connected regions of a multidimensional space containing a relatively high density of points, separated from other such regions by a region containing a relatively low density of points.
  • 9.
    Min Max-Problem Homogeneity: Objectswithin the same cluster should be similar to each other. Separation: Objects in different clusters should be dissimilar from each other. Distance between clusters Distance between objects similarity ⇔ distance
  • 10.
    Types of Clustering Clustering Hierarchical Partitional Clustering Clustering agglomerative divisive
  • 11.
  • 12.
    Distance Measures A metricon a set G is a function d: G x G → R+ that satisfies the following conditions: (D1) d(x, y) = 0 ⇔ x=y (identity) (D2) d(x, y) = d(y, x) ≥ 0 for all x, y ∈ G (symmetry & non-negativity) (D3) d(x, y) ≤ d(x, z) + d(z, y) for all x, y, z ∈ G (triangle inequality) z y x
  • 13.
    Examples Minkowski-Distance 1 _ n r r d r (x, y) = Σ | xi − yi | i=1 , r ∈ [1, ∞) , x, y ∈ Rn. o r = 1: Manhatten distance o r = 2: Euklidian distance
  • 14.
    Euclidean Distance 1 _ n Σ 2 2 d2 (x, y) = ( xi − yi ) , x, y ∈ Rn i=1 y x = (1, 1) y = (4, 3) _ 1 ____ 2 2 2 d2 (x, y) = (1 - 4) + (1 - 3) = √13 x
  • 15.
    Manhatten Distance n d1 (x, y) = Σ | xi − yi | , x, y ∈ Rn i=1 y x = (1, 1) y = (4, 3) d1 (x, y) = | 1 - 4 | + | 1 - 3 | = 3 + 2 = 5 x
  • 16.
    Maximum Distance d∞ (x, y) = max | xi − yi | , x, y ∈ Rn 1≤i≤n y x = (1, 1) y = (4, 3) d∞ (x, y) = max (3, 2) = 3 x
  • 17.
    Similarity Measures A similarityfunction on a set G is a function S: G x G → R that satisfies the following conditions: (S1) S (x, y) ≥ 0 for all x, y ∈ G (positive defined) (S2) S (x, y) ≤ S (x, x) for all x, y ∈ G (auto-similarity) (S3) S (x, y) = S (x, x) ⇔ x=y for all x, y ∈ G (identity) The value of the similarity function is greater when two points are closer.
  • 18.
    Similarity Measures • There are many different definitions of similarity. • Often used (S4) S (x, y) = S (y, x) for all x, y ∈ G (symmetry)
  • 19.
  • 20.
    Dendrogram Cluster Dendrogram Euclidean distance (complete linkage) Euclidean distance (complete linkage) Gross national product of EU countries – agriculture (1993) www.isa.uni-stuttgart.de/lehre/SAHBD
  • 21.
    Hierarchical Clustering Hierarchical clusteringcreates a hierarchy of clusters of the set G. Hierarchical Clustering agglomerative divisive Agglomerative clustering: Clusters are successively merged together Divisive clustering: Clusters are recursively split
  • 22.
    Agglomerative Clustering Merge clusterswith smallest distance between the two clusters Step 3 e1, e2 , e3, e4 1 cluster Step 2 e1, e2, e3 e4 2 clusters Step 1 e1, e2 e3 e4 3 clusters Step 0 e1 e2 e3 e4 4 clusters
  • 23.
    Divisive Clustering Chose acluster, that optimally splits in two particular clusters according to a given criterion. Step 0 e1, e2 , e3, e4 1 cluster Step 1 e1, e2 e3, e4 2 clusters Step 2 e1, e2 e3 e4 3 clusters Step 3 e1 e2 e3 e4 4 clusters
  • 24.
  • 25.
    INPUT Given n objects G = { e1,...,en } represented by p-dimensional feature vectors x1,...,xn ∈ Rp Feature p Feature 1 Feature 2 Feature 3 Object x1 = ( x11 x12 x13 ... x1p ) x2 = ( x21 x22 x23 ... x2p ) ⁞ ⁞ ⁞ ⁞ ⁞ xn = ( xn1 xn2 xn3 ... xnp )
  • 26.
    Example I An onlineshop collects data from its customers. For each of the n customers it exists a p-dimensional feature vector Object
  • 27.
    Example II In aclinical trial laboratory values of a large number of patients are gathered. For each of the n patients it exists a p-dimensional feature vector
  • 28.
    Agglomerative Algorithms • Begin with disjoint clustering C1 = { {e1}, {e2}, ... , {en} } • Terminate when all objects are in one cluster Cn = { {e1, e2, ... , en} } e1 e2 e3 e4 • Iterate find the most similar pair of clusters and merge them into a single cluster. Sequence of clusterings (Ci )i=1,...n of G with Ci ̶ 1 ⊂ Ci for i = 2,...,n.
  • 29.
    What is thedistance between two clusters? A d (A,B) B ⇒ Various hierarchical clustering algorithms
  • 30.
    Agglomerative Hierarchical Clustering Thereexist many metrics to measure the distance between clusters. They lead to particular agglomerative clustering methods: • Single-Linkage Clustering • Complete-Linkage Clustering • Average Linkage Clustering • Centroid Method • ...
  • 31.
    Single-Linkage Clustering Nearest-Neighbor-Method The distancebetween the clusters A und B is the minimum distance between the elements of each cluster: d(A,B) = min { d (a, b) | a ∈ A, b ∈ B } a d(A,B) b
  • 32.
    Single-Linkage Clustering • Advantage: Can detect very long and even curved clusters. Can be used to detect outliers. • Drawback: Chaining phenomen Clusters that are very distant to each other may be forced together due to single elements being close to each other. B C A
  • 33.
    Complete-Linkage Clustering Furthest-Neighbor-Method The distancebetween the clusters A and B is the maximum distance between the elements of each cluster: d(A,B) = max { d(a,b) | a ∈ A, b ∈ B } a d (A, B) b
  • 34.
    Complete-Linkage Clustering • …tends to find compact clusters of approximately equal diameters. • … avoids the chaining phenomen. • … cannot be used for outlier detection.
  • 35.
    Average-Linkage Clustering The distancebetween the clusters A and B is the mean distance between the elements of each cluster: 1 d (A, B) = ⋅ Σ d (a, b) |A| ⋅ |B| a ∈ A, b∈B a b A B d(A,B)
  • 36.
    Centroid Method The distancebetween the clusters A and B is the (squared) Euclidean distance of the cluster centroids. d (A, B) x x
  • 37.
    Agglomerative Hierarchical Clustering d (A, B) d (A, B) a d (A, B) d (A, B) b d (A, B) x (A, B) x
  • 38.
    Bioinformatics Alizadeh et al., Nature 403 (2000): pp.503–511
  • 39.
    Exercise Berlin Kiev Paris Odessa
  • 40.
    Exercise The following tableshows the distances between 4 cities: Kiev Odessa Berlin Paris Kiev ̶ 440 1200 2000 Odessa 440 ̶ 1400 2100 Berlin 1200 1400 ̶ 900 Paris 2000 2100 900 ̶ Determine a hierarchical clustering with the single linkage method.
  • 41.
    Solution - SingleLinkage Step 0: Clustering {Kiev}, {Odessa}, {Berlin}, {Paris} Distances between clusters Kiev Odessa Berlin Paris Kiev ̶ 440 1200 2000 Odessa 440 ̶ 1400 2100 Berlin 1200 1400 ̶ 900 Paris 2000 2100 900 ̶
  • 42.
    Solution - SingleLinkage Step 0: Clustering {Kiev}, {Odessa}, {Berlin}, {Paris} Distances between clusters minimal distance Kiev Odessa Berlin Paris Kiev ̶ 440 1200 2000 Odessa 440 ̶ 1400 2100 Berlin 1200 1400 ̶ 900 Paris 2000 2100 900 ̶ ⇒ Merge clusters { Kiev } and { Odessa } Distance value: 440
  • 43.
    Solution - SingleLinkage Step 1: Clustering {Kiev, Odessa}, {Berlin}, {Paris} Distances between clusters Kiev, Odessa Berlin Paris Kiev, Odessa ̶ 1200 2000 Berlin 1200 ̶ 900 Paris 2000 900 ̶
  • 44.
    Solution - SingleLinkage Step 1: Clustering {Kiev, Odessa}, {Berlin}, {Paris} Distances between clusters minimal distance Kiev, Odessa Berlin Paris Kiev, Odessa ̶ 1200 2000 Berlin 1200 ̶ 900 Paris 2000 900 ̶ ⇒ Merge clusters { Berlin } and { Paris } Distance value: 900
  • 45.
    Solution - SingleLinkage Step 2: Clustering {Kiev, Odessa}, {Berlin, Paris} Distances between clusters minimal distance Kiev, Odessa Berlin, Paris Kiev, Odessa ̶ 1200 Berlin, Paris 1200 ̶ ⇒ Merge clusters { Kiev, Odessa } and { Berlin, Paris } Distance value: 1200
  • 46.
    Solution - SingleLinkage Step 3: Clustering {Kiev, Odessa, Berlin, Paris}
  • 47.
    Solution - SingleLinkage Hierarchy Distance values 2540 1 cluster 1200 1340 2 clusters 900 440 3 clusters 440 0 4 clusters Kiev Odessa Berlin Paris
  • 48.
  • 49.
    Divisive Algorithms • Begin with one cluster C1 = { {e1, e2, ... , en} } e1 e2 e3 e4 • Terminate when all objects are in disjoint clusters Cn = { {e1}, {e2}, ... , {en} } • Iterate Chose a cluster Cf , that optimally splits two particular clusters Ci and Cj according to a given criterion. Sequence of clusterings (Ci )i=1,...n of G with C i ⊃ C i + 1 for i = 1,...,n-1.
  • 50.
  • 51.
    Partitional Clustering K=2 • Aims to partition n observations into K clusters. • The number of clusters and initial partition an initial partition are given. • The initial partition is considered as “not optimal“ and should be K=2 iteratively repartitioned. The number of clusters is given !!! final partition
  • 52.
    Partitional Clustering Difference tohierarchical clustering • number of clusters is fixed. • an object can change the cluster. Initial partition is obtained by • random or • the application of an hierarchical clustering algorithm in advance. Estimation of the number of clusters • specialized methods (e.g., Silhouette) or • the application of an hierarchical clustering algorithm in advance.
  • 53.
    Partitional Clustering -Methods In this course we will introduce the minimal distance methods . . . • K-Means and • Fuzzy-C-Means
  • 54.
  • 55.
    K-Means Aims to partitionn observations into K clusters in which each observation belongs to the cluster with the nearest mean. G C3 Find K cluster centroids µ1 ,..., µK that minimize the objective function K C1 Σ Σ 2 J = dist ( µi, x ) i =1 x ∈ Ci C2
  • 56.
    K-Means Aims to partitionn observations into K clusters in which each observation belongs to the cluster with the nearest mean. G C3 Find K cluster centroids µ1 ,..., µK that minimize the objective function x x K C1 Σ Σ 2 J = dist ( µi, x ) x i =1 x ∈ Ci C2
  • 57.
    K-Means - MinimalDistance Method Given: n objects, K clusters 1. Determine initial partition. 2. Calculate cluster centroids. x x 3. For each object, calculate the distances to all cluster centroids. repartition 4. If the distance to the centroid of another cluster is smaller than the distance to the actual cluster centroid, then assign the object to the other cluster. 5. If clusters are repartitioned: GOTO 2. ELSE: STOP.
  • 58.
    Example ₓ ₓ ₓ ₓ Initial Partition Final Partition
  • 59.
    Exercise ₓ ₓ ₓ ₓ Initial Partition Final Partition
  • 60.
    K-Means • K-Means doesnot determine the global optimal partition. • The final partition obtained by K-Means depends on the initial partition.
  • 61.
    Hard Clustering /Soft Clustering Clustering Hard Clustering Soft Clustering Each object is a member Each object has a fractional of exactly one cluster membership in all clusters K-Means Fuzzy-c-Means
  • 62.
  • 63.
    Fuzzy Clustering vs.Hard Clustering • When clusters are well separated, hard clustering (K-Means) makes sense. • In many cases, clusters are not well separated. In hard clustering, borderline objects are assigned to a cluster in an arbitrary manner.
  • 64.
    Fuzzy Set Theory •Fuzzy Theory was introduced by Lofti Zadeh in 1965. • An object can belong to a set with a degree of membership between 0 and 1. • Classical set theory is a special case of fuzzy theory that restricts membership values to be either 0 or 1.
  • 65.
    Fuzzy Clustering • Isbased on fuzzy logic and fuzzy set theory. • Objects can belong to more than one cluster. • Each object belongs to all clusters with some weight (degree of membership) 1 Cluster 1 Cluster 2 Cluster 3 0
  • 66.
    Hard Clustering • K-Means − The number K of clusters is given. − Each object is assigned to exactly one cluster. Partition Object C3 Cluster e1 e2 e3 e4 e3 e4 C1 0 1 0 0 C2 1 0 0 0 e2 e1 C1 C2 C3 0 0 1 1
  • 67.
    Fuzzy Clustering • Fuzzy-c-means − The number c of clusters is given. − Each object has a fractional membership in all clusters Object Cluster e1 e2 e3 e4 C1 0.8 0.2 0.1 0.0 Fuzzy-Clustering C2 0.2 0.2 0.2 0.0 There is no strict sub-division of clusters. C3 0.0 0.6 0.7 1.0 Σ 1 1 1 1
  • 68.
    Fuzzy-c-Means • Membership Matrix U = ( u i k ) ∈ [0, 1]c x n The entry u i k denotes the degree of membership of object k in cluster i . Object 1 Object 2 … Object n Cluster 1 u11 u12 … u1n Cluster 2 u21 u22 … u2n … … … … Cluster c uc1 uc2 … ucn
  • 69.
    Restrictions (Membership Matrix) 1.All weights for a given object, ek, must add up to 1. c Σ u ik = 1 (k = 1,...,n) i =1 2. Each cluster contains – with non-zero weight – at least one object, but does not contain – with a weight of one – all the objects. n 0< Σ u ik < n (i = 1,...,c) k =1
  • 70.
    Fuzzy-c-Means • Vector of prototypes (cluster centroids) T V = ( v1,...,vc ) ∈ Rc Remark The cluster centroids and the membership matrix are initialized randomly. Afterwards they are iteratively optimized.
  • 71.
    Fuzzy-c-Means ALGORITHM 1. Select aninitial fuzzy partition U = (u i k ) ⇒ assign values to all u i k 2. Repeat 3. Compute the centroid of each cluster using the fuzzy partition 4. Update the fuzzy partition U = (u i k ) 5. Until the centroids do not change. Other stopping criterion: “change in the u i k is below a given threshold”.
  • 72.
    v3 u3k Fuzzy-c-Means xk • K-Means and Fuzzy-c-Means attempt to minimize v1 u1k u2k the sum of the squared errors (SSE). v2 • In K-Means: K Σ Σ 2 SSE = dist ( vi, x ) i =1 x ∈ Ci • In Fuzzy-c-Means: c n m SSE = Σ Σ u ik . 2 dist ( vi, xk ) i =1 k =1 m ∈ [1, ∞] is a parameter (fuzzifier) that determines the influence of the weights.
  • 73.
    v3 u3k Computing Cluster Centroids xk • For each cluster i = 1,...,c the centroid is defined by v1 u1k u2k n v2 Σ u m xk k = 1 ik _________________ ( i = 1,...,c ) vi = n m Σ u ik (V) k=1 • This is an extension of the definition of centroids of k-means. • All points are considered and the contribution of each point to the centroid is weighted by its membership degrees.
  • 74.
    Update of theFuzzy Partition (Membership Matrix) • Minimization of SSE subject to the constraints leads to the following update formula: 1 ______________________________________ u ik = 1 _____ c 2 (U) dist ( v i , xk ) m – 1 Σ __________ 2 s=1 dist ( vs , xk )
  • 75.
    Fuzzy-c-Means Initialization Determine(randomly) • Matrix U of membership grades • Matrix V of cluster centroids. Iteration Calculate updates of • Matrix U of membership grades with (U) • Matrix V of cluster centroids with (V) until cluster centroids are stable or the maximum number of iterations is reached.
  • 76.
    Fuzzy-c-means • Fuzzy-c-means dependson the Euclidian metric ⇒ spherical clusters. • Other metrics can be applied to obtain different cluster shapes. • Fuzzy covariance matrix (Gustafson/Kessel 1979) ⇒ ellipsoidal clusters.
  • 77.
  • 78.
    Cluster Validity Indexes Fuzzy-c-meansrequires the number of clusters as input. Question: How can we determine the “optimal” number of clusters? Idea: Determine the cluster partition for a given number of clusters. Then, evaluate the cluster partition by a cluster validity index. Method: For all possible number of clusters calculate the cluster validity index. Then, determine the optimal number of clusters. Note: CVIs usually do not depend on the clustering algorithm.
  • 79.
    Cluster Validity Indexes • Partition Coefficient (Bezdek 1981) c n 1 2 PC (c) = __ Σ Σ uik , 2 ≤ c ≤ n-1 n i=1 k=1 • Optimal number of clusters c∗ : PC (c∗) = max PC (c) 2 ≤ c ≤ n-1
  • 80.
    Cluster Validity Indexes • Partition Entropy (Bezdek 1974) c n 1 PC (c) = _ __ Σ Σ u i k log2 u i k , 2 ≤ c ≤ n-1 n i=1 k=1 • Optimal number of clusters c∗ : PC (c∗) = min PC (c) 2 ≤ c ≤ n-1 • Drawback of PC and PE: Only degrees of memberships are considered. The geometry of the data set is neglected.
  • 81.
    Cluster Validity Indexes • Fukuyama-Sugeno Index (Fukuyama/Sugeno 1989) c n m Σ Σ 2 Compactness FS (c) = uik dist ( vi , xk ) of clusters i=1 k=1 c n _ m _ Σ Σ Separation 2 uik dist ( vi , v ) of clusters i=1 k=1 • Optimal number of clusters c∗ : _ 1 c PC (c∗) = max PC (c) v = __ Σ vi 2 ≤ c ≤ n-1 c i =1
  • 82.
  • 83.
    Data Mining andDecision Support Systems ̶ Landslide Events (UniBw, Geoinformatics Group: W. Reinhardt, E. Nuhn) → Spatial Data Mining / Early Warning Systems for Landslide Events → Fuzzy clustering approaches (feature weighting) • Measurements (pressure values, tension, deformation vectors) • Simulations (finite-element model)
  • 84.
    Hard Clustering Data Partition Problem: Uncertain data from measurements and simulations
  • 85.
    Fuzzy Clustering Data Fuzzy-Cluster Fuzzy-Partition
  • 86.
  • 87.
    Feature Weighting Nuhn/Kropat/Reinhardt/Pickl: Preparationof complex landslide simulation results with clustering approaches for decision support and early warning. Submitted to Hawaii International Conference on System Sciences (HICCS 45), Grand Wailea, Maui, 2012.
  • 88.