Datamining 8th hclustering
Upcoming SlideShare
Loading in...5
×
 

Datamining 8th hclustering

on

  • 562 views

 

Statistics

Views

Total Views
562
Views on SlideShare
477
Embed Views
85

Actions

Likes
0
Downloads
0
Comments
0

2 Embeds 85

http://togodb.sel.is.ocha.ac.jp 77
http://togodb.seselab.org 8

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Datamining 8th hclustering Datamining 8th hclustering Presentation Transcript

  • 4.1 4• • 1 10 20 30 0.74 0.76 1.34 40 1.75 10 2 2.01 2.62 30 0.87 20 40 0.69 3 0.74 0.87 0.60 1.34 0.76 1.83 1.90 1.75 4 1.73 1.83 0.96 0.93 2.01 2.62 0.87 0.69 4.1 10 20 30 40 0.87 4 0.60 1.83 0 1.90 1.73 1.83 0.96 0.93 2
  • ( )• 2 • • • •• • ( • • • 3
  • •••• (Top-down Clustering, Divisive Clustering) (Bottom-up Clustering, Agglomerative Clustering) C B A F G E D A B C D E F G (A) (B) 4
  • ( )••• 1 C B A 3 2 FA B C D E F G G E D 1 2 3 (C) (D) 5
  • •• • • • • 2 X X (A) 6 (B)
  • • d(x, y) : x, y Ci , Cj Dmin (Ci , Cj ) = min {d(x, y) | x ∈ Ci , y ∈ Cj }1.2. 13. 2 1 7
  • 1 C C C B B B A 3 A 5 A E 4 E E 2 F F F G G G D D D(A) B,C D,F (B) A,E (C) 6 5 4 3 2 1A B C E D F G(D) 8
  • (1/2) •4.5. • 59 (1) N x1 , . . . , xN 1 x1 , . . . , xN C1 , . . . , CN (2) n = N n (3) n=1 (a) C1 , . . . , Cn Ci , Cj i<j1 2 3 4 5 6 7 (b) Ci Cj Ci (c) (d) Cj = Cn n=n−1 4.8 9
  • (1) N x1 , . . . , xN 1(2) n = N x1 , . . . , xN n C1 , . . . , CN (2/2)(3) n=1 (a) C1 , . . . , Cn Ci , Cj i<j (b) Ci Cj Ci (c) (d) Cj = Cn n=n−1 1 4.8 2 3 4 5 6 7 (a) 1 2 3 4 5 6 7 (b) 1 C C C B 1 2 3 B 4 5 6 7 B (d) A 3 A 5 A E E 10 E
  • (A) (B) A B,C D,F E G A 1.2 2.3 1.9 4.1 B,C 1.2 B,C 1.2 3.2 2.0 4.0 A 1.2 D,F 2.3 3.2 2.2 3.5 E 2.2 C E 1.9 2.0 2.2 2.5 A 1.9 B G 4.1 4.0 3.5 2.5 E 2.5 A E A B,C F (C) (D)G D A,B,C D,F E G A,B,C 2.3 1.9 4.0 E 1.9 D,F 2.3 2.2 3.5 E 2.2 E 1.9 2.2 2.5 A,B,C 1.9 G 4.0 3.5 2.5 E 2.5 11 ※
  • • • Ci Cj Ci’ • Ci’ Ck • Ci Ck Cj Ck • • N O(N) • O(N^2)• • Cj Ci’• 1 O(N) O(N^2)• N-1 O(N) O(N^2), O(N^2) 12
  • 35, 48)•(Kruskal’s Algorithm) (Minimum Spanning Tree)• 4.2 G = (V, E)(V E ) T ⊆G G T T G V T T (Spanning Tree) G (u, v) ∈ E w(u, v) G (u,v)∈T w(u, v) T G (Minimum Spanning Tree) 4.13(A) 4.9(P. 59) 4.13(B) 4.13(B) G 1 BC 6 GE 13 4.9(D) P. 59
  • G w(u,v) u,v 14
  • Kruscal( 72 ) 4 (1) G = (V, E) V E (2) A A (3) V ( 1 ) (4) ( (u, v) ∈ E ) (a) A A ∪ {(u, v)} u v (b) u v (5) A C C C B B B 4.14 A A A A E E E C e V-C F F F G G G D D D A={} A={(B,C),(D,F)} A={(B,C),(D,F),(A,B)} {A}, {B}, {C}, {D}, {E}, {F} {A}, {B,C}, {D,F}, {E} e’ {A, B,C}, {D,F}, {E} 15
  • C C B 1 B A 3 A E E 4 F 6 5 F 2 G G D D(A) (B) (A) 6 5 4 3 2 1 A B C E D F G (C) 16
  • C C C B B B A A A E E E F F F G G G D D D(A) (B) A (C) ( AB) E A B,C T B C C B B A A E E Q F F TG G D D(D) C, F T (E) O(E + V log V ) 17
  • X• d(x, y) : x, y Ci , Cj Dmax (Ci , Cj ) = max{d(x, y)|x ∈ Ci , y ∈ Cj }1.2. 13. 2 1 18
  • 1 3 C C C B B B A A A E 5 2 4 E F E F F G G G D D D(A) B,C D,F (B) A,E (C) G 5 (A) (C) (1 5) (D) 1 5 4 3 2 1A B C D F E G (D) 19
  • (A) (B) A B,C D,F E GA 1.3 3.0 1.9 4.1 B,C 1.3B,C 1.3 4.1 2.5 4.5 A 1.3D,F 3.0 4.1 2.3 4.0 E 2.3E 1.9 2.5 2.3 2.5 A 1.9G 4.1 4.5 4.0 2.5 E 2.5 A B,C (C) (D) A,B,C D,F E GA,B,C 4.1 2.5 4.5 E 2.5D,F 4.1 2.3 4.0 E 2.3E 2.5 2.3 2.5 D,F 2.3G 4.5 4.0 2.5 E 2.5 20
  • C C B B A A E E F F G G D D(A) E A (B) A B,C E A C B (C) A B,C E A D,F CE DE E E F G D O (N^2) O(N^3) 21
  • {A},1.9 O(N^2 log N) {B,C}, 2.5 {D,F}, 2.3 ( O(log N) ){G}, 2.5 (A) E {A},1.9 {B,C}, 2.5 (B) 2 {D,F}, 2.3 {G}, 2.5 (C) {A, B,C}, 2.5 (D) {D,F}, 2.3 {G}, 2.5 {A, B,C}, 2.5 (E) 22
  • duces the clusters shown in Figure 12, whereas the complete-link algorithm ob- tains the clustering shown in Figure 13. Data Clustering • 277S The clusters obtained by the complete-iXm2 link algorithm are more compact than X2i those obtained by the single-link algo-la rithm; the cluster labeled 1 obtainedr 2 2 using the 1single-link algorithm 2is elon- 1 11 2i 1 111 2 2 22 2 2 2 22 2t 1 1 11 2 2 gated because 1of the noisy patterns la- 1 1 1 2 2 2 2 2 2y 11 1 1 1 *** * * * * ** 2 2 2 beled “*”. 1 1 1 1 single-link * algorithm 2is 11 The * * * * * * * * 2 2 1 1 2 2 1 2 2 1 1 1 1 1 2 more versatile 1than the complete-link 1 1 1 1 1 2 2 1 2 algorithm, otherwise. For example, the single-link algorithm can extract the concentric clusters shown in Figure 11, A B C D E F***G but the complete-link algorithm cannot.Figure 10. The dendrogram obtained using X1 However, from a pragmatic viewpoint, it 1 Xthe single-link algorithm. Figure 12. A single-link clustering of a pattern has been observed that clustering of a pat- Figure 13. A complete-link the complete- set containing two classes (1 and 2) connected by tern set containing two classes (1 and 2) con- link algorithm produces more useful hi-Y a chain of noisy patterns (*). erarchies inchain of noisy patterns (*). nected by a many applications than the 1 single-link algorithm [Jain and Dubes 1 1988]. 1(3) The output of the algorithm is a well-separated, chain-like, and concen- 2 nested hierarchy of graphs which 2 1 tric clusters, whereas a typicalClus- Agglomerative Single-Link parti- 2 can be cut at a desired dissimilarity 1 2 2 tering Algorithm such as the k -means tional algorithm level forming a partition (clustering) 1 algorithm works well only on data sets 2 (1) Place isotropic clustersits own clus- having each pattern in [Nagy 1968]. identified by simply connected com- 1 ponents in the 1corresponding graph. 1 On theConstruct a list of interpattern ter. other hand, the time and space complexities for all 1992] ofunordered distances [Day distinct the parti-Agglomerative Complete-Link Clus- 23 tional algorithms are typically lower X pairs of patterns, and sort this list
  • • • Average Group Linkage • 1 ￿ ￿ D(Ci , Cj ) = D(x1 , x2 ) |Ci ||Cj | x1 ∈Ci x2 ∈Cj • Ward’s Method • D(Ci , Cj ) = E(Ci ∪ Cj￿ E(Ci ) − E(Cj ) )− where E(Ci ) = (d(x, ci ))2 , x∈Ci 1 ￿ ci = x |Ci | x∈CiAverage Group Linkage 24 Ward’s Method
  • (1)• • •• n • n*(n-1) • • 2^n •
  • (2)• • • ••• DIANA (DIvisive ANAlysis ) • DIANA •
  • DIANA (1) • V(i,S)VS(⊂V):d(i, j) : i jS i∈V-S V(i,S)V (i, S) ￿ ￿ 1 |V |−1 j∈V −{i} d(i, j) if S = φ= ￿ ￿ 1 |V −S|−1 j￿∈S∪{i} d(i, j) − 1 |S| j∈S d(i, j) if S ￿= φ V(i,S) i S - (V-S)
  • DIANA (2): C A B B 1.2 B A C 1.3 1.0 C E D 3.0 4.0 4.1 D F E 1.9 2.0 2.5 2.3 E G F 2.3 3.2 3.4 1.1 2.2 F D G 4.1 4.0 4.5 3.5 2.5 4.0 (A) (B)6 4(1) : S {} S(2) V (i, S) i∈V −S(3) V (i, S) > 0 i S (2)(4) V (i, S) ≤ 0 S i (5)(5) V V −S 4.24
  • DIANA (3): 1 (1) C A C B B 1.2 B B A C 1.3 1.0 C A E D 3.0 4.0 4.1 D E F E 1.9 2.0 2.5 2.3 E F F 2.3 3.2 3.4 1.1 2.2 F G G D G 4.1 4.0 4.5 3.5 2.5 4.0 D (A) (B) (C) G V (i, S) ￿ ￿ 1 |V |−1 j∈V −{i} d(i, j) if S = φ = ￿ ￿ 1 |V −S|−1 j￿∈S∪{i} d(i, j) − 1 |S| j∈S d(i, j) if S ￿= φV (G, {}) = 1/6(d(G, A) + d(G, B) + d(G, C) + d(G, D) + d(G, E) + d(G, F)) = 1/6(4.1 + 4.0 + 4.5 + 3.5 + 2.5 + 4.0) = 3.77
  • DIANA (4): 1 (2) A C B 1.2 B B C 1.3 1.0 C A D 3.0 4.0 4.1 D E E 1.9 2.0 2.5 2.3 E F F 2.3 3.2 3.4 1.1 2.2 F G G 4.1 4.0 4.5 3.5 2.5 4.0 D (B) (D) E V (i, S) ￿ ￿ 1 |V |−1 j∈V −{i} d(i, j) if S = φ = ￿ ￿ 1 |V −S|−1 j￿∈S∪{i} d(i, j) − 1 |S| j∈S d(i, j) if S ￿= φV (E, {}) = 1/6(d(E, A) + d(E, B) + d(E, C) + d(E, D) + d(E, F) + d(E, G)) = 1/6(1.9 + 2.0 + 2.5 + 2.3 + 2.2 + 2.5) = 2.23
  • DIANA (5): 1 (3) • V(i, {})V (A, {}) = 13.8/6 = 2.3, V (B, {}) = 15.4/6 = 2.57,V (C, {}) = 16.8/6 = 2.8, V (D, {}) = 18.0/6 = 3.0,V (E, {}) = 2.23, V (F, {}) = 16.2/6 = 2.7V (G, {}) = 3.77 • V(G, {}) • V(G, {}) > 0 G S • S = {G} • S
  • DIANA (6): 2 (1) C A B B 1.2 B A C 1.3 1.0 C E D 3.0 4.0 4.1 D F E 1.9 2.0 2.5 2.3 E D F 2.3 3.2 3.4 1.1 2.2 F G G 4.1 4.0 4.5 3.5 2.5 4.0V (i, S) ￿ ￿ 1 |V |−1 j∈V −{i} d(i, j) if S = φ= ￿ ￿ 1 |V −S|−1 j￿∈S∪{i} d(i, j) − 1 |S| j∈S d(i, j) if S ￿= φV (A, {G}) = 1/5(d(A, B) + d(A, C)+ d(A, D) + d(A, E) + d(A, F)) − 1/1(d(A, G)) = 1/5(1.2 + 1.3 + 3.0 + 1.9 + 2.3) − 4.1 = −2.16
  • DIANA (7): 2 (2) •V (B, {G}) = 1/5(1.2 + 1.0 + 4.0 + 2.0 + 3.2) − 4.0 = −1.72V (C, {G}) = 1/5(1.3 + 1.0 + 4.1 + 2.5 + 3.4) − 4.5 = −2.04V (D, {G}) = 1/5(3.0 + 4.0 + 4.1 + 2.3 + 1.1) − 3.5 = −0.6V (E, {G}) = 1/5(1.9 + 2.0 + 2.5 + 2.3 + 2.2) − 2.5 = −0.32V (F, {G}) = 1/5(2.3 + 3.2 + 3.4 + 1.1 + 2.2) − 4.0 = −1.56 • V(E {G}) • V(E, {G}) < 0 C B A E F G D
  • DIANA (8): 2 (1)• V {A,B,C,D,E,F,G} {A,B,C,D,E,F} {G}• V = {G}• V = {A,B,C,D,E,F}V (i, S) ￿ ￿ 1 |V |−1 j∈V −{i} d(i, j) if S = φ= ￿ ￿ 1 |V −S|−1 j￿∈S∪{i} d(i, j) − 1 |S| j∈S d(i, j) if S ￿= φ V (A, {}) = 1/5(1.2 + 1.3 + 3.0 + 1.9 + 2.3) = 1.94 V (B, {}) = 1/5(1.2 + 1.0 + 4.0 + 2.0 + 3.2) = 2.28 V (C, {}) = 1/5(1.3 + 1.0 + 4.1 + 2.5 + 3.4) = 2.46 V (D, {}) = 1/5(3.0 + 4.0 + 4.1 + 2.3 + 1.1) = 2.9 V (E, {}) = 1/5(1.9 + 2.0 + 2.5 + 2.3 + 2.2) = 2.18 V (F, {}) = 1/5(2.3 + 3.2 + 3.4 + 1.1 + 2.2) = 2.44 • S={D} S
  • DIANA (9): 2 (2) • V={A,B,C,D,E,F}, S={D} V(i, S)V (i, S) ￿ ￿ 1 |V |−1 j∈V −{i} d(i, j) if S = φ= ￿ ￿ 1 |V −S|−1 j￿∈S∪{i} d(i, j) − 1 |S| j∈S d(i, j) if S ￿= φV (A, {D}) = 1/4(1.2 + 1.3 + 1.9 + 2.3) − 3.0 = −1.325V (B, {D}) = 1/4(1.2 + 1.0 + 2.0 + 3.2) − 4.0 = −2.15V (C, {D}) = 1/4(1.3 + 1.0 + 2.5 + 3.4) − 4.1 = −2.05V (E, {D}) = 1/4(1.9 + 2.0 + 2.5 + 2.2) − 2.3 = −0.15V (F, {D}) = 1/4(2.3 + 3.2 + 3.4 + 2.2) − 1.1 = 1.675 C • S F B • V(F, {D}) > 0 S E A F D G
  • DIANA (10): 2 (3) • V={A,B,C,D,E,F}, S={D,F} V (i, S) ￿ ￿ 1 |V |−1 j∈V −{i} d(i, j) if S = φ = ￿ ￿ 1 |V −S|−1 j￿∈S∪{i} d(i, j) − 1 |S| j∈S d(i, j) if S ￿= φV (A, {D, F}) = 1/3(1.2 + 1.3 + 1.9) − 1/2(3.0 + 2.3) = −1.183V (B, {D, F}) = 1/3(1.2 + 1.0 + 2.0) − 1/2(4.0 + 3.2) = −2.2V (C, {D, F}) = 1/3(1.3 + 1.0 + 2.5) − 1/2(4.1 + 3.4) = −2.15V (E, {D, F}) = 1/3(1.9 + 2.0 + 2.5) − 1/2(2.3 + 2.2) = −0.117 • C B A E F G D G D F E A B C
  • DIANA