Recommended
More Related Content
What's hot
What's hot (16)
Similar to nips勉強会_Toward Property-Based Classification of Clustering Paradigms
Similar to nips勉強会_Toward Property-Based Classification of Clustering Paradigms (20)
Recently uploaded
Recently uploaded (20)
nips勉強会_Toward Property-Based Classification of Clustering Paradigms
- 2. 1 Twitter ◦
- 3. Properties of Clustering Functions A taxonomy of k-clustering fucntions
- 4. Properties of Clustering Functions A taxonomy of k-clustering fucntions
- 5. Clustering: ↓ Clustering Ad-hoc ○Clustering ○ Clustering property
- 6. Clustering clustering
- 7. A Impossibility Theorem for Clustering ◦ Jon Kleinberg, NIPS 2002 Measures of Clustering Quality: A Working Set of Axioms for Clustering ◦ M.Ackerman and S.Ben-David, NIPS 2008 Characterization of Linkage-based Clustering. ◦ M.Ackerman and S.Ben-David, COLT 2010
- 8. X: + d : X × X →R ,d ( x, x ) = 0(∀x ∈ X ) X,d ) ( clustering ( X,d,k ) F clustering € C = F ( X,d,k ) € ⎛ € ⎞ {C1,C2 ,Ck }⎜ Ci = X,1 ≤ k ≤ X ⎟ ⎝ i ⎠
- 9. general clustering function F Input: F ( X,d ) ⎛ ⎞ Output: C = {C1,C2 ,Ck }⎜ Ci = X,1 ≤ k ≤ X ⎟ ⎝ i ⎠ € k-clustering function F € Input: F ( X,d,k ), (1 ≤ k ≤ X ) ⎛ ⎞ Output: = {C1,C2 ,Ck }⎜ Ci = X,1 ≤ k ≤ X ⎟ C ⎝ i ⎠ € €
- 10. Properties of Clustering Functions A taxonomy of k-clustering fucntions
- 11. iso. invariance scale invariance order invariance outer consistency cluster inner consistency cluster k-rich k inner rich richness outer rich threshold rich locality clustering refinement-confined k clustering
- 12. clustering φ : X → X ʹ′ x, y ∈ X,d ( x, y ) = dʹ′(φ (x), φ (y)) F ( X,d,k ),F ( X ʹ′, dʹ′,k ) : isomorphic(∀k) x, y : same → φ (x), φ (y) : same
- 13. clustering
- 14. clustering x, y ∈ X, d ( x, y ) = cdʹ′( x, y ) →F ( X,d,k ) = F ( X ʹ′, dʹ′,k )
- 15. 2 clustering x1, x 2 , x 3 , x 4 ∈ X, d ( x1, x 2 ) < d ( x 3 , x 4 ), dʹ′( x1, x 2 ) < dʹ′( x 3 , x 4 ) →F ( X,d,k ) = F ( X ʹ′, dʹ′,k ) €
- 16. 2 clustering (Single-linkage clustering 0 1 4 9 10 12 15 19 20
- 17. clustering cluster cluster C’
- 18. clustering cluster cluster C = F(X,d,k), Cʹ′ ⊆ C F(Cʹ′,d,| Cʹ′ |)= Cʹ′
- 19. cluster cluster clustering cluster cluster d(x,y) d’(x,y) d(x,y) d’(x,y)
- 20. cluster cluster clustering cluster cluster dʹ′ : (C,d ) − consistent x, y : same → dʹ′( x, y ) ≤ d ( x, y ) x, y : different → dʹ′( x, y ) ≥ d ( x, y )
- 21. cluster clustering cluster d(x,y) d’(x,y)
- 22. cluster clustering cluster dʹ′ : (C,d) − outerconsistent x, y : same → dʹ′( x, y ) = d ( x, y ) x, y : different → dʹ′( x, y ) ≥ d ( x, y )
- 23. cluster clustering cluster d(x,y) d’(x,y)
- 24. cluster clustering cluster dʹ′ : (C,d) − innterconsistent x, y : same → dʹ′( x, y ) ≤ d ( x, y ) x, y : different → dʹ′( x, y ) = d ( x, y )
- 25. clustering any : X1, X 2 X k X ʹ′ = { X1, X 2 X k } →∃d : F ( X ʹ′,d,k ) = { X1, X 2 X k }
- 26. clustering ※ clustering any : (X1,d1 ),(X 2 ,d2 )(X k ,dk ) ⎛ k ⎞ →∃d : F ⎜ X i , d,k ⎟ = { X1, X 2 X k } ˆ ˆ ⎝ i=1 ⎠ ˆ d : entends − d (i ≤ k) i
- 27. clustering ※ clustering ( X,d), X = { X1, X 2 X k } ˆ ˆ ( →∃d : d ( a,b) = d ( a,b) a ∈ X i ,b ∈ X j ,i ≠ j ) ⎛ k ⎞ F ⎜ X i , d,k ⎟ = { X1, X 2 X k } ˆ ⎝ i=1 ⎠
- 28. clustering ∃a < b x, y : same →d(x, y) ≤ a, x, y : different →d(x, y) ≥ b F ( X,d, C ) = C
- 29. k≦k’ F(X,d,k’) F(X,d,k) refinement 1 ≤ k ≤ kʹ′ ≤ X , O( F ( X,d,k')) ≥ O( F ( X,d,k )) €
- 30. k≦k’ F(X,d,k’) F(X,d,k) refinement
- 31. iso. invariance scale invariance order invariance outer consistency cluster inner consistency cluster k-rich k inner rich richness outer rich threshold rich locality clustering refinement-confined k clustering
- 32. Clustering Kleinberg clustering Scale-Invariance Richness Consistency
- 33. single linkage clustering stop condition Consistency + Richness: only link if distance is less than r ◦ clustering cluster Consistency + SI: stop when you have k connected components ◦ clustering / clustering Richness + SI: if x is the diameter of the graph, only add edges with weight βx ◦ cluster / clustering
- 34. Properties of Clustering Functions A taxonomy of k-clustering fucntions
- 35. outer consistent inner consistent scale invariant order invariant threshold rich iso. invariant refinement inner rich out rich k-rich local Singe Linkage ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ Average Linkage ○ × ○ ○ × ○ ○ ○ ○ ○ ○ Complete Linkage ○ × ○ ○ ○ ○ ○ ○ ○ ○ ○ k-median ○ × ○ × × ○ ○ ○ ○ ○ ○ k-means ○ × ○ × × ○ ○ ○ ○ ○ ○ Min sum ○ ○ ○ × × ○ ○ ○ ○ ○ ○ Ratio cut × ○ × × × ○ ○ ○ ○ ○ ○ Nomalize cut × × × × × ○ ○ ○ ○ ○ ○
- 36. outer consistent inner consistent scale invariant order invariant threshold rich iso. invariant refinement inner rich out rich k-rich local Singe Linkage ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ Average Linkage ○ × ○ ○ × ○ ○ ○ ○ ○ ○ Complete Linkage ○ × ○ ○ ○ ○ ○ ○ ○ ○ ○ k-median ○ × ○ × × ○ ○ ○ ○ ○ ○ k-means ○ × ○ × × ○ ○ ○ ○ ○ ○ Min sum ○ ○ ○ × × ○ ○ ○ ○ ○ ○ Ratio cut × ○ × × × ○ ○ ○ ○ ○ ○ Nomalize cut × × × × × ○ ○ ○ ○ ○ ○
- 37. outer consistent inner consistent scale invariant order invariant threshold rich iso. invariant refinement inner rich out rich k-rich local Singe Linkage ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ Average Linkage ○ × ○ ○ × ○ ○ ○ ○ ○ ○ Complete Linkage ○ × ○ ○ ○ ○ ○ ○ ○ ○ ○ k-median ○ × ○ × × ○ ○ ○ ○ ○ ○ k-means ○ × ○ × × ○ ○ ○ ○ ○ ○ Min sum ○ ○ ○ × × ○ ○ ○ ○ ○ ○ Ratio cut × ○ × × × ○ ○ ○ ○ ○ ○ Nomalize cut × × × × × ○ ○ ○ ○ ○ ○
- 38. outer consistent inner consistent scale invariant order invariant threshold rich iso. invariant refinement inner rich out rich k-rich local Singe Linkage ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ Average Linkage ○ × ○ ○ × ○ ○ ○ ○ ○ ○ Complete Linkage ○ × ○ ○ ○ ○ ○ ○ ○ ○ ○ k-median ○ × ○ × × ○ ○ ○ ○ ○ ○ k-means ○ × ○ × × ○ ○ ○ ○ ○ ○ Min sum ○ ○ ○ × × ○ ○ ○ ○ ○ ○ Ratio cut × ○ × × × ○ ○ ○ ○ ○ ○ Nomalize cut × × × × × ○ ○ ○ ○ ○ ○
- 39. outer consistent inner consistent scale invariant order invariant threshold rich iso. invariant refinement inner rich out rich k-rich local Singe Linkage ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ Average Linkage ○ × ○ ○ × ○ ○ ○ ○ ○ ○ Complete Linkage clustering○ ○ × ○ ○ ○ ○ ○ ○ ○ ○ scale invariance k-median ○ × ○ : × × natural ○ ○ ○ ○ ○ ○ isomorphism variance :natural k-means ○ × ○ × × ○ ○ ○ ○ ○ ○ threshold richness Min sum ○ ○ ○ × / × ○ ○ ○ ○ ○ ○ clustering Ratio cut × ○ × × × ○ ○ ○ ○ ○ ○ Nomalize cut × × × × × ○ ○ ○ ○ ○ ○
- 40. Properties of Clustering Functions A taxonomy of k-clustering fucntions
- 41. Invariance properties Consistency properties (C,d) − nice var iant [ ] [ P F ( X, dʹ′, C ) = C ≥ P F ( X,d, C ) = C ] Richness properties ∀ε > 0 € ∃d : P ( F ( X,d,k ) = C ) ≥1 − ε Locality [ ] P F ( X ʹ′,d / X ʹ′, Cʹ′ ) = Cʹ′ € = [ P Cʹ′ ⊆ C F ( X,d, j ) = CandC / X ʹ′isak − clustering ] [ ] P ∃C1,C2 Ck s.t.Ci = X ʹ′ F ( X,d, j ) = C ≠ 0
- 42. k-means
- 43. k-means
- 44. Random Centroids Lloyd Furthest Centroids Lloyd ( maximizemin1≤ j ≤i−1 d c j ,c i ) €
- 45. Clustering Algorithm outer consistent scale invariant threshold rich iso. invariant outer rich k-rich local Optimal k-means ○ ○ ○ ○ ○ ○ ○ Random Centroid Lloyd × × × ○ ○ ○ ○ Furthest Centroid Lloyd × × ○ ○ ○ ○ ○
- 46. Clustering Algorithm outer consistent scale invariant threshold rich iso. invariant outer rich k-rich local Optimal k-means ○ ○ ○ ○ ○ ○ ○ Random Centroid Lloyd × × × ○ ○ ○ ○ Furthest Centroid Lloyd × × ○ ○ ○ ○ ○ threshold richness Furthest Centroid Lloyd Random Centroid Lloyd
- 47. Kleinberg clustering Scale-Invariance Richness Consistency
- 48. clustering Scale-Invariance Richness Outer-Consistency
- 49. Properties of Clustering Functions A taxonomy of k-clustering fucntions
- 50. Clustering Function property clustering axioms scale-invariance, isomorphism-invariance, threshold richness Kleinberg
- 51. Supervised Clustering ◦ 2008 clustering ◦ clustering merge/ split Efficient Robust Feature Selection via Joint L2,1-Norms Minimization ◦ Bio Informatics Feature Selection ◦ L1,2-norm SVM Feature