Density Based Clustering

2,762 views

Published on

AACIMP 2011 Summer School. Operational Research Stream. Lecture by Erik Kropat.

Published in: Education, Technology, Business
1 Comment
6 Likes
Statistics
Notes
No Downloads
Views
Total views
2,762
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
232
Comments
1
Likes
6
Embeds 0
No embeds

No notes for slide

Density Based Clustering

  1. 1. Summer School“Achievements and Applications of Contemporary Informatics, Mathematics and Physics” (AACIMP 2011) August 8-20, 2011, Kiev, Ukraine Density Based Clustering Erik Kropat University of the Bundeswehr Munich Institute for Theoretical Computer Science, Mathematics and Operations Research Neubiberg, Germany
  2. 2. DBSCANDensity based spatial clustering of applications with noise noise arbitrarily shaped clusters
  3. 3. DBSCANDBSCAN is one of the most cited clustering algorithms in the literature.Features− Spatial data geomarketing, tomography, satellite images− Discovery of clusters with arbitrary shape spherical, drawn-out, linear, elongated− Good efficiency on large databases parallel programming− Only two parameters required− No prior knowledge of the number of clusters required.
  4. 4. DBSCANIdea− Clusters have a high density of points.− In the area of noise the density is lower than the density in any of the clusters.Goal− Formalize the notions of clusters and noise.
  5. 5. DBSCANNaïve approachFor each point in a cluster there are at least a minimum number (MinPts)of points in an Eps-neighborhood of that point. cluster
  6. 6. Neighborhood of a Point Eps-neighborhood of a point p NEps(p) = { q ∈ D | dist (p, q) ≤ Eps } Eps p
  7. 7. DBSCAN ‒ DataProblem• In each cluster there are two kinds of points: cluster ̶ points inside the cluster (core points) ̶ points on the border (border points)An Eps-neighborhood of a border point contains significantly less points thanan Eps-neighborhood of a core point.
  8. 8. Better ideaFor every point p in a cluster C there is a point q ∈ C,so that(1) p is inside of the Eps-neighborhood of q border points are connected to core pointsand(2) NEps(q) contains at least MinPts points. core points = high density p q
  9. 9. DefinitionA point p is directly density-reachable from a point qwith regard to the parameters Eps and MinPts, if 1) p ∈ NEps(q) (reachability) 2) | NEps(q) | ≥ MinPts (core point condition) p MinPts = 5 q | NEps(q) | = 6 ≥ 5 = MinPts (core point condition)
  10. 10. RemarkDirectly density-reachable is symmetric for pairs of core points.It is not symmetric if one core point and one border point are involved. Parameter: MinPts = 5 p p directly density reachable from q p ∈ NEps(q) q | NEps(q) | = 6 ≥ 5 = MinPts (core point condition) q not directly density reachable from p | NEps (p) | = 4 < 5 = MinPts (core point condition)
  11. 11. DefinitionA point p is density-reachable from a point qwith regard to the parameters Eps and MinPtsif there is a chain of points p1, p2, . . . ,ps with p1 = q and ps = psuch that pi+1 is directly density-reachable from pi for all 1 < i < s-1. p p1 MinPts = 5 | NEps(q) | = 5 = MinPts (core point condition) q | NEps(p1) | = 6 ≥ 5 = MinPts (core point condition)
  12. 12. Definition (density-connected)A point p is density-connected to a point qwith regard to the parameters Eps and MinPtsif there is a point v such that both p and q are density-reachable from v. p MinPts = 5 v qRemark: Density-connectivity is a symmetric relation.
  13. 13. Definition (cluster)A cluster with regard to the parameters Eps and MinPtsis a non-empty subset C of the database D with 1) For all p, q ∈ D: (Maximality) If p ∈ C and q is density-reachable from p with regard to the parameters Eps and MinPts, then q ∈ C. 2) For all p, q ∈ C: (Connectivity) The point p is density-connected to q with regard to the parameters Eps and MinPts.
  14. 14. Definition (noise)Let C1,...,Ck be the clusters of the database Dwith regard to the parameters Eps i and MinPts I (i=1,...,k).The set of points in the database D not belonging to any cluster C1,...,Ckis called noise: Noise = { p ∈ D | p ∉ Ci for all i = 1,...,k} noise
  15. 15. Two-Step ApproachIf the parameters Eps and MinPts are given,a cluster can be discovered in a two-step approach:1) Choose an arbitrary point v from the database satisfying the core point condition as a seed.2) Retrieve all points that are density-reachable from the seed obtaining the cluster containing the seed.
  16. 16. DBSCAN (algorithm)(1) Start with an arbitrary point p from the database and retrieve all points density-reachable from p with regard to Eps and MinPts.(2) If p is a core point, the procedure yields a cluster with regard to Eps and MinPts and the point is classified.(3) If p is a border point, no points are density-reachable from p and DBSCAN visits the next unclassified point in the database.
  17. 17. Algorithm: DBSCANINPUT: Database SetOfPoints, Eps, MinPtsOUTPUT: Clusters, region of noise(1) ClusterID := nextID(NOISE);(2) Foreach p ∈ SetOfPoints do(3) if p.classifiedAs == UNCLASSIFIED then(4) if ExpandCluster(SetOfPoints, p, ClusterID, Eps, MinPts) then(5) ClusterID++;(6) endif(7) endif(8) endforeachSetOfPoints = the database or a discovered cluster from a previous run.
  18. 18. Function: ExpandClusterINPUT: SetOfPoints, p, ClusterID, Eps, MinPtsOUTPUT: True, if p is a core point; False, else.(1) seeds = NEps(p);(2) if seeds.size < MinPts then // no core point(3) p.classifiedAs = NOISE;(4) return FALSE;(5) else // all points in seeds are density-reachable from p(6) foreach q ∈ seeds do(7) q.classifiedAs = ClusterID(8) endforeach
  19. 19. Function: ExpandCluster (continued)(9) seeds = seeds {p};(10) while seeds ≠ ∅ do(11) currentP = seeds.first();(12) result = NEps(currentP);(13) if result.size ≥ MinPts then(14) foreach resultP ∈ result and resultP.classifiedAs ∈ {UNCLASSIFIED, NOISE} do(15) if resultP.classifiedAs == UNCLASSIFIED then(16) seeds = seeds ∪ {resultP};(17) endif(18) resultP.classifiedAs = ClusterID;(19) endforeach(20) endif(21) seeds = seeds {currentP};(22) endwhile(23) return TRUE;(24) endifSource: A. Naprienko: Dichtebasierte Verfahren der Clusteranalyse raumbezogener Daten am Beispiel von DBSCAN und Fuzzy-DBSCAN. Universität der Bundeswehr München, student’s project, WT2011.
  20. 20. Density Based Clustering ‒ The Parameters Eps and MinPts ‒
  21. 21. Determining the parameters Eps and MinPtsThe parameters Eps and MinPts can be determined by a heuristic.Observation• For points in a cluster, their k-th nearest neighbors are at roughly the same distance.• Noise points have the k-th nearest neighbor at farther distance.⇒ Plot sorted distance of every point to its k-th nearest neighbor.
  22. 22. Determining the parameters Eps and MinPtsProcedure• Define a function k-dist from the database to the real numbers, mapping each point to the distance from its k-th nearest neighbor.• Sort the points of the database in descending order of their k-dist values. k-dist database
  23. 23. Determining the parameters Eps and MinPtsProcedure• Choose an arbitrary point p set Eps = k-dist(p) set MinPts = k.• All points with an equal or smaller k-dist value will be cluster points k-dist p noise cluster points
  24. 24. Determining the parameters Eps and MinPtsIdea: Use the point density of the least dense cluster in the data set as parameters
  25. 25. Determining the parameters Eps and MinPts• Find threshold point with the maximal k-dist value in the “thinnest cluster” of D• Set parameters Eps = k-dist(p) and MinPts = k. Eps noise cluster 1 cluster 2
  26. 26. Density Based Clustering ‒ Applications ‒
  27. 27. Automatic border detection in dermoscopy imagesSample images showing assessments of the dermatologist (red), automated frameworks DBSCAN (blue) and FCM (green).Kockara et al. BMC Bioinformatics 2010 11(Suppl 6):S26 doi:10.1186/1471-2105-11-S6-S26
  28. 28. Literature• M. Ester, H.P. Kriegel, J. Sander, X. Xu A density-based algorithm for discovering clusters in large spatial databases with noise. Proceedings of 2nd International Conference on Knowledge Discovery and Data Mining (KDD96).• A. Naprienko Dichtebasierte Verfahren der Clusteranalyse raumbezogener Daten am Beispiel von DBSCAN und Fuzzy-DBSCAN. Universität der Bundeswehr München, student’s project, WT2011.• J. Sander, M. Ester, H.P. Kriegel, X. Xu Density-based clustering in spatial databases: the algorithm GDBSCAN and its applications. Data Mining and Knowledge Discovery, Springer, Berlin, 2 (2): 169–194.
  29. 29. Literature• J.N Dharwa, A.R. Patel A Data Mining with Hybrid Approach Based Transaction Risk Score Generation Model (TRSGM) for Fraud Detection of Online Financial Transaction. Proceedings of 2nd International Conference on Knowledge Discovery and Data Mining (KDD96). International Journal of Computer Applications, Vol 16, No. 1, 2011.
  30. 30. Thank you very much!

×