Mining Regional Knowledge in Spatial Dataset


Published on

  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Mining Regional Knowledge in Spatial Dataset

  1. 1. CACS Lafayette (LA) , May 1, 2009<br />A Domain-Driven Framework for Clustering with Plug-in Fitness Functions and <br />its Application to Spatial Data Mining<br />Christoph F. Eick<br />Department of Computer Science<br />University of Houston<br />
  2. 2. Talk Outline<br />Domain-driven Data Mining (D3M, DDDM)<br />A Framework for Clustering with Plug-in Fitness Functions<br />MOSAIC---a Clustering Algorithm that Supports Plug-in Fitness Functions<br />Popular Fitness Functions<br />Case Studies: Applications to Spatial Data Mining<br />Co-location Mining <br />Multi-objective Clustering<br />Change Analysis in Spatial Data<br />Summary and Conclusion. <br />
  3. 3. Other Contributors to the Work Presented Today<br />Current PhD Students: <br />Oner-UlviCelepcikay<br />Chun-Shen Chen <br />RachsudaJiamthapthaksin, <br />VadeeratRinsurongkawong<br />Former PhD Student: <br />Wei Ding (Assistant Professor, UMASS, Boston) <br />Former Master Students: <br />RachanaParmar<br />Dan Jiang<br />Seungchan Lee<br />Domain Experts:<br />Jean-Philippe Nicot (Bureau of Economic Geology, UT Austin)<br />Tomasz F. Stepinski(Lunar and Planetary Institute, Houston)<br />Michael Twa (College of Optometry, University of Houston), <br />
  4. 4. DDDM—what is it about?<br />Differences concerning the objectives of data mining created a gap between academia and applications of data mining in business and science.<br />Traditional data mining targets the production of generic, domain-independent algorithms and tools; as a result, data mining algorithms have little capability to adapt to external, domain-specific constraints and evaluation measures.<br />To overcome this mismatch, the need to incorporate domain intelligence into data mining algorithms has been recognized by current research. Domain intelligence requires:<br />the involvement of domain knowledge and experts, <br />the consideration of domain constraints and domain-specific evaluation measures<br />the discovery of in-depth patterns based on a deep domain model<br />On top of the data-driven framework, DDDM aims to develop novel methodologies and techniques for integrating domain knowledge as well as actionability measures into the KDD process and to actively involves humans.<br />
  5. 5. The Vision of DDDM<br /> “DDDM…can assist in a paradigm shift from “data-driven hidden pattern mining” to “domain-driven actionable knowledge discovery”, and provides support for KDD to be translated to the real business situations as widely expected.” [CZ07] <br />
  6. 6. IEEE TKDE Special Issue<br />
  7. 7. 2. Clustering with Plug-in Fitness Functions<br />Motivation:<br />Finding subgroups in geo-referenced datasets has many applications.<br />However, in many applications the subgroups to be searched for do not share the characteristics considered by traditional clustering algorithms, such as cluster compactness and separation.<br />Domain knowledge frequently imposes additional requirements concerning what constitutes a “good” subgroup.<br />Consequently, it is desirable to develop clustering algorithms that provide plug-in fitness functions that allow domain experts to express desirable characteristics of subgroups they are looking for.<br />Only very few clustering algorithms published in the literature provide plug-in fitness functions; consequently existing clustering paradigms have to be modified and extended by our research to provide such capabilities. <br />
  8. 8. Clustering with Plug-In Fitness Functions<br />Clustering<br /> algorithms<br />No fitness function<br />Fixed <br />Fitness<br />Function<br />Provide plug-in<br />fitness function<br />Implicit Fitness Function<br />DBSCAN<br />Hierarchical<br />Clustering<br />K-Means<br />PAM<br />CHAMELEON<br />MOSAIC<br />
  9. 9. Current Suite of Spatial Clustering Algorithms<br />Representative-based: SCEC[1], SPAM[3], CLEVER[4]<br />Grid-based: SCMRG[1]<br />Agglomerative: MOSAIC[2]<br />Density-based: SCDE [4], DCONTOUR[8] (not really plug-in but some fitness functions can be simulated)<br />Density-based<br />Grid-based<br />Agglomerative-based<br />Representative-based<br />Clustering Algorithms<br />Remark: All algorithms partition a dataset into clusters by maximizing a <br />reward-based, plug-in fitness function. <br />
  10. 10. Spatial Clustering Algorithms<br />Datasets are assumed to have the following structure:<br />(<spatial attributes>;<non-spatial attributes>)<br /> e.g. (longitude, latitude; <chemical concentrations>+)<br />Clusters are found in the subspace of the spatial attributes, called regions in the following.<br />The non-spatial attributes are used by the fitness function but neither in distance computations nor by the clustering algorithm itself. <br />Clustering algorithms are assumed to maximize reward-based fitness functions that have the following structure: <br /> where b is a parameter that determines the premium put on cluster size (larger values  fewer, larger clusters)<br />
  11. 11. 3. MOSAIC—a Clustering Algorithm that Supports Plug-in Fitness Functions<br />MOSAIC[2]supports plug-in fitness functions and provides a generic framework that integrates representative-based clustering, agglomerative clustering, and proximity graphs, and which approximates arbitrary shape clusters using unions of small convex polygons. <br />(a) input<br />(b) output<br />Fig. 6: An illustration of MOSAIC’s approach <br />
  12. 12. 3.1 Representative-based Clustering<br />2<br />Attribute1<br />1 <br />3<br />Attribute2<br />4<br />Objective: Find a set of objects OR such that the clustering X <br />obtained by using the objects in OR as representatives minimizes q(X).<br />Properties: <br /><ul><li> Uses 1NN queries to assign objects to a cluster
  13. 13. Cluster shapes are limited to convex polygons</li></ul>Popular Algorithms: K-means, K-medoids, CLEVER, SPAM<br />
  14. 14. 3.2 MOSAIC and Agglomerative Clustering<br />Traditional Agglomerative Clustering Algorithms<br />Decision which clusters to merge next is made solely based on distances between clusters.<br />In particular, two clusters that are closest to each other with respect to a distance measure (single link, group average,…) are merged.<br />Use of some distance measures might lead to non-contiguous clusters.<br />Example: If group average is used, clusters C3 and C4 would be merged next<br />
  15. 15. MOSAIC and Agglomerative Clustering<br />Advantages MOSAIC over traditional agglomerative clustering:<br />Plug-in fitness function<br />Conducts a wider search—considers all neighboring clusters and merges the pair of clusters that enhances fitness the most<br />Clusters are always contiguous <br />Expensive algorithm is only run for 20-1000 iterations<br />Highly generic algorithm <br />
  16. 16. 3.3 Proximity Graphs<br />How to identify neighbouring clusters for representative-based clustering algorithms?<br />Proximity graphs provide various definitions of “neighbour”:<br />NNG = Nearest Neighbour Graph<br />MST = Minimum Spanning Tree<br />RNG = Relative Neighbourhood Graph<br />GG = Gabriel Graph<br />DT = Delaunay Triangulation (neighbours of a 1NN-classifier)<br />
  17. 17. Proximity Graphs: Delaunay<br />The Delaunay Triangulation is the dual of the Voronoi diagram<br />Three points are each others neighbours if their tangent sphere contains no other points<br />Complete: captures all neighbouring clusters<br />Time-consuming to compute; impossible to compute in high dimensions.<br />
  18. 18. Proximity Graphs: Gabriel<br />The Gabriel graph is a subset of the Delaunay Triangulation (some decision boundary might be missed)<br />Points are neighbours only if their (diametral) sphere of influence is empty<br />Can be computed more efficiently: O(k3)<br />Approximate algorithms with faster complexity exist<br />
  19. 19. MOSAIC’s Input<br />Fig. 10: Gabriel graph for clusters generated by <br />a representative-based clustering algorithm<br />
  20. 20. 3.4 Pseudo Code MOSAIC<br />1. Run a representative-based clustering algorithm to create a <br /> large number of clusters.<br />2. Read the representatives of the obtained clusters.<br />3. Create a merge candidate relation using proximity graphs.<br />4. WHILE there are merge-candidates (Ci ,Cj) left <br /> BEGIN<br /> Merge the pair of merge-candidates (Ci,Cj), that <br /> enhances fitness function q the most, into a new cluster C’<br /> Update merge-candidates:<br /> C Merge-Candidate(C’,C)  Merge-Candidate(Ci,C) <br /> Merge-Candidate(Cj,C) <br /> END<br /> RETURN the best clustering X found.<br />
  21. 21. Complexity MOSAIC<br />Let <br />n be the number of objects in the dataset<br />k be the number of clusters generated by the representative-based algorithm<br />Complexity MOSAIC: O(k3 + k2*O(q(x)))<br />Remarks: <br />The above formula assumes that fitness is computed from the scratch when a new clustering is obtained<br />Lower complexities can be obtained with incrementally reusing results of previous fitness computations<br />Our current implementation assumes that only additive fitness functions are used <br />
  22. 22. 4. Interestingness Measure for Spatial Clustering with Plug-in Fitness Functions<br />Clustering algorithms maximize fitness functions that must have the following structure<br />Various interestingness functions i have been introduced in our preliminary work:<br />For supervised clustering [1]<br />Maximizing the variance of a continuous variable [5]<br />For regional association rule scoping [9]<br />For co-location patterns involving continuous variables [4]<br />…. <br />Some examples of fitness functions will be presented in the case studies<br />
  23. 23. 5. Case Studies <br />Co-location patterns involving arsenic pollution <br />Multi-objective Clustering<br />Change analysis involving earth quake patterns<br />
  24. 24. 5.1 Co-location Patterns Involving Arsenic Pollution<br />
  25. 25. Regional Co-location Mining<br /> Goal: To discover regional co-location patterns involving continuous variables in which continuous variables take values from the wings of their statistical distribution<br />Regional<br />Co-location Mining<br />Dataset:<br />(longitude,latitude,<concentrations>+)<br />
  26. 26. Summary Co-location Approach <br />Pattern Interestingness in a region is evaluated using products of (cut-off) z-scores. In general, products of z-scores measure correlation. <br />Additionally, purity is considered that is controlled by a parameter .<br />Finally, the parameter  determines how much premium is put on the size of a region when computing region rewards.<br />
  27. 27. Domain-Driven Clustering for Co-location Mining<br />1. Define problem<br />2. Create/Select a fitness function<br />3. Select a clustering algorithm<br />4. Select parameters of the clustering algorithm,<br />parameters of the fitness function and constraints<br />with respect to which patterns are considered <br />Hydrologist<br />5. Run the clustering algorithm to discover <br />interesting regions and their associated patterns<br />6. Analyze the results<br />
  28. 28. Example: 2 Sets of Results Using Medium/High Rewards for Purity <br />
  29. 29. Challenges Regional RCLM<br />Kind of “seeking a needle in a haystack” problem, because we search for both interesting places and interesting patterns.<br />Our current Interestingness measure is not anti-monotone: a superset of a co-location set might be more interesting. <br />Observation: different fitness function parameter settings lead to quite different results, many of which are valuable to domain experts; therefore, it is desirable combine results of many runs.<br />“Clustering of the future”: run clustering algorithms multiple times with multiple fitness functions, and summarize the resultsmulti-run/multi-objective clustering<br />
  30. 30. 5.2 Multi-Run Clustering<br />Find clusters that good with respect to multiple objectives in automated fashion. Each objective is captured in a reward-based fitness function.<br />To achieve the goal, we run clustering algorithms multiple times with respect to compound fitness functions that capture multiple objectives and store non-dominated clusters in a cluster repository.<br />Summarization tools are provided that create final clusterings with respect to a user’s perspective. <br />
  31. 31. An Architecture for Multi-objective Clustering<br />S2<br />S1<br />Given: set of objectives Q that need to be satisfied; moreover, Q’Q.<br />Clustering <br />Algorithm<br />Goal-driven Fitness <br />Function Generator<br />A Spatial<br />Dataset<br />Q’<br />S3<br />M<br />X<br />Storage Unit<br />Steps in multi-run clustering:<br />S1: Generate a compound fitness <br /> functions. <br />S2: Run a clustering algorithm. <br />S3: Update the cluster list M.<br />S4: Summarize clusters discovered <br /> M’.<br />Q’<br />Cluster<br />Summarization<br />Unit<br />S4<br />M’<br />
  32. 32. Example: Multi-Objective RCLM<br />Example: Finding co-location patterns <br />with respect to Arsenic and a single <br />other chemical is a single objective; <br />we are interested in finding co-location <br />regions that satisfy multiple of those <br />objectives; that is, where high arsenic<br />concentrations are co-located with <br />high concentrations of many other<br />chemicals.<br />AsMoVF-<br />Cl-SO42-TDS<br />(Rank 3)<br />AsMoVBF-<br />Cl-SO42-TDS<br />(Rank 1)<br />AsMo<br />Cl-SO42-TDS<br />(Rank 4)<br />AsMoVBF-<br />Cl-SO42-TDS<br />(Rank 2)<br />AsMoB<br />Cl-SO42-TDS<br />(Rank 5)<br />Figure a: the top 5 regions ordered by rewards using user-defined query {As,Mo}<br />
  33. 33. 5.3 Change Analysis in Spatial Data<br />Question: How do interesting regions where deep earthquakes <br />are in close proximity to shallow earthquakes change?<br />Red: clusters in Oold; Blue: clusters in Onew<br />Cluster Interestingness Measure: Variance of Earthquake Depth<br />
  34. 34. Novelty Regions in Onew<br />Novelty Change Predicate:<br />Novelty(r)  |(r—(r’1  r’k))|>0<br />with rXnew; Xold={r’1,...,r’k}<br />
  35. 35. Domain-Driven Change Analysis in Spatial Data<br />Determine two datasets Oold and Onew for<br />which change patterns have to be extracted <br />2. Cluster both datasets with respect to an interestingness perspective to obtain clusters for each dataset. <br />3. Determine relevant change predicates and <br />select thresholds of change predicates<br />Geologist<br />4. Instantiate change predicates based on<br />the results of step 3. <br />5. Summarize emergent patterns<br />6. Analyze emergent patterns<br />
  36. 36. 6. Conclusion <br />A generic, domain-driven clustering framework has been introduced<br />It incorporates domain intelligence into domain-specific plug-in fitness functions that are maximized by clustering algorithms. <br />Clustering algorithms are independent of the fitness function employed. Several clustering algorithms including prototype-based, agglomerative, and grid-based clustering algorithms have been designed and implemented in our past research. <br />We conducted several case studies in our past research that illustrate the capability of the proposed domain-driven spatial clustering framework to solve challenging problems in planetary sciences, geology, environmental sciences, and optometry. <br />
  37. 37. UH-DMML References <br />C. F. Eick, B. Vaezian, D. Jiang, and J. Wang, Discovery of Interesting Regions in Spatial Datasets Using Supervised Clustering, in Proc. 10th European Conference on Principles and Practice of Knowledge Discovery in Databases (PKDD), Berlin, Germany, September 2006. <br />C. Choo, R. Jiamthapthaksin, C.-S. Chen, O. Celepcikay, C. Giusti, and C. F. Eick, MOSAIC: A Proximity Graph Approach to Agglomerative Clustering, in Proc. 9th International Conference on Data Warehousing and Knowledge Discovery (DaWaK), Regensburg, Germany, September 2007. <br />W. Ding, R. Jiamthapthaksin, R. Parmar, D. Jiang, T. Stepinski, and C. F. Eick, Towards Region Discovery in Spatial Datasets, in Proc. Pacific-Asia Conference on Knowledge Discovery and Data Mining (PAKDD), Osaka, Japan, May 2008. <br />C. F. Eick, R. Parmar, W. Ding, T. Stepinki, and J.-P. Nicot, Finding Regional Co-location Patterns for Sets of Continuous Variables in Spatial Datasets, in Proc. 16th ACM SIGSPATIAL International Conference on Advances in GIS (ACM-GIS), Irvine, California, November 2008.<br />C.-S. Chen, V. Rinsurongkawong, C.F. Eick, and M.D. Twa, Change Analysis in Spatial Data by Combining Contouring Algorithms with Supervised Density Functions, in Proc. Pacific-Asia Conference on Knowledge Discovery and Data Mining (PAKDD), Bangkok, Thailand, April 2009. <br />A. Bagherjeiran, O. U. Celepcikay, R. Jiamthapthaksin, C.-S. Chen, V. Rinsurongkawong, S. Lee, J. Thomas, and C. F. Eick, Cougar**2: An Open Source Machine Learning and Data Mining Development Framework, in Proc. Open Source Data Mining Workshop (OSDM), Bangkok, Thailand, April 2009.<br />C. F. Eick, O. U. Celepcikay, and R. Jiamthapthaksin, A Unifying Domain-driven Framework for Clustering with Plug-in Fitness Functions and Region Discovery, submitted to IEEE TKDE.<br />R. Jiamthapthaksin, C. F. Eick, and R. Vilalta, A Framework for Multi-Objective Clustering and its Application to Co-Location Mining, submitted to Fifth International Conference on Advanced Data Mining and Applications (ADMA), Beijing, China, August 2009. <br />W. Ding, C. F. Eick, X. Yuan, J. Wang, and J.-P. Nicot, A Framework for Regional Association Rule Mining and Scoping in Spatial Datasets, under review for publication in Geoinformatica. <br />
  38. 38. Other References<br />L. Cao and C. Zhang, “The Evolution of KDD: Towards Domain-Driven Data Mining,” Journal of Pattern Recognition and Artificial Intelligence, vol.21, no. 4, pp. 677-692, World Scientific Publishing Company, 2007.<br />O. Thonnard and M. Dacier, Actionable Knowledge Discovery for Threats Intelligence Support using a Multi-Dimensional Data Mining Methodology, DDDM08.<br />
  39. 39. Region Discovery Framework<br />Objective: Develop and implement an integrated framework to automatically discover interesting regional patterns in spatial datasets. Treats region discovery as a clustering problem.<br />
  40. 40. Region Discovery Framework Continued<br />The clustering algorithms we currently investigate solve the following problem:<br />Given:<br />A dataset O with a schema R<br />A distance function d defined on instances of R<br />A fitness function q(X) that evaluates clustering X={c1,…,ck} as follows:<br />q(X)= cXreward(c)=cXinterestingness(c)*size(c) with b>1<br />Objective:<br />Find c1,…,ck O such that:<br />cicj= if ij<br />X={c1,…,ck} maximizes q(X)<br />All cluster ciX are contiguous in the spatial subspace<br />c1,…,ck  O <br />c1,…,ck are usually ranked based on the reward each cluster receives, and low reward clusters are frequently not reported<br />
  41. 41. [CZ07]<br />
  42. 42. Arsenic Water Pollution Problem<br />Arsenic pollution is a serious problem in the Texas water supply.<br />Hard to explain what causes arsenic pollution to occur.<br />Several Datasets were created using the Ground Water Database (GWDB) by Texas Water Development Board (TWDB) that tests water wells regularly, one of which was used in the experimental evaluation in the paper: <br />All the wells have a non-null samples for arsenic<br />Multiple sample values are aggregated using avg/max functions<br />Other chemicals may have null values<br />Format: (Longitude, Latitude, <z-values of chemical concentrations>)<br />