• Save
K-Means, its Variants and its Applications
Upcoming SlideShare
Loading in...5
×
 

K-Means, its Variants and its Applications

on

  • 2,210 views

This presentation was given by our project group at the Lead College competition at Shivaji University. Our project got the 1st Prize. We focused mainly on Rough K-Means and build a ...

This presentation was given by our project group at the Lead College competition at Shivaji University. Our project got the 1st Prize. We focused mainly on Rough K-Means and build a Social-Network-Recommender System based on Rough K-Means.

The Members of the Project group were -
Mansi Kulkarni,
Nikhil Ingole,
Prasad Mohite,
Varad Meru
Vishal Bhavsar.

Wonderful Experience !!!

Statistics

Views

Total Views
2,210
Slideshare-icon Views on SlideShare
2,023
Embed Views
187

Actions

Likes
1
Downloads
0
Comments
0

2 Embeds 187

http://www.linkedin.com 180
https://twitter.com 7

Accessibility

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    K-Means, its Variants and its Applications K-Means, its Variants and its Applications Presentation Transcript

    • K-Means, its Variants and its Applications Group 9 ------------------------------- Varad Meru, Nikhil Ingole, Mansi Kulkarni, Vishal Bhavsar, Prasad Mohite ------------------------------- Guided By: Mrs. V. S. Rupnar ------------------------------- Department of Computer Science and Engineering D. Y. Patil College of Engineering and Technology Kolhapur 1 Monday, 29 July 13
    • Work Completed in the Previous Semester ✓ Selection of Topic and Preliminary Understanding of Clustering. ✓ Implementation of K-Means algorithm with Synthetic Data. ✓ Development of Graphical Representation of Clusters. ✓ Understanding and Implementation of Rough Set Clustering. ✓ Real World Data : Data Collection based on Surveys. ✓ Implementation of Conventional Clustering on Input Surveys Details for Cluster Generation and Recommender System. ✓ Implementing Rough-Set Clustering on Input Surveys Details for Cluster Generation and Recommender System. 2 Monday, 29 July 13
    • Work Completed in this Semester ✓ Study of Genetic Algorithms and its Implementation issues. ✓ Adaption of JavaGAlib for K-Means Clustering. ✓ Verification and Validation of Cluster Quality with all the following Processes : ➡ K-Means, Rough K-Means, GA Rough K-Means. ✓ Recommender System Design and Initial Prototype Evaluation based on K-Means Algorithm. ✓ Verification and Validation of Recommendations and Applying Heuristics on the Results of the Recommendations for Precision ✓ Recommender System Design and Initial Prototype Evaluation based on Rough K-Means Algorithm. 3 Monday, 29 July 13
    • Introduction to Clustering • Organizing data into clusters such that there is • high intra-cluster similarity • low inter-cluster similarity • Informally, finding natural groupings among objects. • Applications of clustering range from various fields • Data Compression, Data Modeling, Expression Analysis and other Fields of Applications. 4 Monday, 29 July 13
    • Introduction to K-Means Algorithm • It was proposed in the year 1956 by Hugo Steinhaus. • It finds partitions such that the Squared Error between the Empirical Mean of a Cluster and the Points in that Cluster is Minimized • Squared Error is defined as : • The Goal of K-Means is to minimize the sum of the Squared Error over all the K-Clusters. • Minimizing this Objective Function is known to be an NP-Hard Problem (even for K=2). 5 Monday, 29 July 13
    • K-Means Clustering Algorithm Stop Start Input: K, no. of Clusters to be Formed Centroid Initialization Find Distance of Objects to Centroids Partition based on Minimum distance New Additions in Group ? Yes No 6 Monday, 29 July 13
    • Graph of Clusters in Synthetic DataResult of K-Means Algorithm 6 Lingras Fig. 2. Synthetic data 7 Monday, 29 July 13
    • 10 20 30 40 50 10 20 30 40 50 Visual Representation of Clusters Formed. k=2 k=6 k=4 k=1 Monday, 29 July 13
    • Demo K-Means Algorithm 9 Monday, 29 July 13
    • Introduction to Rough Sets • It was proposed in the year 1991 by Zdzislaw I. Pawlak. • Formal Approximation of Crisp Sets in terms of a pair of sets. • Pairs gives the Lower and Upper Approximation of original set. • The Rough set are based on Equivalence class partitioning. • The pair A=(U,R) is called Approximation Space. • The lower bound is the union of all the elementary sets which are subsets of X. • The upper bound is the union of all elementary sets which have a non-empty intersection with X • The set X{ , } is the formal representation of regular set X. • It is not possible to differentiate the elements within the same equivalence class. Monday, 29 July 13
    • Adaptation of Rough Sets into K-Means Clustering • We consider the upper and lower bounds for only a few subsets of U. • It is not possible to verify all the properties of the rough sets ( Pawlak, `82,`91). • Lingras et. al. classified these compulsory rules for rough set clustering • An object v can be part of at most one lower bound • • An object v is not part of any lower bound v belongs to two or more upper bounds. Monday, 29 July 13
    • Evolutionary Rough K-means 7 Fig. 3. Rough clusters for the synthetic data ified criterion. The paper demonstrates the use of the proposed algorithm for a Result of Rough Set Clustering Graph of Clusters in Synthetic Data 12 Monday, 29 July 13
    • Lingras’s Absolute Distance Formula • If the distance given by : • Consider the Set T : - • T ≠ Ø, The point X is associated with 2 or more clusters’ upper bounds. • T = Ø, X Exists in lower bound of only one cluster. 1482 G. Peters / Pattern Reco Boundary Area Lower Approximation Upper Approximation Fig. 1. Lower, upper approximation and boundary area. Monday, 29 July 13
    • Peters’s Refinements on Lingras’s Absolute Distance Formula • Limitations of Lingras method- • Outlier in inline position: b = az. • Outlier in an rectangular position. Monday, 29 July 13
    • Modified Rough K-Means • Centroid calculation in Rough Clustering • Membership Assignment on the basis of • Let , the ratio are used to determine the membership of X. • Let and . • T ≠ Ø, The point X is associated with 2 or more clusters’ upper bounds. • T = Ø, X Exists in lower bound of only one cluster. Monday, 29 July 13
    • Working Algorithm of Rough K-Means Implementation Monday, 29 July 13
    • Visual Representation of Rough K-Means Forming 3 Rough Sets Monday, 29 July 13
    • Demo Rough K-Means Algorithm 18 Monday, 29 July 13
    • Genetic Algorithm based Rough Set Clustering • Genetic Algorithms - Introduction • A search process that follows the principles of evolution through natural selection. • Important terms : Genes, Genome, Chromosomes, Populations, Generations, Fitness, Selection, Crossover, Mutation. • This paradigm has the following steps • generate initial population, G(0); evaluate G(0); for (t = 1; solution is not found; t++) generate G(t) using G(t-1); evaluate G(t); 19 Monday, 29 July 13
    • Genetic Algorithm based Rough Set Clustering • Genetic Algorithms for Rough set Clustering • JavaGALib : A Java Library built by Jeff Smith of SoftTechDesign to support GA operations 20 p - Threshold D(n,m) - A Dataset with n objects of m dimensions k - The number of Clusters w_lower, w_upper population - The number of chromosomes to be generated generations - The number of successive populations to be generated Input Fields - A set of clusters. Each cluster is by the objects in the lower region and boundary region(upper bound) Output - • Data Structures used for Genetic Algorithms for Rough set Clustering ... Chromosomes Centroid1* Centroid2* Centroid3* Monday, 29 July 13
    • Genetic Algorithm based Rough Set Clustering • Constructor Description for Genetic Algorithm • super(numOfClusters*numOfDimensions,//no.of genes in a chromosome 100,//population of chromosome 0.7,//crossover probability 6,//random selection chance 50,//stop after these many generations 10,//no. of preliminary runs to build good breeding stack for finding fall run 20,//max preliminary generations 0.1,//chromosome mutation probability Crossover.ctTwoPoint,//crossover type 2,//number of decimal pts of precision false//considers only float numbers ); }//end constructor 21 • Evolve Function computeFitnessRankings(); doGeneticMating(); copyNextGenToThisGen(); Monday, 29 July 13
    • Demo Genetic Algorithm based Rough K-Means Algorithm 22 Monday, 29 July 13
    • Rough Set Clustering based on Kohonen SOM Paradigm • Kohonen network Architecture is used as an Artificial Neural Network Paradigm. • The Single level, One-Dimensional case can be seen in fig. 1. • The weight vector x for a group that is closest to the pattern v is modified using • void update(int winner, int objectID) { for (int j = 0; j < weights[winner].length; j++) weights[winner][j] = (1 - alpha) * weights[winner][j] + alpha * objects[objectID][j]; • The Updates are carried over the previous weights. 23 J 0 0 1 Output Layer Input Layer Fig. 1. Kohonen Neural Network Monday, 29 July 13
    • Rough Set Clustering based on Kohonen SOM Paradigm • The distance metric is calculated by the following code fragment • double dist(int objectID, int weightID) { double d = 0; for (int j = 0; j < weights[0].length; j++) { double o = objects[objectID][j]; double c = weights[weightID][j]; d += (c - o) * (c - o); } if (weights[0].length == 0) return 0; return Math.sqrt(d) / weights[0].length; } • The Flow of the Kohonen K-Means Implementation is as follows • Kohonen m = new Kohonen(numOfRows, numOfCols, numOfClusters, 0.01); m.readObjects(args[0]); m.makeClusters(numOfIterations); m.writeClusters(); m.writeCentroids(); 24 X1 0 01 X2 X3 0 1 0 Monday, 29 July 13
    • Demo Kohonen Self-Organized Maps based K-Means Algorithm 25 Monday, 29 July 13
    • Recommender System based on Clustering • Recommender System is an Information Filtering Technique based System. • It applies Knowledge Discovery Techniques such as Clustering, Classification, and Filtering to find out Recommendations. • Exposing the most interesting items for the user saves time and energy. • Techniques include K-Nearest Neighbor and Collaborative filtering to give Recommendations. • Why Clustering? • Basic feature of clustering algorithm is natural grouping. • Challenges in above two algorithms are overcome. • K-Means works on a P-Time algorithm to give crisp Clusters. 26 Monday, 29 July 13
    • Recommender System based on Clustering • Recommendations for K-Means Algorithm: • All the members of the cluster where the user lies are recommended. • Recommendations for Rough K-Means Algorithm: • If the user lies in lower bound of the cluster, All the members lying in lower bound of that cluster are recommended. • If the user lies in the upper bound of two or more clusters, All the members in the upper bound are recommended. Monday, 29 July 13
    • Recommender System based on Clustering 28 System ArchitectureUser Perspective Monday, 29 July 13
    • Demo Recommender System 29 Monday, 29 July 13
    • References • Completed: ✓ K-Means Algorithm • “Data Clustering: 50 Years Beyond K-Means”, Anil K. Jain, 2010. ✓ Rough Set based K-Means Algorithm • “Precision of Rough Set Clustering”, Pawan Lingras, Min Chen, Duoqian Miao, 2008 • “Some Refinements of Rough K-means Clustering”, George Peters, 2006. • “Interval Set Clustering of Web Users with Rough K-Means”, Pawan Lingras, Chad West, 2003 ✓ Rough K-Means based on Genetic Algorithm and Kohonen Self-Organizing Maps Paradigm • “Applications of Rough Set Based K-Means, Kohonen SOM, GA Clustering”, Pawan Lingras, 2006. • “Evolutionary Rough K-Means Clustering”, Pawan Lingras, 2009. 30 Monday, 29 July 13
    • References (Contd.) • Recommender System • “Enhanced K-means-Based Mobile Recommender System”, Gamal Hussein, International Journal of Information Studies, April 2010. • “Clustering Social Networks”, Nina Mishra, Robert Schreiber, Isabelle Stanton, and Robert E. Tarjan, 2006 • K-Means based on Genetic Algorithms • “Genetic K-Means Algorithm”, K. Krishna and M. Narasimha Murty, IEEE Transactions on Systems, Man and Cybernetics, 1999. • “Initializing K-Means using Genetic Algorithms”, Bashar Al-Shboul, and Sung-Hyon Myaeng, World Academy of Science, Engineering and Technology, 2009. • Advanced Topics • “FGKA- A Fast Genetic K-means Clustering Algorithm”,Yi Lu, Shiyong Lu, Farshad Fotouhi, Youping Deng, Susan J. Brown, 2004. • “Incremental genetic K-means algorithm and its application in gene expression data analysis”, Yi Lu, Shiyong Lu, Farshad Fotouhi, Youping Deng, Susan J. Brown, 2004. • “A Genetic Algorithm for Clustering on Image Data”, Qin Ding and Jim Gasvoda, International Journal of Computational Intelligence,2004. 31 Monday, 29 July 13
    • Thank You Group 9 Have a Nice Day !!! 32 Monday, 29 July 13