An algorithm for clustering symbolic data

137
-1

Published on

For more projects visit @ www.nanocdac.com

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
137
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
0
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

An algorithm for clustering symbolic data

  1. 1. An Algorithm for Clustering Symbolic DataABSTRACT:Clustering is the process of organizing a collection of patterns into groups based on theirsimilarities. Fuzzy clustering techniques aim at finding groups to which every object inthe database belongs to some membership degree. This paper presents a new algorithmfor clustering symbolic data based on ckMeans algorithm. This new algorithm allows thedata entry and the membership degree to be intervals. In order to validate the proposal, itis compared to two other algorithms using the same database.EXISTING SYSTEM:• Even though dynamic clustering method used in large database like web pagecollection which yields better clustering, but it needs additional computationwhich leads to increase in time complexity.• And also when dynamic document clustering adopted for real world applications,sometimes it may not yield the desired output. And also dynamic algorithm workslike static algorithm in initial clustering.www.nanocdac.com www.nsrcnano.com branches: hyderabad nagpur
  2. 2. PROPOSED SYSTEM:An approach for dynamic document clustering based on structured MARDLtechnique is our objective. At first the documents are clustered in Static method usingBisecting K-means algorithm. For clustering of documents in bisecting K-Means, alldocuments should be preprocessed in the initial stage. The preprocessing stage includesstop word removal process and stemming process. In stop word removal process, wordshaving negative influence like adverbs, conjunctions are removed and in stemming processroot word will find out by removing prefixes and suffixes of the word.After the preprocessing process, the documents should grouped into desirednumber of clusters. To make desired number of clusters, bisecting K-Means clusteringmethod is used. In this method, each document is assigning a weight by term frequency andinverse document frequency method using cosine similarity measure. After assigningweight to each document, the documents are first separated into clusters using k-Meansmethod. After clustering of documents using K-means method the largest cluster will splitand forms two sub clusters and this step would be repeated for many times until clustersformed are with high similarity.The overall process is explained in the diagram below.www.nanocdac.com www.nsrcnano.com branches: hyderabad nagpur
  3. 3. HARDWARE REQUIREMENTS• SYSTEM : Pentium IV 2.4 GHz• HARD DISK : 40 GB• MONITOR : 15 VGA colour• MOUSE : Logitech.• RAM : 256 MB• KEYBOARD : 110 keys enhanced.SOFTWARE REQUIREMENTS• Operating system : Windows XP Professional• Front End : JAVAwww.nanocdac.com www.nsrcnano.com branches: hyderabad nagpur
  4. 4. • Tool : NETBEANS IDEMODULES Preprocessing Bisecting K-means Proposed Dynamic AlgorithmMODULE DESCRIPTIONPreprocessingIn preprocessing, stop words removal and stemming process can be done. Stop words areusually given as a word list. Most of these words are conjunctions or adverbs which haveno contribution to cluster process will have negative influence. Words with high frequencywhich can be gotten in word frequency dictionary appear in most documents, so they arenot helpful for cluster either. Such words can be removed in stop word removal process andoutput will send to stemming process. In stemming process we will find the root word byremoving prefixes and suffixes of the word. We will use Porter Stemmer Algorithm for thestemmer process.Bisecting K-meansThe Bisecting k-Means clustering algorithm requires the number of clusters k and thedocuments to be given as input. Hence the clustering is based on the number of clusters givenas an input. If the input document doesn’t match any domain i.e. if it’s not similar, then thedocument is clustered into a separate group of similar documents. According to the numberof clusters as input, the clusters are formed as Cluster 0, Cluster1 up to Cluster k.www.nanocdac.com www.nsrcnano.com branches: hyderabad nagpur
  5. 5. Each document belong to one or the other category of domains including computer,medicine, mathematics, thermodynamics etc., These clusters further act as data points for thenext level of hierarchy which in turn iterates until one global cluster is achieved. It isobserved that the quality of the cluster increases when the number of clusters increases.Proposed Dynamic AlgorithmAfter the clustering of given documents into desired number of clusters, the dynamicdocument clustering algorithm has been used to assign a new document into a cluster withhigh frequency value. The algorithm takes a sample from each cluster and new document are-compared with each sample and we calculate the frequency weight age of the new documentwith each cluster. Then the dynamic document clustering algorithm assigns the newdocument to the cluster with high frequency value if the frequency value is within thethreshold value. Frequency value is calculated using sentence importance calculation ofnewly arrived document docj with each sample and it is clustered with which cluster it hashighest frequency value.REFERENCE:Rogerio R. de Vargas, Benjamin R. C. Bedregal, “Interval ckMeans: An Algorithm forClustering Symbolic Data”, IEEE Ref.: 978-1-61284-968-3/11. IEEE Conference 2011.www.nanocdac.com www.nsrcnano.com branches: hyderabad nagpur
  6. 6. Each document belong to one or the other category of domains including computer,medicine, mathematics, thermodynamics etc., These clusters further act as data points for thenext level of hierarchy which in turn iterates until one global cluster is achieved. It isobserved that the quality of the cluster increases when the number of clusters increases.Proposed Dynamic AlgorithmAfter the clustering of given documents into desired number of clusters, the dynamicdocument clustering algorithm has been used to assign a new document into a cluster withhigh frequency value. The algorithm takes a sample from each cluster and new document are-compared with each sample and we calculate the frequency weight age of the new documentwith each cluster. Then the dynamic document clustering algorithm assigns the newdocument to the cluster with high frequency value if the frequency value is within thethreshold value. Frequency value is calculated using sentence importance calculation ofnewly arrived document docj with each sample and it is clustered with which cluster it hashighest frequency value.REFERENCE:Rogerio R. de Vargas, Benjamin R. C. Bedregal, “Interval ckMeans: An Algorithm forClustering Symbolic Data”, IEEE Ref.: 978-1-61284-968-3/11. IEEE Conference 2011.www.nanocdac.com www.nsrcnano.com branches: hyderabad nagpur

×