Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

KohonAnts: A Self-Organizing Ant Algorithm for Clustering and Pattern Classification


Published on

You can try it at:

This is the presentation of KohonAnts (KANTS), an hybrid Ant Colony and Self-organizing Map algorithm for clustering and pattern classification.

Esta es la presentación de KohonAnts (KANTS), un algoritmo que combina conceptos de los algortimos de hormigas y los mapas autoorganizativos y que se puede utilizar para agrupamiento (clustering) o clasificación de patrones.

Published in: Technology, Business

KohonAnts: A Self-Organizing Ant Algorithm for Clustering and Pattern Classification

  1. 1. KohonAnts: A Self-Organizing Ant Algorithm for Clustering and Pattern Classification Antonio M. Mora , C.M. Fernandes, J.J. Merelo, J.L.J. Laredo Dpto. ATC. UNIVERSITY OF GRANADA (Spain) V. Ramos, A.C. Rosa LASEEB-ISR/IST. UNIVERSITY OF LISBON (Portugal) ALIFE XI
  2. 2. INDEX <ul><li>Ant Algorithms </li></ul><ul><li>Kohonen’s Self Organizing Maps </li></ul><ul><li>KohonAnts </li></ul><ul><li>Experiments and Results </li></ul><ul><li>Conclusions and Future Work </li></ul>ALIFE XI
  3. 3. Ant Algorithms Natural Ants ALIFE XI <ul><li>Ants live in colonies and cooperate searching for the benefit of the colony instead of their own. </li></ul><ul><li>Ants move following (smelling) and depositing a chemical substance known as pheromone (which is also evaporated). This ‘comunication’ method is known as stigmergy . </li></ul><ul><li>They can find the shortest path between their nest and the food location. </li></ul><ul><li>They build clusters with their larvae and also with their dead bodies. </li></ul>
  4. 4. Ant Algorithms Main Features ALIFE XI <ul><li>There are many agents called (artificial) ants . </li></ul><ul><li>All of them move in a graph following and depositing (artificial) pheromone . </li></ul><ul><li>They cooperate to find a solution (usually every ant yields a complete solution). </li></ul><ul><li>There are some formulae applied in the running: </li></ul><ul><ul><li>state transition rule  decides the next step for each ant </li></ul></ul><ul><ul><li>pheromone updating  contribution and evaporation </li></ul></ul><ul><ul><li>evaluation function  assigns the cost to every solution </li></ul></ul>
  5. 5. Ant Algorithms Ant System Model ALIFE XI <ul><li>Introduced by Chialvo and Millonas in 1995. </li></ul><ul><li>Ants move in a grid . Each one is represented by its position r , and orientation to move  . </li></ul><ul><li>They decide where to move without consider their previous movements. </li></ul><ul><li>They apply the so-called response function , which depends on 3 parameters: </li></ul><ul><ul><li>  pheromone density of the cell </li></ul></ul><ul><ul><li>  degree of randomness with which an ant follow the gradient of pheromone </li></ul></ul><ul><ul><li>  factor of ant’s ability to sense pheromone </li></ul></ul>
  6. 6. Kohonen’s SOM <ul><li>It is an unsupervised Artificial Neural Network. </li></ul><ul><li>There is only an input and an output layer. </li></ul><ul><li>There is a Competitive learning (only one neuron is activated for an input pattern). </li></ul><ul><li>It Maps high-dimensional data into a two-dimensional representation space (usually rectangular or hexagonal grid). </li></ul><ul><li>After training phase, neurons with similar weights tend to cluster on the map . </li></ul>Main Features ALIFE XI
  7. 7. Kohonen’s SOM <ul><li>N neurons in the input layer, M neurons in the output layer. Each input neuron is connected with all the output ones. Every input sample is associated to an input neuron. </li></ul><ul><li>Each neuron of the output layer has associated a weight vector Wj , as the same dimension as the input data. </li></ul>General Structure ALIFE XI
  8. 8. Kohonen’s SOM <ul><li>For each input pattern/sample , the most similar neuron (considering its weight vector), and its neighbours , are activated. </li></ul>Working ALIFE XI <ul><li>The weight vector of every one of these neurons is updated to be closer to the input values. </li></ul><ul><li>The SOM evolves , ‘moving’ the neurons in order to fit to the input space (of samples). </li></ul>
  9. 9. KohonAnts Presentation ALIFE XI <ul><li>It merges Ant System with Kohonen’s SOM concepts. </li></ul><ul><li>It is a clustering and classification algorithm. </li></ul>* * INPUT SAMPLES <ul><li>Each input sample (vector of variables) is associated to an ant . </li></ul><ul><li>They move in a toroidal grid which has a vector (of the same dimension) associated to each cell ( pheromone ) . </li></ul>
  10. 10. KohonAnts Working ALIFE XI Initialization Decide Where to Go For each Ant: Pheromone Update Pheromone Evaporation END Stop criteria is TRUE No more Ants Stop criteria is FALSE
  11. 11. KohonAnts Main Functions (I) ALIFE XI <ul><li>Decide Where to Go </li></ul><ul><ul><li>every ant evaluates its neighbourhood (similar to SOM) to find the cell where move to. A probability is associated to every cell . </li></ul></ul><ul><ul><li>the probability of each cell depends on the euclidean distance between the ant vector and the centroid of a zone surrounding the cell. </li></ul></ul>o o o o <ul><li>yellow circle is the neighbourhood (all possible cells to move) with radius = 2. </li></ul><ul><li>green circles are areas to calculate centroid (marked with ’o’). </li></ul><ul><li>the centroid is the arithmetic mean of the vectors inside the circle. </li></ul><ul><li>So, every ant moves with higher probability to cells surrounded by vectors similar to its own . </li></ul>
  12. 12. KohonAnts Main Functions (II) ALIFE XI <ul><li>Pheromone Update </li></ul><ul><ul><li>once an ant chooses the cell to move to (inside its neighbourhood), it updates the vector of that cell and makes it closer to its own vector . </li></ul></ul><ul><ul><li>it applies a formula similar to the learning function of SOM, so it depends on a learning rate  . </li></ul></ul><ul><li>Pheromone Evaporation </li></ul><ul><ul><li>once all the ants have moved, the pheromone tends to disappear (the vectors revert to its initial values). </li></ul></ul><ul><ul><li>it uses a formula similar to the evaporation function of Ant Algorithms, which depends on an evaporation rate  . </li></ul></ul>
  13. 13. KohonAnts Key Features ALIFE XI <ul><li>The neighbourhood radius and the learning rate decreases with the iterations (as in SOM). It means big changes at the beginning ( clustering ) and small changes the rest of the running ( refinement ). </li></ul><ul><li>An ant can ‘jump’ or ‘fly’, since it can move to cells far than one hop . </li></ul><ul><li>The pheromone changes the grid (environment) and makes it closer to the ant’s vector (input pattern/sample). </li></ul><ul><li>The ants tend to form clusters near other similar ants (similar vectors). </li></ul><ul><li>The environment (the grid) also ‘evolves’ and can be used as a classification tool . </li></ul>
  14. 14. Experiments Datasets ALIFE XI <ul><li>IRIS </li></ul><ul><ul><li>data of 3 species of Iris plant, 50 samples of each class, and 4 variables per sample (flower attributes). </li></ul></ul><ul><li>GLASS </li></ul><ul><ul><li>data of 6 classes of glass, 214 samples in total, and 9 variables per sample (chemical composition). </li></ul></ul><ul><li>PIMA </li></ul><ul><ul><li>data of 2 classes of diabetes patients, 768 samples , and 8 variables per sample (medical data). </li></ul></ul>
  15. 15. Experiments Clustering (Study) ALIFE XI <ul><li>performed over IRIS plants dataset (150 samples, 3 classes) </li></ul><ul><li>it shows ants’ final position in the grid after 100 iterations </li></ul><ul><li>a clear line of behaviour can be marked depending on  and  </li></ul><ul><li>it can be seen that clear clusters are formed for some values of these parameters </li></ul>
  16. 16. Experiments Clustering (Results) ALIFE XI <ul><li>performed over IRIS plants dataset (150 samples, 3 classes) </li></ul><ul><li>it shows the evolution of the clusters of ants </li></ul><ul><li>in a few iterations, some clusters appear </li></ul><ul><li>then, they are improved/refined in the following iterations </li></ul>Initial situation Iteration 10 Iteration 40 Iteration 100
  17. 17. Experiments Classification ALIFE XI <ul><li>example performed over IRIS plants dataset (150 samples, 3 classes) </li></ul><ul><li>this is the final situation of the grid pheromone. Each cell is associated to a class </li></ul><ul><li>we apply the KNN method to classify, so the class associated to a sample is the correspondent to the (majority) of the K closest vectors (of the grid) </li></ul>
  18. 18. Experiments Classification (Results) ALIFE XI <ul><li>Every dataset has been transformed into 6: </li></ul><ul><ul><li>3 of 50% training/50% test </li></ul></ul><ul><ul><li>3 of 90% training/10% test </li></ul></ul><ul><li>The results beat the classical KNN method </li></ul><ul><li>In recent works also beat other methods such as Neural Networks, Fuzzy Logic, Linear Discriminant Analysis … </li></ul>
  19. 19. Conclusions And Future Wok ALIFE XI <ul><li>We have presented a new algorithm which combines ideas of two soft-computing methods such as Kohonen’s Self-Organizing Maps and Ant System Model . </li></ul><ul><li>It can be used as a clustering and as a pattern classification method. </li></ul><ul><li>The preliminary results yielded are very promising and beat some other techniques . </li></ul><ul><li>FUTURE WORK : </li></ul><ul><li>Make a comparison with some clustering algorithms and with other classification ones. Try more difficult problems. </li></ul><ul><li>Implement and test some other ideas used in SOMs and AS which may be improve the algorithm. Such a neighbourhood updating, a stop criteria or weights in the decide where to go function. </li></ul>
  20. 20. Thank You! ALIFE XI The sources, bin and data can be found in: