Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

2008: Applied AIS - A Roadmap of AIS Research in Brazil and Sample Applications


Published on

ICARIS 2008 (International Conference on Artificial Immune Systems), Phuket, Thailand

Published in: Technology, Education
  • Be the first to comment

  • Be the first to like this

2008: Applied AIS - A Roadmap of AIS Research in Brazil and Sample Applications

  1. 1. Applied AIS : A Roadmap of AIS Research in Brazil and Sample Applications Leandro Nunes de Castro [email_address] ; [email_address] Mackenzie University NatComp – From Nature to Business
  2. 2. Agenda <ul><li>Main Application Areas and When AIS Should be Applied </li></ul><ul><li>A Worth Knowing Bibliography </li></ul><ul><li>A Roadmap of AIS Research in Brazil </li></ul><ul><li>Sample Projects </li></ul><ul><ul><li>Bi-clustering for text mining </li></ul></ul><ul><ul><li>An AIS for Spam Detection </li></ul></ul><ul><ul><li>Optimal Power Flow </li></ul></ul><ul><ul><li>A Grain Classifier </li></ul></ul><ul><ul><li>Container Scheduling </li></ul></ul>
  3. 3. Application Areas Which and When
  4. 4. Main Application Areas <ul><li>An imprecise and incomplete classification: </li></ul><ul><ul><li>Pattern Recognition and Classification </li></ul></ul><ul><ul><li>Machine Learning </li></ul></ul><ul><ul><li>Knowledge Discovery from Databases </li></ul></ul><ul><ul><li>Search and Optimization </li></ul></ul><ul><ul><li>Robotics </li></ul></ul><ul><ul><li>Control </li></ul></ul><ul><ul><li>Industrial Applications </li></ul></ul><ul><ul><li>Anomaly Detection </li></ul></ul>
  5. 5. Common Features <ul><li>When AIS should be used: </li></ul><ul><ul><li>Difficulty in modeling </li></ul></ul><ul><ul><li>Poorly defined </li></ul></ul><ul><ul><li>Dynamic environments </li></ul></ul><ul><ul><li>Large number of variables </li></ul></ul><ul><ul><li>Missing or noisy variables (attributes) </li></ul></ul><ul><ul><li>Highly nonlinear </li></ul></ul><ul><ul><li>Difficulty in finding derivatives </li></ul></ul><ul><ul><li>Combinatorial solutions (NP-Complete/NP-Hard) </li></ul></ul><ul><ul><li>Multiple simultaneous solutions are required </li></ul></ul>
  6. 6. Where to Find Information
  7. 7. AISWeb: The Online Home of Artificial Immune Systems <ul><li>About the Immune System </li></ul><ul><li>ICARIS </li></ul><ul><li>Immune-Inspired Algorithms </li></ul><ul><li>Jobs and studentships </li></ul><ul><li>Links to Researchers </li></ul><ul><li>Modeling the Immune System </li></ul><ul><li>Publications </li></ul><ul><li>Teaching Resources </li></ul>
  8. 8. The On-line Searchable Bibliography <ul><li> </li></ul><ul><li>You can either search or browse: </li></ul><ul><li>Books </li></ul><ul><li>Book Chapters </li></ul><ul><li>Theses </li></ul><ul><li>Journal Papers </li></ul><ul><li>Conference Papers </li></ul><ul><li>Technical Reports </li></ul>
  9. 9. A Roadmap of AIS Research in Brazil Main Research Groups and Their Focus
  11. 11. A Brief Description of Each Group <ul><li>University : UNICAMP (University of Campinas) </li></ul><ul><li>LBiC : Laboratory for Bio-Inspired Computing </li></ul><ul><li>Leader : Prof. Fernando J. Von Zuben </li></ul><ul><li>Main Application Areas : </li></ul><ul><ul><li>Nonlinear dynamic systems identification </li></ul></ul><ul><ul><li>Combinatorial optimization: vehicle routing, gene ordering, </li></ul></ul><ul><ul><li>Music composition and arts </li></ul></ul><ul><ul><li>Bioinformatics: phylogenetic tree reconstruction, gene expression analysis </li></ul></ul><ul><ul><li>Optimal Wiener equalizers </li></ul></ul><ul><ul><li>ANN design </li></ul></ul><ul><ul><li>Data mining: clustering, classification, text mining </li></ul></ul>
  12. 12. Some LBiC Contributions aiNet opt-aiNet copt-aiNet dopt-aiNet dcopt-aiNet ARIA ABNET RABNET SABNET SaiNet omni-aiNet
  13. 13. A Brief Description of Each Group <ul><li>University : UFMG (Federal University of Minas Gerais) </li></ul><ul><li>LITC : Computational Intelligence Laboratory </li></ul><ul><li>Leader : Prof. Walmir Matos Caminhas </li></ul><ul><li>Main Application Areas : </li></ul><ul><ul><li>Data Mining: spam identification; fault, anomaly and intrusion detection </li></ul></ul><ul><ul><li>Nonlinear system identification and control: induction motors, electromagnetic devices </li></ul></ul>
  14. 14. A Brief Description of Each Group <ul><li>University : UNIFEI (Federal University of Itajuba) </li></ul><ul><li>CRTI : Reference Center on Information Technology </li></ul><ul><li>Leader : Dr. Leonardo de Mello Honório </li></ul><ul><li>Main Application Areas : </li></ul><ul><ul><li>Optimization: optimal power flow, agents scheduling </li></ul></ul><ul><li>University : UTFPR (Technological Federal University of Paraná) </li></ul><ul><li>Leader : Dr. Myriam Regattieri </li></ul><ul><li>Main Application Areas : </li></ul><ul><ul><li>Economic load dispatch, protein folding, breast cancer profiling, protein structure prediction </li></ul></ul>
  15. 15. A Brief Description of Each Group <ul><li>University : Mackenzie & NatComp </li></ul><ul><li>Leader : Prof. Leandro Nunes de Castro </li></ul><ul><li>Main Application Areas : </li></ul><ul><ul><li>Optimization: combinatorial and multimodal </li></ul></ul><ul><ul><li>Intelligent machines </li></ul></ul><ul><ul><li>Optimal Wiener equalizers </li></ul></ul><ul><ul><li>ANN design </li></ul></ul><ul><ul><li>Data mining: clustering, classification, text mining </li></ul></ul>
  16. 16. Sample Applications from the Brazilian Groups From Text Mining to Grain Classification
  17. 17. BIC-aiNet: An AIS for Text Clustering Pablo de Castro et al., Natural Computing journal, in press. A Group from Unicamp
  18. 18. Immune-Inspired Biclustering to Text Mining <ul><li>Standard Clustering; </li></ul><ul><ul><li>Applied to either the rows or columns of a data matrix; that is, clusters either objects or attributes </li></ul></ul><ul><ul><li>‘ Global’ model </li></ul></ul><ul><li>Biclustering: </li></ul><ul><ul><li>Simultaneous row-column clustering; that is, clusters objects and attributes </li></ul></ul><ul><ul><li>‘ Local’ model </li></ul></ul><ul><ul><li>Also called coclustering, bidimensional clustering, subspace clustering </li></ul></ul><ul><ul><li>Terminology introduced for gene expression data analysis </li></ul></ul>
  19. 19. Biclustering <ul><li>When biclustering should be used? </li></ul><ul><ul><li>A set of objects influences (is influenced by) a set of attributes </li></ul></ul><ul><ul><li>An object may belong to more than one cluster </li></ul></ul><ul><li>Restrictions: </li></ul><ul><ul><li>A cluster of objects should be defined with respect to only a subset of attributes </li></ul></ul><ul><ul><li>A cluster of attributes should be defined with respect to only a subset of objects </li></ul></ul><ul><ul><li>The clusters should not be exclusive and/or exhaustive: an object/attribute should be able to belong to more than one cluster or no cluster at all and be grouped using a subset of attributes/objects </li></ul></ul>
  20. 20. Biclustering <ul><li>A bicluster is a subset of rows that exhibit similar behavior across a subset of columns, and vice-versa. </li></ul><ul><li>The bicluster A IJ = ( I , J ) is a subset of rows and a subset of columns where I = { i 1 , ..., i k } is a subset of rows ( I  X and k  n ), and J = { j 1 , ..., j s } is a subset of columns ( J  Y and s  m ). </li></ul><ul><li>A bicluster can be defined as a k by s submatrix of matrix A . </li></ul>
  21. 21. Biclustering <ul><li>Two interpretations: </li></ul><ul><ul><li>As a two-way permutation problem: interactive reordering of rows and columns so as to produce multiple clusters in different regions of the matrix </li></ul></ul><ul><ul><li>As a two-way partition problem: creation of submatrices so as to maximize an index that evaluates clustering properties (e.g., similarity among objects) </li></ul></ul>rows = {1,3} columns = {1,4} rows = {2,3} columns = {3,4}
  22. 22. An Immune-Inspired Algorithm for Biclustering <ul><li>BIC-aiNet: </li></ul><ul><ul><li>Multi-population </li></ul></ul><ul><ul><li>Dynamic control of the population size </li></ul></ul><ul><ul><li>Diversity maintenance </li></ul></ul><ul><li>Encoding: </li></ul><ul><ul><li>Two ordered vectors (rows and columns) </li></ul></ul><ul><ul><li>Each individual represents a single bicluster </li></ul></ul>Row: [2, 3] Column: [1, 3, 4] Data matrix Individual Bicluster
  23. 23. BIC-aiNet <ul><li>Fitness function: </li></ul>N , M are the set of rows and columns, R is the residue of a bicluster,  is a residue threshold, w c is the weight of the number of columns, w r is the weight of the number of rows, a ij is the value in the original data matrix, and I (J) indicates the mean values for row (column) i ( j ) .
  24. 24. The BIC-aiNet Algorithm Mutation : insertion/deletion of rows and columns Suppression : eliminates similar biclusters
  25. 25. BIC-aiNet Performance <ul><li>Application in Collaborative Filtering </li></ul><ul><ul><li>Perform automated recommendations for a user </li></ul></ul><ul><ul><li>Input: matrix R ij , in which each entry represents the rating of user i to item j </li></ul></ul><ul><li>Data sets used: MovieLens, Jester and Dating </li></ul><ul><ul><li>MovieLens: 100,000 ratings assigned by 943 users on 1,682 movies. Range: 1(bad) – 5(excellent) </li></ul></ul><ul><ul><li>The purpose is to group users with similar interests in order to provide a recommendation of a movie when a user asks for. </li></ul></ul><ul><li>Performance Measures: </li></ul><ul><ul><li>RMSE, MAE, Accuracy, Precision </li></ul></ul>
  26. 26. BIC-aiNet Performance
  27. 27. BIC-aiNet Discussion <ul><li>Bi-clustering interpreted as a bipartition problem </li></ul><ul><li>Possibility of using just some attributes per cluster </li></ul><ul><li>Multimodal solutions </li></ul><ul><li>Accurate recommendations </li></ul><ul><li>An attribute may appear in more than one cluster or in none </li></ul>
  28. 28. IA-AIS: An AIS To Detect Spam Thiago S. Guzella et al., BioSystems 92(2008), pp. 215-225 A Group from UFMG
  29. 29. SPAM Detection <ul><li>SPAM messages are constantly ‘evolving’, e.g.: </li></ul><ul><ul><li>free == fr33 </li></ul></ul><ul><ul><li>viagra = v1agra </li></ul></ul><ul><ul><li>casino = casin0 </li></ul></ul><ul><ul><li>watch = w4tch </li></ul></ul><ul><li>SPAM messages can be identified by features such as the sender’s e-mail address, message subject and message body </li></ul>
  30. 30. IA-AIS: Innate and Adaptive AIS <ul><li>Combines features from Negative Selection and Clonal Selection </li></ul><ul><li>Composed of Macrophages, B cells, T cells, Interactions among B and T cells (helper and regulatory) </li></ul>
  31. 31. IA-AIS: Innate and Adaptive AIS
  32. 32. IA-AIS: Innate and Adaptive AIS
  33. 33. IA-AIS: Innate and Adaptive AIS
  34. 34. IA-AIS: Parameters and Configurations
  35. 35. IA-AIS: Experimental Results
  36. 36. IA-AIS: Discussion <ul><li>Macrophages, B cells, T helper cells, T regulatory cells </li></ul><ul><li>Incorporation of user feedback </li></ul><ul><li>Considers interactions of immune cells </li></ul><ul><li>Interesting alternative when high true positive values are relevant </li></ul>
  37. 37. CGbAIS: A Cluster Gradient-Based AIS to Optimal Power Flow Leonardo M. Honorio et al., IEEE Trans. on Power Systems, in press.
  38. 38. Optimal Power Flow Problems <ul><li>OPF Main Features: </li></ul><ul><ul><li>Non-linear, non-convex, large-scale </li></ul></ul><ul><ul><li>Several sets of continuous and discrete variables </li></ul></ul><ul><li>CGbAIS Main Features: </li></ul><ul><ul><li>An individual is related to a set of control variables that define a possible solution, characterized by a set of equations that describe its behavior </li></ul></ul><ul><ul><li>The Jacobian vector associated with the solutions can be used to guide mutation </li></ul></ul><ul><ul><li>A clustering algorithm is used to reduce computational effort </li></ul></ul><ul><ul><li>To ensure the KKT conditions, a modified Lagrangian system is used </li></ul></ul>
  39. 39. The Cluster Gradient-Based AIS (CGbAIS)
  40. 40. CGbAIS for Discrete Optimization <ul><li>An antibody is a partial path over a tree search </li></ul><ul><li>Maintenance of nLocalBest clones and selection of nGlobalBest clones </li></ul><ul><li>Clustering of paths with the same nodes </li></ul><ul><li>Numerical information used to evolve the population: </li></ul>
  41. 41. Combinatorial Optimization with CGbAIS
  42. 42. Solving the Lagrangian Problem with CGbAIS <ul><li>Typical OPF Problem Formulation </li></ul><ul><li>Augmented Lagrangian Function </li></ul>
  43. 43. Solving the Lagrangian Problem with CGbAIS <ul><li>Karush-Kuhn-Tucker Conditions </li></ul><ul><li>The Dual Problem </li></ul>
  44. 44. Solving the Lagrangian Problem with CGbAIS <ul><li>Formulation Mixing the Primal and Dual Problems </li></ul><ul><li>Scenario 1: </li></ul><ul><ul><li>Transmission loss reduction by installing shunt compensation </li></ul></ul><ul><ul><li>Variation of the IEEE 14-bus test system </li></ul></ul><ul><ul><li>mFLoss: mean loss reduction </li></ul></ul>
  45. 45. CGbAIS: Experimental Results Results without clustering Results with clustering
  46. 46. CGbAIS: Discussion <ul><li>A hybridization of na AIS with clustering and numerical information to improve computing effort and search robustness </li></ul><ul><li>Use of a bent augmented Lagrangian </li></ul><ul><li>Good results when compared with traditional interior-point methods </li></ul>
  47. 47. A Grain Classification Machine NatComp – From Nature to Business
  48. 48. Automatic Grain Classification <ul><li>Actors involved: </li></ul><ul><ul><li>Producers; </li></ul></ul><ul><ul><li>Local and global consumers; </li></ul></ul><ul><ul><li>Cooperatives; </li></ul></ul><ul><ul><li>Banks; </li></ul></ul><ul><ul><li>Stock Market. </li></ul></ul><ul><li>Motivation: </li></ul><ul><ul><li>Automatic certification of quality ; </li></ul></ul><ul><ul><li>Avoid classification conflicts; </li></ul></ul><ul><ul><li>No equivalent machine available***; </li></ul></ul><ul><ul><li>Standardization. </li></ul></ul>
  49. 49. Automatic Grain Classification <ul><li>Physical Classification: based on a sample of grains </li></ul><ul><ul><li>Grain quality: endogenous and exogenous defects; </li></ul></ul><ul><ul><li>Grain size. </li></ul></ul>
  50. 50. Examples of Grain Defects
  51. 51. The Grain Classifier Project <ul><li>Public Investor </li></ul><ul><li>The Development Cycle: </li></ul>Conceptual Design Hardware Prototyping Computer Vision Pattern Recognition
  52. 52. HW Prototyping
  53. 53. HW Prototyping
  54. 54. Computer Vision <ul><li>Image Capture: </li></ul><ul><ul><li>Double face capture </li></ul></ul><ul><li>Feature Extraction: </li></ul><ul><ul><li>Color, Texture and Shape attributes </li></ul></ul><ul><ul><li>Based on the RGB histograms </li></ul></ul><ul><ul><li>Total of 70 attributes extracted per grain </li></ul></ul>
  55. 55. Feature Selection and Classification <ul><li>Feature Selection: </li></ul><ul><ul><li>Filter and Wrapper </li></ul></ul><ul><li>Classification: </li></ul><ul><ul><li>Naïve Bayes </li></ul></ul><ul><ul><li>KNN </li></ul></ul><ul><ul><li>Support Vector Machines </li></ul></ul><ul><ul><li>Multi-Layer Perceptrons </li></ul></ul><ul><ul><li>aiNet+RBF </li></ul></ul><ul><ul><li>SRABNET: Supervised RABNET </li></ul></ul>
  56. 56. Experimental Results <ul><li>Estimating the Weight </li></ul>ICS-RBF = aiNet+RBF
  57. 57. Experimental Results <ul><li>Classification Performance </li></ul>E tr % E te % E CV % Std MLP 8,80 1,814 SVM 10,60 3,31 k-NN 15,10 3,00
  58. 58. Sample Certificate
  59. 59. Discussion <ul><li>The immune system approach demonstrated to be competitive </li></ul><ul><li>Experiment with binary classification followed by defect classification </li></ul><ul><li>Experiment hierarchical classification </li></ul><ul><li>Possibility of automating the classification of grains </li></ul>
  60. 60. Operation Planning in a Container Terminal (CONTER) NatComp – From Nature to Business
  61. 61. The Importance of Container Terminals <ul><li>Most World commerce is performed using containers </li></ul><ul><li>The operation of a CONTER is a very complicated and challenging task, involving space and equipment constraints, short time spans for ship docking, pre-specified ship plans, customs procedures, etc. </li></ul>
  62. 62. A Typical Problem: Scheduling RTGs <ul><li>When a Ship Plan is received in the terminal, the operators in have to search and load the selected containers into the ship. </li></ul><ul><li>The RTGs (Rubber Tyred Gantry Crane) are typical container handling equipments and move in three directions: x, y, and z. </li></ul><ul><li>The less movements the RTGs make, the faster and cheaper becomes the ship loading. </li></ul>
  63. 63. RTGs Movements: Productive (a) (b) (c) (c) (d) (e)
  64. 64. RTGs Movements: Unproductive (Set-Up) (a) (b) (c) (d)
  65. 65. Cost to Remove Containers where n x , n y and n z is the number of movements in direction X, Y and Z, respectively; V x , V y and V z is the RTG velocity in direction X, Y and Z, respectively; t T is the time spent to lock or unlock the spreader and n T is the number of spreader locking/unlocking.
  66. 66. The copt-aiNet Algorithm
  67. 67. A Demo on the RTG Scheduling Problem
  68. 68. Discussion <ul><li>Vast number of applications available </li></ul><ul><li>Great potential for further applications and developments </li></ul><ul><li>Some issues that still deserve investigation: </li></ul><ul><ul><li>Formal aspects </li></ul></ul><ul><ul><li>Comparison (theoretical and empirical) with other approaches </li></ul></ul><ul><ul><li>Loads of testing </li></ul></ul><ul><ul><li>Real benefits (Are they really useful?) </li></ul></ul><ul><ul><li>Danger theory </li></ul></ul><ul><ul><li>How far to stretch the metaphor? </li></ul></ul><ul><ul><li>Scalability </li></ul></ul><ul><ul><li>Robustness to high dimensions </li></ul></ul>
  69. 69. Discussion <ul><li>Current Issues at Mackenzie and NatComp </li></ul><ul><ul><li>An optimal clustering algorithm </li></ul></ul><ul><ul><li>AIS applied to recommender systems </li></ul></ul><ul><ul><li>AIS applied to intelligent virtual environments </li></ul></ul><ul><ul><li>AIS applied to virtual simulations for training purposes </li></ul></ul>THANK YOU FOR THE ATTENTION!