2005: A Matlab Tour on Artificial Immune Systems

5,219 views

Published on

BIC 2005 (Biologically Inspired Computing Conference), Johor, Malaysia

Published in: Technology, Education
0 Comments
4 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
5,219
On SlideShare
0
From Embeds
0
Number of Embeds
10
Actions
Shares
0
Downloads
297
Comments
0
Likes
4
Embeds 0
No embeds

No notes for slide

2005: A Matlab Tour on Artificial Immune Systems

  1. 1. A Matlab Tour on Some AIS Algorithms BIC 2005: International Symposium on Bio-Inspired Computing Johor, MY, 10 th September 2005 Dr. Leandro Nunes de Castro [email_address] http://lsin.unisantos.b/lnunes Catholic University of Santos - UniSantos/Brazil
  2. 2. <ul><li>CLONALG: A Clonal Selection Algorithm </li></ul><ul><li>aiNet: An Artificial Immune Network </li></ul><ul><li>ABNET: An Antibody Network </li></ul><ul><li>Opt-aiNet: An Optimization Version of aiNet </li></ul><ul><li>Discussion </li></ul>Outline
  3. 3. CLONALG A Clonal Selection Algorithm
  4. 4. <ul><li>Increasing interest in biologically inspired systems </li></ul><ul><li>Systemic view of the immune system </li></ul><ul><li>Main goals: </li></ul><ul><ul><li>Provide a better understanding of the immune system </li></ul></ul><ul><ul><li>Solve engineering problems </li></ul></ul><ul><ul><li>Study immune learning and memory </li></ul></ul>CLONALG
  5. 5. Clonal Selection Principle
  6. 6. Continuous Learning
  7. 7. Affinity Maturation <ul><li>The cells that are most stimulated by the antigens suffer a hypermutation process </li></ul><ul><ul><li>single point, short deletions and sequence exchange </li></ul></ul><ul><li>The hypermutation is proportional to antigenic affinity </li></ul><ul><li>The higher the cell affinity with the antigen, the greater its probability of being selected for differentiation and memory, thus surviving longer </li></ul><ul><li>The mutation rate is proportional to antigenic affinity </li></ul><ul><li>The editing process promotes a better exploration of the possible antigenic receptors </li></ul>
  8. 8. Hypermutation  Editing
  9. 9. CLONALG: Block Diagram
  10. 10. <ul><ul><ul><li>1) Generate a set (P) of candidate solutions, composed of the subset of memory cells (M) added to the remaining (P r ) population (P = P r  + M) </li></ul></ul></ul><ul><ul><ul><li>2) Determine the n best individuals of the population (P n ), based on an affinity measure </li></ul></ul></ul><ul><ul><ul><li>3) Clone (reproduce) these n best individuals of the population, giving rise to a temporary population of clones ( C ). The clone size is an increasing function of the affinity with the antigen; </li></ul></ul></ul><ul><ul><ul><li>4) Submit the population of clones to a hypermutation scheme, where the hypermutation is proportional to the affinity of the antibody with the antigen. A maturated antibody population is generated ( C* ); </li></ul></ul></ul><ul><ul><ul><li>5) Re-select the improved individuals from C* to compose the memory set. Some members of the P set can be replaced by other improved members of C* ; </li></ul></ul></ul><ul><ul><ul><li>6) Replace d low affinity antibodies of the population, maintaining its diversity. </li></ul></ul></ul>CLONALG: Algorithm
  11. 11. Test Problem I <ul><li>Pattern recognition (learning) </li></ul><ul><li>Cross-reactivity (generalization capability) </li></ul>
  12. 12. <ul><li>Pattern Recognition (Learning) </li></ul>CLONALG - Performance I 00 generations 10 generations 20 generations 50 generations 75 generations 100 generations 150 generations 200 generations 250 generations
  13. 13. <ul><li>Optimization </li></ul><ul><ul><li>function maximization </li></ul></ul>Test Problem II 200 individuals randomly distributed
  14. 14. <ul><li>Multimodal Optimization (Maximization) </li></ul><ul><ul><li>Comparison with the Standard Genetic Algorithm (GA) </li></ul></ul>CLONALG - Performance II GA CLONALG
  15. 15. <ul><li>Optimization </li></ul><ul><ul><li>Travelling Salesman Problem (TSP) </li></ul></ul>Test Problem III Cities Optimal Path (48872 um) 0 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000 0 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000 7 1 8 14 2 15 3 4 11 12 13 17 23 27 30 26 19 21 24 29 28 25 22 20 18 16 6 9 10 5
  16. 16. <ul><li>Travelling Salesman (TSP) (Minimization) </li></ul>CLONALG - Performance III
  17. 17. CLONALG: Discussion <ul><li>General purpose algorithm inspired by the clonal selection and affinity maturation processes </li></ul><ul><li>Capabilities: </li></ul><ul><ul><li>learning and maintenance of high quality memory </li></ul></ul><ul><ul><li>optimization </li></ul></ul><ul><li>Crude version </li></ul><ul><li>GA  CSA: </li></ul><ul><ul><li>same coding schemes </li></ul></ul><ul><ul><li>different sources of inspiration </li></ul></ul><ul><ul><li>related sequence of steps </li></ul></ul>
  18. 18. aiNet An Artificial Immune Network
  19. 19. Immune Network Theory
  20. 20. aiNet: Basic Principles (I) <ul><li>Definition: </li></ul><ul><ul><li>The evolutionary artificial immune network, named aiNet, is an edge-weighted graph , not necessarily fully connected, composed of a set of nodes, called cells , and sets of node pairs called edges with a number assigned called weight , or connection strength , specified to each connected edge. </li></ul></ul>
  21. 21. <ul><li>Features: </li></ul><ul><ul><li>knowledge distributed among the cells </li></ul></ul><ul><ul><li>competitive learning (unsupervised) </li></ul></ul><ul><ul><li>constructive model with pruning phases </li></ul></ul><ul><ul><li>generation and maintenance of diversity </li></ul></ul><ul><li>Growing: </li></ul><ul><ul><li>clonal selection principle </li></ul></ul><ul><li>Learning: </li></ul><ul><ul><li>directed affinity maturation </li></ul></ul><ul><li>Pruning: </li></ul><ul><ul><li>natural death rate (low stimulated cells) </li></ul></ul>aiNet: Basic Principles (II)
  22. 22. aiNet: Training Algorithm <ul><li>At each generation: </li></ul><ul><ul><li>For each Ag </li></ul></ul><ul><ul><ul><li> Affinity with the antigen ( A i ) Ag i -Ab </li></ul></ul></ul><ul><ul><ul><li> Clonal selection ( n cells)  A i </li></ul></ul></ul><ul><ul><ul><li> Cloning  A i </li></ul></ul></ul><ul><ul><ul><li> Directed maturation (mutation)  1/ A i </li></ul></ul></ul><ul><ul><ul><li> Re-selection (  %)  A i </li></ul></ul></ul><ul><ul><ul><li> Natural death (  d )  1/ A i </li></ul></ul></ul><ul><ul><ul><li> Affinity between the network cells ( D ii ) Ab-Ab </li></ul></ul></ul><ul><ul><ul><li> Clonal suppression (  s )  D ii : ( m - memory) </li></ul></ul></ul><ul><ul><ul><li> M t  [M t ; m ] </li></ul></ul></ul><ul><ul><li>Network suppression (  s )  D ii : (M  M t ) </li></ul></ul><ul><ul><li>M  [M;meta] </li></ul></ul>
  23. 23. <ul><ul><ul><li> (affinity) </li></ul></ul></ul><ul><ul><ul><li> (clonal selection) </li></ul></ul></ul><ul><ul><ul><li> (directed mutation) </li></ul></ul></ul><ul><ul><ul><li> (Re-selection) </li></ul></ul></ul><ul><ul><ul><li> (self discrimination) </li></ul></ul></ul><ul><ul><ul><li> (clonal suppression) </li></ul></ul></ul><ul><ul><ul><li>Stopping criterion : or fixed number of generations </li></ul></ul></ul>aiNet: Arithmetic
  24. 24. Test Problem I <ul><li>Five Linearly Separable Classes </li></ul>0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 x y Training Patterns
  25. 25. aiNet - Performance I Minimal Spanning Tree 0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 Number of Clusters (Valleys) 1 1 2 3 4 5 6 7 8 9 10 11 12 13 14
  26. 26. 0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1 1 2 3 4 5 6 7 8 9 10 11 12 13 14 Final Network Structure aiNet - Performance I
  27. 27. <ul><li>2-Donuts: 500 samples </li></ul>Test Problem II -2 -1 0 1 2 -2 0 2 4 -1.5 -1 -0.5 0 0.5 1 1.5
  28. 28. Number of Clusters (Valleys) Minimal Spanning Tree aiNet - Performance II 1 0 5 10 15 20 25 30 35 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9
  29. 29. -1 -0.5 0 0.5 1 -1 0 1 2 3 -1 -0.5 0 0.5 1 1.5 Final Network Structure aiNet - Performance II
  30. 30. aiNet: Discussion <ul><li>Iterative learning </li></ul><ul><li>Robustness with low redundancy (data compression) </li></ul><ul><li>Clustering </li></ul><ul><li>Related with neural networks </li></ul><ul><li>User-defined parameters </li></ul><ul><li>Gave rise to a number of other algorithms </li></ul>
  31. 31. ABNET An Antibody Network
  32. 32. ABNET <ul><li>A single-layer feedforward neural network trained using ideas from the immune system </li></ul><ul><li>Constructive architecture with pruning phases </li></ul><ul><li>Boolean weights </li></ul>
  33. 33. ABNET: Basic Functioning (I)
  34. 34. <ul><li>Affinity measure (Hamming distance): </li></ul><ul><li>Main loop of the algorithm </li></ul><ul><ul><li>Choose randomly an antigen (pattern) </li></ul></ul><ul><ul><li>Determine the cell Ab k with highest affinity </li></ul></ul><ul><ul><li>Update the weight vector of this cell </li></ul></ul><ul><ul><li>Increase the concentration level (  j ) of this cell </li></ul></ul><ul><ul><li>Attribute v a = k </li></ul></ul>ABNET: Basic Functioning (I)
  35. 35. ABNET: Growing
  36. 36. ABNET: Pruning
  37. 37. ABNET: Weight Update
  38. 38. ABNET - Performance 2) Cross-reactivity (generalization) (a)   13.75% Noise tolerance: (b)   13.75%
  39. 39. ABNET: Discussion <ul><li>Performs clustering (data reduction) </li></ul><ul><li>Easily implemented in hardware </li></ul><ul><li>Robust to solve binary tasks </li></ul><ul><li>Adapted to solve real-valued problems, both clustering and classification </li></ul>
  40. 40. Opt-aiNet An Optimization version of aiNet
  41. 41. Introduction <ul><li>The algorithm for opt-aiNet is an adaptation of a discrete artificial immune network usually applied in data analysis </li></ul><ul><li>Features of opt-aiNet: </li></ul><ul><ul><li>population size dynamically adjustable </li></ul></ul><ul><ul><li>exploitation and exploration of the search-space </li></ul></ul><ul><ul><li>capability of locating multiple optima </li></ul></ul><ul><ul><li>automatic stopping criterion </li></ul></ul>
  42. 42. Immune Networks <ul><li>N. Jerne suggested that immune cells and molecules present antigenic peptides </li></ul>
  43. 43. opt-aiNet <ul><li>1. Randomly initialize a population of cells (initial number not relevant) </li></ul><ul><li>2. While not [constant memory population], do </li></ul><ul><li>2.1 Calculate the fitness and normalize the vector of fitnesses. </li></ul><ul><li>2.2 Generate a number Nc of clones for each network cell. </li></ul><ul><li>2.3 Mutate each clone proportionally to the fitness of its parent cell, but keep the parent cell. </li></ul><ul><li>2.4 Determine the fitness of all individuals of the population. </li></ul><ul><li>2.5 For each clone, select the cell with highest fitness and calculate the average fitness of the selected population. </li></ul><ul><li>2.6 If the average fitness of the population is not significantly different from the previous iteration, then continue. Else , return to step 2.1 </li></ul><ul><li>2.7 Determine the affinity of all cells in the network. Suppress all but the highest fitness of those cells whose affinities are less than the suppression threshold  s and determine the number of network cells, named memory cells, after suppression. </li></ul><ul><li>2.8 Introduce a percentage d % of randomly generated cells and return to step 2. </li></ul><ul><li>3. EndWhile </li></ul>
  44. 44. Related Strategies <ul><li>CLONALG: </li></ul><ul><ul><li>encoding, static population size, no inter-cell interaction, different mutation scheme </li></ul></ul><ul><li>Evolution Strategies </li></ul><ul><ul><li>equal to (  +  )-ES, where  = N and  = Nc; both use Gaussian mutation, but with different standard deviations, static population size, no diversity introduction, no direct interaction within the population </li></ul></ul>
  45. 45. Simulation Results (I) <ul><li>Multi Function </li></ul>
  46. 46. Simulation Results (II) <ul><li>Roots Function </li></ul>
  47. 47. Simulation Results (III) <ul><li>Schaffer’s Function </li></ul>
  48. 48. Opt-aiNet: Discussion <ul><li>The algorithm is an adaptation of an immune network model designed to perform data analysis </li></ul><ul><li>Features: </li></ul><ul><ul><li>Exploration and exploitation of the search-space </li></ul></ul><ul><ul><li>Double-plastic search </li></ul></ul><ul><ul><li>Automatic convergence criteria </li></ul></ul><ul><li>Adapted to solve combinatorial and dynamic optimization </li></ul>
  49. 49. Final Comments <ul><li>Biological Inspiration </li></ul><ul><ul><li>utility and extension </li></ul></ul><ul><ul><li>improved comprehension of natural phenomena </li></ul></ul><ul><li>Example based learning, where different pattern categories are represented by adaptive memories of the system </li></ul><ul><li>An iterative artificial immune network </li></ul>
  50. 50. <ul><li>CLONALG </li></ul><ul><ul><li>high degree of parallelism </li></ul></ul><ul><ul><li>by controlling the hipermutation rate an initial search for most general characteristics can be performed, followed by the search for smaller details </li></ul></ul><ul><ul><li>trade off between the clone size and the convergence speed </li></ul></ul><ul><ul><li>possibility of using heuristics to obtain global optima for problems like TSP </li></ul></ul>Final Comments
  51. 51. <ul><li>aiNet </li></ul><ul><ul><li>Models continuous spaces without the need of integration </li></ul></ul><ul><ul><li>Iterative model  dynamic models (DE) </li></ul></ul><ul><ul><li>Robustness with low redundancy </li></ul></ul><ul><ul><li>Clustering without a direct measure of distance* </li></ul></ul><ul><ul><li>RNA: knowledge distributed along the connections </li></ul></ul><ul><ul><li>aiNet: knowledge distributed in the cells </li></ul></ul><ul><ul><li> large amount of user defined parameters </li></ul></ul><ul><ul><li>Specific cells  general cells </li></ul></ul><ul><li>ABNET </li></ul><ul><ul><li>clustering, or grouping of similar patterns </li></ul></ul><ul><ul><li>capability of solving binary tasks </li></ul></ul>Final Comments
  52. 52. Final Comments <ul><li>Opt-aiNet </li></ul><ul><ul><li>;lsdkasdkj </li></ul></ul>
  53. 53. [email_address] Questions? Comments?

×