GECCO'2007: Modeling Selection Pressure in XCS for Proportionate and Tournament Selection

391 views
321 views

Published on

Published in: Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
391
On SlideShare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
4
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • In this work, we revisit the comparison between proportionate and tournament selection. In this case, under some assumptions, we derive a model of both selection schemes that permits us to compare them.
  • GECCO'2007: Modeling Selection Pressure in XCS for Proportionate and Tournament Selection

    1. 1. Modeling Selection Pressure in XCS for Proportionate and Tournament Selection Albert Orriols-Puig 1,2 Kumara Sastry 2 Pier Luca Lanzi 1,3 David E. Goldberg 2 Ester Bernadó-Mansilla 1 1 Research Group in Intelligent Systems Enginyeria i Arquitectura La Salle, Ramon Llull University 2 Illinois Genetic Algorithms Laboratory Department of Industrial and Enterprise Systems Engineering University of Illinois at Urbana Champaign 3 Dipartamento di Elettronica e Informazione Politecnico di Milano
    2. 2. Motivation <ul><li>Facetwise modeling to permit a successful understanding of complex systems (Goldberg, 2002) </li></ul><ul><ul><li>Model of generalization pressures of XCS (Butz, Kovacs, Lanzi, Wilson,04) </li></ul></ul><ul><ul><li>Learning time bound (Butz, Goldberg & Lanzi, 04) </li></ul></ul><ul><ul><li>Population size bound to guarantee niche support (Butz, Goldberg, Lanzi & Sastry, 07) </li></ul></ul>Enginyeria i Arquitectura la Salle
    3. 3. Motivation <ul><li>Analysis of selection schemes in XCS: Proportionate vs. Tournament </li></ul><ul><ul><li>Tournament selection is more robust to parameter settings and noise than proportionate selection (B utz, Goldberg & Tharakunnel, 03) and (Butz, Sastry, Goldberg, 05) </li></ul></ul><ul><ul><li>Proportionate selection is, at least, as robust as tournament selection if the appropriate fitness separation is used (Karbat, Bull & Odeh, 05) </li></ul></ul><ul><li>In GA, these schemes were studied through the analysis of takeover time (Goldberg & Deb, 90; Goldberg, 02) </li></ul>Enginyeria i Arquitectura la Salle
    4. 4. Aim <ul><li>Model selection pressure in XCS through the analysis of the takeover time </li></ul><ul><ul><li>Consider that XCS has converged to an optimal solution </li></ul></ul><ul><ul><li>Write differential equations that describe the change in proportion of the best individual </li></ul></ul><ul><ul><li>Solve the equations and derive a closed form solution </li></ul></ul><ul><ul><li>Validate the model empirically </li></ul></ul>Enginyeria i Arquitectura la Salle
    5. 5. Outline <ul><li>Description of XCS </li></ul><ul><li>Modeling takeover time </li></ul><ul><li>Comparing the two models </li></ul><ul><li>Experimental validation </li></ul><ul><li>Modeling generality </li></ul><ul><li>Conclusions </li></ul>Enginyeria i Arquitectura la Salle
    6. 6. Description of XCS <ul><li>Representation: </li></ul><ul><ul><li>fixed-size rule-based representation </li></ul></ul><ul><ul><li>Rule Parameters: P k , ε k , F k , n k </li></ul></ul><ul><li>Learning interaction: </li></ul><ul><ul><li>At each learning iteration  Sample a new example </li></ul></ul><ul><ul><li>Create the match set [M] </li></ul></ul><ul><ul><li>Each classifier in [M] votes in the prediction array </li></ul></ul><ul><ul><li>Select randomly an action and create the action set [A] </li></ul></ul>Enginyeria i Arquitectura la Salle 1. Description of XCS 2. Modeling Takeover Time 3. Comparing the two Models 4. Experimental Validation 5. Modeling Generality 6. Conclusions
    7. 7. Description of XCS <ul><li>Rule evaluation: reinforcement learning techniques. </li></ul><ul><ul><li>Prediction: </li></ul></ul><ul><ul><li>Prediction error: </li></ul></ul><ul><ul><li>Accuracy of the prediction: </li></ul></ul>Enginyeria i Arquitectura la Salle 1. Description of XCS 2. Modeling Takeover Time 3. Comparing the two Models 4. Experimental Validation 5. Modeling Generality 6. Conclusions
    8. 8. Description of XCS <ul><li>Rule evaluation: reinforcement learning techniques. </li></ul><ul><ul><li>Relative accuracy: </li></ul></ul><ul><ul><li>Fitness computed as a windowed average of the accuracy: </li></ul></ul>Enginyeria i Arquitectura la Salle 1. Description of XCS 2. Modeling Takeover Time 3. Comparing the two Models 4. Experimental Validation 5. Modeling Generality 6. Conclusions
    9. 9. Description of XCS <ul><li>Rule discovering: </li></ul><ul><ul><li>Steady-state, niched GA </li></ul></ul><ul><ul><li>Population-wide deletion </li></ul></ul><ul><ul><li>Proportionate selection </li></ul></ul><ul><ul><ul><li>Probability proportionate to rule’s fitness. </li></ul></ul></ul><ul><ul><li>Tournament selection </li></ul></ul><ul><ul><ul><li>Selects τ percent of classifiers from [A] </li></ul></ul></ul><ul><ul><ul><li>Selects the classifier with higher microclassifier fitness </li></ul></ul></ul>Enginyeria i Arquitectura la Salle 1. Description of XCS 2. Modeling Takeover Time 3. Comparing the two Models 4. Experimental Validation 5. Modeling Generality 6. Conclusions
    10. 10. Outline <ul><li>Description of XCS </li></ul><ul><li>Modeling takeover time </li></ul><ul><li>Comparing the two models </li></ul><ul><li>Experimental validation </li></ul><ul><li>Modeling generality </li></ul><ul><li>Conclusions </li></ul>Enginyeria i Arquitectura la Salle
    11. 11. Modeling Takeover Time <ul><li>In GA: </li></ul><ul><ul><li>Usually, the fitness of an individual is constant </li></ul></ul><ul><ul><li>Selection and replacement are performed over the whole population </li></ul></ul><ul><li>In XCS: </li></ul><ul><ul><li>Fitness depends on the other rule’s fitness in the same niche </li></ul></ul><ul><ul><li>Selection is niched-based, whilst deletion is population-wide </li></ul></ul>Enginyeria i Arquitectura la Salle 1. Description of XCS 2. Modeling Takeover Time 3. Comparing the two Models 4. Experimental Validation 5. Modeling Generality 6. Conclusions
    12. 12. Modeling Takeover Time <ul><li>Assumptions in our model </li></ul><ul><ul><li>XCS has evolved a set of non-overlapping niches </li></ul></ul><ul><ul><li>Simplified scenario: niche with two classifiers cl 1 and cl 2 . </li></ul></ul><ul><ul><li>cl 1 is the best rule in the niche: k 1 > k 2 </li></ul></ul><ul><ul><li>Classifier cl k has: </li></ul></ul><ul><ul><ul><li>prediction error ε k </li></ul></ul></ul><ul><ul><ul><li>fitness F k </li></ul></ul></ul><ul><ul><ul><li>numerosity n k </li></ul></ul></ul><ul><ul><ul><li>microclassifier fitness f k </li></ul></ul></ul><ul><ul><li>cl 1 and cl 2 are equally general  Same reproduction opportunities </li></ul></ul><ul><ul><li>We assume niched deletion </li></ul></ul>Enginyeria i Arquitectura la Salle 1. Description of XCS 2. Modeling Takeover Time 3. Comparing the two Models 4. Experimental Validation 5. Modeling Generality 6. Conclusions
    13. 13. Proportionate Selection <ul><li>Fitness is an average of cl 1 and cl 2 respective accuracies </li></ul><ul><li>Then, the probability of selecting the best classifier cl 1 is: </li></ul><ul><li>Probability of deletion: P del (cl j ) = n j /n where n=n 1 +n 2 </li></ul>Enginyeria i Arquitectura la Salle 1. Description of XCS 2. Modeling Takeover Time 3. Comparing the two Models 4. Experimental Validation 5. Modeling Generality 6. Conclusions Num. ratio : n r = n 2 /n 1 Accuracy ratio: ρ = k 2 /k 1
    14. 14. Proportionate Selection <ul><li>Evolution of cl 1 numerosity </li></ul>Enginyeria i Arquitectura la Salle <ul><li>The numerosity of cl1 increases if cl 1 is selected by the GA and another classifier is selected to be deleted </li></ul><ul><li>The numerosity of cl1 decreases if cl 1 is not selected by the GA but it is selected by the deletion operator </li></ul><ul><li>The numerosity of c1 remain de same otherwise. </li></ul>1. Description of XCS 2. Modeling Takeover Time 3. Comparing the two Models 4. Experimental Validation 5. Modeling Generality 6. Conclusions
    15. 15. Proportionate Selection <ul><li>Grouping the above equations we obtain </li></ul><ul><li>Rewritten in terms of proportion of classifiers cl 1 in the niche: </li></ul><ul><li>Considering P t+1 – P t ≈ dp/dt </li></ul>Enginyeria i Arquitectura la Salle 1. Description of XCS 2. Modeling Takeover Time 3. Comparing the two Models 4. Experimental Validation 5. Modeling Generality 6. Conclusions P t = n 1 /n <ul><li>Integrate: </li></ul><ul><li>Initial proportion: P0 </li></ul><ul><li>Final proportion: P </li></ul>
    16. 16. Proportionate Selection <ul><li>This gives us that the takeover time for proportionate selection is guided by the following expression: </li></ul>Enginyeria i Arquitectura la Salle <ul><li>P 0 : initial proportion of cl 1 in the niche </li></ul><ul><li>P: final proportion of cl 1 in the niche </li></ul><ul><li>ρ : accuracy ratio between cl 2 and cl 1 </li></ul><ul><li>n: niche size </li></ul>A higher separation between fitness enables a higher ability in identifying accurate rules, as announced by Karbat, Bull & Odeh, 2005. n 1. Description of XCS 2. Modeling Takeover Time 3. Comparing the two Models 4. Experimental Validation 5. Modeling Generality 6. Conclusions If ρ  1: t rws ≈ ∞ If ρ  0:
    17. 17. Tournament Selection <ul><li>Assumptions </li></ul><ul><ul><li>Fixed tournament size s </li></ul></ul><ul><ul><li>cl 1 is the best classifier in the niche: f 1 > f 2 , that is, F 1 /n 1 > F 2 /n 2 </li></ul></ul>Enginyeria i Arquitectura la Salle 1. Description of XCS 2. Modeling Takeover Time 3. Comparing the two Models 4. Experimental Validation 5. Modeling Generality 6. Conclusions
    18. 18. Tournament Selection <ul><li>Evolution of cl 1 numerosity </li></ul>Enginyeria i Arquitectura la Salle <ul><li>The numerosity of cl1 increases if cl 1 participates in the tournament and another classifier is selected to be deleted </li></ul><ul><li>The numerosity of cl1 decreases if cl 1 does not participate in the tournament but it is selected by the deletion operator </li></ul><ul><li>The numerosity of c1 remain de same otherwise. </li></ul>1. Description of XCS 2. Modeling Takeover Time 3. Comparing the two Models 4. Experimental Validation 5. Modeling Generality 6. Conclusions
    19. 19. Tournament Selection <ul><li>This gives us that the takeover time for tournament selection is guided by the following expression: </li></ul>Enginyeria i Arquitectura la Salle <ul><li>P 0 : initial proportion of cl 1 in the niche </li></ul><ul><li>P: final proportion of cl 1 in the niche </li></ul><ul><li>s : tournament size </li></ul><ul><li>n: niche size </li></ul>Tournament selection does not depend on the accuracy ratio between the best classifier and the others in the same [A], as pointed by Butz, Sastry & Goldberg, 2005 1. Description of XCS 2. Modeling Takeover Time 3. Comparing the two Models 4. Experimental Validation 5. Modeling Generality 6. Conclusions <ul><li>As s increases, this expression decreases </li></ul><ul><li>It does not depend on the individual fitness </li></ul><ul><li>For large s: </li></ul>
    20. 20. Outline <ul><li>Description of XCS </li></ul><ul><li>Modeling takeover time </li></ul><ul><li>Comparing the two models </li></ul><ul><li>Experimental validation </li></ul><ul><li>Modeling generality </li></ul><ul><li>Conclusions </li></ul>Enginyeria i Arquitectura la Salle
    21. 21. Proportionate vs. Tournament <ul><li>Values of s and ρ for which both schemes result in the same takeover time. Require: t* RWS = t* TS </li></ul><ul><ul><li>We obtain </li></ul></ul>Enginyeria i Arquitectura la Salle For P 0 = 0.01 and P = 0.99 1. Description of XCS 2. Modeling Takeover Time 3. Comparing the two Models 4. Experimental Validation 5. Modeling Generality 6. Conclusions
    22. 22. Outline <ul><li>Description of XCS </li></ul><ul><li>Modeling takeover time </li></ul><ul><li>Comparing the two models </li></ul><ul><li>Experimental validation </li></ul><ul><li>Modeling generality </li></ul><ul><li>Conclusions </li></ul>Enginyeria i Arquitectura la Salle
    23. 23. Design of Test Problems <ul><li>Single-niche problem </li></ul><ul><ul><li>One niche with 2 classifiers: </li></ul></ul><ul><ul><ul><li>Highly accurate classifier cl1: </li></ul></ul></ul><ul><ul><ul><li>Less accurate classifier cl2: </li></ul></ul></ul><ul><ul><ul><li>Varying ρ we are changing the fitness separation between cl 1 and cl 2 </li></ul></ul></ul><ul><ul><li>Population initialized with </li></ul></ul><ul><ul><ul><li>N · P 0 copies of cl1 </li></ul></ul></ul><ul><ul><ul><li>N · (1 – P 0 ) copies of cl2 </li></ul></ul></ul>Enginyeria i Arquitectura la Salle 1. Description of XCS 2. Modeling Takeover Time 3. Comparing the two Models 4. Experimental Validation 5. Modeling Generality 6. Conclusions
    24. 24. Results on the Single-Niche Problem Enginyeria i Arquitectura la Salle 1. Description of XCS 2. Modeling Takeover Time 3. Comparing the two Models 4. Experimental Validation 5. Modeling Generality 6. Conclusions Accuracy ratio: ρ = 0.01 RWS Tournament s=9 Tournament s=3 Tournament s=2
    25. 25. Results on the Single-Niche Problem Enginyeria i Arquitectura la Salle 1. Description of XCS 2. Modeling Takeover Time 3. Comparing the two Models 4. Experimental Validation 5. Modeling Generality 6. Conclusions Accuracy ratio: ρ = 0.50 RWS Tournament s=9 Tournament s=3 Tournament s=2
    26. 26. Results on the Single-Niche Problem Enginyeria i Arquitectura la Salle 1. Description of XCS 2. Modeling Takeover Time 3. Comparing the two Models 4. Experimental Validation 5. Modeling Generality 6. Conclusions Accuracy ratio: ρ = 0.90 RWS Tournament s=9 Tournament s=3 Tournament s=2
    27. 27. Design of Test Problems <ul><li>Multiple-niche problem </li></ul><ul><ul><li>Several niches with 1 maximally accurate classifier each niche. </li></ul></ul><ul><ul><li>One over-general classifier that participates in all niches </li></ul></ul><ul><ul><li>The population contains: </li></ul></ul><ul><ul><ul><li>N · P 0 copies of maximally accurate classifiers </li></ul></ul></ul><ul><ul><ul><li>N · (1 – P 0 ) copies of the overgeneral classifier </li></ul></ul></ul><ul><ul><li>The problem violates two assumptions of the model </li></ul></ul><ul><ul><ul><li>Overlapping niches </li></ul></ul></ul><ul><ul><ul><li>The size of the different niches differ from the population size </li></ul></ul></ul><ul><ul><ul><ul><li>Deletion can select any classifier in [P] </li></ul></ul></ul></ul>Enginyeria i Arquitectura la Salle 1. Description of XCS 2. Modeling Takeover Time 3. Comparing the two Models 4. Experimental Validation 5. Modeling Generality 6. Conclusions
    28. 28. Results on the Multiple-Niche Problem Enginyeria i Arquitectura la Salle <ul><li>For small ρ the theory slightly underestimates the empirical takeover time </li></ul><ul><li>The model of proportionate selection is accurate in general scenarios if the ratio of accuracies is small </li></ul><ul><li>In situations where there is a small proportion of the best classifier in one niche competing with other slightly inaccurate and overgeneral, the overgeneral may take over the population. </li></ul><ul><li>Further experiments, show that for ρ > 0.5, the best classifiers is removed from the population </li></ul>1. Description of XCS 2. Modeling Takeover Time 3. Comparing the two Models 4. Experimental Validation 5. Modeling Generality 6. Conclusions RWS ρ = 0.01 RWS ρ = 0.20 RWS ρ = 0.30 RWS ρ = 0.40 RWS ρ = 0.50
    29. 29. Results on the Multiple-Niche Problem Enginyeria i Arquitectura la Salle <ul><li>For high s the theory slightly underestimates the empirical takeover time </li></ul><ul><li>The model of tournament selection is accurate in general scenarios if the tournament size is high enough </li></ul><ul><li>Only in the extreme case (s=2), the experiments strongly disagree with the theory </li></ul>1. Description of XCS 2. Modeling Takeover Time 3. Comparing the two Models 4. Experimental Validation 5. Modeling Generality 6. Conclusions TS s = 2 TS s = 3 TS s = 9
    30. 30. Outline <ul><li>Description of XCS </li></ul><ul><li>Modeling takeover time </li></ul><ul><li>Comparing the two models </li></ul><ul><li>Experimental validation </li></ul><ul><li>Modeling generality </li></ul><ul><li>Conclusions </li></ul>Enginyeria i Arquitectura la Salle
    31. 31. Modeling Generality <ul><li>Scenario </li></ul><ul><ul><li>The best classifier cl 1 appears in the niche with probability 1 </li></ul></ul><ul><ul><li>cl 2 appears in the niche with probability ρ m </li></ul></ul>Enginyeria i Arquitectura la Salle 1. Description of XCS 2. Modeling Takeover Time 3. Comparing the two Models 4. Experimental Validation 5. Modeling Generality 6. Conclusions
    32. 32. Proportionate Selection <ul><li>The takeover time for proportionate selection is guided by the following expression: </li></ul>Enginyeria i Arquitectura la Salle <ul><li>P 0 : initial proportion of cl 1 in the niche </li></ul><ul><li>P: final proportion of cl 1 in the niche </li></ul><ul><li>ρ : accuracy ratio between cl 2 and cl 1 </li></ul><ul><li>ρm: occurrence probability of cl 2 </li></ul><ul><li>N: niche size </li></ul>If cl 1 is either more accurate or more general than cl 2 , cl 1 will take over the population. 1. Description of XCS 2. Modeling Takeover Time 3. Comparing the two Models 4. Experimental Validation 5. Modeling Generality 6. Conclusions
    33. 33. Tournament Selection <ul><li>The takeover time for tournament selection is guided by the following expression: </li></ul>Enginyeria i Arquitectura la Salle <ul><li>P 0 : initial proportion of cl 1 in the niche </li></ul><ul><li>P: final proportion of cl 1 in the niche </li></ul><ul><li>s : tournament size </li></ul><ul><li>ρm: occurrence probability of cl2 </li></ul><ul><li>n: niche size </li></ul>For low ρ m or high s the right-hand logarithm goes to zero, so that the takeover time mainly depends on P 0 and P 1. Description of XCS 2. Modeling Takeover Time 3. Comparing the two Models 4. Experimental Validation 5. Modeling Generality 6. Conclusions
    34. 34. Results of the Extended Model on the one-niched Problem Enginyeria i Arquitectura la Salle 1. Description of XCS 2. Modeling Takeover Time 3. Comparing the two Models 4. Experimental Validation 5. Modeling Generality 6. Conclusions RWS Tournament
    35. 35. Outline <ul><li>Description of XCS </li></ul><ul><li>Modeling takeover time </li></ul><ul><li>Comparing the two models </li></ul><ul><li>Experimental validation </li></ul><ul><li>Modeling generality </li></ul><ul><li>Conclusions </li></ul>Enginyeria i Arquitectura la Salle
    36. 36. Conclusions <ul><li>We derived theoretical models for proportionate and tournament under some assumptions </li></ul><ul><ul><li>Models are exact in very simple scenarios </li></ul></ul><ul><ul><li>Models can qualitatively explain both selection schemes in more complicated scenarios </li></ul></ul><ul><li>Models support that tournament is more robust (Butz, Sastry & Goldberg, 2005) </li></ul><ul><li>Fitness separation is essential to guarantee that the best classifier will take over the population in proportionate selection (Karbhat, Bull & Oates, 2005) </li></ul><ul><li>Models show that proportionate selection depends on fitness scaling. It may fail in domains where there are slightly inaccurate classifiers (real-world domains) </li></ul>Enginyeria i Arquitectura la Salle 1. Description of XCS 2. Modeling Takeover Time 3. Comparing the two Models 4. Experimental Validation 5. Modeling Generality 6. Conclusions
    37. 37. Modeling Selection Pressure in XCS for Proportionate and Tournament Selection Albert Orriols-Puig 1,2 Kumara Sastry 2 Pier Luca Lanzi 2 David E. Goldberg 2 Ester Bernadó-Mansilla 1 1 Research Group in Intelligent Systems Enginyeria i Arquitectura La Salle, Ramon Llull University 2 Illinois Genetic Algorithms Laboratory Department of Industrial and Enterprise Systems Engineering University of Illinois at Urbana Champaign

    ×