• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Modeling selection pressure in XCS for proportionate and tournament selection
 

Modeling selection pressure in XCS for proportionate and tournament selection

on

  • 2,883 views

In this paper, we derive models of the selection pressure in XCS for proportionate (roulette wheel) selection and tournament selection. We show that these models can explain the empirical results that ...

In this paper, we derive models of the selection pressure in XCS for proportionate (roulette wheel) selection and tournament selection. We show that these models can explain the empirical results that have been previously presented in the literature. We validate the models on simple problems showing that, (i) when the model assumptions hold, the theory perfectly matches the empirical evidence; (ii) when the model assumptions do not hold, the theory can still provide qualitative explanations of the experimental results.

Statistics

Views

Total Views
2,883
Views on SlideShare
2,875
Embed Views
8

Actions

Likes
0
Downloads
108
Comments
0

3 Embeds 8

http://www.kumarasastry.com 3
http://www.slideshare.net 3
http://www.illigal.uiuc.edu 2

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

CC Attribution-NonCommercial-NoDerivs LicenseCC Attribution-NonCommercial-NoDerivs LicenseCC Attribution-NonCommercial-NoDerivs License

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Modeling selection pressure in XCS for proportionate and tournament selection Modeling selection pressure in XCS for proportionate and tournament selection Presentation Transcript

    • Modeling Selection Pressure in XCS for Proportionate and Tournament Selection Albert Orriols-Puig1,2 Kumara Sastry2 Pier Luca Lanzi1,3 David E. Goldberg2 Ester Bernadó-Mansilla1 1Research Group in Intelligent Systems Enginyeria i Arquitectura La Salle, Ramon Llull University 2Illinois Genetic Algorithms Laboratory Department of Industrial and Enterprise Systems Engineering University of Illinois at Urbana Champaign 3Dipartamento di Elettronica e Informazione Politecnico di Milano
    • Motivation Facetwise modeling to permit a successful understanding of complex systems (Goldberg, 2002) Analysis of selection schemes in XCS: Proportionate vs. Tournament – Tournament selection is more robust to parameter settings and noise than proportionate selection (Butz, Sastry & Goldberg, 03, 05) – Proportionate selection is, at least, as robust as tournament selection if the appropriate fitness separation is used (Karbat, Bull & Odeh, 05) In GA, these schemes were studied through the analysis of takeover time (Goldberg & Deb, 90; Goldberg, 02) Aim: Model selection pressure in XCS through the analysis of the takeover time Illinois Genetic Algorithms Laboratory and Group of Research in Intelligent Systems Slide 2 GECCO’07
    • Outline 1. Description of XCS 2. Modeling takeover time 3. Comparing the two models 4. Experimental validation 5. Modeling generality 6. Conclusions Illinois Genetic Algorithms Laboratory and Group of Research in Intelligent Systems Slide 3 GECCO’07
    • 1. Description of XCS 2. Modeling Takeover Time Description of XCS 3. Comparing the two Models 4. Experimental Validation 5. Modeling Generality 6. Conclusions Representation: – fixed-size rule-based representation – Rule Parameters: Pk, εk, Fk, nk Learning interaction: – At each learning iteration Sample a new example – Create the match set [M] – Each classifier in [M] votes in the prediction array – Select randomly an action and create the action set [A] Illinois Genetic Algorithms Laboratory and Group of Research in Intelligent Systems Slide 4 GECCO’07
    • 1. Description of XCS 2. Modeling Takeover Time Description of XCS 3. Comparing the two Models 4. Experimental Validation 5. Modeling Generality 6. Conclusions Rule evaluation: reinforcement learning techniques. – Prediction: – Prediction error: – Accuracy of the prediction: Illinois Genetic Algorithms Laboratory and Group of Research in Intelligent Systems Slide 5 GECCO’07
    • 1. Description of XCS 2. Modeling Takeover Time Description of XCS 3. Comparing the two Models 4. Experimental Validation 5. Modeling Generality 6. Conclusions Rule evaluation: reinforcement learning techniques. – Relative accuracy: – Fitness computed as a windowed average of the accuracy: Illinois Genetic Algorithms Laboratory and Group of Research in Intelligent Systems Slide 6 GECCO’07
    • 1. Description of XCS 2. Modeling Takeover Time Description of XCS 3. Comparing the two Models 4. Experimental Validation 5. Modeling Generality 6. Conclusions Rule discovering: – Steady-state, niched GA – Population-wide deletion Proportionate selection • Probability proportionate to rule’s fitness. Tournament selection • Selects τ percent of classifiers from [A] • Selects the classifier with higher microclassifier fitness Illinois Genetic Algorithms Laboratory and Group of Research in Intelligent Systems Slide 7 GECCO’07
    • Outline 1. Description of XCS 2. Modeling takeover time 3. Comparing the two models 4. Experimental validation 5. Modeling generality 6. Conclusions Illinois Genetic Algorithms Laboratory and Group of Research in Intelligent Systems Slide 8 GECCO’07
    • 1. Description of XCS 2. Modeling Takeover Time Modeling Takeover Time 3. Comparing the two Models 4. Experimental Validation 5. Modeling Generality 6. Conclusions In GA: – Usually, the fitness of an individual is constant – Selection and replacement are performed over the whole population In XCS: – Fitness depends on the other rule’s fitness in the same niche – Selection is niched-based, whilst deletion is population-wide Illinois Genetic Algorithms Laboratory and Group of Research in Intelligent Systems Slide 9 GECCO’07
    • 1. Description of XCS 2. Modeling Takeover Time Modeling Takeover Time 3. Comparing the two Models 4. Experimental Validation 5. Modeling Generality 6. Conclusions Assumptions in our model – XCS has evolved a set of non-overlapping niches – Simplified scenario: niche with two classifiers cl1 and cl2. – cl1 is the best rule in the niche: k1 > k2 – Classifier clk has: • prediction error εk • fitness Fk • numerosity nk • microclassifier fitness fk – cl1 and cl2 are equally general Same reproduction opportunities – We assume niched deletion Illinois Genetic Algorithms Laboratory and Group of Research in Intelligent Systems Slide 10 GECCO’07
    • 1. Description of XCS 2. Modeling Takeover Time Proportionate Selection 3. Comparing the two Models 4. Experimental Validation 5. Modeling Generality 6. Conclusions Fitness is an average of cl1 and cl2 respective accuracies Num. ratio: nr = n2/n1 Accuracy ratio: ρ=k2/k1 Then, the probability of selecting the best classifier cl1 is: Probability of deletion: Pdel(clj) = nj/n where n=n1+n2 Illinois Genetic Algorithms Laboratory and Group of Research in Intelligent Systems Slide 11 GECCO’07
    • 1. Description of XCS 2. Modeling Takeover Time Proportionate Selection 3. Comparing the two Models 4. Experimental Validation 5. Modeling Generality 6. Conclusions Evolution of cl1 numerosity 1. The numerosity of cl1 increases if cl1 is selected by the GA and another classifier is selected to be deleted 2. The numerosity of cl1 decreases if cl1 is not selected by the GA but it is selected by the deletion operator 3. The numerosity of c1 remain de same otherwise. Illinois Genetic Algorithms Laboratory and Group of Research in Intelligent Systems Slide 12 GECCO’07
    • 1. Description of XCS 2. Modeling Takeover Time Proportionate Selection 3. Comparing the two Models 4. Experimental Validation 5. Modeling Generality 6. Conclusions Grouping the above equations we obtain Rewritten in terms of proportion of classifiers cl1 in the niche: Pt = n1/n Considering Pt+1 – Pt ≈ dp/dt Integrate: Initial proportion: P0 Final proportion: P Illinois Genetic Algorithms Laboratory and Group of Research in Intelligent Systems Slide 13 GECCO’07
    • 1. Description of XCS 2. Modeling Takeover Time Proportionate Selection 3. Comparing the two Models 4. Experimental Validation 5. Modeling Generality 6. Conclusions This gives us that the takeover time for proportionate selection is guided by the following expression: n P0: initial proportion of cl1 in the niche If ρ 1: trws ≈ ∞ P: final proportion of cl1 in the niche ρ: accuracy ratio between cl2 and cl1 If ρ 0: n: niche size A higher separation between fitness enables a higher ability in identifying accurate rules, as announced by Karbat, Bull & Odeh, 2005. Illinois Genetic Algorithms Laboratory and Group of Research in Intelligent Systems Slide 14 GECCO’07
    • 1. Description of XCS 2. Modeling Takeover Time Tournament Selection 3. Comparing the two Models 4. Experimental Validation 5. Modeling Generality 6. Conclusions Assumptions – Fixed tournament size s – cl1 is the best classifier in the niche: f1 > f2 , that is, F1/n1 > F2 /n2 Illinois Genetic Algorithms Laboratory and Group of Research in Intelligent Systems Slide 15 GECCO’07
    • 1. Description of XCS 2. Modeling Takeover Time Tournament Selection 3. Comparing the two Models 4. Experimental Validation 5. Modeling Generality 6. Conclusions This gives us that the takeover time for tournament selection is guided by the following expression: P0: initial proportion of cl1 in the niche P: final proportion of cl1 in the niche As s increases, this expression decreases It does not depend on the individual fitness s: tournament size For large s: n: niche size Tournament selection does not depend on the accuracy ratio between the best classifier and the others in the same [A], as pointed by Butz, Sastry & Goldberg, 2005 Illinois Genetic Algorithms Laboratory and Group of Research in Intelligent Systems Slide 16 GECCO’07
    • Outline 1. Description of XCS 2. Modeling takeover time 3. Comparing the two models 4. Experimental validation 5. Modeling generality 6. Conclusions Illinois Genetic Algorithms Laboratory and Group of Research in Intelligent Systems Slide 17 GECCO’07
    • 1. Description of XCS 2. Modeling Takeover Time Proportionate vs. Tournament 3. Comparing the two Models 4. Experimental Validation 5. Modeling Generality 6. Conclusions Values of s and ρ for which both schemes result in the same takeover time. Require: t*RWS = t*TS – We obtain For P0 = 0.01 and P = 0.99 Illinois Genetic Algorithms Laboratory and Group of Research in Intelligent Systems Slide 18 GECCO’07
    • Outline 1. Description of XCS 2. Modeling takeover time 3. Comparing the two models 4. Experimental validation 5. Modeling generality 6. Conclusions Illinois Genetic Algorithms Laboratory and Group of Research in Intelligent Systems Slide 19 GECCO’07
    • 1. Description of XCS 2. Modeling Takeover Time Design of Test Problems 3. Comparing the two Models 4. Experimental Validation 5. Modeling Generality 6. Conclusions Single-niche problem – One niche with 2 classifiers: • Highly accurate classifier cl1: • Less accurate classifier cl2: • Varying ρ we are changing the fitness separation between cl1 and cl2 – Population initialized with • N · P0 copies of cl1 • N · (1 – P0) copies of cl2 Illinois Genetic Algorithms Laboratory and Group of Research in Intelligent Systems Slide 20 GECCO’07
    • 1. Description of XCS Results on the Single-Niche 2. Modeling Takeover Time 3. Comparing the two Models Problem 4. Experimental Validation 5. Modeling Generality 6. Conclusions Accuracy ratio: ρ= 0.01 RWS Tournament s=9 Tournament s=3 Tournament s=2 Illinois Genetic Algorithms Laboratory and Group of Research in Intelligent Systems Slide 21 GECCO’07
    • 1. Description of XCS Results on the Single-Niche 2. Modeling Takeover Time 3. Comparing the two Models Problem 4. Experimental Validation 5. Modeling Generality 6. Conclusions Accuracy ratio: ρ= 0.50 Tournament s=9 Tournament s=3 Tournament s=2 RWS Illinois Genetic Algorithms Laboratory and Group of Research in Intelligent Systems Slide 22 GECCO’07
    • 1. Description of XCS Results on the Single-Niche 2. Modeling Takeover Time 3. Comparing the two Models Problem 4. Experimental Validation 5. Modeling Generality 6. Conclusions Accuracy ratio: ρ= 0.90 Tournament s=9 Tournament s=3 RWS Tournament s=2 Illinois Genetic Algorithms Laboratory and Group of Research in Intelligent Systems Slide 23 GECCO’07
    • 1. Description of XCS 2. Modeling Takeover Time Design of Test Problems 3. Comparing the two Models 4. Experimental Validation 5. Modeling Generality 6. Conclusions Multiple-niche problem – Several niches with 1 maximally accurate classifier each niche. – One over-general classifier that participates in all niches – The population contains: • N · P0 copies of maximally accurate classifiers • N · (1 – P0) copies of the overgeneral classifier – The problem violates two assumptions of the model • Overlapping niches • The size of the different niches differ from the population size – Deletion can select any classifier in [P] Illinois Genetic Algorithms Laboratory and Group of Research in Intelligent Systems Slide 24 GECCO’07
    • 1. Description of XCS Results on the Multiple-Niche 2. Modeling Takeover Time 3. Comparing the two Models Problem 4. Experimental Validation 5. Modeling Generality 6. Conclusions RWS ρ = 0.30 RWS ρ = 0.40 RWS ρ = 0.01 RWS ρ = 0.20 RWS ρ = 0.50 For small ρ the theory slightly underestimates the empirical takeover time The model of proportionate selection is accurate in general scenarios if the ratio of accuracies is small In situations where there is a small proportion of the best classifier in one niche competing with other slightly inaccurate and overgeneral, the overgeneral may take over the population. Further experiments, show that for ρ >= 0.5, the best classifiers is removed from the population Illinois Genetic Algorithms Laboratory and Group of Research in Intelligent Systems Slide 25 GECCO’07
    • 1. Description of XCS Results on the Multiple-Niche 2. Modeling Takeover Time 3. Comparing the two Models Problem 4. Experimental Validation 5. Modeling Generality 6. Conclusions TS s = 9 TS s = 3 TS s = 2 For high s the theory slightly underestimates the empirical takeover time The model of tournament selection is accurate in general scenarios if the tournament size is high enough Only in the extreme case (s=2), the experiments strongly disagree with the theory Illinois Genetic Algorithms Laboratory and Group of Research in Intelligent Systems Slide 26 GECCO’07
    • Outline 1. Description of XCS 2. Modeling takeover time 3. Comparing the two models 4. Experimental validation 5. Modeling generality 6. Conclusions Illinois Genetic Algorithms Laboratory and Group of Research in Intelligent Systems Slide 27 GECCO’07
    • 1. Description of XCS 2. Modeling Takeover Time Modeling Generality 3. Comparing the two Models 4. Experimental Validation 5. Modeling Generality 6. Conclusions Scenario – The best classifier cl1 appears in the niche with probability 1 – cl2 appears in the niche with probability ρm Illinois Genetic Algorithms Laboratory and Group of Research in Intelligent Systems Slide 28 GECCO’07
    • 1. Description of XCS 2. Modeling Takeover Time Proportionate Selection 3. Comparing the two Models 4. Experimental Validation 5. Modeling Generality 6. Conclusions The takeover time for proportionate selection is guided by the following expression: P0: initial proportion of cl1 in the niche P: final proportion of cl1 in the niche ρ: accuracy ratio between cl2 and cl1 ρm: occurrence probability of cl2 N: niche size If cl1 is either more accurate or more general than cl2, cl1 will take over the population. Illinois Genetic Algorithms Laboratory and Group of Research in Intelligent Systems Slide 29 GECCO’07
    • 1. Description of XCS 2. Modeling Takeover Time Tournament Selection 3. Comparing the two Models 4. Experimental Validation 5. Modeling Generality 6. Conclusions The takeover time for tournament selection is guided by the following expression: P0: initial proportion of cl1 in the niche P: final proportion of cl1 in the niche s: tournament size ρm: occurrence probability of cl2 n: niche size For low ρm or high s the right-hand logarithm goes to zero, so that the takeover time mainly depends on P0 and P Illinois Genetic Algorithms Laboratory and Group of Research in Intelligent Systems Slide 30 GECCO’07
    • 1. Description of XCS Results of the Extended Model on 2. Modeling Takeover Time 3. Comparing the two Models the one-niched Problem 4. Experimental Validation 5. Modeling Generality 6. Conclusions Tournament RWS Illinois Genetic Algorithms Laboratory and Group of Research in Intelligent Systems Slide 31 GECCO’07
    • Outline 1. Description of XCS 2. Modeling takeover time 3. Comparing the two models 4. Experimental validation 5. Modeling generality 6. Conclusions Illinois Genetic Algorithms Laboratory and Group of Research in Intelligent Systems Slide 32 GECCO’07
    • 1. Description of XCS 2. Modeling Takeover Time Conclusions 3. Comparing the two Models 4. Experimental Validation 5. Modeling Generality 6. Conclusions We derived theoretical models for proportionate and tournament under some assumptions – Models are exact in very simple scenarios – Models can qualitatively explain both selection schemes in more complicated scenarios Models support that tournament is more robust (Butz, Sastry & Goldberg, 2005) Fitness separation is essential to guarantee that the best classifier will take over the population in proportionate selection (Karbhat, Bull & Oates, 2005) – It may fail in domains where there are slightly inaccurate classifiers (real-world domains). – Eg: Real-world problems where many slightly inaccurate and more general classifiers may take over accurate classifiers. Illinois Genetic Algorithms Laboratory and Group of Research in Intelligent Systems Slide 33 GECCO’07
    • Modeling Selection Pressure in XCS for Proportionate and Tournament Selection Albert Orriols-Puig1,2 Kumara Sastry2 Pier Luca Lanzi2 David E. Goldberg2 Ester Bernadó-Mansilla1 1Research Group in Intelligent Systems Enginyeria i Arquitectura La Salle, Ramon Llull University 2Illinois Genetic Algorithms Laboratory Department of Industrial and Enterprise Systems Engineering University of Illinois at Urbana Champaign
    • 1. Description of XCS 2. Modeling Takeover Time Tournament Selection 3. Comparing the two Models 4. Experimental Validation 5. Modeling Generality 6. Conclusions Evolution of cl1 numerosity Probability that the cl1 participates in the tournament Probability that the cl1 participates in the tournament 1. The numerosity of cl1 increases if cl1 participates in the tournament and another classifier is selected to be deleted 2. The numerosity of cl1 decreases if cl1 does not participate in the tournament but it is selected by the deletion operator 3. The numerosity of c1 remain de same otherwise. Illinois Genetic Algorithms Laboratory and Group of Research in Intelligent Systems Slide 35 GECCO’07
    • 1. Description of XCS Results on the Single-Niche 2. Modeling Takeover Time 3. Comparing the two Models Problem 4. Experimental Validation 5. Modeling Generality 6. Conclusions Empirical results perfectly match theory For ρ=0.01 proportionate selection produces a faster increase Roulette wheel selection is negatively influenced by the increase of ρ Coherently to what was empirically shown in (Butz & Sastry, 2003), tournament selection is more robust to variations on the fitness scaling. Illinois Genetic Algorithms Laboratory and Group of Research in Intelligent Systems Slide 36 GECCO’07
    • Description of XCS Representation: – fixed-size rule-based representation – Michigan-style LCS: solution represented by the entire rule-set – Map of condition-action-predictions. Rule evaluation: reinforcement learning techniques. p k = p k + β(R − p k ) – Prediction: ε k = ε k + β(| R − p k | - ε k ) – Prediction error: – Accuracy of the prediction: Illinois Genetic Algorithms Laboratory and Group of Research in Intelligent Systems Slide 37 GECCO’07
    • 1. Description of XCS 2. Modeling Takeover Time Proportionate vs. Tournament 3. Comparing the two Models 4. Experimental Validation 5. Modeling Generality 6. Conclusions Compare the values of values of s and ρ for which both schemes have the same initial increase of the proportion of the best classifier for which both schemes have the same takeover time Illinois Genetic Algorithms Laboratory and Group of Research in Intelligent Systems Slide 38 GECCO’07
    • 1. Description of XCS 2. Modeling Takeover Time Tournament Selection 3. Comparing the two Models 4. Experimental Validation 5. Modeling Generality 6. Conclusions Grouping the above equations we obtain Rewritten in terms of proportion of classifiers cl1 in the niche: Considering Pt+1 – Pt ≈ dp/dt Illinois Genetic Algorithms Laboratory and Group of Research in Intelligent Systems Slide 39 GECCO’07
    • 1. Description of XCS 2. Modeling Takeover Time Proportionate vs. Tournament 3. Comparing the two Models 4. Experimental Validation 5. Modeling Generality 6. Conclusions Values of s and ρ for which both schemes have the same initial increase of the proportion of the best classifier – Requiring that – We obtain The tournament size s has to increase as ρ decreases For P0 = 0.01 and P = 0.99 Tournament selection produces a stronger pressure toward the best classifier in scenarios in which slightly inaccurate but initially hihgly numerous classifiers are competing against highly accurate classifiers Illinois Genetic Algorithms Laboratory and Group of Research in Intelligent Systems Slide 40 GECCO’07
    • 1. Description of XCS 2. Modeling Takeover Time Modeling Generality 3. Comparing the two Models 4. Experimental Validation 5. Modeling Generality 6. Conclusions Scenario – The best classifier cl1 appears in the niche with probability 1 – cl2 appears in the niche with probability ρm Model for proportionate selection 1. The numerosity of cl1 increases if cl1 and cl2 appear in the niche, cl1 is selected by the GA, and cl2 is chosen by deletion, or when cl1 is the only classifier in the niche and another classifier is deleted. 2. The numerosity of cl1 decreases if cl1 and cl2 are in the niche, cl1 is not selected by the GA but is selected by deletion 3. The numerosity of c1 remain de same otherwise. Illinois Genetic Algorithms Laboratory and Group of Research in Intelligent Systems Slide 41 GECCO’07
    • 1. Description of XCS 2. Modeling Takeover Time Proportionate Selection 3. Comparing the two Models 4. Experimental Validation 5. Modeling Generality 6. Conclusions Which gives us that the takeover time for proportionate selection is guided by the following expression: P0: initial proportion of cl1 in the niche P: final proportion of cl1 in the niche ρ: accuracy ratio between cl2 and cl1 ρm: occurrence probability of cl2 N: niche size If cl1 is either more accurate or more general than cl2, cl1 will take over the population. Illinois Genetic Algorithms Laboratory and Group of Research in Intelligent Systems Slide 42 GECCO’07
    • 1. Description of XCS 2. Modeling Takeover Time Tournament Selection 3. Comparing the two Models 4. Experimental Validation 5. Modeling Generality 6. Conclusions Evolution of cl1 numerosity 1. The numerosity of cl1 increases if cl1 and cl2 appear in the niche, cl1 is selected by the GA, and cl2 is chosen by deletion, or when cl1 is the only classifier in the niche and another classifier is deleted. 2. The numerosity of cl1 decreases if cl1 and cl2 are in the niche, cl1 is not selected by the GA but is selected by deletion 3. The numerosity of cl1 remain de same otherwise. Illinois Genetic Algorithms Laboratory and Group of Research in Intelligent Systems Slide 43 GECCO’07
    • 1. Description of XCS 2. Modeling Takeover Time Tournament Selection 3. Comparing the two Models 4. Experimental Validation 5. Modeling Generality 6. Conclusions Which gives us that the takeover time for tournament selection is guided by the following expression: P0: initial proportion of cl1 in the niche P: final proportion of cl1 in the niche s: tournament size ρm: occurrence probability of cl2 n: niche size For low ρm or high s the right-hand logarithm goes to zero, so that the takeover time mainly depends on P0 and P Illinois Genetic Algorithms Laboratory and Group of Research in Intelligent Systems Slide 44 GECCO’07
    • Aim Model selection pressure in XCS through the analysis of the takeover time – Consider that XCS has converged to an optimal solution – Write differential equations that describe the change in proportion of the best individual – Solve the equations and derive a closed form solution – Validate the model empirically Illinois Genetic Algorithms Laboratory and Group of Research in Intelligent Systems Slide 45 GECCO’07