• Like
  • Save

Loading…

Flash Player 9 (or above) is needed to view presentations.
We have detected that you do not have it on your computer. To install it, go here.

A Practical Schema Theorem for Genetic Algorithm Design and Tuning

  • 2,583 views
Uploaded on

This paper develops the theory that can enable the design of genetic algorithms and choose the parameters such that the proportion of the best building blocks grow. A practical schema theorem has been …

This paper develops the theory that can enable the design of genetic algorithms and choose the parameters such that the proportion of the best building blocks grow. A practical schema theorem has been used for this purpose and its ramification for the choice of selection operator and parameterization of the algorithm is explored. In particular stochastic universal selection, tournament selection, and truncation selection schemes are employed to verify the results. Results agree with the schema theorem and indicate that it must be obeyed in order to ascertain sustained growth of good building blocks. The analysis suggests that schema theorem alone is insufficient to guarantee the success of a selectorecombinative genetic algorithm.

More in: Education , Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
  • Using imagery in this presentation is very effective. You've done a fantastic job here friend.
    Sharika
    http://winkhealth.com http://financewink.com
    Are you sure you want to
    Your message goes here
No Downloads

Views

Total Views
2,583
On Slideshare
0
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
0
Comments
1
Likes
2

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. A Practical Schema Theorem for Genetic Algorithm Design and Tuning David E. Goldberg Kumara Sastry Illinois Genetic Algorithms Laboratory University of Illinois at Urbana-Champaign Urbana, IL 61801 http://www-illigal.ge.uiuc.edu Genetic and Evolutionary Computation Conference (GECCO-2001) July 7-11, 2001 San Francisco, CA
  • 2. 1 Schema Theorem for GA Design & Tuning Foreword • Schema theorem has received wide range of reaction • ...from criticism to overemphasis • But schema theorem is useful, and “easy” to satisfy Illinois Genetic Algorithms Laboratory Department of General Engineering GECCO, July 7-11, 2001 University of Illinois at Urbana-Champaign Urbana, IL 61801. USA. D.E. Goldberg, K. Sastry http://www-illigal.ge.uiuc.edu
  • 3. 2 Schema Theorem for GA Design & Tuning Overview • Background & Motivation • Simplified practical schema theorem • Designing for BB growth • Selection schemes & BB growth – Tournament selection – Truncation selection – Proportionate selection • Summary Illinois Genetic Algorithms Laboratory Department of General Engineering GECCO, July 7-11, 2001 University of Illinois at Urbana-Champaign Urbana, IL 61801. USA. D.E. Goldberg, K. Sastry http://www-illigal.ge.uiuc.edu
  • 4. 3 Schema Theorem for GA Design & Tuning GA Design Decomposition • Goldberg, Deb, & Clark, 1992 1. Know what GAs are processing—building blocks (BBs) 2. Know thy BB challengers—BB-wise difficult problems. 3. Ensure an adequate supply of raw BBs 4. Ensure increased market share for superior BBs 5. Know BB takeover and convergence time 6. Make decisions well among competing BBs 7. Mix BBs well Illinois Genetic Algorithms Laboratory Department of General Engineering GECCO, July 7-11, 2001 University of Illinois at Urbana-Champaign Urbana, IL 61801. USA. D.E. Goldberg, K. Sastry http://www-illigal.ge.uiuc.edu
  • 5. 4 Schema Theorem for GA Design & Tuning Motivation • Ensuring superior BB growth – Important, and necessary for GA success • Usual approach: use schema theorem • Which selection operator should I use ? • What parameter values should I use? • Decisions are usually made ad hoc Illinois Genetic Algorithms Laboratory Department of General Engineering GECCO, July 7-11, 2001 University of Illinois at Urbana-Champaign Urbana, IL 61801. USA. D.E. Goldberg, K. Sastry http://www-illigal.ge.uiuc.edu
  • 6. 5 Schema Theorem for GA Design & Tuning Objective • Design for superior BB growth • Utilize a simplified schema theorem • Study its ramifications on – choice of selection operator, and – parameter tuning • Verify with empirical results Illinois Genetic Algorithms Laboratory Department of General Engineering GECCO, July 7-11, 2001 University of Illinois at Urbana-Champaign Urbana, IL 61801. USA. D.E. Goldberg, K. Sastry http://www-illigal.ge.uiuc.edu
  • 7. 6 Schema Theorem for GA Design & Tuning Simplified Practical Schema Theorem • Schema Theorem (Holland, 1975; De Jong, 1975) – Proportionate selection & one-point crossover f (H, t) δ(H) m(H, t + 1) ≥ m(H, t) 1 − pc −1 f (t) • Simplified practical schema theorem – Goldberg & Deb, 1991 m(H, t + 1) ≥ m(H, t)sp (1 − pc ) Illinois Genetic Algorithms Laboratory Department of General Engineering GECCO, July 7-11, 2001 University of Illinois at Urbana-Champaign Urbana, IL 61801. USA. D.E. Goldberg, K. Sastry http://www-illigal.ge.uiuc.edu
  • 8. 7 Schema Theorem for GA Design & Tuning Designing for Building Block Growth 1 − s−1 m(H, t + 1) p ≥ 1 ⇒ pc ≤ m(H, t) 1 ε = 0.25 ε = 0.5 0.9 ε = 0.75 ε = 1.0 0.8 0.7 Crossover probability, pc 0.6 0.5 0.4 0.3 0.2 0.1 0 0 1 2 10 10 10 Selection pressure, s p Illinois Genetic Algorithms Laboratory Department of General Engineering GECCO, July 7-11, 2001 University of Illinois at Urbana-Champaign Urbana, IL 61801. USA. D.E. Goldberg, K. Sastry http://www-illigal.ge.uiuc.edu
  • 9. 8 Schema Theorem for GA Design & Tuning BB Growth Design Model 1 − s−1 p pc ≤ →∞ • Large selection pressure—s – BB growth ensured even if crossover is fully disruptive (pc = 1) →0 • Small crossover probability—pc – BB growth is ensured for any selection pressure – Easiest way to obey schema theorem – Does not guarantee mixing Illinois Genetic Algorithms Laboratory Department of General Engineering GECCO, July 7-11, 2001 University of Illinois at Urbana-Champaign Urbana, IL 61801. USA. D.E. Goldberg, K. Sastry http://www-illigal.ge.uiuc.edu
  • 10. 9 Schema Theorem for GA Design & Tuning Selection Schemes & Selection Pressure • Initial growth of BBs is important • Later in the GA run – Aggressive growth wears down – Loss of selection pressure – Schema loss due to BB disruption – Stall can occur – Premature convergence • Consider both early and late phases Illinois Genetic Algorithms Laboratory Department of General Engineering GECCO, July 7-11, 2001 University of Illinois at Urbana-Champaign Urbana, IL 61801. USA. D.E. Goldberg, K. Sastry http://www-illigal.ge.uiuc.edu
  • 11. 10 Schema Theorem for GA Design & Tuning Tournament Selection: Early Phase • Goldberg & Deb, 1991 s pt+1 = 1 − (1 − pt ) • Early in the GA run – pt ≈ 0 – sp = s – Good schemata grow exponentially 1 − s−1 pc ≤ Illinois Genetic Algorithms Laboratory Department of General Engineering GECCO, July 7-11, 2001 University of Illinois at Urbana-Champaign Urbana, IL 61801. USA. D.E. Goldberg, K. Sastry http://www-illigal.ge.uiuc.edu
  • 12. 11 Schema Theorem for GA Design & Tuning Tournament Selection: Late Phase pt > 1/s • • Account for self crosses (Shaffer, 1987) • Effect loss due to crossover reduces sp = p−1 [1 − pc (1 − pt )] • t sp ≥ 1 if • 1 − s−1 pc ≤ Illinois Genetic Algorithms Laboratory Department of General Engineering GECCO, July 7-11, 2001 University of Illinois at Urbana-Champaign Urbana, IL 61801. USA. D.E. Goldberg, K. Sastry http://www-illigal.ge.uiuc.edu
  • 13. 12 Schema Theorem for GA Design & Tuning Truncation Selection • Growth is geometric pt+1 = spt • Early in the GA run – sp = s, Same as tournament selection • Late in the GA run – sp = p−1 [1 − pc (1 − pt )] t – Same as tournament selection 1 − s−1 pc ≤ Illinois Genetic Algorithms Laboratory Department of General Engineering GECCO, July 7-11, 2001 University of Illinois at Urbana-Champaign Urbana, IL 61801. USA. D.E. Goldberg, K. Sastry http://www-illigal.ge.uiuc.edu
  • 14. 13 Schema Theorem for GA Design & Tuning Results: Test Function 8-bit Trap 8 7 6 fitness Value 5 4 3 2 1 0 1 2 3456 7 8 No. of ones • 8-bit deceptive trap function • Single building block • Global optimum: 00000000 • Local optimum: 11111111 Illinois Genetic Algorithms Laboratory Department of General Engineering GECCO, July 7-11, 2001 University of Illinois at Urbana-Champaign Urbana, IL 61801. USA. D.E. Goldberg, K. Sastry http://www-illigal.ge.uiuc.edu
  • 15. 14 Schema Theorem for GA Design & Tuning Results: Run Conditions • One-point crossover • Schema disruption rate is known • Crossover probability determined by bisection method 10−5 • Tolerance: • Population size: 5000 • Average of 25 runs Illinois Genetic Algorithms Laboratory Department of General Engineering GECCO, July 7-11, 2001 University of Illinois at Urbana-Champaign Urbana, IL 61801. USA. D.E. Goldberg, K. Sastry http://www-illigal.ge.uiuc.edu
  • 16. 15 Schema Theorem for GA Design & Tuning Results: Tournament & Truncation 1 1 Crossover probability, pc c Crossover probability, p 0.8 0.8 0.6 0.6 ε = 0.95 ε = 1.0 0.4 0.4 Truncation Truncation Tournament WOR Tournament WOR 0.2 0.2 Tournament WR Tournament WR 0 0 0 1 2 0 1 10 10 10 10 10 Selection pressure, s Selection pressure, s p p 1 1 Crossover probability, pc c Crossover probability, p 0.8 0.8 0.6 0.6 ε = 0.9 ε = 0.85 0.4 0.4 Truncation Truncation Tournament WOR Tournament WOR 0.2 0.2 Tournament WR Tournament WR 0 0 0 1 0 1 10 10 10 10 Selection pressure, sp Selection pressure, sp Illinois Genetic Algorithms Laboratory Department of General Engineering GECCO, July 7-11, 2001 University of Illinois at Urbana-Champaign Urbana, IL 61801. USA. D.E. Goldberg, K. Sastry http://www-illigal.ge.uiuc.edu
  • 17. 16 Schema Theorem for GA Design & Tuning Proportionate Selection: Early Phase • Consider only two alternatives – Average (f1 ) & Best (f2 ) individual s pt , s = f2 /f1 pt+1 = (s − 1)pt + 1 • Early in the GA run – sp = s, Similar to tournament & truncation – s varies for different problems – Difficult to determine s before the GA run 1 − s−1 pc ≤ Illinois Genetic Algorithms Laboratory Department of General Engineering GECCO, July 7-11, 2001 University of Illinois at Urbana-Champaign Urbana, IL 61801. USA. D.E. Goldberg, K. Sastry http://www-illigal.ge.uiuc.edu
  • 18. 17 Schema Theorem for GA Design & Tuning Proportionate Selection: Late Phase • Account for self crosses s [1 − pc (1 − pt )] sp = (s − 1)pt + 1 1−s−1 sp ≥ 1 if pc ≤ • s = f2 /f1 is not a constant • f1 increases during the run • s decreases during the run • • Best BB’s domination can stall Illinois Genetic Algorithms Laboratory Department of General Engineering GECCO, July 7-11, 2001 University of Illinois at Urbana-Champaign Urbana, IL 61801. USA. D.E. Goldberg, K. Sastry http://www-illigal.ge.uiuc.edu
  • 19. 18 Schema Theorem for GA Design & Tuning Results: Stall Demonstration 1 0.7 p = 0.225 (b) (a) c 0.6 p = 0.2 0.8 c Proportion of BBs Proportion of BBs 0.5 0.6 0.4 0.3 0.4 0.2 all−zeros 0.2 all−zeros 0.1 all−ones all−ones 0 0 0 50 100 150 200 0 50 100 150 200 No. of generations No. of generations 0.7 1 (c) (d) pc = 0.23 pc = 0.26 0.6 0.8 Proportion of BBs Proportion of BBs 0.5 0.6 0.4 0.3 0.4 0.2 0.2 all−zeros all−zeros 0.1 all−ones all−ones 0 0 0 50 100 150 200 0 50 100 150 200 No. of generations No. of generations Illinois Genetic Algorithms Laboratory Department of General Engineering GECCO, July 7-11, 2001 University of Illinois at Urbana-Champaign Urbana, IL 61801. USA. D.E. Goldberg, K. Sastry http://www-illigal.ge.uiuc.edu
  • 20. 19 Schema Theorem for GA Design & Tuning Results: Proportionate Selection 1 1 (b) (a) Failure Failure Proportion of optimal BB Proportion of optimal BB Success Success 0.8 0.8 0.6 0.6 0.4 0.4 0.2 0.2 pc = 0.215 pc = 0.22 0 0 0 50 100 150 0 50 100 150 No. of generations No. of generations 1 1 (c) (d) Failure Failure Success Proportion of optimal BB Proportion of optimal BB Success 0.8 0.8 0.6 0.6 0.4 0.4 0.2 0.2 pc = 0.225 p = 0.23 c 0 0 0 50 100 150 0 50 100 150 No. of generations No. of generations Illinois Genetic Algorithms Laboratory Department of General Engineering GECCO, July 7-11, 2001 University of Illinois at Urbana-Champaign Urbana, IL 61801. USA. D.E. Goldberg, K. Sastry http://www-illigal.ge.uiuc.edu
  • 21. 20 Schema Theorem for GA Design & Tuning Summary • Schema theorem must be obeyed – Does not guarantee BB mixing • Schema theorem is “easy” to obey • Selection operator & parameter tuning • Use ordinal schemes • Proportionate selection scheme – BB growth can stall – Use only in special cases Illinois Genetic Algorithms Laboratory Department of General Engineering GECCO, July 7-11, 2001 University of Illinois at Urbana-Champaign Urbana, IL 61801. USA. D.E. Goldberg, K. Sastry http://www-illigal.ge.uiuc.edu
  • 22. 21 Schema Theorem for GA Design & Tuning Acknowledgments • Air Force Office of Scientific Research, Air Force Materiel Command, USAF, under grant F49620-00-1-0163. • National Science Foundation under grant DMI-9908252. • U. S. Army Research Laboratory under the Federated Laboratory Program, Cooperative Agreement DAAL01-96-2-0003. Illinois Genetic Algorithms Laboratory Department of General Engineering GECCO, July 7-11, 2001 University of Illinois at Urbana-Champaign Urbana, IL 61801. USA. D.E. Goldberg, K. Sastry http://www-illigal.ge.uiuc.edu