• Like
  • Save

Loading…

Flash Player 9 (or above) is needed to view presentations.
We have detected that you do not have it on your computer. To install it, go here.

Evaluation Relaxation as an Efficiency-Enhancement Technique: Handling Variance and Bias

  • 1,525 views
Uploaded on

This study develops a decision-making strategy for deciding between fitness functions with differing bias values. Simple, yet practical facetwise models are derived to aid the decision-making process. …

This study develops a decision-making strategy for deciding between fitness functions with differing bias values. Simple, yet practical facetwise models are derived to aid the decision-making process. The decision making strategy is designed to provide maximum speed-up and thereby enhance the efficiency of GA search processes. Results indicate that bias can be handled temporally and that significant speed-up values can be obtained.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
1,525
On Slideshare
0
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
0
Comments
0
Likes
0

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Evaluation Relaxation as an Efficiency-Enhancement Technique: Handling Variance and Bias Kumara Sastry, and David E. Goldberg {ksastry,deg}@uiuc.edu Illinois Genetic Algorithms Laboratory University of Illinois at Urbana-Champaign Urbana, IL 61801 http://www-illigal.ge.uiuc.edu July 9−13, 2002. New York, NY
  • 2. Evaluation Relaxation as an EET 1 Background • Competent GAs (Goldberg, 1999) – Solve hard problems quickly, reliably, and accurately – Subquadratic number of function evaluations • Costly function evaluations – Subquadratic function evaluations can be high – = 100, Time/Eval. = 1 s. Need 2.5 hours • Efficiency-enhancement techniques (EETs) – Evaluation relaxation is one such EET Illinois Genetic Algorithms Laboratory Department of General Engineering K. Sastry, D. E. Goldberg University of Illinois at Urbana−Champaign Urbana, IL 61801. USA. http://www−illigal.ge.uiuc.edu July 9−13, 2002. New York, NY
  • 3. Evaluation Relaxation as an EET 2 Motivation • Evaluation Relaxation – Replace accurate, but costly function evaluation – Use approximate, but cheap function evaluation – Approximation introduces error • Practical guidelines are lacking – Which, When and How? – What’s the speed-up? Illinois Genetic Algorithms Laboratory Department of General Engineering K. Sastry, D. E. Goldberg University of Illinois at Urbana−Champaign Urbana, IL 61801. USA. http://www−illigal.ge.uiuc.edu July 9−13, 2002. New York, NY
  • 4. Evaluation Relaxation as an EET 3 Overview • Evaluation relaxation • Decomposition of evaluation relaxation • Problem statement • Key assumptions • Part I: Handling Variance • Part II: Handling Bias • Conclusions Illinois Genetic Algorithms Laboratory Department of General Engineering K. Sastry, D. E. Goldberg University of Illinois at Urbana−Champaign Urbana, IL 61801. USA. http://www−illigal.ge.uiuc.edu July 9−13, 2002. New York, NY
  • 5. Evaluation Relaxation as an EET 4 Evaluation Relaxation • Use approximate, but low-cost fitness function • Approximation introduces error • Previous work: – Grefenstette & Fitzpatrick (1985) – Aizawa & Wah (1993, 1994) – Miller & Goldberg (1995, 1996, 1997) – Jin, Olhofer, & Sendhoff (2001) – Smith, Dike, & Stegmann (1996) – Sastry, Goldberg, & Pelikan (2001) – Albert & Goldberg (2001, 2002) Illinois Genetic Algorithms Laboratory Department of General Engineering K. Sastry, D. E. Goldberg University of Illinois at Urbana−Champaign Urbana, IL 61801. USA. http://www−illigal.ge.uiuc.edu July 9−13, 2002. New York, NY
  • 6. Evaluation Relaxation as an EET 5 Decomposition of Evaluation Relaxation • Fitness function approximation – Endogenous: Error introduced by the GA ∗ Eg., fitness inheritance – Exogenous: Error in fitness functions ∗ Eg., response surface methodology • Components of error: (Geman et al, 1992) – Variance: Changes the fitness value – Bias: Changes the optimal solution Illinois Genetic Algorithms Laboratory Department of General Engineering K. Sastry, D. E. Goldberg University of Illinois at Urbana−Champaign Urbana, IL 61801. USA. http://www−illigal.ge.uiuc.edu July 9−13, 2002. New York, NY
  • 7. Evaluation Relaxation as an EET 6 Problem Statement • Two fitness functions – High cost, but low error – Low cost, but high error • Which fitness function to use? – Spatial vs. temporal decision making – Strategy that yields maximum speed-up – Utilize facetwise models ∗ Convergence-time and population-sizing models Illinois Genetic Algorithms Laboratory Department of General Engineering K. Sastry, D. E. Goldberg University of Illinois at Urbana−Champaign Urbana, IL 61801. USA. http://www−illigal.ge.uiuc.edu July 9−13, 2002. New York, NY
  • 8. Evaluation Relaxation as an EET 7 Assumptions • Non-overlapping population of fixed size • Generationwise selectorecombinative GAs – Uniform crossover, tournament selection • Binary encoding and fixed string length • Stationary fitness functions • models for OneMax domain • Tight linkage or usage of a competent GA Illinois Genetic Algorithms Laboratory Department of General Engineering K. Sastry, D. E. Goldberg University of Illinois at Urbana−Champaign Urbana, IL 61801. USA. http://www−illigal.ge.uiuc.edu July 9−13, 2002. New York, NY
  • 9. Evaluation Relaxation as an EET 8 Part I: Handling Variance • Noisy OneMax function – OneMax function + Gaussian noise f1 , and f2 • Two fitness functions: 2 – f1 : High cost (c1 ), and low noise variance (σN1 ). 2 – f2 : Low cost c2 , and high noise variance (σN2 ). 2 2 c1 > c2 , σN1 < σN2 – Higher the accuracy, costlier the evaluation. • Averaging eliminates variance effects Illinois Genetic Algorithms Laboratory Department of General Engineering K. Sastry, D. E. Goldberg University of Illinois at Urbana−Champaign Urbana, IL 61801. USA. http://www−illigal.ge.uiuc.edu July 9−13, 2002. New York, NY
  • 10. Evaluation Relaxation as an EET 9 Convergence-Time Model - I • Selection-intensity based model (Bulmer, 1980) – M¨hlenbein & Schlierkamp-Voosen (1993); u Thierens & Goldberg (1993); B¨ck (1994); Miller & a Goldberg (1996) 2 2 • Gaussian fitness distribution: Fi ∼ N µt , σt + σNi • Prop. of correct BBs: (Miller & Goldberg, 1996) I √ pt+1 − pt = pt (1 − pt ) ρe 2 2 – Elongation factor, ρe (t) = 1 + σNi /σt • Exact solution is complex Illinois Genetic Algorithms Laboratory Department of General Engineering K. Sastry, D. E. Goldberg University of Illinois at Urbana−Champaign Urbana, IL 61801. USA. http://www−illigal.ge.uiuc.edu July 9−13, 2002. New York, NY
  • 11. Evaluation Relaxation as an EET 10 Convergence-Time Model - II • Prop. of correct BBs: (Miller & Goldberg, 1996) I √ pt+1 − pt = pt (1 − pt ) ρe ρe (t) = ρe (t = 0) • Constant elongation factor: • Convergence time √ 2 σ Ni π 2 1+ 2 tconv σNi = 2I σ0 • Convergence-time ratio 1 2 2 2 tconv σN1 σ0 + σ N1 2 tc,r = = 2 2 2 σ0 + σ N2 tconv σN2 Illinois Genetic Algorithms Laboratory Department of General Engineering K. Sastry, D. E. Goldberg University of Illinois at Urbana−Champaign Urbana, IL 61801. USA. http://www−illigal.ge.uiuc.edu July 9−13, 2002. New York, NY
  • 12. Evaluation Relaxation as an EET 11 Convergence-Time Model Verification Tournament size, s = 2 Tournament size, s = 3 1 1 c,r c,r Convergence time ratio, t Convergence time ratio, t 0.8 0.8 0.6 0.6 Theory Theory l = 50 l = 50 l = 100 l = 100 0.4 0.4 l = 200 l = 200 l = 300 l = 300 l = 400 l = 400 0.2 0.2 0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1 2 2 2 2 σN /σN σN /σN 1 2 1 2 Tournament size, s = 4 Tournament size, s = 5 1 1 c,r c,r Convergence time ratio, t Convergence time ratio, t 0.8 0.8 0.6 0.6 Theory Theory l = 50 l = 50 l = 100 l = 100 0.4 0.4 l = 200 l = 200 l = 300 l = 300 l = 400 l = 400 0.2 0.2 0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1 2 2 2 2 σN /σN σN /σN 1 2 1 2 Illinois Genetic Algorithms Laboratory Department of General Engineering K. Sastry, D. E. Goldberg University of Illinois at Urbana−Champaign Urbana, IL 61801. USA. http://www−illigal.ge.uiuc.edu July 9−13, 2002. New York, NY
  • 13. Evaluation Relaxation as an EET 12 Population-Sizing Model • Gambler’s ruin population-sizing model – Harik, Cant´-Paz, Goldberg & Miller (1997) u – Building-block supply + Decision-making model • External noise (Miller & Goldberg, 1997) √ π k−1 2 2 2 n σ Ni = − 2 ln(α) σ 0 + σ Ni d • Population-size ratio 1 2 2 2 n σ N1 σ0 + σ N1 2 nr = = 2 2 2 σ0 + σ N2 n σ N2 Illinois Genetic Algorithms Laboratory Department of General Engineering K. Sastry, D. E. Goldberg University of Illinois at Urbana−Champaign Urbana, IL 61801. USA. http://www−illigal.ge.uiuc.edu July 9−13, 2002. New York, NY
  • 14. Evaluation Relaxation as an EET 13 Population-Sizing Model Verification Tournament size, s = 2 Tournament size, s = 3 1 1 Population size ratio, nr Population size ratio, nr 0.8 0.8 0.6 0.6 Theory l = 50 Theory l = 100 l = 50 0.4 0.4 l = 200 l = 100 l = 300 l = 200 l = 400 l = 300 0.2 0.2 0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1 σ2 /σ2 σ2 /σ2 N N N N 1 2 1 2 Tournament size, s = 4 Tournament size, s = 5 1 1 Population size ratio, nr Population size ratio, nr 0.8 0.8 0.6 0.6 Theory Theory l = 50 0.4 0.4 l = 50 l = 100 l = 100 l = 200 l = 200 l = 300 0.2 0.2 0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1 σ2 /σ2 σ2 /σ2 N N N N 1 2 1 2 Illinois Genetic Algorithms Laboratory Department of General Engineering K. Sastry, D. E. Goldberg University of Illinois at Urbana−Champaign Urbana, IL 61801. USA. http://www−illigal.ge.uiuc.edu July 9−13, 2002. New York, NY
  • 15. Evaluation Relaxation as an EET 14 Number of Function Evaluations • Convergence-time, and population-size ratios 1 2 2 σ0 + σ N1 2 tc,r = nr = 2 2 σ0 + σ N2 • No. of function evaluations ratio 2 2 σ 0 + σ N1 nf e,r = nr tc,r = 2 2 σ 0 + σ N2 Illinois Genetic Algorithms Laboratory Department of General Engineering K. Sastry, D. E. Goldberg University of Illinois at Urbana−Champaign Urbana, IL 61801. USA. http://www−illigal.ge.uiuc.edu July 9−13, 2002. New York, NY
  • 16. Evaluation Relaxation as an EET 15 Number of Function Evaluations Tournament size, s = 2 Tournament size, s = 3 Number of function evaluations ratio, nfe,r Number of function evaluations ratio, nfe,r 1 1 0.8 0.8 0.6 0.6 Theory 0.4 0.4 l = 50 Theory l = 100 l = 50 l = 200 l = 100 0.2 0.2 l = 300 l = 200 l = 400 l = 300 0 0 0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1 σ2 /σ2 σ2 /σ2 N N N N 1 2 1 2 Tournament size, s = 4 Tournament size, s = 5 Number of function evaluations ratio, nfe,r Number of function evaluations ratio, nfe,r 1 1 0.8 0.8 0.6 0.6 0.4 0.4 Theory Theory l = 50 l = 50 l = 100 0.2 0.2 l = 100 l = 200 l = 200 l = 300 0 0 0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1 σ2 /σ2 σ2 /σ2 N N N N 1 2 1 2 Illinois Genetic Algorithms Laboratory Department of General Engineering K. Sastry, D. E. Goldberg University of Illinois at Urbana−Champaign Urbana, IL 61801. USA. http://www−illigal.ge.uiuc.edu July 9−13, 2002. New York, NY
  • 17. Evaluation Relaxation as an EET 16 Optimal Decision for Handling Variance • Total computational cost ratio 2 2 σ 0 + σ N1 c1 c1 ctot,r = nf e,r = 2 2 c2 c2 σ 0 + σ N2 2 2 2 2 > (σ0 + σN1 )/(σ0 + σN2 ), then use f1 • If c2 /c1 2 2 2 2 < (σ0 + σN1 )/(σ0 + σN2 ), then use f2 • If c2 /c1 2 2 2 2 = (σ0 + σN1 )/(σ0 + σN2 ), then use f1 or f2 • If c2 /c1 Illinois Genetic Algorithms Laboratory Department of General Engineering K. Sastry, D. E. Goldberg University of Illinois at Urbana−Champaign Urbana, IL 61801. USA. http://www−illigal.ge.uiuc.edu July 9−13, 2002. New York, NY
  • 18. Evaluation Relaxation as an EET 17 Verification of Optimal Decision Tournament size, s = 2 Tournament size, s = 3 1 1 21 Function evaluation cost ratio, c2/c1 Function evaluation cost ratio, c /c Theory Theory l = 50 l = 50 0.8 0.8 l = 100 l = 100 Choose f Choose f 1 1 l = 200 l = 200 l = 300 l = 300 0.6 0.6 l = 400 l = 400 0.4 0.4 Choose f2 Choose f2 0.2 0.2 0 0 0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1 No. of function evaluations ratio, n No. of function evaluations ratio, n fe,r fe,r Tournament size, s = 4 Tournament size, s = 5 1 1 21 Function evaluation cost ratio, c2/c1 Function evaluation cost ratio, c /c Theory Theory l = 50 l = 50 0.8 0.8 l = 100 l = 100 Choose f Choose f 1 1 l = 200 l = 200 l = 300 l = 300 0.6 0.6 0.4 0.4 Choose f Choose f 2 2 0.2 0.2 0 0 0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1 No. of function evaluations ratio, n No. of function evaluations ratio, n fe,r fe,r Illinois Genetic Algorithms Laboratory Department of General Engineering K. Sastry, D. E. Goldberg University of Illinois at Urbana−Champaign Urbana, IL 61801. USA. http://www−illigal.ge.uiuc.edu July 9−13, 2002. New York, NY
  • 19. Evaluation Relaxation as an EET 18 Model to Predict Speed-Up • Optimal vs. Na¨ ıve decision – Use low-variance, high-cost fitness function 2 2  σ0 +σN1 c2  ctot,r < 2 2 c1 σ0 +σN ηs = 2 1 otherwise Illinois Genetic Algorithms Laboratory Department of General Engineering K. Sastry, D. E. Goldberg University of Illinois at Urbana−Champaign Urbana, IL 61801. USA. http://www−illigal.ge.uiuc.edu July 9−13, 2002. New York, NY
  • 20. Evaluation Relaxation as an EET 19 Verification of Speed-Up Tournament size, s = 2 Tournament size, s = 3 8 8 Theory Theory 7 7 l = 50 l = 50 l = 100 l = 100 6 6 l = 200 l = 200 s s Speed−up, η Speed−up, η l = 300 l = 300 5 5 l = 400 l = 400 4 4 3 3 2 2 1 1 0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1 2 2 2 2 σN σN /σN /σN 1 2 1 2 Tournament size, s = 4 Tournament size, s = 5 8 8 Theory Theory 7 7 l = 50 l = 50 l = 100 l = 100 6 6 l = 200 l = 200 s s Speed−up, η Speed−up, η l = 300 l = 300 5 5 4 4 3 3 2 2 1 1 0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1 σ2 /σ2 σ2 /σ2 N N N N 1 2 1 2 Illinois Genetic Algorithms Laboratory Department of General Engineering K. Sastry, D. E. Goldberg University of Illinois at Urbana−Champaign Urbana, IL 61801. USA. http://www−illigal.ge.uiuc.edu July 9−13, 2002. New York, NY
  • 21. Evaluation Relaxation as an EET 20 Aside: Sampling Fitness Functions • Noisy fitness function 2 f = f + N 0, σN • Sampling fitness function ns 1 fns (S) = f (S) ns j=1 • Effect of sampling 2 – Variance: σN /ns – Cost: α + ns β Illinois Genetic Algorithms Laboratory Department of General Engineering K. Sastry, D. E. Goldberg University of Illinois at Urbana−Champaign Urbana, IL 61801. USA. http://www−illigal.ge.uiuc.edu July 9−13, 2002. New York, NY
  • 22. Evaluation Relaxation as an EET 21 Optimal sampling • Total computational cost 2 2 ctot ∝ (α + ns β) σ0 + σN /ns • Optimal sampling rate ∂ctot = 0 ∂ns 1 2 α σN 2 n∗ = s 2 β σ0 • Same result as Miller & Goldberg (1997) – Does not account for convergence-time – Uses BB decision-making model Illinois Genetic Algorithms Laboratory Department of General Engineering K. Sastry, D. E. Goldberg University of Illinois at Urbana−Champaign Urbana, IL 61801. USA. http://www−illigal.ge.uiuc.edu July 9−13, 2002. New York, NY
  • 23. Evaluation Relaxation as an EET 22 Handling Variance: Summary • Decision-making strategy: – Fitness functions with differing variances – Utilized facetwise models – Provides maximum speed-up • Key parameters: – Fitness-variance and fitness-cost ratios • Variance can be handled spatially Illinois Genetic Algorithms Laboratory Department of General Engineering K. Sastry, D. E. Goldberg University of Illinois at Urbana−Champaign Urbana, IL 61801. USA. http://www−illigal.ge.uiuc.edu July 9−13, 2002. New York, NY
  • 24. Evaluation Relaxation as an EET 23 Ingredients of Error • Variance: Changes the fitness value – Sampling eliminates effects of variance • Bias: Changes the optimal solution – Bias cannot be averaged out – Can be handled temporally ∗ Use high-bias function initially ∗ Then switch to low-bias function Illinois Genetic Algorithms Laboratory Department of General Engineering K. Sastry, D. E. Goldberg University of Illinois at Urbana−Champaign Urbana, IL 61801. USA. http://www−illigal.ge.uiuc.edu July 9−13, 2002. New York, NY
  • 25. Evaluation Relaxation as an EET 24 Bias in Fitness Functions 1.2 1.2 Bias Bias 1 1 0.8 0.8 Fitness value Fitness value 0.6 0.6 0.4 0.4 0.2 0.2 Accurate fitness function Accurate fitness function Fitness function with bias Fitness function with bias 0 0 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 Decision variable value Decision variable value (a) Bias in optimal value (b) Bias in optimal solution Illinois Genetic Algorithms Laboratory Department of General Engineering K. Sastry, D. E. Goldberg University of Illinois at Urbana−Champaign Urbana, IL 61801. USA. http://www−illigal.ge.uiuc.edu July 9−13, 2002. New York, NY
  • 26. Evaluation Relaxation as an EET 25 Part II: Handling Bias f1 , and f2 • Two fitness functions: – f1 : High cost (c1 ), and low bias (b1 ) – f2 : Low cost (c2 ), and high bias (b2 ). c1 > c2 , b2 > b1 – Higher the accuracy, costlier the evaluation. f2 initially • Use f1 later • Switch to • Key parameter: Switching time Illinois Genetic Algorithms Laboratory Department of General Engineering K. Sastry, D. E. Goldberg University of Illinois at Urbana−Champaign Urbana, IL 61801. USA. http://www−illigal.ge.uiuc.edu July 9−13, 2002. New York, NY
  • 27. Evaluation Relaxation as an EET 26 Weighted OneMax function • Weights associated with each bit wi ∈ {−1, +1} f (X) = wi xi i=1 • Different bias values = Different weights • OneMax properties – Unimodal, uniformly scaled – Analytical tractability • Bias does not affect population sizing Illinois Genetic Algorithms Laboratory Department of General Engineering K. Sastry, D. E. Goldberg University of Illinois at Urbana−Champaign Urbana, IL 61801. USA. http://www−illigal.ge.uiuc.edu July 9−13, 2002. New York, NY
  • 28. Evaluation Relaxation as an EET 27 GA Run: Initial Stages (t < ts ) f2 • High-bias, low-cost function • Gaussian fitness distribution pt − ( − − b) µf2 ,t = 1 2 pt (1 − pt ) σf2 ,t = • Selection-intensity based model 1 It √ pt = 1 + sin 2 Illinois Genetic Algorithms Laboratory Department of General Engineering K. Sastry, D. E. Goldberg University of Illinois at Urbana−Champaign Urbana, IL 61801. USA. http://www−illigal.ge.uiuc.edu July 9−13, 2002. New York, NY
  • 29. Evaluation Relaxation as an EET 28 GA Run: At Switching Time (t = ts ) • Proportion of correct building blocks 1 Its √ pts = 1 + sin 2 f2 to f1 • Switch from − b BBs: pts – – b BBs: 1 − pts • Corrected proportion of correct BBs b b 1−2 pts = pts + Illinois Genetic Algorithms Laboratory Department of General Engineering K. Sastry, D. E. Goldberg University of Illinois at Urbana−Champaign Urbana, IL 61801. USA. http://www−illigal.ge.uiuc.edu July 9−13, 2002. New York, NY
  • 30. Evaluation Relaxation as an EET 29 GA Run: Later Stages (t > ts ) • Proportion of correct BBs – Unbiased and biased portions are normally distributed – Overall fitness distribution is normal I(t − ts ) 1 + 2 sin−1 √ 1 − cos pt = pts 2 • Convergence time √ π − 2 sin−1 tconv = ts + pts I Illinois Genetic Algorithms Laboratory Department of General Engineering K. Sastry, D. E. Goldberg University of Illinois at Urbana−Champaign Urbana, IL 61801. USA. http://www−illigal.ge.uiuc.edu July 9−13, 2002. New York, NY
  • 31. Evaluation Relaxation as an EET 30 Run-Duration Model Verification - I 1 1 Proportion of correct BBs Proportion of correct BBs 0.9 0.9 0.8 0.8 0.7 0.7 Theory: b = 10 Theory: l = 50 Expt: b = 10 Expt: l = 50 0.6 0.6 Theory: b = 50 Theory: l = 400 Expt: b = 50 Expt: l = 400 0.5 0.5 0 10 20 30 40 50 0 20 40 60 No. of generations, t No. of generations, t 1 1 Proportion of correct BBs Proportion of correct BBs 0.9 0.9 0.8 0.8 0.7 0.7 Theory: ts = 5 0.6 Theory: s = 3 Expt: ts = 5 Expt: s = 3 Theory: ts = 20 0.6 0.5 Theory: s = 5 Expt: ts = 20 Expt: s = 5 0.5 0.4 0 10 20 30 40 50 0 10 20 30 No. of generations, t No. of generations, t Illinois Genetic Algorithms Laboratory Department of General Engineering K. Sastry, D. E. Goldberg University of Illinois at Urbana−Champaign Urbana, IL 61801. USA. http://www−illigal.ge.uiuc.edu July 9−13, 2002. New York, NY
  • 32. Evaluation Relaxation as an EET 31 Run-Duration Model Verification - II Bias, b = 10 Bias, b = 50 40 Expt Expt 55 Expt, 2XO Expt, 2XO Convergence time, tconv conv 38 Theory Theory 50 Convergence time, t 36 45 34 40 32 35 30 30 28 0 5 10 15 20 0 5 10 15 20 Switching time, t Switching time, t s s Problem size, l = 50 Problem size, l = 300 Expt Expt 27 75 Expt, 2XO Expt, 2XO Convergence time, tconv conv 26 Theory Theory 70 Convergence time, t 25 65 24 23 60 22 55 21 50 20 0 5 10 0 10 20 30 40 Switching time, t Switching time, t s s Illinois Genetic Algorithms Laboratory Department of General Engineering K. Sastry, D. E. Goldberg University of Illinois at Urbana−Champaign Urbana, IL 61801. USA. http://www−illigal.ge.uiuc.edu July 9−13, 2002. New York, NY
  • 33. Evaluation Relaxation as an EET 32 When Should We Switch? • Total evaluation cost ctot = n [c2 ts + c1 (tconv − ts )] • Optimal switching time ∂ctot = 0 ∂ts t∗ β(1 − β) 4 ∗ s ≈ 1− τs = tconv,1 π (1 − 2β) c2 − 1 r – Bias proportion β = b/ √ – f1 is used alone: tconv,1 = π /(2I) Illinois Genetic Algorithms Laboratory Department of General Engineering K. Sastry, D. E. Goldberg University of Illinois at Urbana−Champaign Urbana, IL 61801. USA. http://www−illigal.ge.uiuc.edu July 9−13, 2002. New York, NY
  • 34. Evaluation Relaxation as an EET 33 Optimal Switching Time Cost ratio, C = 1.5 Cost ratio, C = 2.5 r r 0.5 0.7 Theory Theory 0.6 Expt: l = 50 Expt: l = 50 0.4 Expt: l = 100 Expt: l = 100 0.5 0.3 ts / tconv,1 conv,1 0.4 t* / t 0.3 0.2 s * 0.2 0.1 0.1 0 0 0 0.1 0.2 0.3 0.4 0.5 0 0.1 0.2 0.3 0.4 0.5 Bias proportion Bias proportion Cost ratio, Cr = 3.5 Cost ratio, Cr = 5.0 0.7 0.7 Theory Theory 0.6 0.6 Expt: l = 50 Expt: l = 50 Expt: l = 100 Expt: l = 100 0.5 0.5 ts / tconv,1 conv,1 0.4 0.4 t* / t 0.3 0.3 s * 0.2 0.2 0.1 0.1 0 0 0 0.1 0.2 0.3 0.4 0.5 0 0.1 0.2 0.3 0.4 0.5 Bias proportion Bias proportion Illinois Genetic Algorithms Laboratory Department of General Engineering K. Sastry, D. E. Goldberg University of Illinois at Urbana−Champaign Urbana, IL 61801. USA. http://www−illigal.ge.uiuc.edu July 9−13, 2002. New York, NY
  • 35. Evaluation Relaxation as an EET 34 Model for Predicting Speed-Up • Optimal vs. Na¨ ıve decision – Use high-cost, low-bias function cr ηs = tconv ∗ − (cr − 1)τs tconv,1 – Cost proportion: cr = c1 /c2 – Valid when cr > (1 − 2β)−1 Illinois Genetic Algorithms Laboratory Department of General Engineering K. Sastry, D. E. Goldberg University of Illinois at Urbana−Champaign Urbana, IL 61801. USA. http://www−illigal.ge.uiuc.edu July 9−13, 2002. New York, NY
  • 36. Evaluation Relaxation as an EET 35 Verification of Speed-Up Cost ratio, c = 1.5 Cost ratio, c = 2.5 r r 1.2 Theory Theory 1.4 Experiment Experiment 1.15 1.3 Speed−up, ηs s Speed−up, η 1.1 1.2 1.05 1.1 1 1 0.95 0 0.1 0.2 0.3 0.4 0.5 0 0.1 0.2 0.3 0.4 0.5 Bias proportion, β Bias proportion, β Cost ratio, c = 3.5 Cost ratio, c = 5.0 r r 1.6 1.7 Theory Theory 1.6 1.5 Experiment Experiment 1.5 1.4 Speed−up, ηs s Speed−up, η 1.4 1.3 1.3 1.2 1.2 1.1 1.1 1 1 0 0.1 0.2 0.3 0.4 0.5 0 0.1 0.2 0.3 0.4 0.5 Bias proportion, β Bias proportion, β Illinois Genetic Algorithms Laboratory Department of General Engineering K. Sastry, D. E. Goldberg University of Illinois at Urbana−Champaign Urbana, IL 61801. USA. http://www−illigal.ge.uiuc.edu July 9−13, 2002. New York, NY
  • 37. Evaluation Relaxation as an EET 36 Handling Bias: Summary • Decision-making strategy: – Fitness functions with differing bias values – Utilized facetwise models – Provides maximum speed-up • Key parameters: – Bias proportion and fitness-cost ratios • Bias can be handled temporally Illinois Genetic Algorithms Laboratory Department of General Engineering K. Sastry, D. E. Goldberg University of Illinois at Urbana−Champaign Urbana, IL 61801. USA. http://www−illigal.ge.uiuc.edu July 9−13, 2002. New York, NY
  • 38. Evaluation Relaxation as an EET 37 Conclusions • Evaluation-relaxation can yield good speed-up • Error has two parts: Variance, and bias • Handle variance spatially – Fitness-variance and fitness-cost ratios • Handle bias temporally – Use high-bias, low-cost function initially – Switch to low-bias, high-cost function later – Bias proportion and fitness-cost ratios Illinois Genetic Algorithms Laboratory Department of General Engineering K. Sastry, D. E. Goldberg University of Illinois at Urbana−Champaign Urbana, IL 61801. USA. http://www−illigal.ge.uiuc.edu July 9−13, 2002. New York, NY
  • 39. Evaluation Relaxation as an EET 38 Acknowledgments • Air Force Office of Scientific Research, Air Force Materiel Command, USAF, F49620-00-1-0163. • National Science Foundation, DMI-9908252. Illinois Genetic Algorithms Laboratory Department of General Engineering K. Sastry, D. E. Goldberg University of Illinois at Urbana−Champaign Urbana, IL 61801. USA. http://www−illigal.ge.uiuc.edu July 9−13, 2002. New York, NY