Slideshare uses cookies to improve functionality and performance, and to provide you with relevant advertising. If you continue browsing the site, you agree to the use of cookies on this website. See our User Agreement and Privacy Policy.

Slideshare uses cookies to improve functionality and performance, and to provide you with relevant advertising. If you continue browsing the site, you agree to the use of cookies on this website. See our Privacy Policy and User Agreement for details.

Like this presentation? Why not share!

- Sub-Structural Niching in Estimatio... by kknsastry 1794 views
- Linkage Learning, Overlapping Build... by kknsastry 1689 views
- AGUARELAS DA FIGUEIRA DA FOZ by pedronr3 1569 views
- Combating User Fatigue in iGAs: Par... by kknsastry 1127 views
- FESTIVAL DA LARANJA by pedronr3 612 views
- Accidentes de moto by frani 7881 views

1,348 views

1,213 views

1,213 views

Published on

No Downloads

Total views

1,348

On SlideShare

0

From Embeds

0

Number of Embeds

50

Shares

0

Downloads

0

Comments

0

Likes

1

No embeds

No notes for slide

- 1. Combining Competent Crossover and Mutation Operators: A Probabilistic Model Building Approach Claudio Limal, Kumara Sastryz, David Goldbergz, and Fernando Lobe‘ ‘Department of Electronics and ‘Illinois Genetic Algorithm Laboratory C°mP“l°" Sdence E“3in¢¢"i“8 Department of General Engineering U“iV°’5"Y °f A183?“ University of Illinois at Urbana- Champaign
- 2. Overview Motivation Background ° Crossover VS Mutation ' Extended Compact GA ° BB-wise Mutation Algorithm Probabilistic Model-Building Hybrid GA Experiments Conclusions Extensions
- 3. Motivation 0 EDAs outperform simple GA 0 EDAS solve decomposable problems within a low- order polynomial number of function evaluations 0 However, for large-scale problems, even a low-order polynomial number of FE can be very demanding. ..
- 4. Motivation 0 Efficiency-enhancement techniques ' Hybridization ‘ Time Continuation ° Parallclization ° Evaluation-Relaxation 0 Use Model information to perform local search = > Building-Block-wise Local Search 0 A more general Hybridization
- 5. Background 0 Crossover versus Mutation 0 Extended Compact Genetic Algorithm 0 Building-Block-wise Mutation Algorithm
- 6. Crossover vs Mutation o Assumption of knowledge of the problem decomposition (Sastry and Goldberg, 2004) o For deterministic decomposable problems: ' Mutation outperforms Crossover ' Speedup of O( sqrt(k)*log(m) ) o For decomposable problems with additive Gaussian noise: ' Crossover outperforms Mutation ' Speedup of O( sqrt(k)*m / log(m) )
- 7. Extended Compact GA 0 Estimation of distribution 5 Linkage learning 0 Estimate the distribution of the population using Marginal Product Models (MPMS) ° For a linear problem: lxillxzllxsl [X4] [Xsl[X. «,l lX7llX3l lX9llXwllX11llX12l ‘ For an order-3 additively decomposable problem: lx1x2x3l lx4X5xel lX7x3X9l lx10x1lx12l ° For an order-4 additively decomposable problem: lX1x2"3x-ll lX5xex7"3l lX9x10X11"12l
- 8. Which Model is Better? 0 Metric: Minimum Description Length (MDL) ‘ Preference for accurate and simpler models 0 Search for the model that minimizes the storage needed to represent the population ° Overall Complexity = Model Complexity + Population Compression Complexity 1 Ill 2*’ C', ,, : log.2(n + ’l ) E(‘. .k’ — I) C', . : n —l}i_j l(_)g‘.2(])i_, ‘) : =l _)= l
- 9. Model Building 0 In each generation, after selection, perform a greedy search for the best MPM 0 Start with the simplest model: [x1][x2][x3]. ..[x, _1][xl] 0 Consider all possible merges of two subsets and choose the one that leads to a lower overall complexity 0 Stop when no further improvement is possible
- 10. eCGA steps Extended Compact Genetic Algorithm (eCGA) (1) Create A mudoui population of n lll(ll’l(llIil. l:~’. (2) Evaluate all lll(ll'l(llldlb' in the population. (3) Apply s-wise tournauieiit selc-ction (4) llo(, leI the S(! lL‘('[~L‘(, l ll. l(ll'l(, lllzll: i using it gl'Ut'(, l)' . lPll . s:_. -nrcll proce«lure. (5) Generate a new population according to the . IPIl found in step 4. ((5) If stopping criteria is not. satisfied. return to step 2.
- 11. BB-wise Mutation Algorithm 0 Induces good neighborhoods as linkage groups (Sastry & Goldberg, 2004) 0 Use linkage learning procedures developed for selectorecombinative GAS o Mutation: Bit-wise = > Building-Block-wise 0 Search: Hillclimbing = > Deterministic or Random
- 12. BB Neighborhood Consider a order-3 decomposable problem with4 subfunctions (BB partitions): 331 332 333 334 / 101 011 001 110 Solution A 101 011 010 110 Neighborhood of Solution A within BB partition #3 101 011 011 110 101 011 100 110 101 011 101 110 How can we get the best BB in partition #3? 101 011 110 110 Evaluate all the neighborhood and choose 101 011 111 110 the best individual
- 13. Extended Compact Mutation Algorithm mien e Cmnpacl. 'utati01i gorit mi e " A (1) Create it rzunloiii population of n iu<livi<lIml. ~ and eval- uate their ﬁtness. ('2) Apply . -c-vvisu IU| ll‘Il? |Il| (.‘l| l. §| .'l(‘(‘l. lt)ll (3) . Io«lel the select. e«l inclivitlmils using :1 greedy llPll h0tll‘(‘ll proculure. (-I} C'vlIoo. ~‘e the heat. iI| (li'l<lII: l of the poplllution for BB- wise mutation. (5) For earl: (lvt('«‘t(-(l llli pnrtiiiioll: (-5.1) Create '2" — 1 unique lll(llVl(lllill. ' with all pU. ': ~’ll)lC selieuiata in the current BB partition. Note that the rest of the individual remains the 3211110 and equal to the lust solution found so far. (5.2) l‘: ’allllM. l(. ' nll 2* - I ll| (ll’l(lll2|l. 's' and I1-t. Mi11 the lu-st for inutatiou in the other BB pa1‘titions.
- 14. A Probabilistic Model Building Hybrid GA 0 The probabilistic model of eCGA is used for two distinct purposes: l . Effective recombination of BB3 that provide rapid global-search capabilities 2. Effective search in the BB neighborhod that locally provides high-quality solutions 0 The key idea is to obtain the benefits from both approaches
- 15. Joining Both. .. 0 Learn the model o Perform BB-wise local search on the best individual of the population 0 Update the model parameters (frequencies) according with the BB-wise mutated individual ° Increase the BB instances frequencies of the mutated individual by s ° Decrease the BB instances frequencies of the previous best solution by s
- 16. Updating the Model Model found: [x1x2x3][x4x5x6][x7xS][x9] Best individual of the population 000 011 01 0 Mutated version of the best individual 111 111 00 0
- 17. Hybﬁd eCGA Hybrid Eattended Compact. Genetic Algorithm (heCGA) (1) Create a random population of 11 individuals. (2) Evaluate all individuals in the population. (3) Apply s-wise tournament selection (4) . VIodel the selected individuals using a greedy MP. 'I Search procedure. (5) Apply BB-wise mutation to the best individual. (6) Update the frequencies of the MPM found on step 4 according to the BB5 instances present. on the mutated individual: (6.1) Increase the BB instmices frequencies of the mu- tated individual by s. (6.2) Decrease the BB Instances frequencies of the pre- vious best individual by 3. (7) Generate a new population according to the updated . 'IPl»[. (8) If stopping criteria is not satisﬁed, return to step 2.
- 18. Experiments 0 m concatenated deceptive trap functions of order-k 1 if u = I. ‘ ftrup(U) : { l_d_ u * l—d 3 othc-rwisc u - # of 15 in the string k — size of the trap function d — fitness signal For our experiments, k=4 and d= ]/k=0.25
- 19. Experiments 0 Problem 1: Deception III I . fd(»’) = Z frn-p(-1‘a. -s--1‘I. -.'+| - - - ---7'1.-«'+5:—l 1 i=0 0 Problem 2: Deception + Scaling m—1 fa. .(<': | = Z 1?ifr. .., .(J‘i. ..l'kun. -.. .J‘uu. - ll (:0 0 Problem 3: Deception + Noise fdr1(. .Y) = fd(. ') + G((). aj-1.-)
- 20. Uniformly Scaled BBs k-4 5000 v V ' v ' L’ C . ..‘ § C man 3 v’ _ 2 2 § 300 S 5 ' 5 3 .5 8 *5 I00 8 ’ E10 . 5;: 3;; : . : : '. . ': . 5° . J O ocGA. ou'm"’) : ' 1:‘ 1: I O ¢CGAO1m“} I: . V ,5 2 . . , ,5 ; IIoCMAO(m'I7 8 I : ﬂeCIM. Cxm) _ I _ _ hoCGA"O{mx) O M; ‘ H ‘ 0 hoOGA.0[m”) 2 3 4 6 10 15 20 i 3 4 6 to 15 20 Number of building Mocks. m Number of building blocks. m k=4
- 21. Number at iunclnon evalualrons, nk Behaviour of Hybrid eCGA Uniformly Scaled BB5 . More Flexible: can solve the problem within a bigger range of pop size than eCMA k=4.m=10 o 4 lines, 4 different hitting times (# gen. ) 2500 Population 5128. n who was 2500 aocc
- 22. Uniformly Scaled BBs o eCGA needs smaller populations, but takes more FES than eCMA and heCGA l. BBs discovery in a progressive way 2. Mixing time o eCMA scales better than eCGA o heCGA behaves similarly to eCMA
- 23. Population size. n Exponentially Scaled BBs Number 01 function evaluations. rye I. . . _ 1 .1 . . . o . co». ;oqm“; 0 -csA: <:<m“'> . .. .. .. . . . . . . D QCMA; cum”; 9 ocMA'OIm'; |1 1°, ‘ O nece: Loum") ‘O. 9 MCGNOW I 2 3 4 5 1a 15 2o 2 3 4 6 10 15 20 Number of building blocks. m Number or building blocks. in
- 24. Exponentially Scaled BBs o eCMA needs exponentially pop sizes and NFES 0 Hybrid eCGA performs similar to regular eCGA o I-Iybrid eCGA changed his behaviour
- 25. Number of function evaluations, nm Behaviour of Hybrid eCGA P’ in 1-! r N U! N in v 1 Exponentially Scaled BBs I10‘ k=4.m=1O I . . . _l 1000 1500- 2000 2500 3000 3503 4000 450:) 5000 Population size. n 0 Number of PE grows almost linearly with pop size 0 Increasing pop size will not reveal much more BB information
- 26. BBs with additive Gaussian Noise . .. 39- ': ~ eCMA ~ -'°'. ".°‘?9é Population size. n all Number ol lunction evaluations. nu Noise-to-signal ratio aflxiogdi Noise-to-signal ratio. crznllcr/ cl)2 10'
- 27. BBs with additive Gaussian Noise 0 Trasition phase on Hybrid eCGA behaviour 0 For small values of noise, similar case to the deterministic function: ' Hybrid eCGA do similar to eCMA, which is the best e For larger values of noise: ’ Hybrid eCGA do similar to eCGA, which is the best
- 28. Summary and Conclusions 0 Integration of BB-wise mutation on eCGA o Probabilistic Model Building Hybrid GA allows a more general hybridization o In the absence of domain knowledge, the hybrid eCGA is more robust than either single- operator-based approach
- 29. Extensions 0 Other hybridization configurations 0 Combination with other enhancement techniques for EDAS 0 Problems with overlapping BBS 0 Application to real-world problems
- 30. Acknowledgements o GECCO reviewers for helpful comments and suggestions 0 FCT/ MCES under grants POSI / SR1 / 42065 / 2001 and SFRH / BD / 16980 / 2004, AF OSR/ USAF under grants F49620—O0-0163 and F49620-03-1-0129, NSF under grant DMI-9908252, and ITR grants DMR-99- 76550 and DMR-0121695.

No public clipboards found for this slide

×
### Save the most important slides with Clipping

Clipping is a handy way to collect and organize the most important slides from a presentation. You can keep your great finds in clipboards organized around topics.

Be the first to comment