Analysis of Evolutionary Algorithms on the One-Dimensional Spin Glass with Power-Law Interactions
Upcoming SlideShare
Loading in...5
×
 

Analysis of Evolutionary Algorithms on the One-Dimensional Spin Glass with Power-Law Interactions

on

  • 997 views

 

Statistics

Views

Total Views
997
Views on SlideShare
979
Embed Views
18

Actions

Likes
0
Downloads
4
Comments
0

2 Embeds 18

http://medal.cs.umsl.edu 11
http://medal-lab.org 7

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Analysis of Evolutionary Algorithms on the One-Dimensional Spin Glass with Power-Law Interactions Analysis of Evolutionary Algorithms on the One-Dimensional Spin Glass with Power-Law Interactions Presentation Transcript

  • Analysis of Evolutionary Algorithms on the One-Dimensional Spin Glass with Power-Law Interactions Martin Pelikan and Helmut G. Katzgraber Missouri Estimation of Distribution Algorithms Laboratory (MEDAL) University of Missouri, St. Louis, MO http://medal.cs.umsl.edu/ pelikan@cs.umsl.edu Download MEDAL Report No. 2009004 http://medal.cs.umsl.edu/files/2009004.pdf Martin Pelikan and Helmut G. Katzgraber Analysis of EAs on 1D Spin Glass with Power-Law Interactions
  • Motivation Testing evolutionary algorithms Adversarial problems on the boundary of design envelope. Random instances of important classes of problems. Real-world problems. This study Use one-dimensional spin glass with power-law interactions. This allows the user to tune the effective range of interactions. Short-range to long-range interactions. Generate large number of instances of proposed problem class. Solve all instances with branch and bound and hybrids. Test evolutionary algorithms on the generated instances. Analyze the results. Martin Pelikan and Helmut G. Katzgraber Analysis of EAs on 1D Spin Glass with Power-Law Interactions
  • Outline 1. Sherrington-Kirkpatrick (SK) spin glass. 2. Power-law interactions. 3. Problem instances. 4. Experiments. 5. Conclusions and future work. Martin Pelikan and Helmut G. Katzgraber Analysis of EAs on 1D Spin Glass with Power-Law Interactions
  • SK Spin Glass SK spin glass (Sherrington & Kirkpatrick, 1978) Contains n spins s1 , s2 , . . . , sn . Ising spin can be in two states: +1 or −1. All pairs of spins interact. Interaction of spins si and sj specified by real-valued coupling Ji,j . Spin glass instance is defined by set of couplings {Ji,j }. Spin configuration is defined by the values of spins {si }. Martin Pelikan and Helmut G. Katzgraber Analysis of EAs on 1D Spin Glass with Power-Law Interactions
  • Ground States of SK Spin Glasses Energy Energy of a spin configuration C is given by H(C) = − Ji,j si sj i<j Ground states are spin configurations that minimize energy. Finding ground states of SK instances is NP-complete. Compare with other standard spin glass types 2D: Spin interacts with only 4 neighbors in 2D lattice. 3D: Spin interacts with only 6 neighbors in 3D lattice. SK: Spin interacts with all other spins. 2D is polynomially solvable; 3D and SK are NP-complete. Martin Pelikan and Helmut G. Katzgraber Analysis of EAs on 1D Spin Glass with Power-Law Interactions
  • Random Spin Glass Instances Generating random spin glass instances Generate couplings {Ji,j } using a specific distribution. Study the properties of generated spin glasses. Example study Find ground states and analyze their properties. Example coupling distributions Each coupling is generated from N (0, 1). Each coupling is +1 or -1 with equal probability. Each coupling is generated from a power-law distribution. Martin Pelikan and Helmut G. Katzgraber Analysis of EAs on 1D Spin Glass with Power-Law Interactions
  • Power-Law Interactions Power-law interactions Spins arranged on a circle. Couplings generated according to i,j Ji,j = c(σ) σ , ri,j i,j are generated according to N (0, 1), c(σ) is a normalization constant, σ > 0 is a parameter to control effective range of interactions, ri,j = n sin(π|i − j|/n)/π is geometric Figure 1: One-dimensional spin glass of size n = 10 ar distance between si and sj Magnitude ofwhere ǫi,j are generated decreases with their distance. zero spin-spin couplings according to normal distribution with Effects of distance on magnitude of couplings increase withparameter t is a normalization constant, σ > 0 is the user-specified σ. interactions, and ri,j = n sin(π|i − j|/n)/π denotes the geometric d figure 1). The magnitude of spin-spin couplings decreases with th discussed shortly, the effects EAsdistance on the magnitude of coupli Martin Pelikan and Helmut G. Katzgraber Analysis of of on 1D Spin Glass with Power-Law Interactions
  • Power-Law Interactions: Illustration Example for n = 10 (normalized) Distance on Coupling variance circle σ = 0.0 σ = 0.5 σ = 2.0 1 1.00 1.00 1.00 2 1.00 0.73 0.28 3 1.00 0.62 0.15 4 1.00 0.57 0.11 5 1.00 0.56 0.10 Martin Pelikan and Helmut G. Katzgraber Analysis of EAs on 1D Spin Glass with Power-Law Interactions
  • Problem Instances Parameters n = 20 to 150. σ ∈ {0.00, 0.55, 0.75, 0.83, 1.00, 1.50, 2.00}. σ = 0 denotes standard SK spin glass with N(0,1) couplings. σ = 2 enforces short-range interactions. Variety of instances For each n and σ, generate 10,000 random instances. Overall 610,000 unique problem instances. Finding optima Small instances solved using branch and bound. For large instances, use heuristic methods to find reliable (but not guaranteed) optima. Martin Pelikan and Helmut G. Katzgraber Analysis of EAs on 1D Spin Glass with Power-Law Interactions
  • Compared Algorithms Basic algorithms Hierarchical Bayesian optimization algorithm (hBOA). Genetic algorithm with uniform crossover (GAU). Genetic algorithm with twopoint crossover (G2P). Local search Single-bit-flip hill climbing (DHC) on each solution. Improves performance of all methods. Niching Restricted tournament replacement (niching). Martin Pelikan and Helmut G. Katzgraber Analysis of EAs on 1D Spin Glass with Power-Law Interactions
  • Experimental Setup All algorithms Bisection determines adequate population size for each instance. Ensure 10 successful runs out of 10 independent runs. In RTR, use window size w = min{N/20, n}. GA Probability of crossover, pc = 0.6. Probability of bit-flip in mutation, pm = 1/n. Martin Pelikan and Helmut G. Katzgraber Analysis of EAs on 1D Spin Glass with Power-Law Interactions
  • Results: Evaluations until Optimum Number of evaluations (GA, twopoint) Number of evaluations (GA, twopoint) Number of evaluations (GA, twopoint) Number of evaluations (GA, twopoint) Number of evaluations (GA, twopoint) Number of evaluations (GA, twopoint) 5 5 5 10 10 10 σ=2.00 10 10 10 σ=2.00 10 10 10 σ=2.00 5 5 5 σ=2.00 σ=2.00 σ=2.00 Number of evaluations (hBOA) 5 5 5 σ=2.00 σ=2.00 σ=2.00 Number of evaluations (hBOA) Number of evaluations (hBOA) σ=1.50 σ=1.50σ=1.50 σ=1.50 σ=1.50 σ=1.50 σ=1.50 σ=1.50 σ=1.50 4 σ=1.00 4 4 σ=1.00 σ=1.00 4 σ=1.00 4 4 σ=1.00σ=1.00 4 4 σ=1.00 σ=1.00 4 σ=1.00 10 10 10 10 10 10 10 10 10 σ=0.83 σ=0.83σ=0.83 σ=0.83 σ=0.83 σ=0.83 σ=0.83 σ=0.83 σ=0.83 σ=0.75 σ=0.75σ=0.75 σ=0.75 σ=0.75 σ=0.75 σ=0.75 σ=0.75 σ=0.75 3 3 3 3 3 3 3 3 3 10 10 10 σ=0.55 σ=0.55σ=0.55 10 10 10 σ=0.55 σ=0.55 σ=0.55 10 10 10 σ=0.55 σ=0.55 σ=0.55 σ=0.00 σ=0.00σ=0.00 σ=0.00 σ=0.00 σ=0.00 σ=0.00 σ=0.00 σ=0.00 2 2 2 2 2 2 2 2 2 10 10 10 10 10 10 10 10 10 1 1 1 1 1 1 1 1 1 10 10 10 10 10 10 10 10 10 16 16 16 32 32 32 64 64 64 128 128 128 16 16 16 32 32 32 64 64 64 128 128 128 16 16 16 32 32 32 64 64 64 128128 128 Problem size Problem size size Problem Problem size Problem size size Problem Problem sizesize Problem Problem size (a) hBOA (a)(a) hBOA hBOA (b) GA (twopoint) (b) GA (twopoint) (b) GA (twopoint) (c)(c) GA (uniform) GA (uniform) (c) GA (uniform) Scalability of hBOA and GA with twopoint crossover better forFigure 2:2: 2: Growththe the numberevaluations withwith problem size. short-range interactions. of of evaluations problem size. Figure Growth ofof of number of evaluations with problem size. Figure Growth the number 6 6 σ=2.00 σ=2.00 10 10 10 σ=2.00 Linkage tightens 10 σ=2.00grows. as σ 10 10 σ=2.00 σ=2.00 σ=2.00 σ=2.00 10 10 10 σ=2.00 Number of flips (GA, twopoint) Number of flips (GA, twopoint) Number of flips (GA, twopoint) Number of flips (GA, twopoint) Number of flips (GA, twopoint) Number of flips (GA, twopoint) 6 6 6 6 6 6 6 σ=1.50σ=1.50 σ=1.50 σ=1.50σ=1.50 σ=1.50 σ=1.50 σ=1.50 σ=1.50 Number of flips (hBOA) Number of flips (hBOA) Tighter linkage makes problem easier (if good recombination). Number of flips (hBOA) 5 5 σ=1.00 σ=1.00 5 σ=1.00 σ=1.00σ=1.00 σ=1.00 5 σ=1.00 σ=1.00 σ=1.00 5 5 5 5 5 10 10 10 10 10 10 10 10 10 σ=0.83σ=0.83 σ=0.83 σ=0.83 4 4 4 σ=0.83 σ=0.75σ=0.75 σ=0.75 Twopoint crossoverσ=0.75 respects tight linkage. σ=0.75σ=0.83 σ=0.75 σ=0.83 σ=0.83 σ=0.75 4 σ=0.83 4 σ=0.75 σ=0.75 4 4 4 4 σ=0.55σ=0.55 10 10 10 σ=0.55 σ=0.55 σ=0.55 10 10 10 σ=0.55 σ=0.55 σ=0.55 10 10 10 σ=0.55 3 σ=0.00 10 103 10 3 σ=0.00 σ=0.00 GA with uniform 10 10 10σ=0.00σ=0.00 with shorter-range σ=0.00 gets σ=0.00 worse 3 σ=0.00 10 10 10 3 interactions. σ=0.00 3 3 3 3 2 2 2 2 2 2 2 2 2 10 10 10 10 10 10 10 10 10 16 16 16 32 32 32 64 64 64 128 128 128 16 16 16 32 32 32 64 64 64 128 128 128 16 16 16 32 32 32 64 64 64 128128 128 Problem size size Problem Problem size Problem size size Problem Problem size Problem sizesize Problem size Problem (a) hBOA (a)(a) hBOA hBOA (b) GA (twopoint) (b) GA (twopoint) (b) GA (twopoint) (c)(c) GA (uniform) GA (uniform) (c) GA (uniform) Martin Pelikan and Helmut G. Katzgraber Analysis of EAs on 1D Spin Glass with Power-Law Interactions
  • 10 2 2 2 1010 10 10 1010 2 2 2 Number Numbe 10 10 Number of Number of Number Number of Number of Number of Number of Results: LS Steps until Optimum (Flips) 10 1 10 10 16 1 16 16 1 32 32 32 64 64 64 Problem size 128 128 128 1010 10 1 1 1616 16 1 32 32 32 64 64 64 128 128 Problem size 128 10 1010 16 1616 1 1 1 32 3232 64 6464 128128 128 Problem size size Problem Problem size size Problem Problem sizesize Problem size Problem (a) hBOA (a) (a) hBOA hBOA (b) GA (twopoint) (b) GA GA (twopoint) (b) (twopoint) (c)(c) GA (uniform) GA GA (uniform) (c) (uniform) Figure 2: Growth ofofof the numberevaluations with problem size. Figure 2: 2: Growththe number ofofof evaluations with problem size. Figure Growth the number evaluations with problem size. 6 σ=2.00 σ=2.00 6 6 6 σ=2.00 σ=2.00 6 6 σ=2.00 σ=2.00 Number of flips (GA, twopoint) Number of flips (GA, twopoint) 10 10 σ=2.00 1010 10σ=2.00 10 1010σ=2.00 6 Number of flips (GA, twopoint) Number of flips (GA, twopoint) Number of flips (GA, twopoint) Number of flips (GA, twopoint) 6 6 10 σ=1.50 σ=1.50 σ=1.50 σ=1.50 σ=1.50 σ=1.50 σ=1.50 σ=1.50 σ=1.50 Number of flips (hBOA) Number of flips (hBOA) Number of flips (hBOA) 5 5 5 σ=1.00 σ=1.00 σ=1.00 5 σ=1.00 5 5 σ=1.00σ=1.00 5 5 σ=1.00 σ=1.00 5 σ=1.00 10 10 10 σ=0.83 1010 10 σ=0.83 10 1010 σ=0.83 σ=0.83 σ=0.83 σ=0.83 σ=0.83 σ=0.83 σ=0.83 σ=0.75 σ=0.75 σ=0.75 σ=0.75 σ=0.75 σ=0.75 σ=0.75 σ=0.75 σ=0.75 4 4 4 4 4 4 4 4 4 10 σ=0.55 σ=0.55 10 10 σ=0.55 σ=0.55 1010 10σ=0.55 σ=0.55 σ=0.55 10 1010σ=0.55 σ=0.55 σ=0.00 σ=0.00 σ=0.00 σ=0.00 σ=0.00 σ=0.00 σ=0.00 σ=0.00 σ=0.00 3 3 3 3 3 3 3 3 3 10 10 10 1010 10 10 1010 2 2 2 2 2 2 2 2 2 10 10 10 1010 10 10 1010 16 16 16 32 32 32 64 64 64 128 128 128 1616 16 32 32 32 64 64 64 128 128 128 16 1616 32 3232 64 6464 128128 128 Problem size Problem size size Problem Problem size Problem size size Problem Problem sizesize Problem size Problem (a) (a) hBOA hBOA (a) hBOA (b) GA GA (twopoint) (b) (twopoint) (b) GA (twopoint) (c)(c) GA (uniform) GA GA (uniform) (c) (uniform) Scalability 3:3:Growth ofofof the numberflipsflips with problem size. better Figure of hBOA and GA with twopoint crossover Figure 3: Growththe number ofofof with problem size. Figure Growth the number flips with problem size. for short-range interactions. and how thethe effects σuniform gets worse the algorithm under consideration; this is the topic and how theeffects of ofchange depending on the algorithm under consideration; this is is the topic effects of change depending on the algorithm under consideration; this the topic and howGA with σ σ change depending on with shorter-range interactions. discussed in thethe following few paragraphs. discussed in in following few paragraphs. discussed the following few paragraphs. Based on thethe definitionthe the 1D spin glass with power-law interactions,the the value of σ grows, Based onon definition of of 1D spin glass with power-law interactions, asas the value of grows, Based the definition of the 1D spin glass with power-law interactions, as value of σ σ grows, thethe rangethethe most significant interactions is reduced. With reduction of the range of interactions, therange of of most significant interactions isis reduced. With reduction of the range of interactions, range of the most significant interactions reduced. With reduction of the range of interactions, thethe problem should become easier both for selectorecombinative GAs capablelinkage learning, theproblem should become easier both for selectorecombinative GAs capable of ofof linkage learning, problem should become easier both for selectorecombinative GAs capable linkage learning, such as hBOA, as well as for for selectorecombinative GAs which rarely break interactionsbetween such as as hBOA, as well as selectorecombinative GAs which rarely break interactions between such hBOA, as well as for selectorecombinative GAs which rarely break interactions between closely located bits, such as GAGA with twopoint crossover. This isis clearlydemonstrated by the closely located bits, such asas with twopoint crossover. This is clearly demonstrated by the the closely located bits, such GA with twopoint crossover. This clearly demonstrated by results for for these two algorithms presentedfigures 2 2 2 andAlthough for many problem sizes, the the results forthese two algorithms presented ininin figures and 3.3. Although for many problem sizes, results these two algorithms presented figures and 3. Although for many problem sizes, the Martinabsolute number evaluations and the the number flips are of EAs on smaller Glassfor larger valuesσ, σ, absolute number of G. Katzgraber Pelikan and Helmut of evaluations and number of of flips are factfact smaller larger values of of Analysis in in 1D Spin for with Power-Law Interactions
  • 1 1 1 σ=1.00σ=1.00 σ=1.00 for hBOA (co for hBOA (co for hBOA (co Slowdown factor Slowdown factor Slowdown factor Slowdown factor Slowdown factor Slowdown factor Slowdown factor Slowdown factor Slowdown factor for GA with twopoi for GA with twopoi for GA with twopoi for GA with unifor for GA with unifor for GA with unifor 1 1 1 0.06250.06250.0625 σ=0.83σ=0.83 σ=0.83 σ=0.75σ=0.75 σ=0.75 Comparison: Evaluations until Optimum 0.5 0.5 0.5 0.5 0.5 0.5 0.0312 0.0156 0.0312 0.0156 0.0312 σ=0.55σ=0.55 0.0156 σ=0.00 σ=0.55 σ=0.00 σ=0.00 16 16 32 16 32 64 32 64 128 128 64 128 16 16 32 16 32 64 32 64 128 128 64 128 16 16 3216 32 6432 64 128 128 64 128 Problem sizeProblem size Problem size Problem sizeProblem size Problem size Problem sizeProblem size Problem size (a) hBOA hBOA (a) hBOA (a) (b) GA (twopoint) (b) GA (twopoint) (b) GA (twopoint) (c) GA (uniform) (c) GA (uniform) (c) GA (uniform) Figure 6: Comparison of the the the number for flips 2.00 2.00 2.00 with that 2.00.2.00.2.00. Figure 6: Comparison of number of flipsflips σ = σ = σ = with that σ < σ < σ < Figure 6: Comparison of number of of for for with that for for for Number of evaluations (GA, twopoint) / Number of evaluations (GA, twopoint) / Number of evaluations (GA, twopoint) / Number of evaluations (GA, uniform) / Number of evaluations (GA, uniform) / Number of evaluations (GA, uniform) / Number of evaluations (GA, uniform) / Number of evaluations (GA, uniform) / Number of evaluations (GA, uniform) / Number of evaluations (GA, twopoint) Number of evaluations (GA, twopoint) Number of evaluations (GA, twopoint) 3.5 3.5 3.5 30 30 30 17.5 17.5 17.5 Number of evaluations (hBOA) Number of evaluations (hBOA) Number of evaluations (hBOA) Number of evaluations (hBOA) Number of evaluations (hBOA) Number of evaluations (hBOA) σ=2.00 σ=2.00 σ=2.00 σ=2.00 σ=2.00 σ=2.00 σ=2.00 σ=2.00 σ=2.00 σ=1.503 σ=1.50 σ=1.50 25 σ=1.50 25 σ=1.50 25 σ=1.50 15 15 15 σ=1.50 σ=1.50 σ=1.50 3 3 σ=1.00 σ=1.00 σ=1.00 σ=1.00 σ=1.00 σ=1.00 σ=1.00 σ=1.00 σ=1.00 12.5 12.5 12.5 σ=0.83 σ=0.83 σ=0.83 20 20 20 σ=0.83 σ=0.83 σ=0.83 σ=0.83 σ=0.83 σ=0.83 2.5 2.5 2.5 σ=0.75 σ=0.75 σ=0.75 σ=0.75 σ=0.75 σ=0.75 10 10 10 σ=0.75 σ=0.75 σ=0.75 σ=0.552 σ=0.55 σ=0.55 15 15 15 σ=0.55 σ=0.55 σ=0.55 σ=0.55 σ=0.55 σ=0.55 2 2 7.5 7.5 7.5 σ=0.00 σ=0.00 σ=0.00 σ=0.00 σ=0.00 σ=0.00 σ=0.00 σ=0.00 σ=0.00 10 10 10 1.5 1.5 1.5 5 5 5 5 5 5 2.5 2.5 2.5 1 1 1 0 0 0 0 0 0 0 0 40 0 40 80 40 80120 80120 120 160 160 160 0 0 40 0 40 80 40 8012080120 120 160 160 160 0 0 40 0 40 80 40 80120 80120 120 160 160 160 Problem sizeProblem size Problem size Problem sizeProblem size Problem size Problem sizeProblem size Problem size (a) GA (twopoint) and hBOA hBOA (b) GA (uniform) and hBOA hBOA GA GA GA (uniform) and GA (two- (a) GA (twopoint) and hBOA (a) GA (twopoint) and (b) GA (uniform) and hBOA (b) GA (uniform) and (c) (c) (uniform) and and GA (two- (c) (uniform) GA (two- point) point) point) hBOA outperforms both GA variants. Figure 7: Ratio for the the the number of evaluations pairs of compared algorithms. Figure 7: Ratio for number of evaluations for pairs of pairs of compared algorithms. Figure 7: Ratio for number of evaluations for for compared algorithms. Biggest differences for short-range interactions (expected). N. Friedman Friedman M. Goldszmidt. Learning Bayesian [10] N. Friedman M. Goldszmidt. Learning Bayesian Bayesian [10] N. and and and M. Goldszmidt. Learning networks with with with local structure.Jordan,I. Jordan, editor, networks local local structure. In M. I.M. editor, networks structure. In M. I. In Jordan, editor, GA with uniform crossover performs 2008.2008.2008. view.ofJournal of Conf.Conf.Conf. Series 95:012004, A worst. A one-dimensional view.view. Journal of Physics: Series, A one-dimensional Journal one-dimensional 95:012004, 95:012004, Physics: Physics: Series, Graphical Graphical models, pages MIT MIT MIT Press, Cambridge, Graphical models, pages 421–459. Press, Cambridge, models, pages 421–459. 421–459. Press, Cambridge, [21] H. G. Katzgraber, M. K¨rner, rner, rner, M. Liers, unger,nger, and [21] H. G. Katzgraber, M. K¨ F. Liers, F. J¨nger, and and [21] H. G. Katzgraber, M. K¨ F. Liers, u J¨ J¨ o o o M. M. u MA, MA, MA, 1999. 1999.1999. A. K. Hartmann. Universality-class dependence of energy of energy A. K. Hartmann. Universality-class dependence of energy A. K. Hartmann. Universality-class dependence D. E.[11] D. E. Goldberg.algorithms in search,in search, [11] D. E. Goldberg. Genetic algorithms in search, Goldberg. Genetic Genetic algorithms distributions in spin glasses. Phys. Rev. Rev. Rev. B, 72:094421, 200 distributions in spin glasses. Phys. B, 72:094421, 2005.2005. distributions in spin glasses. Phys. B, 72:094421, optimization, and machine learning. Addison-Wesley, optimization, and machine learning. Addison-Wesley, optimization, and machine learning. Addison-Wesley, [22] H. G. Katzgraber and A. P.A. P.A. P. Young. Monte Carlo studies o [22] H. G. Katzgraber and and Young. Monte Carlo studies of [22] H. G. Katzgraber Young. Monte Carlo studies of Reading, MA, MA, MA, 1989. Reading, 1989.1989. Reading, the one-dimensional Ising Ising Ising glass glass with power-law the one-dimensional spin glass spin with power-law the one-dimensional spin with power-law D. E.[12] D. E. Goldberg,and Deb, J. H.J. H. Clark. Genetic [12] D. E. Goldberg, K. Deb, J. H. and Clark. Genetic Goldberg, K. Deb, K. and Clark. Genetic interactions. Phys. Rev. Rev. Rev. B, 67:134410, 2003. interactions. Phys. B, 67:134410, 2003.2003. interactions. Phys. B, 67:134410, algorithms, noise, andnoise, the sizing of populations. Complex algorithms, noise, the sizing thepopulations. Complex Complex algorithms, and and of sizing of populations. [23] S. Kirkpatrick and D. Sherrington. Infinite-ranged models models [23] S. Kirkpatrick and D. Sherrington. Infinite-ranged models [23] S. Kirkpatrick and D. Sherrington. Infinite-ranged Systems, 6:333–362, 1992.1992.1992. Systems, 6:333–362, Systems, 6:333–362, of spin-glasses. Phys. Rev. Rev. Rev. B, 17(11):4384–4403, 1978.1978. of spin-glasses. Phys. B, 17(11):4384–4403, Jun 1978.Jun of spin-glasses. Phys. B, 17(11):4384–4403, Jun D. E.[13] D. E. Goldberg M. Rudnick. Genetic algorithms and and [13] D. E. Goldberg M. Rudnick. Genetic algorithms and Goldberg and and and M. Rudnick. Genetic algorithms [24] S. Kobe. Ground-state energy and frustration of the the the [24] S. Kobe. Ground-state energy and frustration of of [24] S. Kobe. Ground-state energy and frustration Martin variance and Helmut G. Complex Systems, 5(3):265–278, the Pelikan of fitness. Complex Systems, 5(3):265–278, the variance Complex Katzgraber the variance of fitness. of fitness. Systems, 5(3):265–278, Sherrington-KirkpatrickSpin Glassmodel Power-Law ArXiv Analysis of EAs on 1D model and with related models. ArXiv Sherrington-Kirkpatrick model related models. Interactions Sherrington-Kirkpatrick and and related models. ArXiv
  • Comparison: LS Steps until Optimum 140 140 140 80 80 80 1.8 1.8 1.8 Number of flips (GA, twopoint) / Number of flips (GA, twopoint) / Number of flips (GA, twopoint) / Number of flips (GA, uniform) / Number of flips (GA, uniform) / Number of flips (GA, uniform) / Number of flips (GA, uniform) / Number of flips (GA, uniform) / Number of flips (GA, uniform) / Number of flips (GA, twopoint) Number of flips (GA, twopoint) Number of flips (GA, twopoint) σ=2.00 σ=2.00 σ=2.00 σ=2.00 σ=2.00 σ=2.00 σ=2.00 σ=2.00 σ=2.00 σ=1.50 σ=1.50 σ=1.50 120 120 120 σ=1.50 σ=1.50 σ=1.50 σ=1.50 σ=1.50 σ=1.50 Number of flips (hBOA) Number of flips (hBOA) Number of flips (hBOA) Number of flips (hBOA) Number of flips (hBOA) Number of flips (hBOA) 1.6 1.6 1.6 σ=1.00 σ=1.00 σ=1.00 σ=1.00 σ=1.00 σ=1.00 60 60 60 σ=1.00 σ=1.00 σ=1.00 100 100 100 σ=0.83 σ=0.83 σ=0.83 σ=0.83 σ=0.83 σ=0.83 σ=0.83 σ=0.83 σ=0.83 1.4 1.4 1.4 80 80 80 σ=0.75 σ=0.75 σ=0.75 σ=0.75 σ=0.75 σ=0.75 σ=0.75 σ=0.75 σ=0.75 40 40 40 σ=0.55 σ=0.55 σ=0.55 60 σ=0.55 60 σ=0.55 60 σ=0.55 σ=0.55 σ=0.55 σ=0.55 1.2 1.2 1.2 σ=0.00 σ=0.00 σ=0.00 σ=0.00 σ=0.00 σ=0.00 σ=0.00 σ=0.00 σ=0.00 40 40 40 1 1 1 20 20 20 20 20 20 0.8 0.8 0.8 0 0 0 0 0 0 0 0 40 0 40 80 40 80120 80120 120 160 160 160 0 0 40 0 40 80 40 80120 80120 120 160 160 160 0 0 40 0 40 80 40 8012080120 120 160 160 160 Problem sizeProblem size Problem size Problem sizeProblem size Problem size Problem sizeProblem size Problem size (a) GA (twopoint) and hBOA hBOA (b) GA (uniform) and hBOA hBOA GA GA GA (uniform) and GA (two- (a) GA (twopoint) and hBOA (a) GA (twopoint) and (b) GA (uniform) and hBOA (b) GA (uniform) and (c) (c) (uniform) and and GA (two- (c) (uniform) GA (two- point) point) point) hBOA outperforms both GA variants. Figure 8: Ratio for the the the number for flips pairs of compared algorithms. Figure 8: Ratio for number of flipsflips pairs of pairs of compared algorithms. Figure 8: Ratio for number of of for for compared algorithms. Biggest differences for short-range of interactions (expected). 1.2 1.2 1.2 8 8 8 8 8 8 Population size (GA, twopoint) / Population size (GA, twopoint) / Population size (GA, twopoint) / Population size (GA, uniform) / Population size (GA, uniform) / Population size (GA, uniform) / Population size (GA, uniform) / Population size (GA, uniform) / Population size (GA, uniform) / Population size (GA, twopoint) Population size (GA, twopoint) Population size (GA, twopoint) σ=2.00 σ=2.00 σ=1.50 σ=2.00 σ=1.50 σ=1.50 GA with uniform 7crossover performs worst. σ=2.00 7 σ=2.00 σ=1.507 σ=2.00 σ=1.50 σ=1.50 7 σ=2.00 σ=2.00 σ=1.507 7 σ=2.00 σ=1.50 σ=1.50 Population size (hBOA) Population size (hBOA) Population size (hBOA) Population size (hBOA) Population size (hBOA) Population size (hBOA) 1.1 1.1 1.1 σ=1.00 σ=1.00 σ=1.00 6 σ=1.006 6 σ=1.00 σ=1.00 6 σ=1.006 6 σ=1.00 σ=1.00 1 σ=0.831 1 σ=0.83 σ=0.83 σ=0.83 σ=0.83 σ=0.83 σ=0.83 σ=0.83 σ=0.83 5 5 5 5 5 5 σ=0.75 σ=0.75 σ=0.75 σ=0.75 σ=0.75 σ=0.75 σ=0.75 σ=0.75 σ=0.75 0.9 0.9 0.9 σ=0.55 σ=0.55 σ=0.55 4 σ=0.554 4 σ=0.55 σ=0.55 4 σ=0.554 4 σ=0.55 σ=0.55 σ=0.00 σ=0.00 σ=0.00 3 σ=0.003 3 σ=0.00 σ=0.00 3 σ=0.003 3 σ=0.00 σ=0.00 0.8 0.8 0.8 2 2 2 2 2 2 0.7 0.7 0.7 1 1 1 1 1 1 0.6 0.6 0.6 0 0 0 0 0 0 0 0 40 0 40 80 40 80120 80120 120 160 160 160 0 0 40 0 40 80 40 80 12080 120160 160 120 160 0 0 40 0 40 80 40 80 12080 120160 160 120 160 Problem sizeProblem size Problem size Problem sizeProblem size Problem size Problem sizeProblem size Problem size Martin(a) (twopoint) and hBOA hBOA (b) GA (uniform) andAnalysis of EAs (c) 1D Spin Glass with Power-Law Interactions (a) GA GA (twopoint) and Katzgraber Pelikan GA (twopoint) hBOA (a) and Helmut G. and (b) GA (uniform) hBOA hBOA GA GA GA (uniform) and GA (two- (b) GA (uniform) hBOA and and on (c) (uniform) and and GA (two- (c) (uniform) GA (two-
  • Comparison: Correlation w.r.t. LS Steps Long range (σ = 0.55) Short range (σ = 2.00) Long-range problems lead to stronger correlations. Short-range problems deviate less than long-range ones. Correlations between GAs stronger than for hBOA. Martin Pelikan and Helmut G. Katzgraber Analysis of EAs on 1D Spin Glass with Power-Law Interactions
  • Comparison: Short-Range vs. Long-Range hBOA GA (twopoint) GA (uniform) for GA with twopoint (compared to σ=2.00) for GA with uniform (compared to σ=2.00) Slowdown factor for the number of flips Slowdown factor for the number of flips Slowdown factor for the number of flips 4 4 for hBOA (compared to σ=2.00) σ=1.50 σ=1.50 1 σ=1.00 σ=1.00 σ=0.83 σ=0.83 0.5 2 2 σ=0.75 σ=0.75 0.25 σ=0.55 σ=0.55 0.125 σ=1.50 σ=0.00 σ=0.00 1 σ=1.00 1 0.0625 σ=0.83 0.0312 σ=0.75 0.5 σ=0.55 0.0156 σ=0.00 0.5 16 32 64 128 16 32 64 128 16 32 64 128 Problem size Problem size Problem size (a) hBOA (b) GA (twopoint) (c) GA (uniform) Figure 6: Comparison of the number of flips for σ = 2.00 with that for σ < 2.00. Number of evaluations (GA, twopoint) / For hBOA and GA (twopoint), performance17.5 σ=2.00 with better Number of evaluations (GA, uniform) / Number of evaluations (GA, uniform) / Number of evaluations (GA, twopoint) 3.5 30 σ=2.00 σ=2.00 Number of evaluations (hBOA) Number of evaluations (hBOA) σ=1.50 σ=1.50 σ=1.50 3 short-range interactions. σ=1.00 25 σ=1.00 15 12.5 σ=1.00 σ=0.83 20 σ=0.83 σ=0.83 2.5 For GA (uniform), behavior is opposite. σ=0.75 15 σ=0.75 10 σ=0.75 2 σ=0.55 σ=0.55 σ=0.55 Related to properties10 ofσ=0.00 σ=0.00 recombination and problem structure. 7.5 5 σ=0.00 1.5 5 2.5 1 0 0 0 40 80 120 160 0 40 80 120 160 0 40 80 120 160 Problem size Problem size Problem size Martin Pelikan GA (twopoint) and hBOA (a) and Helmut G. Katzgraber (b) GA (uniform) and hBOA Analysis of EAs on(c) GA (uniform) Power-Law(two- 1D Spin Glass with and GA Interactions
  • Conclusions and Future Work Summary and conclusions Considered class of SK spin glasses with power-law interactions as class of random test problems. Can tune effective size of overlap. NP complete. Generated a broad range of problem instances. Analyzed results using hybrids of GEAs. Future work Use generated problems to test other algorithms. Design new hybrid optimizers for broad range of spin glasses. Relate results to existing theory (problem difficulty, landscape analysis, population sizing, time to convergence). Analyze obtained instances from computational physics perspective. Martin Pelikan and Helmut G. Katzgraber Analysis of EAs on 1D Spin Glass with Power-Law Interactions
  • Acknowledgments Acknowledgments NSF; NSF CAREER grant ECS-0547013. U.S. Air Force, AFOSR; FA9550-06-1-0096. University of Missouri; High Performance Computing Collaboratory sponsored by Information Technology Services; Research Award; Research Board. Martin Pelikan and Helmut G. Katzgraber Analysis of EAs on 1D Spin Glass with Power-Law Interactions