Finding Ground States of Sherrington-Kirkpatrick
Spin Glasses with hBOA and GAs
Martin Pelikan, Helmut G. Katzgraber, & Si...
Background
Background
Spin glasses are prototypical models for disordered systems.
Important topic in theoretical physics ...
Purpose
Purpose
Develop and test a robust approach to reliably solving large
instances of SK spin glass and other NP compl...
Outline
1. Sherrington-Kirkpatrick (SK) spin glass.
2. Branch and bound for SK spin glass.
3. Approaches to reliable solut...
SK Spin Glass
SK spin glass (Sherrington & Kirkpatrick, 1978)
Contains n spins s1, s2, . . . , sn.
Ising spin can be in tw...
Ground States of SK Spin Glasses
Energy
Energy of a spin configuration C is given by
H(C) = −
i<j
Ji,jsisj
Ground states ar...
Random Instances of SK Spin Glass
Random spin glass instances
Spin glass models usually studied over large sets of random
...
Branch and Bound for SK Spin Glass
Basic idea
Traverse the entire search space
(try all spin configurations).
Each level de...
Iterative Branch and Bound
Basic idea
Hartwig, Daske, and Kobe (1984).
Reduce the system to consider only first i spins.
So...
Current Situation and Goal
Current situation
We have BB which is guaranteed solve small instances.
We have hBOA and other ...
Basic Approach
Step 1: Branch and bound
Generate many instances for small problems solvable with BB.
Solve each instance w...
Step 1: Solve Small Problems with BB
Prepare instances
Generate 10,000 random SK instances for n = 20 to 80.
This gives a ...
Step 2: Run hBOA and Analyze Parameters
Basic setup
hBOA with default parameters.
Only population size and number of gener...
Step 2: Results
Population size appears to follow log-normal distribution.
Number of generations is very small in all case...
Step 2: Results
Estimate parameters of pop. size distribution for each n.
Derive upper bound from 0.001% tail of the distr...
Step 3: Find Reliable Optima of Larger Instances
Starting point
Predicted bound on pop. size to solve 99.999% instances.
P...
Step 2 Revisited: Run hBOA and Analyze Parameters
Run and analyze hBOA
Run hBOA for n = 100 to 200 as for smaller instance...
Step 2 Revisited: Results
Estimate parameters of pop. size distribution for each n.
Derive upper bound from 0.001% tail of...
So How Does It Work?
How does it work?
Incrementally increase problem size.
Set parameters using model based on smaller pr...
hBOA Results for n ≤ 300
10
1
10
2
10
3
10
1
10
2
10
3
10
4
Problem size
Meannumberofevaluations
Martin Pelikan, Helmut G....
Other Approaches: Fit Distribution Parameters
Basic idea
Fit distribution of a quantity (e.g. pop. size).
Fit a model to t...
Other Approaches: Population Doubling
Basic idea
Related to parameter-less genetic algorithms.
Start with a reasonable pop...
Comparison: hBOA vs. GA (Uniform Crossover)
Number of evaluations Number of flips
20 40 60 80 100 120 140 160 180 200
0.8
1...
Comparison: Uniform vs. Two-Point Crossover
Number of evaluations Number of flips
20 40 60 80 100 120 140 160 180 200
0.75
...
Conclusions and Future Work
Conclusions
The proposed approaches hold big promise for reliable solution
of extremely large ...
Acknowledgments
Acknowledgments
NSF; NSF CAREER grant ECS-0547013.
U.S. Air Force, AFOSR; FA9550-06-1-0096.
University of ...
Upcoming SlideShare
Loading in …5
×

Finding Ground States of Sherrington-Kirkpatrick Spin Glasses with Hierarchical BOA and Genetic Algorithms

1,995 views
1,890 views

Published on

This study focuses on the problem of finding ground states of random instances of the Sherrington-Kirkpatrick (SK) spin-glass model with Gaussian couplings. While the ground states of SK spin-glass instances can be obtained with branch and bound, the computational complexity of branch and bound yields instances of not more than about 90 spins. We describe several approaches based on the hierarchical Bayesian optimization algorithm (hBOA) to reliably identifying ground states of SK instances intractable with branch and bound, and present a broad range of empirical results on such problem instances. We argue that the proposed methodology holds a big promise for reliably solving large SK spin-glass instances to optimality with practical time complexity. The proposed approaches to identifying global optima reliably can also be applied to other problems and they can be used with many other evolutionary algorithms. Performance of hBOA is compared to that of the genetic algorithm with two common crossover operators.

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,995
On SlideShare
0
From Embeds
0
Number of Embeds
25
Actions
Shares
0
Downloads
12
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Finding Ground States of Sherrington-Kirkpatrick Spin Glasses with Hierarchical BOA and Genetic Algorithms

  1. 1. Finding Ground States of Sherrington-Kirkpatrick Spin Glasses with hBOA and GAs Martin Pelikan, Helmut G. Katzgraber, & Sigismund Kobe Missouri Estimation of Distribution Algorithms Laboratory (MEDAL) University of Missouri, St. Louis, MO http://medal.cs.umsl.edu/ pelikan@cs.umsl.edu Theoretische Physik ETH Z¨urich, Switzerland katzgraber@phys.ethz.ch Institut fr Theoretische Physik Technische Universit¨at Dresden, Germany kobe@physik.tu-dresden.de Martin Pelikan, Helmut G. Katzgraber, Sigismund Kobe Finding Ground States of SK Spin Glasses with hBOA and GAs
  2. 2. Background Background Spin glasses are prototypical models for disordered systems. Important topic in theoretical physics for several decades. Popular also as test problem for evolutionary algorithms Can generate many random instances of varying difficulty. Highly multimodal landscape. Strong interactions between variables. Similarities with other difficult NP-complete problems. Usually spins arranged on 2D or 3D lattices, but only few studies for the infinitely dimensional SK spin glass. Yet the infinitely dimensional systems are most difficult and interesting. Martin Pelikan, Helmut G. Katzgraber, Sigismund Kobe Finding Ground States of SK Spin Glasses with hBOA and GAs
  3. 3. Purpose Purpose Develop and test a robust approach to reliably solving large instances of SK spin glass and other NP complete problems. Don’t compromise problem size or reliability. Two target areas Computational physics. Optimization. Martin Pelikan, Helmut G. Katzgraber, Sigismund Kobe Finding Ground States of SK Spin Glasses with hBOA and GAs
  4. 4. Outline 1. Sherrington-Kirkpatrick (SK) spin glass. 2. Branch and bound for SK spin glass. 3. Approaches to reliable solution of large SK instances. 4. Future work. 5. Summary and conclusions. Martin Pelikan, Helmut G. Katzgraber, Sigismund Kobe Finding Ground States of SK Spin Glasses with hBOA and GAs
  5. 5. SK Spin Glass SK spin glass (Sherrington & Kirkpatrick, 1978) Contains n spins s1, s2, . . . , sn. Ising spin can be in two states: +1 or −1. All pairs of spins interact. Interaction of spins si and sj specified by real-valued coupling Ji,j. Spin glass instance is defined by set of couplings {Ji,j}. Spin configuration is defined by the values of spins {si}. Martin Pelikan, Helmut G. Katzgraber, Sigismund Kobe Finding Ground States of SK Spin Glasses with hBOA and GAs
  6. 6. Ground States of SK Spin Glasses Energy Energy of a spin configuration C is given by H(C) = − i<j Ji,jsisj Ground states are spin configurations that minimize energy. Finding ground states of SK instances is NP-complete. Compare with other standard spin glass types 2D: Spin interacts with only 4 neighbors in 2D lattice. 3D: Spin interacts with only 6 neighbors in 3D lattice. SK: Spin interacts with all other spins. 2D is polynomially solvable; 3D and SK are NP-complete. Martin Pelikan, Helmut G. Katzgraber, Sigismund Kobe Finding Ground States of SK Spin Glasses with hBOA and GAs
  7. 7. Random Instances of SK Spin Glass Random spin glass instances Spin glass models usually studied over large sets of random instances. Two most common distributions for couplings Gaussian: N(0, 1). ±J: +1 or −1 with equal probability. Sometimes a distance metric is imposed and coupling strength decreases with distance. Instances used in this work We use Gaussian couplings from N(0, 1). Martin Pelikan, Helmut G. Katzgraber, Sigismund Kobe Finding Ground States of SK Spin Glasses with hBOA and GAs
  8. 8. Branch and Bound for SK Spin Glass Basic idea Traverse the entire search space (try all spin configurations). Each level decides on one spin (+1 or -1). Each leaf encodes a unique spin configuration. Branches that lead to provably suboptimal solutions are cut. Why? BB is inefficient, but can verify the global optimum. Martin Pelikan, Helmut G. Katzgraber, Sigismund Kobe Finding Ground States of SK Spin Glasses with hBOA and GAs
  9. 9. Iterative Branch and Bound Basic idea Hartwig, Daske, and Kobe (1984). Reduce the system to consider only first i spins. Solve for i = 2 to i = n with step 1. Use previous results to provide better bounds. Denote best energy for for first i spins by f∗ i . Lower bound on best energy for first j spins given by f∗ j ≥ f∗ j−1 − j−1 i=1 |Ji,j|. Effects of iterative approach We must solve n − 1 problems instead of 1. But the overall performance much better. Martin Pelikan, Helmut G. Katzgraber, Sigismund Kobe Finding Ground States of SK Spin Glasses with hBOA and GAs
  10. 10. Current Situation and Goal Current situation We have BB which is guaranteed solve small instances. We have hBOA and other evolutionary algorithms which can solve larger instances but we need to set Population size. Number of generations. Goal Find reliable optima of relatively large instances. Don’t stick with small problems because of BB. Don’t compromise reliability by guessing EA parameters wildly. Martin Pelikan, Helmut G. Katzgraber, Sigismund Kobe Finding Ground States of SK Spin Glasses with hBOA and GAs
  11. 11. Basic Approach Step 1: Branch and bound Generate many instances for small problems solvable with BB. Solve each instance with iterative BB. Step 2: hBOA with optimal settings Apply hBOA to each new instance. Find accurate statistical model for hBOA parameters. Use model to predict sufficient parameters for larger problems. Step 3: Going to larger problems Apply hBOA with the conservative settings from step 2 to find reliable global optima of larger instances. Go to step 2 (to get to larger and larger problems). Martin Pelikan, Helmut G. Katzgraber, Sigismund Kobe Finding Ground States of SK Spin Glasses with hBOA and GAs
  12. 12. Step 1: Solve Small Problems with BB Prepare instances Generate 10,000 random SK instances for n = 20 to 80. This gives a total of 310,000 unique problem instances. Solve each instance with BB to find global optimum. Martin Pelikan, Helmut G. Katzgraber, Sigismund Kobe Finding Ground States of SK Spin Glasses with hBOA and GAs
  13. 13. Step 2: Run hBOA and Analyze Parameters Basic setup hBOA with default parameters. Only population size and number of generations tuned. Deterministic 1-bit hill climber improves all solutions. Maximum number of generations is set to n. Population size set with bisection for each instance (10 successes in 10 independent runs). Analysis Total of 3,100,000 hBOA runs to analyze. Analyze the distribution of the following Population size. Number of generations. Number of evaluations. Number of flips of hill climber. Martin Pelikan, Helmut G. Katzgraber, Sigismund Kobe Finding Ground States of SK Spin Glasses with hBOA and GAs
  14. 14. Step 2: Results Population size appears to follow log-normal distribution. Number of generations is very small in all cases. n = 20 n = 80 0 20 40 60 80 0 500 1000 1500 2000 2500 Population size Frequency 0 100 200 300 400 500 0 500 1000 1500 2000 Population size Frequency Martin Pelikan, Helmut G. Katzgraber, Sigismund Kobe Finding Ground States of SK Spin Glasses with hBOA and GAs
  15. 15. Step 2: Results Estimate parameters of pop. size distribution for each n. Derive upper bound from 0.001% tail of the distribution, which sould solve 99.999% instances. Find a fit of this upper bound. Predict pop. size for larger problems (up to n = 200). Fit of 99.999% percentile Prediction for larger instances 20 30 40 50 60 70 80 100 150 200 250 300 350 400 450 500 550 600 Populationsize Problem size Power−law fit 95% prediction bounds 99.999 percentile 20 40 60 80 100 120 140 160 180 200 0 250 500 750 1000 1250 1500 1750 2000 Populationsize Problem size Power−law fit 95% prediction bounds Martin Pelikan, Helmut G. Katzgraber, Sigismund Kobe Finding Ground States of SK Spin Glasses with hBOA and GAs
  16. 16. Step 3: Find Reliable Optima of Larger Instances Starting point Predicted bound on pop. size to solve 99.999% instances. Prepare larger instances Generate 1,000 instances for n = 100 to 200. For each instance Use estimated upper bound of the population size. Use maximum number of generations of n. Make 10 hBOA runs on each instance to find global optimum. Record the best solution found. All runs should agree. Martin Pelikan, Helmut G. Katzgraber, Sigismund Kobe Finding Ground States of SK Spin Glasses with hBOA and GAs
  17. 17. Step 2 Revisited: Run hBOA and Analyze Parameters Run and analyze hBOA Run hBOA for n = 100 to 200 as for smaller instances. Repeat bisection 10 times for each instance. Analysis Total of 2,100,000 successful hBOA runs. Do the analysis as for smaller problems. Martin Pelikan, Helmut G. Katzgraber, Sigismund Kobe Finding Ground States of SK Spin Glasses with hBOA and GAs
  18. 18. Step 2 Revisited: Results Estimate parameters of pop. size distribution for each n. Derive upper bound from 0.001% tail of the distribution. Find a fit of this upper bound. Predict pop. size for larger problems (up to n = 300). Fit of 99.999% percentile Prediction for larger instances 20 40 60 80 100 120 140 160 180 200 0 250 500 750 1000 1250 1500 1750 2000 2250 2500 Populationsize Problem size Power−law fit 95% prediction bounds 99.999 percentile 20 60 100 140 180 220 260 300 0 500 1000 1500 2000 2500 3000 3500 4000 Populationsize Problem size Power−law fit 95% prediction bounds Martin Pelikan, Helmut G. Katzgraber, Sigismund Kobe Finding Ground States of SK Spin Glasses with hBOA and GAs
  19. 19. So How Does It Work? How does it work? Incrementally increase problem size. Set parameters using model based on smaller problems. If distributions are easy to model and the growth of different parameters can be fit reliably, this allows us to reliably solve large instances even when no complete algorithm is tractable. Ultimate goal Go to problems with 4,000 spins or so. Important Don’t make too big steps to ensure tractability and reliability. Martin Pelikan, Helmut G. Katzgraber, Sigismund Kobe Finding Ground States of SK Spin Glasses with hBOA and GAs
  20. 20. hBOA Results for n ≤ 300 10 1 10 2 10 3 10 1 10 2 10 3 10 4 Problem size Meannumberofevaluations Martin Pelikan, Helmut G. Katzgraber, Sigismund Kobe Finding Ground States of SK Spin Glasses with hBOA and GAs
  21. 21. Other Approaches: Fit Distribution Parameters Basic idea Fit distribution of a quantity (e.g. pop. size). Fit a model to the parameters of the distribution. Estimate parameters for larger problems from the fit. Compute tails from estimated parameters. 20 40 60 80 100 120 140 160 180 200 0 1 2 3 4 5 Problem size Populationsize Log mean Power−law fit for mean Log standard deviation Power−law fit for std. dev. 20 40 60 80 100 120 140 160 180 200 0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 Problem size Numberofiterations Log mean Power−law fit (mean) Log standard deviation Power−law fit (std. dev.) Martin Pelikan, Helmut G. Katzgraber, Sigismund Kobe Finding Ground States of SK Spin Glasses with hBOA and GAs
  22. 22. Other Approaches: Population Doubling Basic idea Related to parameter-less genetic algorithms. Start with a reasonable population size. Make 10 runs (can change). Double the population and repeat. Terminate doubling when All 10 runs result in the same solution. Last couple of rounds resulted in the same solution. Martin Pelikan, Helmut G. Katzgraber, Sigismund Kobe Finding Ground States of SK Spin Glasses with hBOA and GAs
  23. 23. Comparison: hBOA vs. GA (Uniform Crossover) Number of evaluations Number of flips 20 40 60 80 100 120 140 160 180 200 0.8 1 1.2 1.4 1.6 1.8 2 Number of spins Num.GA(U)evals./num.hBOAevals. 20 40 60 80 100 120 140 160 180 200 0.7 0.8 0.9 1 1.1 1.2 1.3 1.4 Number of spins Num.GA(U)flips/num.hBOAflips Martin Pelikan, Helmut G. Katzgraber, Sigismund Kobe Finding Ground States of SK Spin Glasses with hBOA and GAs
  24. 24. Comparison: Uniform vs. Two-Point Crossover Number of evaluations Number of flips 20 40 60 80 100 120 140 160 180 200 0.75 0.8 0.85 0.9 0.95 1 1.05 Number of spins Num.GA(U)evals./num.GA(2P)evals. 20 40 60 80 100 120 140 160 180 200 0.9 0.95 1 1.05 1.1 Number of spins Num.GA(U)flips/num.GA(2P)flips Martin Pelikan, Helmut G. Katzgraber, Sigismund Kobe Finding Ground States of SK Spin Glasses with hBOA and GAs
  25. 25. Conclusions and Future Work Conclusions The proposed approaches hold big promise for reliable solution of extremely large problems. The proposed approaches can be used with other optimization techniques which require adequate parameter settings. SK spin glass closely related to other difficult problems, such as protein folding. Future work Compare hBOA & GA to other techniques Extremal optimization (EO). Hysteretic optimization (HO). Create efficient hybrids of hBOA, GA, EO, HO, and BB. Apply other efficiency enhancement techniques. Further increase problem size to 1,000–4,000 and more. Martin Pelikan, Helmut G. Katzgraber, Sigismund Kobe Finding Ground States of SK Spin Glasses with hBOA and GAs
  26. 26. Acknowledgments Acknowledgments NSF; NSF CAREER grant ECS-0547013. U.S. Air Force, AFOSR; FA9550-06-1-0096. University of Missouri; High Performance Computing Collaboratory sponsored by Information Technology Services; Research Award; Research Board. Martin Pelikan, Helmut G. Katzgraber, Sigismund Kobe Finding Ground States of SK Spin Glasses with hBOA and GAs

×