Slideshare uses cookies to improve functionality and performance, and to provide you with relevant advertising. If you continue browsing the site, you agree to the use of cookies on this website. See our User Agreement and Privacy Policy.

Slideshare uses cookies to improve functionality and performance, and to provide you with relevant advertising. If you continue browsing the site, you agree to the use of cookies on this website. See our Privacy Policy and User Agreement for details.

1,995 views

1,890 views

1,890 views

Published on

Published in:
Technology

No Downloads

Total views

1,995

On SlideShare

0

From Embeds

0

Number of Embeds

25

Shares

0

Downloads

12

Comments

0

Likes

1

No embeds

No notes for slide

- 1. Finding Ground States of Sherrington-Kirkpatrick Spin Glasses with hBOA and GAs Martin Pelikan, Helmut G. Katzgraber, & Sigismund Kobe Missouri Estimation of Distribution Algorithms Laboratory (MEDAL) University of Missouri, St. Louis, MO http://medal.cs.umsl.edu/ pelikan@cs.umsl.edu Theoretische Physik ETH Z¨urich, Switzerland katzgraber@phys.ethz.ch Institut fr Theoretische Physik Technische Universit¨at Dresden, Germany kobe@physik.tu-dresden.de Martin Pelikan, Helmut G. Katzgraber, Sigismund Kobe Finding Ground States of SK Spin Glasses with hBOA and GAs
- 2. Background Background Spin glasses are prototypical models for disordered systems. Important topic in theoretical physics for several decades. Popular also as test problem for evolutionary algorithms Can generate many random instances of varying diﬃculty. Highly multimodal landscape. Strong interactions between variables. Similarities with other diﬃcult NP-complete problems. Usually spins arranged on 2D or 3D lattices, but only few studies for the inﬁnitely dimensional SK spin glass. Yet the inﬁnitely dimensional systems are most diﬃcult and interesting. Martin Pelikan, Helmut G. Katzgraber, Sigismund Kobe Finding Ground States of SK Spin Glasses with hBOA and GAs
- 3. Purpose Purpose Develop and test a robust approach to reliably solving large instances of SK spin glass and other NP complete problems. Don’t compromise problem size or reliability. Two target areas Computational physics. Optimization. Martin Pelikan, Helmut G. Katzgraber, Sigismund Kobe Finding Ground States of SK Spin Glasses with hBOA and GAs
- 4. Outline 1. Sherrington-Kirkpatrick (SK) spin glass. 2. Branch and bound for SK spin glass. 3. Approaches to reliable solution of large SK instances. 4. Future work. 5. Summary and conclusions. Martin Pelikan, Helmut G. Katzgraber, Sigismund Kobe Finding Ground States of SK Spin Glasses with hBOA and GAs
- 5. SK Spin Glass SK spin glass (Sherrington & Kirkpatrick, 1978) Contains n spins s1, s2, . . . , sn. Ising spin can be in two states: +1 or −1. All pairs of spins interact. Interaction of spins si and sj speciﬁed by real-valued coupling Ji,j. Spin glass instance is deﬁned by set of couplings {Ji,j}. Spin conﬁguration is deﬁned by the values of spins {si}. Martin Pelikan, Helmut G. Katzgraber, Sigismund Kobe Finding Ground States of SK Spin Glasses with hBOA and GAs
- 6. Ground States of SK Spin Glasses Energy Energy of a spin conﬁguration C is given by H(C) = − i<j Ji,jsisj Ground states are spin conﬁgurations that minimize energy. Finding ground states of SK instances is NP-complete. Compare with other standard spin glass types 2D: Spin interacts with only 4 neighbors in 2D lattice. 3D: Spin interacts with only 6 neighbors in 3D lattice. SK: Spin interacts with all other spins. 2D is polynomially solvable; 3D and SK are NP-complete. Martin Pelikan, Helmut G. Katzgraber, Sigismund Kobe Finding Ground States of SK Spin Glasses with hBOA and GAs
- 7. Random Instances of SK Spin Glass Random spin glass instances Spin glass models usually studied over large sets of random instances. Two most common distributions for couplings Gaussian: N(0, 1). ±J: +1 or −1 with equal probability. Sometimes a distance metric is imposed and coupling strength decreases with distance. Instances used in this work We use Gaussian couplings from N(0, 1). Martin Pelikan, Helmut G. Katzgraber, Sigismund Kobe Finding Ground States of SK Spin Glasses with hBOA and GAs
- 8. Branch and Bound for SK Spin Glass Basic idea Traverse the entire search space (try all spin conﬁgurations). Each level decides on one spin (+1 or -1). Each leaf encodes a unique spin conﬁguration. Branches that lead to provably suboptimal solutions are cut. Why? BB is ineﬃcient, but can verify the global optimum. Martin Pelikan, Helmut G. Katzgraber, Sigismund Kobe Finding Ground States of SK Spin Glasses with hBOA and GAs
- 9. Iterative Branch and Bound Basic idea Hartwig, Daske, and Kobe (1984). Reduce the system to consider only ﬁrst i spins. Solve for i = 2 to i = n with step 1. Use previous results to provide better bounds. Denote best energy for for ﬁrst i spins by f∗ i . Lower bound on best energy for ﬁrst j spins given by f∗ j ≥ f∗ j−1 − j−1 i=1 |Ji,j|. Eﬀects of iterative approach We must solve n − 1 problems instead of 1. But the overall performance much better. Martin Pelikan, Helmut G. Katzgraber, Sigismund Kobe Finding Ground States of SK Spin Glasses with hBOA and GAs
- 10. Current Situation and Goal Current situation We have BB which is guaranteed solve small instances. We have hBOA and other evolutionary algorithms which can solve larger instances but we need to set Population size. Number of generations. Goal Find reliable optima of relatively large instances. Don’t stick with small problems because of BB. Don’t compromise reliability by guessing EA parameters wildly. Martin Pelikan, Helmut G. Katzgraber, Sigismund Kobe Finding Ground States of SK Spin Glasses with hBOA and GAs
- 11. Basic Approach Step 1: Branch and bound Generate many instances for small problems solvable with BB. Solve each instance with iterative BB. Step 2: hBOA with optimal settings Apply hBOA to each new instance. Find accurate statistical model for hBOA parameters. Use model to predict suﬃcient parameters for larger problems. Step 3: Going to larger problems Apply hBOA with the conservative settings from step 2 to ﬁnd reliable global optima of larger instances. Go to step 2 (to get to larger and larger problems). Martin Pelikan, Helmut G. Katzgraber, Sigismund Kobe Finding Ground States of SK Spin Glasses with hBOA and GAs
- 12. Step 1: Solve Small Problems with BB Prepare instances Generate 10,000 random SK instances for n = 20 to 80. This gives a total of 310,000 unique problem instances. Solve each instance with BB to ﬁnd global optimum. Martin Pelikan, Helmut G. Katzgraber, Sigismund Kobe Finding Ground States of SK Spin Glasses with hBOA and GAs
- 13. Step 2: Run hBOA and Analyze Parameters Basic setup hBOA with default parameters. Only population size and number of generations tuned. Deterministic 1-bit hill climber improves all solutions. Maximum number of generations is set to n. Population size set with bisection for each instance (10 successes in 10 independent runs). Analysis Total of 3,100,000 hBOA runs to analyze. Analyze the distribution of the following Population size. Number of generations. Number of evaluations. Number of ﬂips of hill climber. Martin Pelikan, Helmut G. Katzgraber, Sigismund Kobe Finding Ground States of SK Spin Glasses with hBOA and GAs
- 14. Step 2: Results Population size appears to follow log-normal distribution. Number of generations is very small in all cases. n = 20 n = 80 0 20 40 60 80 0 500 1000 1500 2000 2500 Population size Frequency 0 100 200 300 400 500 0 500 1000 1500 2000 Population size Frequency Martin Pelikan, Helmut G. Katzgraber, Sigismund Kobe Finding Ground States of SK Spin Glasses with hBOA and GAs
- 15. Step 2: Results Estimate parameters of pop. size distribution for each n. Derive upper bound from 0.001% tail of the distribution, which sould solve 99.999% instances. Find a ﬁt of this upper bound. Predict pop. size for larger problems (up to n = 200). Fit of 99.999% percentile Prediction for larger instances 20 30 40 50 60 70 80 100 150 200 250 300 350 400 450 500 550 600 Populationsize Problem size Power−law fit 95% prediction bounds 99.999 percentile 20 40 60 80 100 120 140 160 180 200 0 250 500 750 1000 1250 1500 1750 2000 Populationsize Problem size Power−law fit 95% prediction bounds Martin Pelikan, Helmut G. Katzgraber, Sigismund Kobe Finding Ground States of SK Spin Glasses with hBOA and GAs
- 16. Step 3: Find Reliable Optima of Larger Instances Starting point Predicted bound on pop. size to solve 99.999% instances. Prepare larger instances Generate 1,000 instances for n = 100 to 200. For each instance Use estimated upper bound of the population size. Use maximum number of generations of n. Make 10 hBOA runs on each instance to ﬁnd global optimum. Record the best solution found. All runs should agree. Martin Pelikan, Helmut G. Katzgraber, Sigismund Kobe Finding Ground States of SK Spin Glasses with hBOA and GAs
- 17. Step 2 Revisited: Run hBOA and Analyze Parameters Run and analyze hBOA Run hBOA for n = 100 to 200 as for smaller instances. Repeat bisection 10 times for each instance. Analysis Total of 2,100,000 successful hBOA runs. Do the analysis as for smaller problems. Martin Pelikan, Helmut G. Katzgraber, Sigismund Kobe Finding Ground States of SK Spin Glasses with hBOA and GAs
- 18. Step 2 Revisited: Results Estimate parameters of pop. size distribution for each n. Derive upper bound from 0.001% tail of the distribution. Find a ﬁt of this upper bound. Predict pop. size for larger problems (up to n = 300). Fit of 99.999% percentile Prediction for larger instances 20 40 60 80 100 120 140 160 180 200 0 250 500 750 1000 1250 1500 1750 2000 2250 2500 Populationsize Problem size Power−law fit 95% prediction bounds 99.999 percentile 20 60 100 140 180 220 260 300 0 500 1000 1500 2000 2500 3000 3500 4000 Populationsize Problem size Power−law fit 95% prediction bounds Martin Pelikan, Helmut G. Katzgraber, Sigismund Kobe Finding Ground States of SK Spin Glasses with hBOA and GAs
- 19. So How Does It Work? How does it work? Incrementally increase problem size. Set parameters using model based on smaller problems. If distributions are easy to model and the growth of diﬀerent parameters can be ﬁt reliably, this allows us to reliably solve large instances even when no complete algorithm is tractable. Ultimate goal Go to problems with 4,000 spins or so. Important Don’t make too big steps to ensure tractability and reliability. Martin Pelikan, Helmut G. Katzgraber, Sigismund Kobe Finding Ground States of SK Spin Glasses with hBOA and GAs
- 20. hBOA Results for n ≤ 300 10 1 10 2 10 3 10 1 10 2 10 3 10 4 Problem size Meannumberofevaluations Martin Pelikan, Helmut G. Katzgraber, Sigismund Kobe Finding Ground States of SK Spin Glasses with hBOA and GAs
- 21. Other Approaches: Fit Distribution Parameters Basic idea Fit distribution of a quantity (e.g. pop. size). Fit a model to the parameters of the distribution. Estimate parameters for larger problems from the ﬁt. Compute tails from estimated parameters. 20 40 60 80 100 120 140 160 180 200 0 1 2 3 4 5 Problem size Populationsize Log mean Power−law fit for mean Log standard deviation Power−law fit for std. dev. 20 40 60 80 100 120 140 160 180 200 0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 Problem size Numberofiterations Log mean Power−law fit (mean) Log standard deviation Power−law fit (std. dev.) Martin Pelikan, Helmut G. Katzgraber, Sigismund Kobe Finding Ground States of SK Spin Glasses with hBOA and GAs
- 22. Other Approaches: Population Doubling Basic idea Related to parameter-less genetic algorithms. Start with a reasonable population size. Make 10 runs (can change). Double the population and repeat. Terminate doubling when All 10 runs result in the same solution. Last couple of rounds resulted in the same solution. Martin Pelikan, Helmut G. Katzgraber, Sigismund Kobe Finding Ground States of SK Spin Glasses with hBOA and GAs
- 23. Comparison: hBOA vs. GA (Uniform Crossover) Number of evaluations Number of ﬂips 20 40 60 80 100 120 140 160 180 200 0.8 1 1.2 1.4 1.6 1.8 2 Number of spins Num.GA(U)evals./num.hBOAevals. 20 40 60 80 100 120 140 160 180 200 0.7 0.8 0.9 1 1.1 1.2 1.3 1.4 Number of spins Num.GA(U)flips/num.hBOAflips Martin Pelikan, Helmut G. Katzgraber, Sigismund Kobe Finding Ground States of SK Spin Glasses with hBOA and GAs
- 24. Comparison: Uniform vs. Two-Point Crossover Number of evaluations Number of ﬂips 20 40 60 80 100 120 140 160 180 200 0.75 0.8 0.85 0.9 0.95 1 1.05 Number of spins Num.GA(U)evals./num.GA(2P)evals. 20 40 60 80 100 120 140 160 180 200 0.9 0.95 1 1.05 1.1 Number of spins Num.GA(U)flips/num.GA(2P)flips Martin Pelikan, Helmut G. Katzgraber, Sigismund Kobe Finding Ground States of SK Spin Glasses with hBOA and GAs
- 25. Conclusions and Future Work Conclusions The proposed approaches hold big promise for reliable solution of extremely large problems. The proposed approaches can be used with other optimization techniques which require adequate parameter settings. SK spin glass closely related to other diﬃcult problems, such as protein folding. Future work Compare hBOA & GA to other techniques Extremal optimization (EO). Hysteretic optimization (HO). Create eﬃcient hybrids of hBOA, GA, EO, HO, and BB. Apply other eﬃciency enhancement techniques. Further increase problem size to 1,000–4,000 and more. Martin Pelikan, Helmut G. Katzgraber, Sigismund Kobe Finding Ground States of SK Spin Glasses with hBOA and GAs
- 26. Acknowledgments Acknowledgments NSF; NSF CAREER grant ECS-0547013. U.S. Air Force, AFOSR; FA9550-06-1-0096. University of Missouri; High Performance Computing Collaboratory sponsored by Information Technology Services; Research Award; Research Board. Martin Pelikan, Helmut G. Katzgraber, Sigismund Kobe Finding Ground States of SK Spin Glasses with hBOA and GAs

No public clipboards found for this slide

×
### Save the most important slides with Clipping

Clipping is a handy way to collect and organize the most important slides from a presentation. You can keep your great finds in clipboards organized around topics.

Be the first to comment