Computational complexity and simulation of rare events of Ising spin glasses
Upcoming SlideShare
Loading in...5
×

Like this? Share it with your network

Share

Computational complexity and simulation of rare events of Ising spin glasses

  • 2,438 views
Uploaded on

We discuss the computational complexity of random 2D Ising spin glasses, which represent an interesting class of constraint satisfaction problems for black box optimization. Two extremal cases are......

We discuss the computational complexity of random 2D Ising spin glasses, which represent an interesting class of constraint satisfaction problems for black box optimization. Two extremal cases are considered: (1) the +/- J spin glass, and (2) the Gaussian spin glass. We also study a smooth transition between these two extremal cases. The computational complexity of all studied spin glass systems is found to be dominated by rare events of extremely hard spin glass samples. We show that complexity of all studied spin glass systems is closely related to Frechet extremal value distribution. In a hybrid algorithm that combines the hierarchical Bayesian optimization algorithm (hBOA) with a deterministic bit-flip hill climber, the number of steps performed by both the global searcher (hBOA) and the local searcher follow Frechet distributions. Nonetheless, unlike in methods based purely on local search, the parameters of these distributions confirm good scalability of hBOA with local search. We further argue that standard performance measures for optimization algorithms---such as the average number of evaluations until convergence---can be misleading. Finally, our results indicate that for highly multimodal constraint satisfaction problems, such as Ising spin glasses, recombination-based search can provide qualitatively better results than mutation-based search.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
  • Thanks for the good comment. This presentation focused on hBOA and its performance analysis since it was an EC conference.

    hBOA actually outperforms even state of the art methods based on small random perturbations used typically in statistical physics on this problem like flat histogram Markov chain Monte Carlo, the complexity of which grows exponentially. Spin glasses are tough for all algorithms that make small randomized changes in solutions and are not capable of changing blocks of spins; this is because of their properties---there are energy barriers, many good local optima are relatively far from local optima, etc. This was investigated before in e.g. Dayal et al. (2004) who showed that flat histogram MCMC takes exponential time. Also, SA won't do better than flat histogram MCMC. There is more on this in the paper associated with this presenttion:

    http://arxiv.org/abs/cs.NE/0402030

    There is also an extended version of the paper which I can provide if this is not sufficient.

    Of course there is more to say to this, and for example cluster exact approximation works very well on spin glasses, and can in fact be combined with hBOA (http://medal.cs.umsl.edu/files/2006002. pdf). But that's a very different method than the usual SA techniques (it's based on maximum flow), it changes large domains of spins optimally in each step. If you want more pointers, I can provide :-)
    Are you sure you want to
    Your message goes here
  • Very interesting, but perhaps you should summarize how this method is related to previous works linked to the use of statistical physics in optimization (simulated annealing, notably). There is a lot of interesting works around SA, in particular for the sampling of density probability functions...
    Are you sure you want to
    Your message goes here
No Downloads

Views

Total Views
2,438
On Slideshare
2,435
From Embeds
3
Number of Embeds
3

Actions

Shares
Downloads
34
Comments
2
Likes
1

Embeds 3

http://www.slideshare.net 1
http://www.slashdocs.com 1
https://www.linkedin.com 1

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Computational complexity and simulation of rare events of Ising spin glasses Pelikan, M., Ocenasek, J., Trebst, S., Troyer, M., Alet, F.
  • 2. Motivation Spin glass Origin in physics, but interesting for optimization as well Huge number of local optima and plateaus Local search fails miserably Some classes can be scalably solved using analytical methods Some classes provably NP-complete This paper Extends previous work to more classes of spin glasses Provides a thorough statistical analysis of results
  • 3. Outline Hierarchical BOA (hBOA) Spin glasses Definition Difficulty Considered classes of spin glasses Experiments Summary and conclusions
  • 4. Hierarchical BOA (hBOA) Pelikan, Goldberg, and Cantu-Paz (2001, 2002) Evolve population of candidate solutions Operators Selection Variation Build a Bayesian network with local structures for selected solutions Sample the built network to generate new solutions Replacement Restricted tournament replacement for niching
  • 5. hBOA: Basic algorithm Bayesian New Current network population Selection population Restricted tournament replacement
  • 6. Spin glass (SG) Spins arranged on a lattice (1D, 2D, 3D) Each spin si is +1 or -1 Neighbors connected Periodic boundary conditions Each connection (i,j) contains number Ji,j (coupling) Couplings usually initialized randomly +/- J couplings ~ uniform on {-1, +1} Gaussian couplings ~ N(0,1)
  • 7. Finding ground states of SGs Energy ∑s J E= sj i i, j <i , j > Ground state Configuration of spins that minimizes E for given couplings Configurations can be represented with binary vectors Finding ground states Find ground states given couplings
  • 8. 2-dimensional +/- J SG As constraint satisfaction problem ≠ ≠ = Spins: ≠ = ≠ = Constraints: ≠ = ≠ ≠ ≠ = ≠ General case Periodic boundary cond. (last and first connected) Constraints can be weighted
  • 9. SG Difficulty 1D Trivial, deterministic O(n) algorithm 2D Local search fails miserably (exponential scaling) Good recombination-based EAs should scale-up Analytical method exists, O(n3.5) 3D NP-complete But methods exist to solve SGs of 1000s spins
  • 10. Test SG classes Dimensions n=6x6 to n=20x20 1000 random instances for each n and distribution 2 basic coupling distributions +/- J, where couplings are randomly +1 or -1 Gaussian, where couplings ~N(0,1) Transition between the distributions for n=10x10 4 steps between the bounding cases
  • 11. Coupling distribution 2-component normal mixture with overall σ2=1 N (μ1 , σ 12 ) + N (μ 2 , σ 2 ) Vary μ2-μ1 is from 0 to 2 2 p(J ) = 2 μ = 0.60 μ = 0.80 Pure Gaussian (μ=0) -3 -2 -1 0 1 2 3 -3 -2 -1 0 1 2 3 -3 -2 -1 0 1 2 3 μ = 0.95 μ = 0.99 ±J -3 -2 -1 0 1 2 3 -3 -2 -1 0 1 2 3 -3 -2 -1 0 1 2 3
  • 12. Analysis of running times Traditional approach Run multiple times, estimate the mean Often works well, but sometimes misleading Performance on SGs MCMC performance shown to follow Frechet distr. All distribution moments ill-defined (incl. the mean)! Here Identify distribution of running times Estimate parameters of the distribution
  • 13. Frechet distribution Central limit theorem for extremal values ⎞ ⎛ 1 x−μ ⎞ ε⎟ ⎜⎛ H ξ ;μ ;β = exp⎜ − ⎜1 + ξ ⎟⎟ ⎜ β ⎟⎟ ⎜⎝ ⎠ ⎠ ⎝ ξ = shape, μ = location, β = scale ξ determines speed of tail decay Our case ξ<0: Frechet distribution (polynomial decay) ξ=0: Gumbel distribution (exponential decay) ξ>0: Weibull distribution (faster than exponential decay) Frechet: mth moment exists iff |ξ|<m
  • 14. Results +/- J vs. Gaussian couplings Distribution of the number of evaluations Location scale-up Shape Transition Location change Shape change 10 independent runs for each instance Minimum population size to converge in all runs
  • 15. Number of evaluations
  • 16. Location, μ
  • 17. Shape, ξ
  • 18. Transition: Location & Shape
  • 19. Discussion Performance on +/- J SGs Number of evaluations grows approx. as O(n1.5) Agrees with BOA theory for uniform scaling Performance on Gaussian SGs Number of evaluations grows approx. as O(n2) Agrees with BOA theory for exponential scaling Transition Transition is smooth as expected
  • 20. Important implications Selection+Recombination scales up great Exponential number of optima easily escaped Global optimum found reliably Overall time complexity similar to best analytical method Selection+Mutation fails to scale up Easily trapped in local minima Exponential scaling
  • 21. Conclusions Average running time anal. might be insufficient In-depth statistical analysis confirms past results hBOA scales up well on all tested classes of SGs hBOA scalability agrees with theory Promising direction for solving other challenging constraint satisfaction problems
  • 22. Contact Martin Pelikan Dept. of Math and Computer Science, 320 CCB University of Missouri at St. Louis 8001 Natural Bridge Rd. St. Louis, MO 63121 E-mail: pelikan@cs.umsl.edu WWW: http://www.cs.umsl.edu/~pelikan/