Your SlideShare is downloading.
×

×
# Saving this for later?

### Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime - even offline.

#### Text the download link to your phone

Standard text messaging rates apply

Like this presentation? Why not share!

- Presentation by guest6072cc 307 views
- Order Or Not: Does Parallelization ... by Martin Pelikan 1378 views
- Presentation by guest6072cc 307 views
- Order Or Not: Does Parallelization ... by Martin Pelikan 1378 views
- Analyzing probabilistic models in h... by kknsastry 994 views
- Analysis of Evolutionary Algorithms... by Martin Pelikan 577 views
- Hierarchical BOA on Random Decompos... by kknsastry 1581 views
- Estimation of Distribution Algorith... by Martin Pelikan 8680 views
- Sporadic Model Building for Efficie... by kknsastry 798 views
- Hybrid Evolutionary Algorithms on M... by Martin Pelikan 2767 views
- Intelligent Bias of Network Structu... by Martin Pelikan 490 views
- Effects of a Deterministic Hill cli... by Martin Pelikan 419 views

Like this? Share it with your network
Share

1,243

views

views

Published on

We discuss the computational complexity of random 2D Ising spin glasses, which represent an interesting class of constraint satisfaction problems for black box optimization. Two extremal cases are …

We discuss the computational complexity of random 2D Ising spin glasses, which represent an interesting class of constraint satisfaction problems for black box optimization. Two extremal cases are considered: (1) the +/- J spin glass, and (2) the Gaussian spin glass. We also study a smooth transition between these two extremal cases. The computational complexity of all studied spin glass systems is found to be dominated by rare events of extremely hard spin glass samples. We show that complexity of all studied spin glass systems is closely related to Frechet extremal value distribution. In a hybrid algorithm that combines the hierarchical Bayesian optimization algorithm (hBOA) with a deterministic bit-flip hill climber, the number of steps performed by both the global searcher (hBOA) and the local searcher follow Frechet distributions. Nonetheless, unlike in methods based purely on local search, the parameters of these distributions confirm good scalability of hBOA with local search. We further argue that standard performance measures for optimization algorithms---such as the average number of evaluations until convergence---can be misleading. Finally, our results indicate that for highly multimodal constraint satisfaction problems, such as Ising spin glasses, recombination-based search can provide qualitatively better results than mutation-based search.

No Downloads

Total Views

1,243

On Slideshare

0

From Embeds

0

Number of Embeds

2

Shares

0

Downloads

34

Comments

2

Likes

1

No embeds

No notes for slide

- 1. Computational complexity and simulation of rare events of Ising spin glasses Pelikan, M., Ocenasek, J., Trebst, S., Troyer, M., Alet, F.
- 2. Motivation Spin glass Origin in physics, but interesting for optimization as well Huge number of local optima and plateaus Local search fails miserably Some classes can be scalably solved using analytical methods Some classes provably NP-complete This paper Extends previous work to more classes of spin glasses Provides a thorough statistical analysis of results
- 3. Outline Hierarchical BOA (hBOA) Spin glasses Definition Difficulty Considered classes of spin glasses Experiments Summary and conclusions
- 4. Hierarchical BOA (hBOA) Pelikan, Goldberg, and Cantu-Paz (2001, 2002) Evolve population of candidate solutions Operators Selection Variation Build a Bayesian network with local structures for selected solutions Sample the built network to generate new solutions Replacement Restricted tournament replacement for niching
- 5. hBOA: Basic algorithm Bayesian New Current network population Selection population Restricted tournament replacement
- 6. Spin glass (SG) Spins arranged on a lattice (1D, 2D, 3D) Each spin si is +1 or -1 Neighbors connected Periodic boundary conditions Each connection (i,j) contains number Ji,j (coupling) Couplings usually initialized randomly +/- J couplings ~ uniform on {-1, +1} Gaussian couplings ~ N(0,1)
- 7. Finding ground states of SGs Energy ∑s J E= sj i i, j <i , j > Ground state Configuration of spins that minimizes E for given couplings Configurations can be represented with binary vectors Finding ground states Find ground states given couplings
- 8. 2-dimensional +/- J SG As constraint satisfaction problem ≠ ≠ = Spins: ≠ = ≠ = Constraints: ≠ = ≠ ≠ ≠ = ≠ General case Periodic boundary cond. (last and first connected) Constraints can be weighted
- 9. SG Difficulty 1D Trivial, deterministic O(n) algorithm 2D Local search fails miserably (exponential scaling) Good recombination-based EAs should scale-up Analytical method exists, O(n3.5) 3D NP-complete But methods exist to solve SGs of 1000s spins
- 10. Test SG classes Dimensions n=6x6 to n=20x20 1000 random instances for each n and distribution 2 basic coupling distributions +/- J, where couplings are randomly +1 or -1 Gaussian, where couplings ~N(0,1) Transition between the distributions for n=10x10 4 steps between the bounding cases
- 11. Coupling distribution 2-component normal mixture with overall σ2=1 N (μ1 , σ 12 ) + N (μ 2 , σ 2 ) Vary μ2-μ1 is from 0 to 2 2 p(J ) = 2 μ = 0.60 μ = 0.80 Pure Gaussian (μ=0) -3 -2 -1 0 1 2 3 -3 -2 -1 0 1 2 3 -3 -2 -1 0 1 2 3 μ = 0.95 μ = 0.99 ±J -3 -2 -1 0 1 2 3 -3 -2 -1 0 1 2 3 -3 -2 -1 0 1 2 3
- 12. Analysis of running times Traditional approach Run multiple times, estimate the mean Often works well, but sometimes misleading Performance on SGs MCMC performance shown to follow Frechet distr. All distribution moments ill-defined (incl. the mean)! Here Identify distribution of running times Estimate parameters of the distribution
- 13. Frechet distribution Central limit theorem for extremal values ⎞ ⎛ 1 x−μ ⎞ ε⎟ ⎜⎛ H ξ ;μ ;β = exp⎜ − ⎜1 + ξ ⎟⎟ ⎜ β ⎟⎟ ⎜⎝ ⎠ ⎠ ⎝ ξ = shape, μ = location, β = scale ξ determines speed of tail decay Our case ξ<0: Frechet distribution (polynomial decay) ξ=0: Gumbel distribution (exponential decay) ξ>0: Weibull distribution (faster than exponential decay) Frechet: mth moment exists iff |ξ|<m
- 14. Results +/- J vs. Gaussian couplings Distribution of the number of evaluations Location scale-up Shape Transition Location change Shape change 10 independent runs for each instance Minimum population size to converge in all runs
- 15. Number of evaluations
- 16. Location, μ
- 17. Shape, ξ
- 18. Transition: Location & Shape
- 19. Discussion Performance on +/- J SGs Number of evaluations grows approx. as O(n1.5) Agrees with BOA theory for uniform scaling Performance on Gaussian SGs Number of evaluations grows approx. as O(n2) Agrees with BOA theory for exponential scaling Transition Transition is smooth as expected
- 20. Important implications Selection+Recombination scales up great Exponential number of optima easily escaped Global optimum found reliably Overall time complexity similar to best analytical method Selection+Mutation fails to scale up Easily trapped in local minima Exponential scaling
- 21. Conclusions Average running time anal. might be insufficient In-depth statistical analysis confirms past results hBOA scales up well on all tested classes of SGs hBOA scalability agrees with theory Promising direction for solving other challenging constraint satisfaction problems
- 22. Contact Martin Pelikan Dept. of Math and Computer Science, 320 CCB University of Missouri at St. Louis 8001 Natural Bridge Rd. St. Louis, MO 63121 E-mail: pelikan@cs.umsl.edu WWW: http://www.cs.umsl.edu/~pelikan/

hBOA actually outperforms even state of the art methods based on small random perturbations used typically in statistical physics on this problem like flat histogram Markov chain Monte Carlo, the complexity of which grows exponentially. Spin glasses are tough for all algorithms that make small randomized changes in solutions and are not capable of changing blocks of spins; this is because of their properties---there are energy barriers, many good local optima are relatively far from local optima, etc. This was investigated before in e.g. Dayal et al. (2004) who showed that flat histogram MCMC takes exponential time. Also, SA won't do better than flat histogram MCMC. There is more on this in the paper associated with this presenttion:

http://arxiv.org/abs/cs.NE/0402030

There is also an extended version of the paper which I can provide if this is not sufficient.

Of course there is more to say to this, and for example cluster exact approximation works very well on spin glasses, and can in fact be combined with hBOA (http://medal.cs.umsl.edu/files/2006002. pdf). But that's a very different method than the usual SA techniques (it's based on maximum flow), it changes large domains of spins optimally in each step. If you want more pointers, I can provide :-)