The Bayesian Optimization Algorithm with Substructural Local Search

The Bayesian Optimization Algorithm with Substructural Local Search Claudio Lima, Martin Pelikan, Kumara Sastry, Martin Butz, David Goldberg, and Fernando Lobo

Overview Motivation Bayesian Optimization Algorithm (BOA) Modeling fitness in BOA Substructural Neighborhoods BOA with Substuctural Hillclimbing Results Conclusions Future Work 2 OBUPM 2006

Motivation Probabilistic models of EDAs allow better recombination of subsolutions Get we can more from these models? Yes! Efficiency enhancement on EDAs Evaluation relaxation Local search in substructural neighborhoods 3 OBUPM 2006

Bayesian Optimization Algorithm Pelikan, Goldberg, and Cantú-Paz (1999) Use Bayesian networks to model good solutions Model structure => acyclic directed graph Nodes represent variables Edges represent conditional dependencies Model parameters => conditional probabilities Conditional Probability Tables based on the observed frequencies Local structures: Decision Trees or Graphs 4 OBUPM 2006

Learning a Bayesian Network Start with an empty network (independence assumption) Perform operation that improves the metric the most Edge addition, edge removal, edge reversal Metric quantifies the likelihood of the model wrt data (good solutions) Stop when no more improvement is possible 5 OBUPM 2006

A 3-bit Example Model Structure Model Parameters Directed Acyclic Graph Conditional Probability Tables Decision Trees X2X3 P(X1=1|X2X3) X2 X2 X3 00 0.20 0 1 01 0.20 X3 P(x1=1) = 0.20 10 0.15 X1 0 1 11 0.45 P(x1=1) = 0.15 P(x1=1) = 0.45 6 OBUPM 2006

Modeling Fitness in BOA Bayesian networks extended to store a surrogate fitness model (Pelikan & Sastry,2004) The surrogate fitness is learned from a proportion of the population... ...and is used to estimate the fitness of the remaining individuals (therefore reducing evals) 7 OBUPM 2006

The same 3-bit Example X2X3 P(X1=1|X2X3) f(X1=0|X2X3) f(X1=1|X2X3) X2 00 0.20 -0.49 0.53 0 1 01 0.20 -0.38 0.51 10 0.15 -0.55 0.47 X3 P(X1=1) = 0.20 f(X1=0) = -0.48 11 0.45 -0.52 0.62 f(X1=1) = 0.54 0 1 P(X1=1) = 0.15 P(X1=1) = 0.45 f(X1=0) = -0.55 f(X1=0) = -0.52 f(X1=1) = 0.47 f(X1=1) = 0.62 Estimated fitness: 8 OBUPM 2006

Why Substructural Neighborhoods? An efficient mutation operator should search in the correct neighborhood Oftentimes this is done by incorportaring domain- or problem-specific knowledge However, efficiency typically does not generalize beyond a small number of applications Bitwise local search have more general applicability but with inferior results 9 OBUPM 2006

Substructural Neighborhoods Neighborhoods defined by the probabilistic model of EDAs Exploits the underlying problem structure while not loosing generality of application Exploration of neighborhoods respect dependencies between variables If [X1X2X3] form a linkage group, the neighborhood considered will be 000, 001, 010, ..., 111 10 OBUPM 2006

Substructural Local Search For uniformly-scaled decomposable problems, substructural local search scales as 0(2km1.5) (Sastry & Goldberg, 2004) Bitwise hillclimber: O( mk log(m) ) Extended Compact GA with substructural local search is more robust than either single- operator-based aproaches (Lima et al., 2005) 11 OBUPM 2006

Substructural Neighborhoods in BOA Model is more complex than in eCGA What is a linkage group? Which dependencies to consider? Is order relevant? Example: topology of 3 different substructural neighborhoods for variable X2: 12 OBUPM 2006

BOA + Substructural Hillclimbing After model sampling each offspring undergoes local search with a certain probability pls Current model is used to define the neighborhoods Choice of best subsolutions => surrogate fitness model Cost of performing local search is then minimal 13 OBUPM 2006

Substructural Hillclimbing in BOA 14 OBUPM 2006

Substructural Hillclimbing in BOA Use reverse ancestral ordering of variables 2 different versions of the substructural hillclimber (step 3) Evaluated fitness Estimated fitness Result of local search is evaluated 15 OBUPM 2006

Experiments Additively decomposable problems Two important bounds: Onemax and concatenated k-bit traps Many things in between 16 OBUPM 2006

Onemax Results (l=50) 17 OBUPM 2006

Onemax Results (l=50) Correctness of substructural neighborhoohs is not relevant... ...but the choice of subsolutions relies on the accuracy of the surrogate fitness model More important, the acceptance of the best subsolutions depends also on the surrogate, if using estimated fitness 18 OBUPM 2006

10x5-bit trap Results (l=50) 19 OBUPM 2006

10x5-bit trap Results (l=50) Correct identification of problem substructure is crucial Different versions of the hillclimber perform similar (for small pls) Cost of using evaluated fitness increases significatively with pls (and with problem size) Phase transition in the population size required 20 OBUPM 2006

Scalability Results (5-bit traps) 21 OBUPM 2006

Scalability Results (5-bit traps) Substancial speedups are obtained (η=6 for l=140) Speedup scales as O(l0.45) for l<80 For bigger problem sizes the speedup is more moderate pls=5x10-4 adequate for range of problems tested, but optimal proportion should decrease for higher problem sizes 22 OBUPM 2006

The Bayesian Optimization Algorithm with Substructural Local Search

0 comments

Notes on slide 1

3 Favorites & 2 Groups