Background Objectives rADPs Experiments Conclusions
Hierarchical BOA on Random Decomposable
Problems
Martin Pelikan1 , Kumara Sastry2 ,
Martin V. Butz3 , and David E. Goldberg2
1
Missouri Estimation of Distribution Algorithms Laboratory
University of Missouri, St. Louis, MO
http://medal.cs.umsl.edu/
pelikan@cs.umsl.edu
2
Illinois Genetic Algorithms Laboratory (IlliGAL)
University of Illinois at Urbana-Champaign, Urbana, IL
{ksastry,deg}@uiuc.edu
3
Department of Cognitive Psychology
Bayerische Julius-Maximilians University of W¨rzburg
u
mbutz@psychologie.uni-wuerzburg.de
M. Pelikan, K. Sastry, M. V. Butz, and D. E. Goldberg hBOA on Random Decomposable Problems

Background Objectives rADPs Experiments Conclusions
Background
Testing optimization algorithms
Testing on the boundary of the design envelope using artificial
test problems.
Examples: Concatenated traps, hierarchical traps.
Testing on classes of random problems.
Examples: MAXSAT, Ising spin glass.
Testing on real-world problems or their approximations.
Examples: Military antenna design, cluster optimization.
M. Pelikan, K. Sastry, M. V. Butz, and D. E. Goldberg hBOA on Random Decomposable Problems

Background Objectives rADPs Experiments Conclusions
Objectives
Objectives
Propose random additively decomposable problems (rADPs).
Three objectives
Scalability.
Known optimum.
Easy generation of random instances.
Introduce overlap between subproblems to study its effects.
Test various genetic and evolutionary algorithms on rADPs
Hierarchical Bayesian optimization algorithm (hBOA).
Genetic algorithm (GA).
Univariate marginal distribution algorithm (UMDA).
Hill climbing (HC).
M. Pelikan, K. Sastry, M. V. Butz, and D. E. Goldberg hBOA on Random Decomposable Problems

Background Objectives rADPs Experiments Conclusions
Additively Decomposable Problems (rADPs)
Additively Decomposable Problems
Fitness defined as
m
f (X1 , X2 , . . . , Xn ) = fi (Si ),
i=1
where
n is the number of bits (variables),
m is the number of subproblems,
Si is the subset of variables in ith subproblem.
M. Pelikan, K. Sastry, M. V. Butz, and D. E. Goldberg hBOA on Random Decomposable Problems

Background Objectives rADPs Experiments Conclusions
ADPs of Bounded Order With and Without Overlap
Order-k ADPs with and without overlap
Each subproblem contains k bits.
Separable problems contain non-overlapping subproblems:
There may be overlap in o bits between neighboring
subproblems with subproblems defined in contiguous blocks:
Bits may be shuffled to not enforce tight subproblems.
Verifying optimum of order-k ADPs
Need to use problem-specific knowledge.
Use dynamic programming to solve in O(2k n) evaluations.
M. Pelikan, K. Sastry, M. V. Butz, and D. E. Goldberg hBOA on Random Decomposable Problems

Background Objectives rADPs Experiments Conclusions
Random ADPs (rADPs)
Generating random ADPs
User sets m, k, and o.
Randomly generate 2k values for each subproblem from [0,1).
Reorder bits randomly.
Test Problems
Order of subproblems, k = 4.
Overlap o = 0, o = 1, and o = 2.
Vary problem size to study scalability.
Generate 1000 random instances for each m, k, and o.
M. Pelikan, K. Sastry, M. V. Butz, and D. E. Goldberg hBOA on Random Decomposable Problems

Background Objectives rADPs Experiments Conclusions
Experiments
Population-based methods
Population size
Minimum population size to ensure convergence in 10 out of
10 independent runs (by bisection method).
Selection and replacement strategies
Binary tournament selection.
Restricted tournament replacement.
Genetic algorithm
One-point or uniform crossover (pc = 0.6), bit-flip mutation
(pm = 1/n).
Hill climbing
Bit-flip mutation with pm = k/n.
Results averaged over 10 runs.
M. Pelikan, K. Sastry, M. V. Butz, and D. E. Goldberg hBOA on Random Decomposable Problems

Background Objectives rADPs Experiments Conclusions
Comparison of Population-Based Methods for o = 0 and 2
o=0 o=2
UMDA UMDA
5
6
10
10 GA (two−point) GA (two−point)
Number of evaluations
GA (uniform)
Number of evaluations
GA (uniform)
hBOA: O(n 1.85) hBOA: O(n 2.02)
5 4
10 10
4
10 3
10
3
10
2
10
15 30 60 120 8 16 32 64
Problem size (n) Problem size (n)
M. Pelikan, K. Sastry, M. V. Butz, and D. E. Goldberg hBOA on Random Decomposable Problems

Background Objectives rADPs Experiments Conclusions
Effects of Overlap on hBOA
2.02
hBOA (overlap=2): O(n )
5
10 1.92
hBOA (overlap=1): O(n )
Number of evaluations
1.85
hBOA (overlap=0): O(n )
4
10
3
10
2
10
7.5 15 30 60 120
Problem size (n)
M. Pelikan, K. Sastry, M. V. Butz, and D. E. Goldberg hBOA on Random Decomposable Problems

Background Objectives rADPs Experiments Conclusions
Effects of Overlap on GA and HC
GA HC
6
10
GA (overlap=2) HC (overlap=2)
GA (overlap=1) HC (overlap=1)
7
10
Number of evaluations
GA (overlap=0)
Number of evaluations
5 HC (overlap=0)
10
6
10
4 5
10 10
4
10
3
10
3
10
2 2
10 10
10 20 40 80 7.5 15 30 60
Problem size (n) Problem size (n)
M. Pelikan, K. Sastry, M. V. Butz, and D. E. Goldberg hBOA on Random Decomposable Problems

Background Objectives rADPs Experiments Conclusions
Sensitivity of hBOA and GA to Problem Difficulty
GA (uniform), worst 3.125%−100% runs
7
10 hBOA, worst 3.125%−100% runs
Number of evaluations
6
10
5
10
4
10
3
10
15 30 60 120
Problem size (n)
M. Pelikan, K. Sastry, M. V. Butz, and D. E. Goldberg hBOA on Random Decomposable Problems

Background Objectives rADPs Experiments Conclusions
Summary and Conclusions
Summary
Proposed random additively decomposable problems (rADPs)
Problem size, difficulty, and overlap easily controlled.
Problem instances deterministically solvable in linear time.
Large numbers of random instances can be generated easily.
Tested hBOA, GA, UMDA, and HC on rADPs.
Code & papers available at http://medal.cs.umsl.edu
Conclusions
Best performance achieved with hBOA (O(n2.02 ) evaluations).
Deception not enforced, but linkage learning still important.
Recombination is less sensitive to overlap than mutation.
Difficulty of rADPs does not seem to vary much.
Can be used to thoroughly test other optimization methods.
M. Pelikan, K. Sastry, M. V. Butz, and D. E. Goldberg hBOA on Random Decomposable Problems