Empirical Analysis of Ideal Crossover on
Random Additively Decomposable Problems
Kumara Sastry1, Martin Pelikan2, David E. Goldberg1
1Illinois Genetic Algorithms Laboratory (IlliGAL)
University of Illinois at Urbana-Champaign, Urbana, IL 61801
2MissouriEstimation of Distribution Algorithm Lab (MEDAL)
University of Missouri at St. Louis, St. Louis, MO
ksastry@uiuc.edu, pelikan@cs.umsl.edu, deg@uiuc.edu
http://www.illigal.uiuc.edu, http://medal.cs.umsl.edu
Supported by AFOSR FA9550-06-1-0096, NSF DMR
03-25939, and CAREER ECS-0547013. Computational
results obtained using CSE’s Turing cluster.

Roadmap
Adversarial test problem design
Random additively decomposable problems
Ideal crossover
Scalability of selectorecombinative GAs
Population sizing and Run duration
Experimental Procedure
Key Results
Summary and Conclusions
2

Adversarial Test Problem Design
Test systems on boundary of design envelope
Common approach in designing complex systems
GAs are complex systems [Goldberg, 2002]
GA design envelope characterized by different dimensions
of problem difficulty
Thwart the mechanism of GAs to the extreme
Fluctuating
R
P Noise
Deception Scaling
3

Random Additively Decomposable Problem
Focus on nearly decomposable problems [Simon, 1960]
Three desired features
Scalability: Able to control problem size and difficulty
Known optimum: Allows comparison of different solvers
Easy problem instance generation
rADP fitness function:
Si represents variable subset for ith subproblem
Each subset consists of k bits
gi is the fitness of the ith subproblem
gi is sampled from uniform distribution U[0,1]
4

Ideal Crossover: Exchange Building Blocks
Population sizing and run duration models assume good
exchange of building blocks
Simulate what we ideally want to achieve with model-
building GAs
For example, extended compact GA [Harik, 1999]
Ideal recombination operator
Effectively exchange building blocks
Don’t disrupt any building block
Uniform building-block-wise crossover
BBs #1 and #3 exchanged
Exchange BBs with probability 0.5
5

Purpose: Analyze Ideal Crossover on rADPs
Analyze behavior of selectorecombinative GAs on rADPs
Verify the validity of lessons learned from adversarial test
problems
Expand the pool of test problems
6

Selectorecombinative GA Population Sizing
Noise-to-fitness
variance ratio
Error tolerance # Components (# BBs)
Signal-to-Noise ratio # Competing sub-components
Gambler’s ruin model
[Harik, et al, 1997]
Combines decision
making and supply
models
Additive Gaussian noise
with variance σ2N
7

GA Run Duration (Selection)
Selection-Intensity based model [Bulmer, 1980; Mühlenbein &
Schlierkamp-Voosen, 1993; Thierens & Goldberg, 1994; Bäck, 1994; Miller & Goldberg,
1995 & 1996]
Derived for the OneMax problem
Applicable to additively-separable problems [Miller, 1997]
Problem size (m·k )
Selection Intensity
[Miller & Goldberg, 1995;
Sastry & Goldberg, 2002]
8

GA Run Duration (Drift)
Accumulation of stochastic errors due to finite population
Proportion of competing sub-solutions change due to drift
Drift time [Goldberg & Segrest, 1987]:
Substituting population sizing bound
9

Signal-to-Noise Ratio for rADPs
Signal d is the fitness difference between best and second
best sub-solutions
jth order statistic follows a Beta distribution with
α = j and β = 2k-j+1
Probability density function (p.d.f) of d:
p.d.f. of sub-solution fitness variance approximation
E[1/d] = 2k and E[σ2BB] ≈ 1/12
10

Assumptions and Experimental Setup
Non-overlapping sub-problems
Identical sub-problems across different partitions
g1 = g2 = … = gm
Selectorecombinative GA
Binary tournament selection
10,000 random problem instances
m = 5 – 50, k = 3, 4, and 5
Minimum population size determined by bisection method
Population correctly converges to at least m-1 out of m BBs in
49 out of 50 independent runs
Averaged over 30 bisection runs
Results averaged over 1,500 GA runs
11

Population Sizing & Run Duration Histograms
Population size Run duration
m = 50 m = 50
Tail increases with m
0.15-0.59% of rADP instances require # evals greater
than 3σ from the median
12

Easy and Hard Problem Instances
Hard instance
Subsolution fitness
Min signal
Max noise
Sorted subsolution index
Easy instance
Subsolution fitness
Max signal
Min noise
Sorted subsolution index
13

Population Sizing Scalability
Gambler’s ruin model bounds population sizing
14

Run Duration Scalability
Selection-intensity based run-duration model bounds
median convergence time
Drift-time model bounds convergence time
15

Number of Function Evaluations Scalability
Facetwise models are applicable to rADPs
Testing on adversarial problems bounds performance
of GAs on rADPs
16

Easy and Hard Scalable Problem Instances
Easy Scalable Hard Scalable
Instances Instances
Easy instances have large signal difference
Hard instances have very small signal difference
17

Summary and Conclusions
Empirically analyzed behavior of selectorecombinative GA
with ideal crossover:
Class of random additively decomposable problems
Sub-solution fitness sampled from uniform distribution
Verified applicability of facetwise models:
Developed based on adversarial problems
GA scales subquadratically with problem size
Analyzed easy and hard problem instances:
Easy problem instances have large signal, small variance.
Hard problem instances have small signal, large variance
18