Sub-Structural Niching in Estimation of
Distribution Algorithms
Kumara Sastry, H. A. Abbass, D. E. Goldberg, D. D. Johnson
Illinois Genetic Algorithms Laboratory
Department of General Engineering
University of Illinois at Urbana-Champaign
http://www-illigal.ge.uiuc.edu
ksastry@uiuc.edu
Supported by AFOSR F49620-03-1-0129, NSF/DMR at MCC DMR-99-76550, NSF/ITR at
CPSD DMR-0121695, DOE at FS-MRL DEFG02-91ER45439.

Outline
Background & purpose
Sub-structural niching framework
Sub-structure identification
Sub-structure quality assessment
Sub-structure niche preservation
Results & discussion
Niche preservation capability
Scalability
Summary & conclusions

Background
Niching is an essential component
Hierarchical, multiobjective, dynamic, multimodal problems
Existing methods maintain diversity at individual level
Distance metric based on entire chromosome
Not usually adaptive to niche size and distribution
Need diversity at sub-structural level
Exploits the working mechanism of EDAs
Adaptive niche size and distribution

Purpose: Design Sub-Structural Niching
Develop sub-structural niching framework
Sub-structure identification
What are the components?
Sub-structure quality estimation
What are the relative quality of sub-components?
Sub-structure niche preservation
Which sub-structures to preserve?
What market share to allot to competing sub-structures?

Sub-Structure Identification:
Use Linkage-Learning Techniques
Sub-structure identification
What are the components?
Use linkage-learning techniques
Demonstrate using extended compact GA [Harik, 1999]
Can also use DSMGA [Yu et al, 2003]
Brief description of eCGA next…

Extended Compact Genetic Algorithm (eCGA)
A Probabilistic model building GA (Harik, 1999)
Builds models of good solutions as linkage groups
Key idea:
Good probability distribution ≡ Linkage learning
Key components:
Representation: Marginal product model (MPM)
Marginal distribution of a gene partition
Quality: Minimum description length (MDL)
Occam’s razor principle
All things being equal, simpler models are better
Search Method: Greedy heuristic search

Marginal Product Model (MPM)
Partition variables into clusters
Product of marginal distributions on a partition of genes
Gene partition maps to linkage groups
MPM: [1, 2, 3], [4, 5, 6], … [l-2, l -1, l]
...
x1 x2 x3 x4 x5 x6 xl-2 xl-1 xl
{p000, p001, p010, p100, p011, p101, p110, p111}

Minimum Description Length Metric
Hypothesis: For an optimal model
Model size and error is minimum
Model complexity, Cm
# of bits required to store all marginal probabilities
Compressed population complexity, Cp
Entropy of the marginal distribution over all partitions
MDL metric, Cc = Cm + Cp

Building an Optimal MPM
1. Assume independent genes ([1],[2],…,[l])
2. Compute MDL metric, Cc
3. All combinations of two subset merges
Eg., {([1,2],[3],…,[l]), ([1,3],[2],…,[l]), ([1],[2],…,[l-1,l])}
4. Compute MDL metric for all model candidates
5. Select the set with minimum MDL,
6. If , accept the model and go to step 2.
7. Else, the current model is optimal

Extended Compact Genetic Algorithm [Harik, 1999]
1. Initialize the population (usually random initialization)
2. Evaluate the fitness of individuals
3. Select promising solutions (e.g., tournament selection)
4. Build the probabilistic model
Optimize structure & parameters to best fit selected
individuals
Automatic identification of sub-structures
5. Sample the model to create new candidate solutions
Effective exchange of building blocks
6. Repeat steps 2–7 till some convergence criteria are met

Sub-Structure Identification: Models built by EDAs
Use model-building procedure of extended compact GA
Partition genes into (mutually) independent groups
Start with the lowest complexity model
Search for a least-complex, most-accurate model
Model Structure Metric
[X0] [X1] [X2] [X3] [X4] [X5] [X6] [X7] [X8] [X9] [X10] [X11] 1.0000
[X0] [X1] [X2] [X3] [X4X5] [X6] [X7] [X8] [X9] [X10] [X11] 0.9933
[X0] [X1] [X2] [X3] [X4X5X7] [X6] [X8] [X9] [X10] [X11] 0.9819
[X0] [X1] [X2] [X3] [X4X5X6X7] [X8] [X9] [X10] [X11] 0.9644
M M
[X0] [X1] [X2] [X3] [X4X5X6X7] [X8X9X10X11] 0.9273
M M
[X0X1X2X3] [X4X5X6X7] [X8X9X10X11] 0.8895

Purpose: Design Sub-Structural Niching
Develop sub-structural niching framework
Sub-structure identification
What are the components?
Sub-structure quality estimation
What are the relative quality of sub-components?
Sub-structure niche preservation
Which sub-structures to preserve?
What marker share to allot to competing sub-structures?

Designing Sub-Structural Niching Framework
Sub-structure identification
Use linkage-learning techniques
E.g., eCGA
Sub-structure quality estimation
What are the relative quality of sub-components?
Use fitness-estimation technique used in developing
probabilistic endogenous fitness model
Pelikan et al 2004, Sastry et al 2004

Sub-Structure Quality Estimation
Identify key sub-structures of the search problem
Estimate the fitness of sub-structure instances
Average fitness of all Average fitness of all
Fitness of
= – evaluated individuals in
evaluated individuals with
substructure instance
the schema instance the population
Used to estimate fitness of individuals [Sastry, Pelikan, &
Goldberg, 2004. Pelikan, Sastry & Goldberg, 2004]

Designing Sub-Structural Niching Framework
Sub-structure identification
Use linkage-learning techniques
Sub-structure quality estimation
Schema fitness estimation
Sub-structure niche preservation
Proportional to sub-structure fitness
Top κ sub-structures (truncation selection)
Tournament selection

Sub-Structure Niche Preservation
Allocate more copies to highly-fit sub-structures
Sub-structure market share proportional to fitness
Fitness of sub-structure instance
Market-share of a
=
substructure instance Sum of all sub-structure fitnesses
[Sastry, Abbass, Goldberg, 2004]

Test Problem: Modified m-k Trap Functions
Linkage learning is critical to GA success
Massively multimodal: 2m distinct optima
[Ackley, 1987; Goldberg, 1987; Deb & Goldberg, 1993]

Performance Comparison
Compare performance of sub-structural niching to that of
restricted tournament selection (RTS) [Harik, 1995]
Used in hierarchical BOA [Pelikan & Goldberg, 2001]
For every offspring
Randomly select w individuals from the parent population
Find the parent closest to the offspring
If the offspring is better than the closest parent
Replace the parent with the offspring

Experimental Methodolgy
Sub-structural niching Vs. RTS
Stability of maintaining multiple niches
Market share allocation of different niches
Population size scalability
Empirical results:
Tournament selection w/o replacement (s = 8)
Results are averaged over 50—100 independent runs

Sub-Structural Niching: Stably Maintains All Niches
RTS SSN
Significantly less stable Stable maintenance of all
niches over long duration
Loses niches over time
Smaller population sizes
Faster first hitting times

SSN is Stable in Maintaining Multiple Niches
SSN is more stable than RTS
Stably maintains all global optima at desired level
Sub-Structural Niching (SSN)
RTS

Market Share Allocation Of A Niche
SSN consistently allocates market share to all niches at
desired level
RTS has large variance in market share allocation
RTS loses some of the niches
RTS
Sub-Structural
Niching (SSN)

Population Sizing and Success Probability
Probability of maintaining at least one copy of all niches
Different population sizes and generations
SSN requires significantly less population size than RTS and
maintains diversity longer
SSN
RTS

Population Size Scalability of SSN
Minimum population size required to maintain at least
one copy of nopt-1 or more global optima
SSN requires scales significantly better than RTS
RTS Population Sizing
[Mahfoud, 1995]
RTS
SSN

Sub-Structural Niching Enhances Niching Efficiency
Niching: Maintain diversity at individual level
Linkage learning: Problem decomposition + BB identification
Substructural niching: Diversity among building blocks
Linkage learning + BB ranking + niching strategy
[Sastry et al, 2005]

Future Work
Designing sub-structural niching for BOA/hBOA
Not so straightforward
Co-evolutionary niching might be a good alternative
Implicit sub-structural niching
Performance of sub-structural niching on multiobjective
and hierarchical problems
Which niche preserving methods to use and when?