A two stage memetic computing approach for real-valued domains with a high performance

1. Re-sampling Search: A Seriously Simple
Memetic Approach with a High
Performance
Fabio Caraffini, Ferrante Neri, Mario Gongora and Benjamin
N. Passow
De Montfort University
United Kingdom
17.04.2013
(SSCI2013, Singapore)

2. Outline
Background
Ockham’s Razor in Memetic Computing
Re-sampling Search (RS)
Numerical/Graphical Results
Conclusions and Future Developments

3. Background
Memetic Computing (MC): structured set of heterogeneous
components for solving problems
Ockham’s Razor in MC: simple algorithms can display a
performance which is as good as that of complex algorithms 1
1
G. Iacca, F. Neri, E. Mininno, Y.S. Ong, M.H. Lim, Ockham’s Razor in Memetic Computing: Three Stage
Optimal Memetic Exploration, Information Sciences, Elsevier, Volume 188, pages 17-43, April 2012

4. Ockham’s Razor in MC: why simplicity?
Algorithmic Design Issues
A simple structure is easier to control and allows us to
understand the actual importance of certain operators
Complex structures often employ redundant operators which
are hard to identify and whose coordination logic is not clear
The increase of the algorithmic complexity is not always worth
the improvement of the performances

5. Ockham’s Razor in MC: why simplicity?
Engineering Applications Issues
Complex structures:
Usually require many fitness
evaluations
Are computationally expensive
(high algorithmic overhead)
Make extensive use of hardware
resources
(large memory footprint)

6. Re-sampling Search (RS)
Multi-start single-solution structure with high performances
Requires only 3 memory slots:
Xe (elite): global best solution
Xt (trial): trial solution
Xs (solution): obtained by
perturbing Xt
The local searcher’s performance
highly depends on the starting
point, even with uni-modal
functions

7. Re-sampling Search (RS)
Multi-start single-solution structure with high performances
Requires only 3 memory slots:
Xe (elite): global best solution
Xt (trial): trial solution
Xs (solution): obtained by
perturbing Xt

8. Re-sampling Search (RS)
Multi-start single-solution structure with high performances
Requires only 3 memory slots:
Xe (elite): global best solution
Xt (trial): trial solution
Xs (solution): obtained by
perturbing Xt

9. Re-sampling Search (RS)
Multi-start single-solution structure with a high performance
Requires only 3 memory slots:
Xe (elite): global best solution
Xt (trial): trial solution
Xs (solution): obtained by
perturbing Xt

10. Re-sampling Search (RS)
Multi-start single-solution structure with a high performance
Requires only 3 memory slots:
Xe (elite): global best solution
Xt (trial): trial solution
Xs (solution): obtained by
perturbing Xt

11. Re-sampling Search (RS)
Multi-start single-solution structure with a high performance
Requires only 3 memory slots:
Xe (elite): global best solution
Xt (trial): trial solution
Xs (solution): obtained by
perturbing Xt
Stop criterion:
n
i=1
ρ[i]
bi −ai
2
< ε
where (bi − ai ) is the width of the
decision space D along the ith
dimension.
parameters setting:
Initial exploratory radius ρ = 40%
of the width of the decision space
Precision threshold ε = 10−6

12. Numerical Results
We considered a set of 76 problems:
The CEC2005 benchmark in 30 dimensions
(25 test problems)
The BBOB2010 benchmark in 100 dimensions
(24 test problems)
The CEC2008 benchmark in 1000 dimensions
(7 test problems)
The CEC2010 benchmark in 1000 dimensions
(20 test problems)
We compared RS against 7 complex algorithms by performing the
average value and the standard deviation over 100 runs, the
Wilcoxon Rank-Sum test and the Holm-Bonferroni procedure.

13. Numerical results
Table : Components and memory requirement of the algorithms under
consideration
Algorithm Features Memory slots
CLPSO PSO structure 2 × Np
modified velocity rule
JADE DE structure Np+archive
samples from distribution
archive
MA-CMA-Chains GA structure Np 1 + n2
covariance matrix driven search
for multiple individuals
MA-SSW-Chains GA structure Np (1 + n)
Solis-Wets Local Search
CCPSO2 PSO structure 2 × Np
variable decomposition
MDE-pBX DE structure Np+neighborhood
multiple mutation strategies
self adaptive parameters
3SOME single-solution structure 3
3 sequential operators
trial and error coordination
RS single-solution structure 3
2 operators

14. Numerical Results
Table : Holm-Bonferroni test on the Fitness, reference algrithm = RS
(Rank = 4.72e+00 )
j Optimizer Rank zj pj δ/j Hypothesis
1 CCPSO2 4.55e+00 -5.64e-01 2.87e-01 5.00e-02 Accepted
2 3SOME 4.24e+00 -1.60e+00 5.43e-02 2.50e-02 Accepted
3 MACh 4.13e+00 -1.95e+00 2.55e-02 1.67e-02 Accepted
4 MDE-pBX 3.82e+00 -2.99e+00 1.39e-03 1.25e-02 Rejected
5 CLPSO 3.43e+00 -4.25e+00 1.07e-05 1.00e-02 Rejected
6 JADE 3.09e+00 -5.38e+00 3.81e-08 8.33e-03 Rejected

15. Graphical results
Function f5 from BBOB2010 in 100 dimensions
0 1e+5 2e+5 3e+5 4e+5 5e+5
0
200
400
600
800
1000
1200
1400
1600
1800
2000
Fitness function call
Fitnessvalue
RS
CLPSO
JADE
3SOME
MDE pBX
CCPSO2
MACh

16. Graphical results
Function f14 from CEC2010 in 1000 dimensions
0 1e+6 2e+6 3e+6 4e+6 5e+6
10
8
10
9
10
10
10
11
10
12
Fitness function call
Fitnessvalue[Logarithmicscale]
RS
CLPSO
JADE
3SOME
MDE pBX
CCPSO2
MACh

17. Graphical results
Average computational complexity (overhead VS dimensionality)

18. Conclusions and Future Developments
We proposed an extremely simple Memetic Computing
approach, competitive with the state-of-the-art optimization
algorithms
RS involves a minimal memory footprint and modest
computational overhead
RS is meant to be used on-board of embedded systems.
Future work will involve robotic applications. At the moment
we are trying to use this logic to tune the PID regulator of the
heading control system of an indoor helicopter

19. Thank you for your attention!
Any questions?