Simple MC algorithm

1. Handling Non-Separability in Three
Stage Optimal Memetic Exploration
Ilpo Poikolainen Fabio Caraffini Ferrante Neri
Matthieu Weber
Department of Mathematical Information Technology,
University of Jyv¨askyl¨a
24.05.2012

2. 1 Summary
2 Separability of a function
3 Rotation Invariant search in Three Stage Optimal Memetic
Exploration
4 Numerical Results

3. Separability of a function
If function is separable then
min f (x) =
min f (x1) + ... + min f (xn), where
x ∈ Rn.
In optimization fully separable
functions can be optimized one
variable at time.
For optimizer to be able to be
effective on non-separable
problems it needs to perform
diagonal movements.
Effect of function rotation on
exponential crossover in
Differential Evolution (DE).
se
st st2
st1
s't2
s't1
x
x'
yy'

4. Operators of Three stage optimal memetic exploration: an
overview
Composed of three different search operators.

5. Operators of Three stage optimal memetic exploration: an
overview
Composed of three different search operators.
Stochastic long distance exploration. Exponential crossover
with high crossover rate: higher chance to perform diagonal
movements.

6. Operators of Three stage optimal memetic exploration: an
overview
Composed of three different search operators.
Stochastic long distance exploration. Exponential crossover
with high crossover rate: higher chance to perform diagonal
movements.
Stochastic medium/short distance exloration. Exponential
crossover with low crossover rate: diagonal movements are
limited.

7. Operators of Three stage optimal memetic exploration: an
overview
Composed of three different search operators.
Stochastic long distance exploration. Exponential crossover
with high crossover rate: higher chance to perform diagonal
movements.
Stochastic medium/short distance exloration. Exponential
crossover with low crossover rate: diagonal movements are
limited.
Deterministic local search. Searches along the axis, no
diagonal movements performed.

8. Operators of Three stage optimal memetic exploration: an
overview
Composed of three different search operators.
Stochastic long distance exploration. Exponential crossover
with high crossover rate: higher chance to perform diagonal
movements.
Stochastic medium/short distance exloration. Exponential
crossover with low crossover rate: diagonal movements are
limited.
Deterministic local search. Searches along the axis, no
diagonal movements performed.
Framework for co-operation between operators.

9. Stochastic long distance search operator
This exploration move attempts to
detect new promising solution
within the entire search space.
Utilizes exponential crossover from
DE with high crossover rate.
Repeated until better solution is
found.
Xe
Xt1
Xt
Xt2

10. Rotationally invariant stochastic medium distance search
operator
Diagonal movements required?

11. Rotationally invariant stochastic medium distance search
operator
Diagonal movements required?
Creates hypercube around current
solution with sidewidth of 0.2
times total space width.
Xe
Xr
Xs
Xv
Xt
K
F'

12. Rotationally invariant stochastic medium distance search
operator
Diagonal movements required?
Creates hypercube around current
solution with sidewidth of 0.2
times total space width.
Exponential crossover is replaced
with DE/current-to-rand/1
mutation.
xt = xe +K(xv −xe)+F (xr −xs),
where K ∈ [0, 1] and F = K ∗ F.
Xe
Xr
Xs
Xv
Xt
K
F'

13. Rotationally invariant stochastic medium distance search
operator
Diagonal movements required?
Creates hypercube around current
solution with sidewidth of 0.2
times total space width.
Exponential crossover is replaced
with DE/current-to-rand/1
mutation.
xt = xe +K(xv −xe)+F (xr −xs),
where K ∈ [0, 1] and F = K ∗ F.
Repeated for given budget and no
better solution is found.
Xe
Xr
Xs
Xv
Xt
K
F'

14. Deterministic short distance exploration
Attempts to exploit the promising
search directions (along the axis).
Repeated for given budget and
depending if better solution was
found one of the earlier operators
is activated.
Xe
p p/2
Xs

15. Functioning scheme of rotation invariant RI3SOME
Long Stochastic short
Deterministic short
S
F
S
S or F

16. Comparison algorithms
Real-parameter black-box optimization benchmark 2010.
Consisting 24 problems for 10,40 and 100 dimensions run for
Dim ∗ 5000 fitness evaluations.
Algorithms are compared using Wilcoxon Rank-sum test on
the fitness values over 100 runs.
N. Hansen, A. Auger, S. Finck, R. Ros, et al. Real-parameter
black-box optimization benchmarking 2010: Noiseless
functions definitions, Technical Report RR-6829, INRIA, 2010.
Comparison algorithms: 3SOME, Computational Efficient
Covariance Matrix Evolution Strategy (1+1)-CMAES and
DE/current-to-rand/1.

17. Numerical results
RI3SOME is equal or better on all non-separable problems
(f6-f24) in 40 and 100 dimensions than 3SOME while gets
outperformed on some of the separable problems (f1-f5) as
expected.

18. Numerical results
RI3SOME is equal or better on all non-separable problems
(f6-f24) in 40 and 100 dimensions than 3SOME while gets
outperformed on some of the separable problems (f1-f5) as
expected.
RI3SOME Outperforms DE/current-to-rand/1 on atleast
22 test problems in each of dimensions 10,40 and 100.

19. Numerical results
RI3SOME is equal or better on all non-separable problems
(f6-f24) in 40 and 100 dimensions than 3SOME while gets
outperformed on some of the separable problems (f1-f5) as
expected.
RI3SOME Outperforms DE/current-to-rand/1 on atleast
22 test problems in each of dimensions 10,40 and 100.
(1+1)-CMAES gets outperformed in most of the separable
problems (f2-f5) while (1+1)-CMAES outperforms
RI3SOME on 22 non-separable problems, gets outperformed
on 8 and is equal on 27.

20. Computational overhead
Table : Computational overhead and memory slot requirements
Algorithm Computational Overhead[s] Memoty Slots
(1+1)-CMA-ES 3.2706 n+2
RI3SOME 1.5246 3

21. Conclusions
Proposed integration with 3SOME algorithm and a
rotationally invariant mutation from DE shows promising
results on handling non-separable problems within 3SOME
framework.
RI3SOME algorithm retains all original properties of 3SOME:
Simple structure with low computational requirements is
preferable for applications characterized by limited hardware.

22. Thanks for your attention.
Questions?