1. Introduction An Optimization CSA KLd , Rd and VNd Entropic Divergences Results Conclusions
An Information-Theoretic Approach
for Clonal Selection Algorithms
V. Cutello, G. Nicosia, M. Pavone, G. Stracquadanio
Department of Mathematics and Computer Science
University of Catania
Viale A. Doria 6, 95125 Catania, Italy
(cutello, nicosia, mpavone, stracquadanio)@dmi.unict.it
Seminar - DMI - Catania, 12 July 2010
An Information-Theoretic Approach for Clonal Selection Algorithms
Mario Pavone – mpavone@dmi.unict.it – http://www.dmi.unict.it/mpavone/

2. Introduction An Optimization CSA KLd , Rd and VNd Entropic Divergences Results Conclusions
Outline
1 Introduction
Global Optimization
Numerical Minimization Problem
Artificial Immune System
2 An Optimization CSA
i-CSA operators
pseudo-code of CSA
3 KLd , Rd and VNd Entropic Divergences
Learning Process
Kullback-Leibler divergence
Rényi generalized divergence
Von Neumann divergence
Comparison of the divergences
4 Results
5 Conclusions
An Information-Theoretic Approach for Clonal Selection Algorithms
Mario Pavone – mpavone@dmi.unict.it – http://www.dmi.unict.it/mpavone/

3. Introduction An Optimization CSA KLd , Rd and VNd Entropic Divergences Results Conclusions
Global Optimization
Global Optimization (GO): finding the best set of
parameters to optimize a given objective function
GO problems are quite difficult to solve: there exist
solutions that are locally optimal but not globally
GO requires finding a setting x = (x1 , x2 , . . . , xn ) ∈ S,
where S ⊆ Rn is a bounded set on Rn , such that a certain
n-dimensional objective function f : S → R is optimized.
GOAL: findind a point xmin ∈ S such that f (xmin ) is a global
minimum on S, i.e. ∀x ∈ S : f (xmin ) ≤ f (x).
It is difficult to decide when a global (or local) optimum has
been reached
There could be very many local optima where the
algorithm can be trapped
the difficulty increases proportionally with the problem dimension
An Information-Theoretic Approach for Clonal Selection Algorithms
Mario Pavone – mpavone@dmi.unict.it – http://www.dmi.unict.it/mpavone/

4. Introduction An Optimization CSA KLd , Rd and VNd Entropic Divergences Results Conclusions
Numerical Minimization Problem
Let be x = (x1 , x2 , . . . , xn ) the variables vector in Rn ;
Bl = (Bl1 , Bl2 , . . . , Bln ) and Bu = (Bu1 , Bu2 , . . . , Bun ) the lower
and the upper bounds of the variables, such that
xi ∈ Bli , Bui (i = 1, . . . , n).
GOAL: minimizing f (x) the objective function
min(f (x)), Bl ≤ x ≤ Bu
The first 13 functions taken from [Yao et al., IEEE TEC, 1999],
were used as benchmark to evaluate the performances and
convergence ability.
These functions belong to two different categories: unimodal,
and multimodal with many local optima.
An Information-Theoretic Approach for Clonal Selection Algorithms
Mario Pavone – mpavone@dmi.unict.it – http://www.dmi.unict.it/mpavone/

5. Introduction An Optimization CSA KLd , Rd and VNd Entropic Divergences Results Conclusions
Artificial Immune System
Articial Immune Systems - AIS
Immune System (IS) is the main responsible to protect the
organism against the attack from external microorganisms,
that might cause diseases;
The biological IS has to assure recognition of each
potentially dangerous molecule or substance
Artificial Immune Systems are a new paradigm of the
biologically-inspired computing
Three immunological theory: immune networks, negative
selection, and clonal selection
AIS have been successfully employed in a wide variety of
different applications
[Timmis et al.: J.Ap.Soft.Comp., BioSystems, Curr. Proteomics, 2008]
An Information-Theoretic Approach for Clonal Selection Algorithms
Mario Pavone – mpavone@dmi.unict.it – http://www.dmi.unict.it/mpavone/

6. Introduction An Optimization CSA KLd , Rd and VNd Entropic Divergences Results Conclusions
Artificial Immune System
Clonal Selection Algorithms - CSA
CSA represents an effective mechanism for search and
optimization
[Cutello et al.: IEEE TEC & J. Comb. Optimization, 2007]
Cloning, Hypermutation and Aging operators: features key
of CSA
Cloning: triggers the growth of a new population of
high-value B cells centered on a higher affinity value
Hypermutation: can be seen as a local search procedure
that leads to a faster maturation during the learning phase.
Aging: aims to generate diversity inside the population,
and thus to avoid getting trapped in a local optima.
increasing or decreasing the allowed time to stay in the
population (δ) influences the convergence process
An Information-Theoretic Approach for Clonal Selection Algorithms
Mario Pavone – mpavone@dmi.unict.it – http://www.dmi.unict.it/mpavone/

7. Introduction An Optimization CSA KLd , Rd and VNd Entropic Divergences Results Conclusions
Artificial Immune System
Learning Capability of CSA
Same classes of functions were used to analyze the
learning process
Three relative entropies were used: Kullback-Leibler, Rényi
generalized and Von Neumann divergences
The learning analysis was made studying the gain both
with respect to the initial distribution, and the ones based
on the information obtained in previous step.
An Information-Theoretic Approach for Clonal Selection Algorithms
Mario Pavone – mpavone@dmi.unict.it – http://www.dmi.unict.it/mpavone/

8. Introduction An Optimization CSA KLd , Rd and VNd Entropic Divergences Results Conclusions
i-CSA operators
Cloning Operator
Cloning operator clones each B cell dup times (P (clo) )
Thanks to this operator CSA produces individuals with higher
affinities (higher fitness function values)
The strategy to set the age is crucial for the search inside the
landscape
What age to assign to each clone?
Each clone was assigned a random age chosen into the range
[0, τB ] (labelled IA) [Cutello et al.: SAC 2006 & ICANNGA 2007]
Same age of parents, whilst zero when the fitness of the
mutated clones is improved [Cutello et al.: IEEE TEC 2007 & CEC 2004]
The improved proposed version: choosing the age of each clone
2
into the range [0, 3 τB ] (labelled i-CSA)
An Information-Theoretic Approach for Clonal Selection Algorithms
Mario Pavone – mpavone@dmi.unict.it – http://www.dmi.unict.it/mpavone/

9. Introduction An Optimization CSA KLd , Rd and VNd Entropic Divergences Results Conclusions
i-CSA operators
Hypermutation Operator
Tries to mutate any B cell receptor: isn’t based on an
explicit usage of a mutation probability.
There exist several different kinds of hypermutation
operator [Cutello et al.: ICARIS & CEC, 2004]
Inversely Proportional Hypermutation was designed
Mutation rate:
ˆ
α = e−ρ f (x) ,
with ˆ(x) the normalized fitness function in [0, 1]
f
Number of mutations:
M = (α × ) + 1 ,
where is the length of the B cell
An Information-Theoretic Approach for Clonal Selection Algorithms
Mario Pavone – mpavone@dmi.unict.it – http://www.dmi.unict.it/mpavone/

10. Introduction An Optimization CSA KLd , Rd and VNd Entropic Divergences Results Conclusions
i-CSA operators
Number of mutations produced
Number of Mutations of the Inversely Hypermutation Operator
220
dim=30, ρ=3.5
200 dim=50, ρ=4.0
dim=100, ρ=6.0
dim=200, ρ=7.0
180
160
140 10
9
120 8
M
7
100 6
5
80 4
3
60
2
1
40 0.4 0.5 0.6 0.7 0.8 0.9 1
20
0
0 0.2 0.4 0.6 0.8 1
normalized fitness
An Information-Theoretic Approach for Clonal Selection Algorithms
Mario Pavone – mpavone@dmi.unict.it – http://www.dmi.unict.it/mpavone/

11. Introduction An Optimization CSA KLd , Rd and VNd Entropic Divergences Results Conclusions
i-CSA operators
How the Hypermutation Operator works
Choose randomly a variable xi (i ∈ {1, . . . , = n})
Choose randomly an index j (j ∈ {1, . . . , = n}, and j = i)
Replace xi in according to the following rule:
(t+1) (t) (t)
xi = (1 − β) xi + β xj ,
Normalization of fitness: best current fitness value into the
population decreased of an user-defined threshold Θ,
This strategy is due to making CSA as blind as possible,
since is not known a priori any additional information
concerning the problem [Cutello et al.: SAC 2006]
An Information-Theoretic Approach for Clonal Selection Algorithms
Mario Pavone – mpavone@dmi.unict.it – http://www.dmi.unict.it/mpavone/

12. Introduction An Optimization CSA KLd , Rd and VNd Entropic Divergences Results Conclusions
i-CSA operators
Aging Operator
(t) (hyp)
Eliminates all old B cells, in the populations Pd , and PNc
Depends on the parameter τB : maximum number of
generations allowed
when a B cell is τB + 1 old it is erased
GOAL: produce an high diversity into the current
population to avoid premature convergence
static aging operator: when a B cell is erased
independently from its fitness value quality
elitist static aging operator: the selection mechanism does
not allow the elimination of the best B cell
An Information-Theoretic Approach for Clonal Selection Algorithms
Mario Pavone – mpavone@dmi.unict.it – http://www.dmi.unict.it/mpavone/

13. Introduction An Optimization CSA KLd , Rd and VNd Entropic Divergences Results Conclusions
pseudo-code of CSA
Clonal Selection Algorithm
CSA (d, dup, ρ, τB , Tmax )
fen := 0;
Nc := d × dup;
(t=0)
Pd := init_pop(d);
// xi = Bli + β(Bui − Bli ), – β ∈ [0, 1] is a real random value
(t=0)
comp_fit(Pd );
fen := fen + d;
while (fen < Tmax )do
(clo) (t)
PNc := Cloning (Pd , dup);
(hyp) (clo)
PNc := Hypermutation(PNc , ρ);
(hyp)
comp_fit(PNc );
fen := fen + Nc;
(t) (hyp) (t) (hyp)
(Pad , PaNc ) := aging(Pd , PNc , τB );
(t+1) (t) (hyp)
Pd := (µ + λ)-selection(Pad , PaNc );
t := t + 1;
end_while
An Information-Theoretic Approach for Clonal Selection Algorithms
Mario Pavone – mpavone@dmi.unict.it – http://www.dmi.unict.it/mpavone/

14. Introduction An Optimization CSA KLd , Rd and VNd Entropic Divergences Results Conclusions
Learning Process
To analyze the learning process were used three entropic
divergence metrics:
1 Kullback-Leibler (KLd ), or Information Gain
2 Rényi generalized (Rd ),
3 Von Neumann (VNd )
The learning process of CSA was studied with respect both
to the initial distribution, and the ones to the previous step
Distribution function:
t
Bm Bmt
(t)
fm = h
= ,
t
m=0 Bm
d
t
Bm is the number of B cells at time step t with fitness
function value m
An Information-Theoretic Approach for Clonal Selection Algorithms
Mario Pavone – mpavone@dmi.unict.it – http://www.dmi.unict.it/mpavone/

15. Introduction An Optimization CSA KLd , Rd and VNd Entropic Divergences Results Conclusions
Kullback-Leibler divergence
KLd or Information Gain
Is one of the most frequently used information-theoretic
distance measure
Is based on two probability distributions of discrete random
variable, called relative information, which found many
applications in setting important theorems in information
theory and statistics
measures the quantity of information the system discovers
during the learning phase [Cutello et al.: journal of Combinatorial
Optimization, 2007]
Formally defined as:
(t) (t ) (t) (t) (t )
KLd (fm , fm 0 ) = fm log(fm /fm 0 )
m
An Information-Theoretic Approach for Clonal Selection Algorithms
Mario Pavone – mpavone@dmi.unict.it – http://www.dmi.unict.it/mpavone/

16. Introduction An Optimization CSA KLd , Rd and VNd Entropic Divergences Results Conclusions
Kullback-Leibler divergence
Maximum KLd Principle
The gain is the amount of information the system has
already learned during its search process
Once the learning process begins, the information gain
increases monotonically until it reaches a final steady state
Maximum Kullback-Leibler principle [Cutello et al., GECCO 2003]:
dK
≥0
dt
The learning process will end when the above equation is
zero
Maximum KLd principle as termination criterion [Cutello et al.,
GECCO 2003 & JOCO 2007]
An Information-Theoretic Approach for Clonal Selection Algorithms
Mario Pavone – mpavone@dmi.unict.it – http://www.dmi.unict.it/mpavone/

17. Introduction An Optimization CSA KLd , Rd and VNd Entropic Divergences Results Conclusions
Kullback-Leibler divergence
KLd on f5 , f7 , and f10
Information Gain
25
f5
f7
f10
20
15
10
5
0
1 2 4 8 16 32
Generations
An Information-Theoretic Approach for Clonal Selection Algorithms
Mario Pavone – mpavone@dmi.unict.it – http://www.dmi.unict.it/mpavone/

18. Introduction An Optimization CSA KLd , Rd and VNd Entropic Divergences Results Conclusions
Kullback-Leibler divergence
KLd and standard deviation on f5
Clonal Selection Algorithm: i-CSA
25
20
300
15
250
200
10 150
100
50
5
0
16 32 64
0
16 32 64
Generations
An Information-Theoretic Approach for Clonal Selection Algorithms
Mario Pavone – mpavone@dmi.unict.it – http://www.dmi.unict.it/mpavone/

19. Introduction An Optimization CSA KLd , Rd and VNd Entropic Divergences Results Conclusions
Kullback-Leibler divergence
Average fitness vs. best fitness on f5
Clonal Selection Algorithm: i-CSA
4e+09
avg fitness
best fitness
3.5e+09
3e+09
25
2.5e+09 gain
20 entropy
Fitness
2e+09 15
10
1.5e+09
5
1e+09
0
16 32 64
5e+08
0
0 2 4 6 8 10
Generations
An Information-Theoretic Approach for Clonal Selection Algorithms
Mario Pavone – mpavone@dmi.unict.it – http://www.dmi.unict.it/mpavone/

20. Introduction An Optimization CSA KLd , Rd and VNd Entropic Divergences Results Conclusions
Rényi generalized divergence
Rényi generalized divergence (Rd )
Formally defined as:
(t) α
(t) (t ) 1 fm
Rd (fm , fm 0 , α) = log ,
α−1 (t ) α−1
m fm 0
with α > 0 and α = 1
[Rényi, proc. 4th Berkeley Symposium on Mathematics, Statistics
and Probability, 1961]
An Information-Theoretic Approach for Clonal Selection Algorithms
Mario Pavone – mpavone@dmi.unict.it – http://www.dmi.unict.it/mpavone/

21. Introduction An Optimization CSA KLd , Rd and VNd Entropic Divergences Results Conclusions
Von Neumann divergence
Von Neumann divergence (VNd )
Formally defined as:
(t) (t ) 1 (t) (t ) 1 (t) (t)
VNd (fm , fm 0 ) = − (fm log fm 0 )− (fm log fm )
n m
n m
[Kopp, et al., Annals of Physics, 2007]
An Information-Theoretic Approach for Clonal Selection Algorithms
Mario Pavone – mpavone@dmi.unict.it – http://www.dmi.unict.it/mpavone/

22. Introduction An Optimization CSA KLd , Rd and VNd Entropic Divergences Results Conclusions
Comparison of the divergences
KLd vs. Rd and VNd , with respect t0
Learning with respect to initial distribution (t0)
30
Von Neumann distribution (VNd)
Renyi generalized
Kullback-Leibler entropy
25
20 1.2
VNd
1
15 0.8
0.6
10
0.4
0.2
5
0
16 32 64
0
-5
16 32 64
Generations
An Information-Theoretic Approach for Clonal Selection Algorithms
Mario Pavone – mpavone@dmi.unict.it – http://www.dmi.unict.it/mpavone/

23. Introduction An Optimization CSA KLd , Rd and VNd Entropic Divergences Results Conclusions
Comparison of the divergences
KLd vs. Rd and VNd , with respect (t − 1)
Learning with respect (t-1) time step
30
Von Neumann divergence (VNd)
Renyi generalized
Kullback-Leibler entropy
25
0.9
20 0.8 VNd
0.7
0.6
15
0.5
0.4
0.3
10
0.2
0.1
5 0
16 32 64
0
-5
16 32 64
Generations
An Information-Theoretic Approach for Clonal Selection Algorithms
Mario Pavone – mpavone@dmi.unict.it – http://www.dmi.unict.it/mpavone/

24. Introduction An Optimization CSA KLd , Rd and VNd Entropic Divergences Results Conclusions
Results
In order to study the search capability, and the ability to
escape from a local optima, the first 13 functions of the
classical benchmark were taken into account from [Yao et al.,
IEEE TEC, 1999]
There are many evolutionary methodologies able to tackle
effectively the global numerical function optimization.
Among such algorithms, differential evolution (DE) has
shown better performances than other evolutionary
algorithms on complex and continuous search space
[Price, et al., journal of Global Optimization 1997, & Springer-Verlag
2005]
An Information-Theoretic Approach for Clonal Selection Algorithms
Mario Pavone – mpavone@dmi.unict.it – http://www.dmi.unict.it/mpavone/

25. Introduction An Optimization CSA KLd , Rd and VNd Entropic Divergences Results Conclusions
i-CSA vs. IA, and DE rand/1/bin [Thomsen, et al., CEC 2004]
IA i-CSA DE rand/1/bin
30 variables
f1 0.0 0.0 0.0
0.0 0.0 0.0
f2 0.0 0.0 0.0
0.0 0.0 0.0
f3 0.0 0.0 2.02 × 10−9
0.0 0.0 8.26 × 10−10
f4 0.0 0.0 3.85 × 10−8
0.0 0.0 9.17 × 10−9
f5 12 0.0 0.0
13.22 0.0 0.0
f6 0.0 0.0 0.0
0.0 0.0 0.0
f7 1.521 × 10−5 7.48 × 10−6 4.939 × 10−3
2.05 × 10−5 6.46 × 10−6 1.13 × 10−3
An Information-Theoretic Approach for Clonal Selection Algorithms
Mario Pavone – mpavone@dmi.unict.it – http://www.dmi.unict.it/mpavone/

26. Introduction An Optimization CSA KLd , Rd and VNd Entropic Divergences Results Conclusions
i-CSA vs. IA, and DE rand/1/bin [Thomsen, et al., CEC 2004]
IA i-CSA DE rand/1/bin
30 variables
f8 −1.256041 × 10+4 −9.05 × 10+3 −1.256948 × 10+4
25.912 1.91 × 104 2.3 × 10−4
f9 0.0 0.0 0.0
0.0 0.0 0.0
f10 0.0 0.0 −1.19 × 10−15
0.0 0.0 7.03 × 10−16
f11 0.0 0.0 0.0
0.0 0.0 0.0
f12 0.0 0.0 0.0
0.0 0.0 0.0
f13 0.0 0.0 −1.142824
0.0 0.0 4.45 × 10−8
An Information-Theoretic Approach for Clonal Selection Algorithms
Mario Pavone – mpavone@dmi.unict.it – http://www.dmi.unict.it/mpavone/

27. Introduction An Optimization CSA KLd , Rd and VNd Entropic Divergences Results Conclusions
i-CSA vs. IA, and DE rand/1/bin [Thomsen, et al., CEC 2004]
IA i-CSA DE rand/1/bin
100 variables
f1 0.0 0.0 0.0
0.0 0.0 0.0
f2 0.0 0.0 0.0
0.0 0.0 0.0
f3 0.0 0.0 5.87 × 10−10
0.0 0.0 1.83 × 10−10
f4 6.447 × 10−7 0.0 1.128 × 10−9
3.338 × 10−6 0.0 1.42 × 10−10
f5 74.99 22.116 0.0
38.99 39.799 0.0
f6 0.0 0.0 0.0
0.0 0.0 0.0
f7 1.59 × 10−5 1.2 × 10−6 7.664 × 10−3
3.61 × 10−5 1.53 × 10−6 6.58 × 10−4
An Information-Theoretic Approach for Clonal Selection Algorithms
Mario Pavone – mpavone@dmi.unict.it – http://www.dmi.unict.it/mpavone/

28. Introduction An Optimization CSA KLd , Rd and VNd Entropic Divergences Results Conclusions
i-CSA vs. IA, and DE rand/1/bin [Thomsen, et al., CEC 2004]
IA i-CSA DE rand/1/bin
100 variables
f8 −4.16 × 10+4 −2.727 × 10+4 −4.1898 × 10+4
2.06 × 10+2 7.63 × 10−4 1.06 × 10−3
f9 0.0 0.0 0.0
0.0 0.0 0.0
f10 0.0 0.0 8.023 × 10−15
0.0 0.0 1.74 × 10−15
f11 0.0 0.0 5.42 × 10−20
0.0 0.0 5.42 × 10−20
f12 0.0 0.0 0.0
0.0 0.0 0.0
f13 0.0 0.0 −1.142824
0.0 0.0 2.74 × 10−8
An Information-Theoretic Approach for Clonal Selection Algorithms
Mario Pavone – mpavone@dmi.unict.it – http://www.dmi.unict.it/mpavone/

29. Introduction An Optimization CSA KLd , Rd and VNd Entropic Divergences Results Conclusions
i-CSA and IA vs. several DE variants [Coello et al., GECCO 2006]
Unimodal Functions
f1 f2 f3 f4 f6 f7
i-CSA 0.0 0.0 0.0 0.0 0.0 2.79 × 10−5
IA 0.0 0.0 0.0 0.0 0.0 4.89 × 10−5
DE rand/1/bin 0.0 0.0 0.02 1.9521 0.0 0.0
DE rand/1/exp 0.0 0.0 0.0 3.7584 0.84 0.0
DE best/1/bin 0.0 0.0 0.0 0.0017 0.0 0.0
DE best/1/exp 407.972 3.291 10.6078 1.701872 2737.8458 0.070545
DE current-to-best/1 0.54148 4.842 0.471730 4.2337 1.394 0.0
DE current-to-rand/1 0.69966 3.503 0.903563 3.298563 1.767 0.0
DE current-to-rand/1/bin 0.0 0.0 0.000232 0.149514 0.0 0.0
DE rand/2/dir 0.0 0.0 30.112881 0.044199 0.0 0.0
An Information-Theoretic Approach for Clonal Selection Algorithms
Mario Pavone – mpavone@dmi.unict.it – http://www.dmi.unict.it/mpavone/

30. Introduction An Optimization CSA KLd , Rd and VNd Entropic Divergences Results Conclusions
i-CSA and IA vs. several DE variants [Coello et al., GECCO 2006]
Multimodal Functions
f5 f9 f10 f11 f12 f13
i-CSA 16.2 0.0 0.0 0.0 0.0 0.0
IA 11.69 0.0 0.0 0.0 0.0 0.0
DE rand/1/bin 19.578 0.0 0.0 0.001117 0.0 0.0
DE rand/1/exp 6.696 97.753938 0.080037 0.000075 0.0 0.0
DE best/1/bin 30.39087 0.0 0.0 0.000722 0.0 0.000226
DE best/1/exp 132621.5 40.003971 9.3961 5.9278 1293.0262 2584.85
DE current-to-best/1 30.984666 98.205432 0.270788 0.219391 0.891301 0.038622
DE current-to-rand/1 31.702063 92.263070 0.164786 0.184920 0.464829 5.169196
DE current-to-rand/1/bin 24.260535 0.0 0.0 0.0 0.001007 0.000114
DE rand/2/dir 30.654916 0.0 0.0 0.0 0.0 0.0
An Information-Theoretic Approach for Clonal Selection Algorithms
Mario Pavone – mpavone@dmi.unict.it – http://www.dmi.unict.it/mpavone/

31. Introduction An Optimization CSA KLd , Rd and VNd Entropic Divergences Results Conclusions
Immune systems have been related to swarm systems,
since many immune algorithms operate in a very similar
manner
i-CSA was compared with some swarm intelligence
algorithms [Karaboga et al., journal of Global Optimization, 2007]:
1 particle swam optimization (PSO)
2 particle swarm inspired evolutionary algorithm (PS-EA)
3 artificial bee colony
Experimental protocol: n = {10, 20, 30} and as termination
criterion 500, 750, and 1000 generations
five functions were taken into account: f5 , f9 , f10 f11 , and
the new function:
n
H(x) = (418.9829 × n) + −xi sin( |xi |)
i=1
An Information-Theoretic Approach for Clonal Selection Algorithms
Mario Pavone – mpavone@dmi.unict.it – http://www.dmi.unict.it/mpavone/

32. Introduction An Optimization CSA KLd , Rd and VNd Entropic Divergences Results Conclusions
i-CSA vs. three Swarm Intelligence algorithms [Karaboga et al., journal of
Global Optimization, 2007]
f11 f9 f5 f10 H
10 variables
GA 0.050228 1.3928 46.3184 0.59267 1.9519
0.029523 0.76319 33.8217 0.22482 1.3044
PSO 0.079393 2.6559 4.3713 9.8499 × 10−13 161.87
0.033451 1.3896 2.3811 9.6202 × 10−13 144.16
PS-EA 0.222366 0.43404 25.303 0.19209 0.32037
0.0781 0.2551 29.7964 0.1951 1.6185
i-CSA 0.0 0.0 0.0 0.0 1.27 × 10−4
0.0 0.0 0.0 0.0 1.268 × 10−14
ABC1 0.00087 0.0 0.034072 7.8 × 10−11 1.27 × 10−9
0.002535 0.0 0.045553 1.16 × 10−9 4 × 10−12
ABC2 0.000329 0.0 0.012522 4.6 × 10−11 1.27 × 10−9
0.00185 0.0 0.01263 5.4 × 10−11 4 × 10−12
An Information-Theoretic Approach for Clonal Selection Algorithms
Mario Pavone – mpavone@dmi.unict.it – http://www.dmi.unict.it/mpavone/

33. Introduction An Optimization CSA KLd , Rd and VNd Entropic Divergences Results Conclusions
i-CSA vs. three Swarm Intelligence algorithms [Karaboga et al., journal of
Global Optimization, 2007]
f11 f9 f5 f10 H
20 variables
GA 1.0139 6.0309 103.93 0.92413 7.285
0.026966 1.4537 29.505 0.22599 2.9971
PSO 0.030565 12.059 77.382 1.1778 × 10−6 543.07
0.025419 3.3216 94.901 1.5842 × 10−6 360.22
PS-EA 0.59036 1.8135 72.452 0.32321 1.4984
0.2030 0.2551 27.3441 0.097353 0.84612
i-CSA 0.0 0.0 0.0 0.0 237.5652
0.0 0.0 0.0 0.0 710.4036
ABC1 2.01 × 10−8 1.45 × 10−8 0.13614 1.6 × 10−11 19.83971
6.76 × 10−8 5.06 × 10−8 0.132013 1.9 × 10−11 45.12342
ABC2 0.0 0.0 0.014458 0.0 0.000255
0.0 0.0 0.010933 1 × 10−12 0
An Information-Theoretic Approach for Clonal Selection Algorithms
Mario Pavone – mpavone@dmi.unict.it – http://www.dmi.unict.it/mpavone/

34. Introduction An Optimization CSA KLd , Rd and VNd Entropic Divergences Results Conclusions
i-CSA vs. three Swarm Intelligence algorithms [Karaboga et al., journal of
Global Optimization, 2007]
f11 f9 f5 f10 H
30 variables
GA 1.2342 10.4388 166.283 1.0989 13.5346
0.11045 2.6386 59.5102 0.24956 4.9534
PSO 0.011151 32.476 402.54 1.4917 × 10−6 990.77
0.014209 6.9521 633.65 1.8612 × 10−6 581.14
PS-EA 0.8211 3.0527 98.407 0.3771 3.272
0.1394 0.9985 35.5791 0.098762 1.6185
i-CSA 0.0 0.0 0.0 0.0 2766.804
0.0 0.0 0.0 0.0 2176.288
ABC1 2.87 × 10−9 0.033874 0.219626 3 × 10−12 146.8568
8.45 × 10−10 0.181557 0.152742 5 × 10−12 82.3144
ABC2 0.0 0.0 0.020121 0.0 0.000382
0.0 0.0 0.021846 0.0 1 × 10−12
An Information-Theoretic Approach for Clonal Selection Algorithms
Mario Pavone – mpavone@dmi.unict.it – http://www.dmi.unict.it/mpavone/

35. Introduction An Optimization CSA KLd , Rd and VNd Entropic Divergences Results Conclusions
Conclusion 1/3
Global numerical optimization has been taken into account
to prove the effectiveness of a derivative-free clonal
selection algorithm (i-CSA)
The main features of i-CSA can be summarized as:
1 cloning operator, which explores the neighbourhood of a
given solution
2 inversely proportional hypermutation operator, that perturbs
each candidate solution as a function of its fitness function
value (inversely proportionally)
3 aging operator, that eliminates the oldest candidate
solutions from the current population in order to introduce
diversity and thus avoiding local minima during the search
process
An Information-Theoretic Approach for Clonal Selection Algorithms
Mario Pavone – mpavone@dmi.unict.it – http://www.dmi.unict.it/mpavone/

36. Introduction An Optimization CSA KLd , Rd and VNd Entropic Divergences Results Conclusions
Conclusion 2/3
The decision of the age to assign to any B cell receptor is crucial
for the quality of the search inside the space of solutions
We have presented a simple variant of CSA able to effectively
improve its performances leaving a longer maturation for each B
cell
A large set of classical numerical functions was taken into
account
i-CSA was compared with several variants of the DE algorithm,
since it has been shown to be effective on many optimization
problems.
i-CSA was also compared with state-of-the-art swarm
algorithms.
An Information-Theoretic Approach for Clonal Selection Algorithms
Mario Pavone – mpavone@dmi.unict.it – http://www.dmi.unict.it/mpavone/

37. Introduction An Optimization CSA KLd , Rd and VNd Entropic Divergences Results Conclusions
Conclusion 3/3
The analysis of the results shows that i-CSA is comparable, and often
outperforms, all nature-inspired algorithms in terms of accuracy to the accuracy,
and effectiveness in solving large-scale instances
By analyzing one of the most difficult function of the benchmark, (f5 ), we
characterize the learning capability of i-CSA.
The obtained gain has been analyzed both with respect the initial distribution,
and the ones obtained in the previous step.
For this study, three different entropic metrics were used
1 Kullback-Leibler
2 Rényi
3 von Neumann
By the relative curves, is possible to observe a strong correlation between optima
(peaks in the search landscape) discovering, and high relative entropy values.
An Information-Theoretic Approach for Clonal Selection Algorithms
Mario Pavone – mpavone@dmi.unict.it – http://www.dmi.unict.it/mpavone/