Slides presented at the course Software Testing at Polytechnique Montreal, 2016.

1. Metaheuristic Algorithms for
Optimization
Francisco de Melo jr

2. Problem
Introduction
Research Question
Meta heuristics
Comparative evaluation
Conclusion
References
Outline

3. Problem
Generate data to test a program
<non linear function>

4. Solution
Consider the problem a minimization vs maximization problem
We can apply Simplex! (From Linear programing)

5. Solution
Simplex!
From Linear programing
Well, not quite

6. Research Question
How to optimize the generation?
Compare the several way to generate the data?

7. Introduction
Meta heuristic algorithms*
To solve the optimization problem (1), efficient search or optimization algorithms are needed. There are many optimization algorithms
which can be classified in many ways, depending on the focus and characteristics.
Name controversy*

8. Meta-heuristics
Components
Two major components of any metaheuristic algorithms are: intensification and diversification, or exploitation and
exploration (Blum and Roli, 2003).

9. Meta heuristic I
Particle Swarm Optimization
PSO searches the space of an objective function by adjusting the trajectories of individual agents, called particles.
Each particle traces a piecewise path which can be modelled as a time-dependent positional vector.
The movement of a swarming particle consists of two major components: a stochastic component and a deterministic component.
Each particle is attracted toward the position of the current global best g∗ and its own best known location x∗i , while exhibiting at the
same time a tendency to move randomly.

10. Meta heuristic I
Particles

11. Meta heuristic I
Algorithm
for each particle i = 1, ..., S do
Initialize the particle's position with a uniformly distributed random vector: xi ~ U(blo, bup)
Initialize the particle's best known position to its initial position: pi ← xi
if f(pi) < f(g) then
update the swarm's best known position: g ← pi
Initialize the particle's velocity: vi ~ U(-|bup-blo|, |bup-blo|)
while a termination criterion is not met do:
for each particle i = 1, ..., S do
for each dimension d = 1, ..., n do
Pick random numbers: rp, rg ~ U(0,1)
Update the particle's velocity: vi,d ← ω vi,d + φp rp (pi,d-xi,d) + φg rg (gd-xi,d)
Update the particle's position: xi ← xi + vi
if f(xi) < f(pi) then
Update the particle's best known position: pi ← xi
if f(pi) < f(g) then
Update the swarm's best known position: g ← pi

12. Meta heuristic I
Demo

13. Meta heuristic I
After finding the two best values, the particle updates its velocity and
positions with following equation (a) and (b).
Fitness function update
v[] = v[] + c1 * rand() * (pbest[] - present[]) + c2 * rand() * (gbest[] - present[]) (a)
present[] = persent[] + v[] (b)

14. Five principles of swarm intelligence
First is the proximity principle: the population should be able to carry out simple space and time
computations.
Second is the quality principle: the population should be able to respond to quality factors in the
environment.
Third is the principle of diverse response: the population should not commit its activities along excessively
narrow channels.
Fourth is the principle of stability: the population should not change its mode of behavior every time the
environment changes.
Fifth is the 1946 principle of adaptability: the population must be able to change behavior mode when it’s
worth the computational price.

15. Meta heuristic II
Bee Colony Optimization Metaheuristic (BCO)

16. Meta heuristic II
Agents (bees) collaborate in order to solve a difficult problem together
Problems that can be applied:
● Traveling Salesman Problem (TSP)
● Quadratic Assignment Problem (QAP)
● Binary Knapsack Problem

17. Meta heuristic II
Bees
During the search process, artificial bees communicate directly.
Each artificial bee makes a series of local moves, and in this way incrementally constructs
a solution of the problem.
Bees are adding solution components to the current partial solution until they create one or
more feasible solutions.
The search process is composed of iteration

18. Meta heuristic II
Algorithms Pseudocode for the standard bees algorithm[2]
1 for i=1,…,ns
i scout[i]=Initialise_scout()
ii flower_patch[i]=Initialise_flower_patch(scout[i])
2 do until stopping_condition=TRUE
i Recruitment()
ii for i =1,...,nb
1 flower_patch[i]=Local_search(flower_patch[i])
2 flower_patch[i]=Site_abandonment(flower_patch[i])
3 flower_patch[i]=Neighbourhood_shrinking(flower_patch[i])
iii for i = nb,...,ns
1 flower_patch[i]=Global_search(flower_patch[i])}

19. Meta heuristic II Traveling Salesman Problem

20. Meta heuristic II
Demo

21. Meta heuristic III
Ant Colony
Optimization

22. Meta heuristic III
Ant Colony Optimization - ACO
It is a probabilistic technique for solving computational problems which can be
reduced to finding good paths through graphs.

23. Meta heuristic IV
Cuckoo Search
Cuckoos are fascinating birds, not only because of the beautiful sounds they make, but also because of their aggressive reproduction
strategy. Some species named ani and Guira lay their eggs in communal nests, though they may remove others' eggs to increase the
hatching probability of their own eggs.
Quite a number of species engage in the mandatory brood parasitism by laying their eggs in the nests of other host birds (often other
species).
Cuckoo search (CS) is one of the latest nature-inspired metaheuristic algorithms, developed by Xin-She Yang and Suash Deb in 2009.

24. Comparative evaluation
CS is based on the brood parasitism of some cuckoo species (Yang and Deb 2009). In addition, this algorithm is enhanced by the so-
called Lévy flights, rather than by simple isotropic random walks (Pavlyukevich 2007). Recent studies show that CS is potentially far
more efficient than PSO and genetic algorithms (Yang and Deb 2010).

25. Summary
- GA and SA are not considered swarm methods
- Thousands of algorithms with Swarm optimization methods
- Each particle is a solution
- It is not guaranteed to find the best solution

26. Conclusion
- Metaheuristic vs heuristic
- “Standard” metaheuristic
- CS and recent developments
- Several other techniques

27. References
Blum and Roli, 2003
Yang and Deb 2009
Pavlyukevich 2007
Fred Glover in the 1970s
James McCaffrey from Microsoft Research

28. Contact
Francisco de Melo
francisco.melo.jr@gmail.com