ANT COLONY

1. Ant Colony Optimization

2. Swarm Intelligence
“Swarm intelligence (SI) is artificial
intelligence based on the collective
behaviour of decentralized, self-
organized systems”

3. Swarm Intelligence General
Characteristics
Composed of many individuals
Individuals are homogeneous
Local interaction based on simple rules
Self-organization
Constituting a natural model particularly
suited to distributed problem solving

4. Swarm Intelligence General
Characteristics
 Collective system capable of accomplishing
difficult tasks in dynamic and varied
environments without any external guidance or
control and with no central coordination
 Achieving a collective performance which could
not normally be achieved by an individual acting
alone
 Constituting a natural model particularly suited
to distributed problem solving

6. http://www.scs.carleton.ca/~arpwhite/courses/95590Y/notes/SI%20Lecture%203.pdf

7. http://www.scs.carleton.ca/~arpwhite/courses/95590Y/notes/SI%20Lecture%203.pdf

8. http://www.scs.carleton.ca/~arpwhite/courses/95590Y/notes/SI%20Lecture%203.pdf

9. Ant Colony System
 First introduced by Marco Dorigo in 1992 as a
method for solving hard combinatorial
optimization problems (COPs).
 Progenitor to “Ant Colony System,” later
discussed
 Result of research on computational intelligence
approaches to combinatorial optimization
 Originally applied to Traveling Salesman
Problem
 Applied later to various hard optimization
problems

10. Inherent Features
Inherent parallelism
Stochastic nature
Adaptivity
Use of positive feedback
Autocatalytic in nature

11. Real Ant actual scenario
Almost blind.
Incapable of achieving complex tasks alone.
Rely on the phenomena of swarm intelligence for
survival.
Capable of establishing shortest-route paths from
their colony to feeding sources and back.
Use stigmergic communication via pheromone trails.
Follow existing pheromone trails with high
probability.
What emerges is a form of autocatalytic behavior:
the more ants follow a trail, the more attractive that
trail becomes for being followed.

12. Real Ant actual scenario
The process is thus characterized by a positive
feedback loop, where the probability of a
discrete path choice increases with the number
of times the same path was chosen before.

13. Natural behavior of an ant :
Foraging modes
Wander mode
Search mode
Return mode
Attracted mode
Trace mode
Carry mode

14. Behavior of Ant colony
regulation of nest temperature within 1 degree celsius
range;
forming bridges;
raiding specific areas for food;
building and protecting nest;
sorting brood and food items;
cooperating in carrying large items;
emigration of a colony;
finding shortest route from nest to food
source;
preferentially exploiting the richest food source
available.

15. Autocatalyzation
Autocatalysis is a positive feedback
loop that drives the ants to explore
promising aspects of the search space
over less promising areas.

16. A key concept: Stigmergy
Stigmergy is: indirect communication via
interaction with the environment.
 A problem gets solved bit by bit ..
 Individuals communicate with each other in the
above way, affecting what each other does on the
task.
 Individuals leave markers or messages – these
don’t solve the problem in themselves, but they
affect other individuals in a way that helps them
solve the problem.

17. Stigmergy in Ants
Ants are behaviourally unsophisticated, but
collectively they can perform complex tasks.
Ants have highly developed sophisticated sign-
based stigmergy.
– They communicate using pheromones;
– They lay trails of pheromone that can be
followed by other ants.

18. Pheromone Trails
 Individual ants lay pheromone trails while travelling
from the nest, to the nest or possibly in both
directions.
 The pheromone trail gradually evaporates over time.
 But pheromone trail strength accumulate with
multiple ants using path.
Food source
Nest

19. Naturally Observed Ant Behavior
All is well in the world of the ant.

20. Naturally Observed Ant Behavior
An obstacle has blocked our path

21. Naturally Observed Ant Behavior

22. Naturally Observed Ant Behavior

23. ACO Algorithms: Basic Ideas
 Ants are agents that: Move along between nodes in a
graph.
 They choose where to go based on pheromone
strength (and maybe other things)
 An ant’s path represents a specific candidate solution.
 When an ant has finished a solution, pheromone is laid
on its path, according to quality of solution.
 This pheromone trail affects behaviour of other ants
by `stigmergy’

24. Artificial Ants
• artifcial ants may simulate pheromone
laying by modifying appropriate pheromone
variables associated with problem states
they visit while building solutions to the
optimization problem. Also, according to the
stigmergic communication model, the
artifcial ants would have only local access
tothese pheromone variables.

25. Artificial Ants
Main characteristics of stigmergy can be extended to
artificial agents by
• Associating state variables with different problem
states; and
• Giving the agents only local access to these
variables.
• Coupling between the autocatalytic mechanism
and the implicit evaluation of solutions
• Just like real ants, artificial ants create their
solutions sequentially by moving from one
problem state to another

26. Differences between real and artificial
ants:
 Artificial ants live in a discrete world| they move
sequentially through a finite set of problem states.
 The pheromone update (i.e., pheromone
depositing and evaporation) is not accomplished
in exactly the same way by artificial ants as by real
ones. Sometimes the pheromone update is done
only by some of the artificial ants, and often only
after a solution has been constructed.
 Some implementations of artificial ants use
additional mechanisms that do not exist in the
case of real ants. Examples include look-ahead,
local search, backtracking, etc.

27. Nature/Technology

28. SHORTEST PATH
Ants deposit pheromones on ground that form
a trail. The trail attracts other ants.
Pheromones evaporate faster on longer paths.
Shorter paths serve as the way to food for
most of the other ants.

29. Minimum Cost Path

30. General ACO
• A stochastic construction procedure
• Probabilistically build a solution
• Iteratively adding solution components to partial
solutions
- Heuristic information
- Trace/Pheromone trail
• Reinforcement Learning reminiscence
• Modify the problem representation at each
iteration
• Ants work concurrently and independently
• Collective interaction via indirect communication
leads to good solutions

31. Some inherent advantages
• Positive Feedback accounts for rapid
discovery of good solutions
• Distributed computation avoids premature
convergence
• The greedy heuristic helps find acceptable
solution in the early solution in the early
stages of the search process.
• The collective interaction of a population of
agents.

32. Disadvantages in Ant Systems
Slower convergence than other Heuristics
Performed poorly for TSP problems larger
than 75 cities.
No centralized processor to guide the AS
towards good solutions

33. Ant System (AS) Algorithm
1. Initialization
2. Randomly place ants
3. Build tours
4. Deposit trail
5. Update trail
6. Loop or exit

34. Ant with Binary Bridge
• Let the amount of pheromone on a branch be proportional
to the number of ants that used the branch in the past and
let ms(t) and ml(t) be the numbers of ants that have used
the short and the long branches after a total of t ants have
crossed the bridge, with ms(t) þ ml(t) =t.The probability
ps(t) with which the (t+1) th ant chooses the short branch
can then be written as

35. Ant with Binary Bridge
The number of ants choosing the short branch is
given by
The number of ants choosing the long branch by
where q is a uniform random number drawn from the interval [0; 1].
Mote Carlo Simulation method will give good solution

36. ACO Parameters

37. ACO Parameters

38. Application
Traveling Salesman Problem
Quadratic Assignment Problem
Network Model Problem
Vehicle routing
Graph coloring

39. ACO system -PSEUDOCODE
 Often applied to TSP (Travelling Salesman Problem):
shortest path between n nodes
 Algorithm in Pseudocode:
– Initialize Trail
– Do While (Stopping Criteria Not Satisfied) – Cycle Loop
• Do Until (Each Ant Completes a Tour) – Tour Loop
• Local Trail Update
• End Do
• Analyze Tours
• Global Trail Update
– End Do

40. ACO Algorithm
• Ant Colony Algorithms are typically use to solve
minimum cost problems.
• We may usually have N nodes and A undirected arcs
• There are two working modes for the ants: either
forwards or backwards
• The ants memory allows them to retrace the path it
has followed while searching for the destination node
• Before moving backward on their memorized path,
they eliminate any loops from it. While moving
backwards, the ants leave pheromones on the arcs
they traversed.

41. ACO Algorithm
• At the beginning of the search process, a constant amount of
pheromone is assigned to all arcs. When located at a node i an
ant k uses the pheromone trail to compute the probability of
choosing j as the next node:
where is the neighborhood of ant k when in node i.

42. ACO Algorithm
k
ij ij
  
  
• When the arc (i,j) is traversed , the pheromone value changes
as follows:
• By using this rule, the probability increases that forthcoming
ants will use this arc.
• After each ant k has moved to the next node, the pheromones
evaporate by the following equation to all the arcs:
(1 ) , ( , )
ij ij
p i j A
 
   

43. Steps for Solving a Problem by ACO
1. Represent the problem in the form of sets of
components and transitions, or by a set of weighted
graphs, on which ants can build solutions
2. Define the meaning of the pheromone trails
3. Define the heuristic preference for the ant while
constructing a solution
4. If possible implement a efficient local search
algorithm for the problem to be solved.
5. Choose a specific ACO algorithm and apply to
problem being solved
6. Tune the parameter of the ACO algorithm.

44. Combinatorial optimization
• Find values of discrete variables
• Optimizing a given objective function
Π = (S, f, Ω) – problem instance
S – set of candidate solutions
f – objective function
Ω – set of constraints
set of feasible solutions (with respect to Ω)
Find globally optimal feasible solution s*

45. Combinatorial optimization
problem mapping
• Combinatorial problem (S, f, Ω(t))
• Ω(t) – time-dependent constraints
 Example – dynamic problems
• Goal – find globally optimal feasible solution
s*
• Minimization problem
• Mapped on another problem

46. Combinatorial optimization
problem mapping
• C = {c1, c2, …, cNc} – finite set of
components
• States of the problem:
X = {x = <ci, cj, …, ch, …>, |x| < n < +∞}
• Set of candidate solutions:

47. Combinatorial optimization
problem mapping

48. Combinatorial optimization
problem mapping
• X – states
• S – candidate solutions
• – feasible states
• S* – optimal solutions
S* X
~
S
X

49. Combinatorial optimization
problem mapping
• Cost g(s, t) for each
• In most cases – g(s, t) ≡ f(s, t)
• GC = (C, L) – completely connected graph
• C – set of components
• L – edges fully connecting the components
(connections)
• GC – construction graph
• Artificial ants build solutions by performing
randomized walks on GC(C, L)

50. ACO for Traveling Salesman Problem
The first ACO algorithm was called the Ant system and
it was aimed to solve the travelling salesman problem,
in which the goal is to find the shortest round-trip to
link a series of cities. At each stage, the ant chooses to
move from one city to another according to some
rules:
 It must visit each city exactly once;
 A distant city has less chance of being chosen (the visibility);
 The more intense the pheromone trail laid out on an edge between
two cities, the greater the probability that that edge will be chosen;
 Having completed its journey, the ant deposits more pheromones
on all edges it traversed, if the journey is short;
 After each iteration, trails of pheromones evaporate.

51. ACO for Traveling Salesman Problem
TSP PROBLEM : Given N cities, and a distance function d between cities,
find a tour that:
1. Goes through every city once and only once
2. Minimizes the total distance.

52. HOW TO IMPLEMENT IN A PROGRAM
• Ants: Simple computer agents
• Move ant: Pick next component in the const. solution
• Pheromone:
• Memory: MK or TabuK
• Next move: Use probability to move ant
• Graph (N,E): where N = cities/nodes, E = edges
• = the tour cost from city i to city j (edge weight)
• Ant move from one city i to the next j with some transition probability.

53. A simple TSP example
A
D
C
B
1
[]
4
[]
3
[]
2
[]
dAB =8;dBC = 4;dCD =15;dDA =6
53

54. Iteration 1
A
D
C
B
1
[A]
3
[C]
2
[B]
4
[D]
54

55. How to build next sub-solution?
1
[A]
A
D
C
B
55

56. Iteration 2
A
D
C
B
2
[B,C]
3
[C,D]
1
[A,B]
4
[D,A]
56

57. Iteration 3
A
D
C
B
2
[B,C,D]
3
[B,C,A]
4
[D,A,B]
1
[A,B,C]
57

58. Iteration 4
A
D
C
B
1
[A,B,C,D]
2
[B,C,D,A]
3
[C,D,A,B]
4
[D,A,B,C]
58

59. Path and Pheromone Evaluation
1
[A,B,C,D
L1 =27
L2 =25
L3 =29
L4 =18
2
[B,C,D,A]
3
[C,D,A,B]
4
[D,A,B,C]
59
Best tour

60. MAX–MIN Ant System
• MAX–MIN Ant System (MMAS) (Stu¨ tzle & Hoos,
1997, 2000; Stu¨ tzle, 1999) introduces four main
modifications with respect to AS. First, it strongly
exploits the best tours found: only either the
iteration-best ant, that is, the ant that produced
the best tour in the current iteration, or the best-
so-far ant is allowed to deposit pheromone.
Unfortunately, such a strategy may lead to a
stagnation situation in which all the ants follow
the same tour, because of the excessive growth of
pheromone trails on arcs of a good, although
suboptimal, tour.

61. MAX–MIN Ant System
• To counteract this effect, a second modification
introduced by MMAS is that it limits the possible
range of pheromone trail values to the interval
[Ʈmin; Ʈmax]. Third, the pheromone trails are
initialized to the upper pheromone trail limit,
which, together with a small pheromone
evaporation rate, increases the exploration of
tours at the start of the search. Finally, in MMAS,
pheromone trails are reinitialized each time the
system approaches stagnation or when no
improved tour has been generated for a certain
number of consecutive iterations.

62. Greedy Search Algorithm
• A greedy algorithm is an algorithm that
follows the problem solving heuristic of
making the locally optimal choice at each
stage[1] with the hope of finding a global
optimum. In many problems, a greedy
strategy does not in general produce an
optimal solution, but nonetheless a greedy
heuristic may yield locally optimal solutions
that approximate a global optimal solution
in a reasonable time.

63. Greedy Search Algorithm
• For example, a greedy strategy for the
traveling salesman problem (which is of a
high computational complexity) is the
following heuristic: "At each stage visit an
unvisited city nearest to the current city".
This heuristic need not find a best solution,
but terminates in a reasonable number of
steps; finding an optimal solution typically
requires unreasonably many steps. In
mathematical optimization, greedy
algorithms solve combinatorial problems
having the properties of matroids.

64. Constructive Heuristics
• Start from an “empty solution”
• Repeatedly, extend the current solution until a
complete solution is constructed
• Use heuristics to try to extend in such a way that
the final solution is a good one
It is essential to know the difference between:
• Constructive methods
Extend empty solution until get complete
solution
• Local search
Take complete solution and try to improve it
via local moves