The document discusses swarm intelligence and particle swarm optimization. It defines a swarm as a collection of interacting agents and provides examples of swarms in nature like bee hives and bird flocks. Particle swarm optimization is introduced as an algorithm inspired by swarm behavior that uses a population of particles that improve potential solutions based on their own experience and the experience of neighboring particles. The algorithm is explained in detail including concepts like personal best positions and global best positions that guide particles toward better solutions. Variations of the algorithm including different neighborhood topologies and multi-swarm approaches are also summarized.

1. Swarm Intelligence

2. What is a Swarm?
A loosely structured collection of interacting agents
 Agents:
 Individuals that belong to a group
 They contribute to and benefit from the group
 They can recognize, communicate, and/or interact with each other

3. Examples of Swarms in Nature:
Classic Example: Swarm of Bees
Can be extended to other similar systems:
 Ant colony
 Agents: ants
 Flock of birds
 Agents: birds
 Traffic
 Agents: cars
 Crowd
 Agents: humans

4. Swarm Intelligence (SI)
SI is the discipline that deals with artificial systems
composed of many agents that coordinate using
decentralized control and self-organization.
Generally made up of agents who interact with each
other and the environment.
No centralized control structures.
Based on group behavior.

5. SI system properties:
It is composed of many individuals.
The individuals are relatively homogeneous .
The interactions among individuals are based on
simple behavioral rules.
The overall behavior of the system results from the
interactions of individuals with each other and with
their environment.

6. Three Common SI Algorithms
Particle Swarm Optimization
Ant Colony Optimization
Bee Colony Optimization

7. Particle Swarm Optimization (PSO)
The Idea is similar to bird flocks searching for food.
 Bird=Particle and Position of food=a solution

8. Particle Swarm Optimization (PSO)
• PSO is a population-based, self-adaptive search
optimization technique.
• It was developed in 1995 by James Kennedy and Russell
Eberhart.
• It uses a number of agents (particles) that constitute a
swarm moving around in the search space looking for the
best solution.

9. Particle Swarm Optimization (PSO)
A swarm consists of N particles in a D-dimensional
search space. Each particle holds
 a position (which is a candidate solution to the problem) and
 a velocity (which means the flying direction and step of the
particle).

10. Particle Swarm Optimization (PSO)
Each particle successively adjust its position toward
the global optimum based on two factors:
 The best position visited by itself (pbest).
 The best position visited by the whole swarm (gbest) .
Each particle moves forward by adjusting its
“flying” according to
 its own flying experience known as cognitive component as
well as
 the flying experience of other particles known as social
component.

11. Personal best (Cognitive behavior)
Personal best position of a particle expresses the
cognitive behavior of particle.
It is defined as the best position found by the
particle.
It will be updated whenever the particle reaches a
position with better fitness value than the fitness
value of the previous personal best(pbest).

12. Global best (Social behavior)
Global best position expresses the social behavior.
It is defined as the best position found by all the
particles in the swarm.
It will be updated whenever a particle reaches a
position with better fitness value than the fitness
value of the previous global best (gbest).

13. In each timestep, a particle has to move to a new
position. It does this by adjusting its velocity.
 The current velocity PLUS
 A weighted random portion in the direction of its
personal best PLUS
 A weighted random portion in the direction of the
global best.
Having worked out a new velocity, its position is
simply its old position plus the new velocity.
What a particle does

14. Particle Swarm Optimization (PSO)
• C1 and C2 = acceleration coefficients. (Typically (C1
+ C2)<=4.
• rand(.) and rand(.) = random number between
(0,1).
• Pi –Xi(t)=“cognitive” component represents the
personal thinking of each particle which
encourages the particles to move their best
positions found so far
• Pg-Xg(t)=“social” component which helps to find
the global best position found so far.

15. Particle Swarm Optimization (PSO)
Here I
am! The best
perf. of
team
My best
perf.
x
pg
pi
v
PBest
gBest
Eq.(1)
Eq.(3)

16. Single Particle

17. Particle Swarm Optimization (PSO)
Flow chart depicting the General PSO Algorithm:
Start
Initialize particles with random position
and velocity vectors.
For each particle’s position (p)
evaluate fitness
If fitness(p) better than
fitness(pbest) then pbest= p
Loopuntilall
particlesexhaust
Set best of pBests as gBest
Update particles velocity (eq. 1) and
position (eq. 3)
Loopuntilmaxiter
Stop: giving gBest, optimal solution.

18. Some functions often used for testing real-valued optimisation
algorithms
Sphere Rastrigin
Ackley
Schewefel

19. Problems in multimodal functions
In the original PSO-
 All particles learns from gBest in updating velocities and
positions.
 So the algorithm exhibits a fast-converging behavior.
But on multimodal problems,
 A gBest located at a local optimum may trap the whole swarm
and lead to premature convergence.

20. PSO variants
Tuning the control parameters to maintain the
balance between local search and global search.
Defining different neighborhood topologies to
replace the traditional global topology.
Hybridizing PSO with other meta-heuristic search
techniques.
Multi-swarm techniques.

21. Tuning the control parameters[2]
A new parameter is introduced, namely the inertia
weight ω to influence the convergence.

22. Comments on the Inertia weight factor:Comments on the Inertia weight factor:
A large inertia weight (A large inertia weight (ω) facilitates a global search while a) facilitates a global search while a
small inertia weight facilitates a local search.small inertia weight facilitates a local search.
LargerLarger ω ----------- greater global search ability----------- greater global search ability
SmallerSmaller ω ------------ greater local search ability.------------ greater local search ability.
A scheme is proposed to decrease ω linearly
from 0.9 to 0.4 over the course of search
process.
Particle Swarm Optimization (PSO)

23. Neighborhood Topologies[1]
Gbest Swarm:
• In the gbest swarm, all the particles are neighbors of each
other; thus, the position of the best overall particle in the
swarm is used in the social term of the velocity update
equation.
• It is assumed that gbest swarms converge fast, as all the
particles are attracted simultaneously to the best part of
the search space.
• However, if the global optimum is not close to the best
particle, it may be impossible to the swarm to explore
other areas; this means that the swarm can be trapped in
local optima.

24. Neighborhood Topologies
In the Lbest swarm, only a specific number of
particles can affect the velocity of a given particle.
The swarm will converge slower but can locate the
global optimum with a greater chance.

25. Neighborhood Topologies
Gbest Swarm Lbest Swarm

26. Hybridizing PSO
Hybridization of PSO with ACO [4]
Hybridization of PSO with GA[5]

27. Multi-Swarm PSO (MPSO)[10][11]

28. Multi-Swarm PSO (MPSO)
A set of independent swarms.
Method:
 Run PSO for a number of iteration.
 Have an interaction.
 K best particles in the sender swarm is sent to the receiver
swarm.
 The new particles replaces the worst k ones in the receiver
swarm.

29. References
1.R. C. Eberhart and J. Kennedy, “A new optimizer using particle swarm theory,” in Proc. 6th Int. Symp. Micromachine
Hum. Sci., 1995, pp. 39–43
2.Y. Shi and R. C. Eberhart, “A modified particle swarm optimizer,” in Proc. IEEE Congr. Evol. Comput., May 1998, pp. 69–
73.
3. J. Kennedy and R. Mendes, “Population structure and particle swarm performance,” in Proc. IEEE Congr. Evol. Comput.,
vol. 2. May 2002, pp. 1671–1676.
4. P.S. Shelokar, Patrick Siarry, V.K. Jayaraman, B.D. Kulkarni,"Particle swarm and ant colony algorithms hybridized for
improved continuous optimization",Applied Mathematics and Computation 188 (2007) 129–142
5. A. Gandelli, F. Grimaccia, M. Mussetta, P. Pirinoli, R.E. Zich, “Devel-opment and Validation of Different Hybridization
Strategies betweenGA and PSO”, Proc. of the 2007 IEEE Congress on EvolutionaryComputation , Sept. 2007,
Singapore, pp. 2782–2787.
6. Jacques Riget, Jakob S. Vesterstrøm , "A Diversity-Guided Particle Swarm Optimizer - the ARPSO", EVALife Technical
Report,2002(2002-02)
7. T.M. Blackwell and P.J. Bentley., “Dynamic Search with Charged Swarms”, In Proceedings of the Genetic and
Evolutionary Computation Conference, 2002.
8.J. J. Liang, and P. N. Suganthan, "Dynamic Multi-Swarm Particle Swarm Optimizer," Proc. of IEEE Int. Swarm
Intelligence Symposium, pp. 124-129, 2005.
9.Shi-Zheng Zhao, Ponnuthurai Nagaratnam Suganthan, Swagatam Das, "Dynamic multi-swarm particle swarm optimizer
with sub-regional harmony search",Journal Expert Systems with Applications: An International Journal archiveVolume
38 Issue 4, April, 2011 Pages 3735-3742
10.Leonardo VANNESCHI, Daniele CODECASA and Giancarlo MAURI, "A study of parallel and distributed particle swarm
optimization methods", Proceedings of the 2nd workshop on Bio-inspired algorithms for distributed systems Pages 9-16
11. http://cse.unl.edu/~lksoh/Classes/CSCE990AMAS_Spring13/Seminar08_Kahrobaee.pdf
12. Wei-neng Chen, Jun Zhang, Ying Lin, Ni Chen, Zhi-hui Zhan, Henry Shu-Hung Chung, Yun Li, Yu-hui Shi: Particle
Swarm Optimization With an Aging Leader and Challengers. IEEE Trans. Evolutionary Computation 17(2): 241-258
(2013)