The slide provides some introductory information about the nature-inspired algorithm called Particle Swarm Optimization.

1. Particle Swarm Optimization
By Natnael Tilahun
1

2. Contents
• Competitive and cooperative population
• Flocking
• Introduction to Particle Swarm Optimization
• Mathematical Model of PSO
• PSO Algorithm
• Application of PSO
• Summary of PSO
• Further Reading
• References
2

3. Cooperative Populations
• Work together to find an optimal
solutions
• A fixed set of individuals will improve
its solutions as the iterations progress
• Adjust its positions
Cooperative Populations
• PSO (Particle Swarm Optimization)
• ACO (Ant Colony Optimization)
Populations
Competitive Populations
Other Chapters
• Improved by creating successively
better generations of solutions
• Adjust the genetic code
3

4. Flocking
• Flocks of birds in the sky illustrate the idea of cooperative behavior
• Flocking appears as a very complex behavior, and many animals
exhibits it
Shows the identical behavior of grouping
through the various names for it
4

5. Flocking (Cont.)
• Craig Reynolds (1986) first replicated flocking behavior
on a computer with his simulation program, Boids
• Boids= A shortened version of ‘bird-oid objects’, which
refers to a bird-like object
The simulation algorithm has three simple rules
Alignment Separation Cohesion
If objects are too
close, move away
Each agents tries to steers toward the
average heading of its neighbors
Each agents tries to move toward the
average heading of its neighbors

6. Introduction to the PSO: Authors
6

7. • Inspired from the nature social behavior and dynamic movements
with communications of insects, birds and fish
Introduction to the PSO: Origins
7

8. • Collection of flying particles (swarm) - Changing solutions
• Search area - Possible solutions
• Movement towards a promising area to get the global optimum
• Each particle adjusts its travelling speed dynamically corresponding to
the flying experiences of itself and its colleagues
Introduction to the PSO: Concept
8

9. Each particle modifies its position according to:
• its current position
• its current velocity
• the distance between its current position and pbest
• the distance between its current position and gbest
Introduction to the PSO: Concepts
9

10. Particle Swarm Optimization (PSO)
• The idea is similar to bird flocks searching for food.
• Bird = a particle, Food = a solution
• pbest = the best solution (fitness) a particle has achieved so far.
• gbest = the global best solution of all particles within the
swarm
10

11. Mathematical Model of PSO
11

12. Mathematical Model of PSO (Cont.)
Where
V(t) velocity of the particle at time t
X(t) Particle position at time t
w Inertia weight
c1 , c2 learning factor or accelerating factor
r1, r2 uniformly distributed random number
between 0 and 1
pij(t) particle’s best position at time t
g(t) global best
12

13. Particle’s velocity:
• Makes the particle move in the same
direction and with the same velocity
1. Inertia
Term
2. Personal
Influence (CC)
3. Social
Influence(SC)
• Improves the individual
• Makes the particle return to a previous
position, better than the current
• Conservative
• Makes the particle follow the best
neighbors direction
Introduction to the PSO: Algorithm
13

14. Simple Example of PSO
14

15. Consider the function f(x, y) = x² + (y + 1)² - 5cos(1.5x + 1.5) - 3cos(2x - 1.5)
Randomly initialize N=30
Introduction to the PSO: Example
15

16. Comparison of the W (Inertia weight )
• It is recommended to avoid w >1 which can lead to a
divergence of our particles.
• The inertia weight w thus makes a balance between the
exploration and the exploitation of the best solutions found
so far
Introduction to the PSO: Example (Cont.)
16

17. C1 and C2 (Acceleration coefficient)comparison(Inertia weight )
• Each species has an overall tendency to follow its instinct (personal)
and a tendency to focus on the group experience (social).
• C1: Defining the ability of the group to be influenced by the best
personal solutions found over the iterations
• C2: The ability of the group to be influenced by the best global
solution found over the iterations
Introduction to the PSO: Example
17

18. PSO Algorithm
• Relatively easy understanding algorithm
• Uses a fixed population of particles
• Often the number of population is 30, however the PSO handles
smaller or larger values
Each particles in PSO holds several values
• Current position (or model parameters)
• Best position & score
• Velocity Vector
18

19. The PSO algorithm
19

20. Applications of PSO
• Hyper-parameter tuning in DNN
• Solution for optimization problems in
engineering problems
• Cloud computing
• Robotics
• Telecommunication
WSN Network
Coverage
Hyper-parameter tuning
in DNN
Task Scheduling in a Cloud
Computing
Swarm Intelligent Robotics
Form-finding analysis of a
suspension bridge installation
20

21. PSO Algorithm Summary
• The flocking and PSO algorithm uses particles
• Flocking utilize individual particles to simulate flocks of birds.
• Three simple rules (Separation, Alignment, and Cohesion) govern the
complex flocking behavior
• PSO extends the flocking behavior to become an optimization
algorithm
• PSO is one of the cooperative population like ACO (Ant Colony
Optimization)
21

22. Recommended Research Area
Hyper-parameter Selection
22

23. Further Reading on Variants of PSO
• Hybrid of Genetic Algorithm and PSO (GA-PSO)
• Hybrid of Evolutionary Programming and PSO (EPSO)
• Adaptive PSO (APSO)
• Multi Objective PSO (MOPSO) and others
23

24. Reference Materials
24

25. 25