Optimizations technique genetic technique swarm technique

1. optimiza
tion
2016
AhmedMohamed Abdel-Hakeem Elkholy
economic

2. Table of Contents
Contents
definition_______________________________________________________________________ 1
step of optimization ____________________________________________________________ 1
Classification of optimizations technique ____________________________________________ 2
deterministic algorithms – classical algorithms______________________________________ 2
Classification of Meta Heuristic Algorithms _________________________________________ 3
Genetic Algorithm (GA)___________________________________________________________ 5
The implementation of GA is considered as follows _________________________________ 7
Particle Swarm Optimization Algorithm (PSO) ________________________________________ 8
Over view ____________________________________________________________________ 8
What is the different between the genetic and swarm? _______________________________ 8
PSO population _______________________________________________________________ 8
The implementation of PSO _____________________________________________________ 9

3. Pg. 01 definition
definition
an optimization problem is the problem of finding the best solution from all feasible solutions
step of optimization
1. The first step for using any successful optimization algorithm is to develop an
appropriate objective function to be optimized. This objective function can be a
combination of quantities, which are represented by a single variable such as cost,
weight, potential energy, number of devices, power losses, voltage deviation and so
on.
2. The second step in optimization is to determine the unknown variables that affect the
objective function. Once the objective is determined and variables are defined, the goal
of optimization is to find the values of variables that give the best solution in logical
time limit. The optimization process is limited by certain constraints related either to the
variables themselves or generally to the power system operation.
3. The last step in the optimization process is to choose the suitable optimization
algorithm to solve the problem and get the optimal solution.
During the evolution process, two types of solutions can be recognized.
The first one is the feasible solutions that satisfies all constraints
The second is the infeasible solutions that violate constraints

4. Pg. 02 Classification of optimizations technique
Classification of optimizationstechnique
The optimization technique is divided in two main classifiction
deterministic algorithms – classicalalgorithms
firstly, the main way of optimization is
 Nonlinear programming (NLP)
 Linear programming (LP)
 Interior point method (IP)
 Quadratic method (QM)
 Gradient method (GM)
 Lagrange decomposition based method
Defect of classical optimization technique
1. they may be trapped in local optimal and hence, they may fail to find optimal solution.
2. they require additional information about objective function and constraints with
derivative level. Look note 1
3. For small search space, the classical optimization method will succeed to find optimal
solution, however if the problem is difficult and the search space is large, the classical
method will fail to find optimal solution.
4. the classical optimization techniques fail to handle optimization problems that have
discontinues nature or that have no derivative.
Note 1 on 2 -Before selecting any optimization algorithm, some information must be known
about the optimization problem. Firstly, due to the existence of more than one local solution, it
is important to use a suitable optimization method to avoid misguide in search process and
stuck in local solution. So, the optimization algorithm should differentiate between local and
global search points.

5. Pg. 03 Classification of optimizations technique
Note 2 on 4- Continuous and discrete variables.
In mathematics, a variable may be continuous or discrete. If it can take on two particular real
values such that it can also take on all real values between them (even values that are
arbitrarily close together), the variable is continuous in that interval. If it can take on a value
such that there is a non-infinitesimal gap on each side of it containing no values that the
variable can take on, then it is discrete around that value. In some contexts, a variable can be
discrete in some ranges of the number line and continuous in others.
Classification ofMeta Heuristic Algorithms
Meta Heuristic algorithms is divided in to
1. Evolution based techniques:
1.1 Genetic Algorithm (GA)
1.2 Genetic Programming (GP)
1.3 Evolution Strategy (ES)
1.4 Differential Evolution Algorithm (DEA)
1.5 Paddy Field Algorithm (PFA)
2. Swarm based techniques:
2.1 Group Search Algorithm (GSA)
2.2 Particle Swarm Algorithm (PSA)
2.3 Ant Colony Optimization (ACO)
2.4 Fish Swarm Algorithm (FSA)
2.5 Firefly Algorithm (FA)
2.6 Artificial Bee Colony Algorithm (ABCA)

6. Pg. 04 Classification of optimizations technique
2.7 Shuffled Frog Leaping Algorithm (SFLA)
2.8 Seeker Optimization Algorithm (SOA)
3. Ecology algorithms:
3.1 Biography- Based Optimization (BBO)
3.2 Invasive Weed Colony Optimization
3.3 Multi Species Optimizer
The general steps for any algorithms
1. Initialization by random values
2. Calculating the fitness function of each individual of population.
3. Creating new population by modifying individual through simply reproduction.
4. Looping to the second step till stopping criteria is satisfied.
Heuristics are classified into two main categories
the first one is constructive algorithm,
The concept of constructive algorithm is based on building a solution by joining components of
solution together, which is annexed one after the other until completing the solution.
the second category is local search algorithms.
the local search algorithms begin with current solution (preexistent solution) then improve it by
some modifications, called moves, for its components.

7. Pg. 05 Genetic Algorithm (GA)
Genetic Algorithm (GA)
is a parallel and global search technique, which solves optimization problems based on natural
evolution such as mutation, crossover and selection.
GA differs from other optimization algorithms and procedures in four points as
follows
1. Since GA depends on the coding of parameter set not on the parameters themselves,
GA can easily handle integer and discrete variables.
2. GA provides a global optimal solution because it searches within a population of points
not in a single point similar to some other optimization algorithms.
3. GA utilizes only objective function information, so it is used to deal with non-smooth,
discontinuous and non-differentiable functions, while many algorithms need derivatives
or auxiliary knowledge about functions.
4. GA uses probabilistic transition rules rather than deterministic transition rules.
GA evaluates a number of solutions simultaneously. The first step is to initialize a
random population having P individuals containing feasible and infeasible solutions. The
individuals are evaluated seeking to find the value of the objective function. The constrained
optimization problem can be converted to unconstrained one by adding penalty functions.
Then, the objective function is mapped into fitness function and finally GA operators are
applied.
GA depends on many operators, such as selection and reproduction, where good strings
in a population are chosen and a mating pool is formed. Individuals with better fitness have
higher probability to participate in the reproduction process. Among selection operators,
Roulette-Wheel represents the most popular selection operator. A string is selected with a
probability, which proportional to its fitness.
Regarding the reproduction process, the crossover is used to mix two strings to
produce a better string. A combination between two individuals is performed to create different
individuals; while new strings are created by exchanging information between strings in mating
pool. The strings before recombination are called parent strings; on the other hand, the
resultant string is called children strings or offspring. Commonly, all present strings that are in
the mating pool are not used in crossover

8. Pg. 06 Genetic Algorithm (GA)
Many types of crossover are used in GA, where the most popular are the one-site
crossover and two-site crossover. To carry out crossover operator, two strings are selected
randomly from mating pool, and then some portions of strings are exchange between them. To
execute the one-site crossover operator, a crossing site along the string is picked randomly,
then all bits on the right side are exchanged. On the other hand, twosite crossover is illustrated
in Figure 2-3. If the string length is small, the onesite crossover is the suitable choice, while in
case long-length string, the twosite crossover is preferred.
The third important operator which affects GA performance is the mutation. This
operator has two functions; the first function is adding randomly new information to the search
process, while the other one is helping to eliminate trapping in local optimal. Mutation operator
plays an important role in alternating population homogeneously. To perform this operation, it
regards the bit level specially when copying the bits from current string to form new string.

9. Pg. 07 Genetic Algorithm (GA)
Mutation probability is defined as the probability of bits to be mutated. To carry out
mutation process a mechanism, which is called coin toss, is used. The bit is inverted (zero
becomes one or one becomes zero) when the random number which selected between zero
and one is less than mutation probability.
The implementation ofGA is consideredas follows
Step 1: Initialization: where the algorithm begins by a population, which includes n
chromosomes that represents candidate solutions.
Step 2: Assigning fitness values: where the objective function for each chromosome is
calculated and the best value of the objective function is identified, the chromosome of the best
value is defined as marginal global best.
Step 3: Defining new population: where the new population is created
Step 4: Selection: where two parents’ chromosomes are selected for the population based on
their fitness.
Step 5: Recombination: in the crossover process, a new child is created based on crossover
probability. Through the mutation process, a new gene at each chromosome is mutated by
using mutation probability.
Step 6: Replacement: where new children are inserted in the population
Step 7: Termination: where the process stops if the stopping criteria is satisfied

10. Pg. 08 Particle Swarm Optimization Algorithm (PSO)
Particle Swarm Optimization Algorithm
(PSO)
Over view
PSO is inspired by the nature such as bird flocking, section interaction behavior of human and
fish schooling. In case of birds and fishes, each member adjusts its position and velocity
according to group information, so they travel in a group without collision. Thus, the efforts of
searching for food are reduced.
What is the different between the genetic and swarm?
1. PSO has high efficiency of computation.
2. PSO needs small memory space.
3. PSO needs low speed of CPU.
4. PSO includes few numbers of parameters.
5. PSO deals with discrete and analogue systems, while GA deals with discrete
variables.
PSO population
the population members are called particles without mass and volumes. Each particle has four
vectors:
the first one is defined as current position, while the second one is the best position found
so far. In addition, the third one is the best position found by its neighborhood so far and the
fourth vector is the velocity. The particle adjusts its position according to the best position
reached by it (Pbest) and the best position reached by its neighborhood (gbest) through the
search space. Each particle modifies its position and its velocity after each iteration as follows

11. Pg. 09 Particle Swarm Optimization Algorithm (PSO)
Figure below provides a schematic representation of updating the velocity of particles
The implementation ofPSO
1. Initialization: each particle is taking a random position in the search space.
2. Evaluation: to evaluate the objective function for each particle.
3. Defining the marginal global best: search the best value of the objective function and
set the particle associated with the best objective function as the global best.
4. Updating the inertia weight.
5. Updating the velocity of the particle according to
6. Updating the position of the particle according to
7. Defining the minimum or maximum value of fitness function and updating the global
best.
8. Terminating the process if the stopping criterion is satisfied.