SlideShare a Scribd company logo
1 of 45
YAHYE ABUKAR AHMED
Supervisor: Yrd. Doç.Dr. MUSTAFA SERVET KIRAN
FEUTURE SELECTION
USING PARTICLE SWARM
OPTIMIZATION (PSO)
Date: 16 /12/ 2015
Presentation outline
1.0 • Introduction
2.0 • Concept of PSO
3.0 • Literature Review
4.0
• PSO Algorithm-Parameter Selection
5.0
• DWC’s application
6.0
• Summary of reviewed paper7.0
• Implementing PSO on the Text
 A group of birds are randomly
searching food in an area. There is only
one piece of food in the area being
searched.
 All the birds do not know where the
food is. But they know how far the
food is in each iteration.
 So what's the best strategy to find the
food?????????????????????
 The effective one is to follow the bird
which is nearest to the food.
Problem Definition
Suppose the following scenario:
optimization of continuous nonlinear functions
↓
finding the best solution in problem space
Problem Definition
5
 Developed by James
Kennedy((social-psychologist),
Bureau of Labor Statistics, U.S.
Department of Labor.
 and Russ Eberhart (electrical
engineer), Purdue University at
1995.
Origins and Inspiration from Natural Systems
INTRODCUTION
• PSO is a robust stochastic optimization technique based
on the movement and intelligence of swarms.
• PSO applies the concept of social interaction to problem
solving.
• It uses a number of agents (particles) that constitute a
swarm moving around in the search space looking for the
best solution.
• Each particle is treated as a point in a N-dimensional space
which adjusts its “flying” according to its own flying
experience as well as the flying experience of other
particles.
What is Particle Swarm Optimization (PSO)?
INTRODCUTION
• Each solution is considered as bird, called particle
All the particles have a fitness value.
• The fitness values can be calculated using objective
function.
• All the particles preserve their individual best performance
They also know the best performance of their group
They adjust their velocity considering their best
performance and also considering the best performance of
the best particle
What is Particle Swarm Optimization (PSO)?
INTRODCUTION
• Each particle keeps track of its coordinates in the
solution space which are associated with the best
solution (fitness) that has achieved so far by that
particle. This value is called personal best , pbest.
• Another best value that is tracked by the PSO is the
best value obtained so far by any particle in the
neighborhood of that particle. This value is called
gbest.
• The basic concept of PSO lies in accelerating each
particle toward its pbest and the gbest locations, with
a random weighted accelaration at each time step as
shown in Fig.1
INTRODCUTION
What is Particle Swarm Optimization (PSO)?
Fig.1 Concept of modification of a searching point by PSO
sk : current searching point.
sk+1: modified searching point.
vk: current velocity.
vk+1: modified velocity.
vpbest : velocity based on pbest.
vgbest : velocity based on gbest
sk
vk
vpbest
vgbest
sk+1
vk+1
sk
vk
vpbest
vgbest
sk+1
vk+1
x
y
CONCEPT OF PSO
• Each particle tries to modify its position
using the following
information:
 the current positions,
 the current velocities,
 the distance between the current position and pbest,
 the distance between the current position and the gbest.
CONCEPT OF PSO
• The modification of the particle’s position can be
mathematically
• modeled according the following equation :
• Vi
k+1 = wVi
k +c1 rand1(…) x (pbesti-si
k) + c2 rand2(…) x
(gbest-si
k) ….. (1)
• where, vi
k : velocity of agent i at iteration k,
w: weighting function,
cj : weighting factor,
rand : uniformly distributed random number between 0 and 1,
si
k : current position of agent i at iteration k,
pbesti : pbest of agent i,
gbest: gbest of the group.
CONCEPT OF PSO
The following weighting function is usually utilized in (1)
w = wMax-[(wMax-wMin) x iter]/maxIter (2)
where wMax= initial weight,
wMin = final weight,
maxIter = maximum iteration number,
iter = current iteration number.
si
k+1 = si
k + Vi
k+1 (3)
CONCEPT OF PSO
A large inertia weight (w) facilitates a global search while
a small inertia weight facilitates a local search.
By linearly decreasing the inertia weight from a relatively
large value to a small value through the course of the
PSO run gives the best PSO performance compared
with fixed inertia weight settings.
Larger w ----------- greater global search ability
Smaller w ------------ greater local search ability.
Inertial weight factor
Flow chart depicting
the General PSO Algorithm:
Start
Initialize particles with random position
and velocity vectors.
For each particle’s position (p)
evaluate fitness
If fitness(p) better than
fitness(pbest) then pbest= p
Loopuntilall
particlesexhaust
Set best of pBests as gBest
Update particles velocity (eq. 1) and
position (eq. 3)
Loopuntilmaxiter
Stop: giving gBest, optimal solution.
Particles Adjust their positions according to a ``Psychosocial
compromise’’ between what an individual is comfortable with, and
what society reckons
Here I
am!
The best
perf. of my
neighbours
My best
performance
x
pg
pi
v
How It Works
Example
Example
Example
Example
Example
Example
Example
Example
 Algorithm parameters
– A : Population of agentspi : Position of agent ai in the solution space
– pi : Position of agent ai in the solution space
– f : Objective function
– vi : Velocity of agent’s ai
– V(ai) : Neighborhood of agent ai (fixed)
 The neighborhood concept in PSO is not the same as the one
used in other meta-heuristics search, since in PSO each
particle’s neighborhood never changes (is fixed)
Algorithm - Parameters
[x*] = PSO()
P = Particle_Initialization();
For i=1 to it_max
For each particle p in P do
fp = f(p);
If fp is better than f(pBest)
pBest = p;
end
end
gBest = best p in P;
For each particle p in P do
v = v + c1*rand*(pBest – p) + c2*rand*(gBest – p);
p = p + v;
end
end
Algorithm - Parameters
 Particle update rule
p = p + v
 with
v = v + c1 * rand * (pBest – p) + c2 * rand * (gBest – p)
 where
• p: particle’s position
• v: path direction
• c1: weight of local information
• c2: weight of global information
• pBest: best position of the particle
• gBest: best position of the swarm
• rand: random variable
Algorithm - Parameters
Nebojša Trpković
trx.lists@gmail.com
Slide 27 of 18
Single Particle
 Number of particles usually between 10 and 50
 C1 is the importance of personal best value
 C2 is the importance of neighborhood best value
 Usually C1 + C2 = 4 (empirically chosen value)
 If velocity is too low → algorithm too slow
 If velocity is too high → algorithm too unstable
 Cognitive coefficient c1 and social coefficient c2 are constants
known as acceleration coefficients,
Algorithm - Parameters
1. Create a ‘population’ of agents (particles) uniformly distributed
over X
2. Evaluate each particle’s position according to the objective
function
3. If a particle’s current position is better than its previous best
position, update it
4. Determine the best particle (according to the particle’s previous
best positions)
Algorithm - Parameters
5. Update particles’ velocities:
6. Move particles to their new positions:
7. Go to step 2 until stopping criteria are satisfied
Algorithm - Parameters
Particle’s velocity:
• Makes the particle move in the same
direction and with the same velocity
1. Inertia
2. Personal
Influence
3. Social
Influence
• Improves the individual
• Makes the particle return to a previous
position, better than the current
• Conservative
• Makes the particle follow the best
neighbors direction
Algorithm - Parameters
 Intensification: explores the previous solutions, finds the best
solution of a given region
 Diversification: searches new solutions, finds the regions with
potentially the best solutions
 In PSO:
Algorithm - Parameters
• Number of particles (swarmsize)
• C1 (importance of personal best)
• C2 (importance of neighbourhood best)
• Vmax: limit on velocity
• c1, c2 are learning factors
DWC’s Parameters
Play with DWC’s app for a while
TEXT FEATURE SELECTION
USING
PARTICLE SWARM OPTIMIZATION
36
Machine
learning
Unsupervised
supervised
Nature-Inspired
Optimization
Algorithms
GA-
Algorithm
Differential
evolution
algorithms
Particle
Swarm
Optimization
Firefly
Algorithms
Bat
Algorithm
feature
Selection
Biomedical
data
Data
mining
Text
Categorization
Where??????
• Feature selection is used to identify a powerfully
predictive subset of fields within the database and to
reduce the number of fields presented to the mining
process.
• Affects several aspects of pattern classification:
1.The accuracy of classification algorithm learned
2.The time needed for learning a classification
function
3.The number of examples needed for learning
4.The cost associated with feature
Feature Selection
• Filter
• Wrapper
Feature
Generation
Learning
Algorithm
Testing
Learning
Algorithm
Good ?
Phase 1
Phase 2
Subset
Classifier Best Subset
Training
Data
Training
Data
Testing
Data
Accuracy
Yes
No
AccuracyFull Set
Feature Selection
39
In text classification, we usually represent documents in a
high-dimensional space, with each dimension corresponding to
a term.
In this lecture: axis = dimension = word = term = feature
Many dimensions correspond to rare words.
Rare words can mislead the classifier.
Rare misleading features are called noise features.
Eliminating noise features from the representation increases
efficiency and effectiveness of text classification.
Eliminating features is called feature selection.
Feature Selection
Types of Feature
Document Frequency Gain Ratio (GR) Fisher Score
Filter-basedWrapper based Embedded methods
Feature Selection
41
 A feature selection method is mainly defined by the
feature utility measure it employs
Feature utility measures:
Frequency – select the most frequent terms
Mutual information – select the terms with the highest
mutual information
Mutual information is also called information gain in
this context.
Chi-square
Document Frequency (DF)
Different feature selection
methods
Methods used in the reviewed
papers
The paper of Ferruh which about A New Feature Selection Method For Text
Categorization Based On Information Gain And Particle Swarm
Optimization
Summary of reviewed parpers on particle swarm
optimization algorithm
Summary of reviewed parpers on particle swarm
optimization algorithm
THANKS

More Related Content

What's hot

Particle swarm optimization
Particle swarm optimizationParticle swarm optimization
Particle swarm optimization
Hanya Mohammed
 
Sequence Modelling with Deep Learning
Sequence Modelling with Deep LearningSequence Modelling with Deep Learning
Sequence Modelling with Deep Learning
Natasha Latysheva
 
Support Vector Machines
Support Vector MachinesSupport Vector Machines
Support Vector Machines
nextlib
 
Intro to Feature Selection
Intro to Feature SelectionIntro to Feature Selection
Intro to Feature Selection
chenhm
 

What's hot (20)

Variational inference intro. (korean ver.)
Variational inference intro. (korean ver.)Variational inference intro. (korean ver.)
Variational inference intro. (korean ver.)
 
Deep Q-Learning
Deep Q-LearningDeep Q-Learning
Deep Q-Learning
 
Particle swarm optimization
Particle swarm optimizationParticle swarm optimization
Particle swarm optimization
 
Sequence Modelling with Deep Learning
Sequence Modelling with Deep LearningSequence Modelling with Deep Learning
Sequence Modelling with Deep Learning
 
Policy gradient
Policy gradientPolicy gradient
Policy gradient
 
Decision tree and random forest
Decision tree and random forestDecision tree and random forest
Decision tree and random forest
 
Genetic Algorithm
Genetic AlgorithmGenetic Algorithm
Genetic Algorithm
 
Methods of Optimization in Machine Learning
Methods of Optimization in Machine LearningMethods of Optimization in Machine Learning
Methods of Optimization in Machine Learning
 
Support Vector Machines
Support Vector MachinesSupport Vector Machines
Support Vector Machines
 
Genetic algorithm fitness function
Genetic algorithm fitness functionGenetic algorithm fitness function
Genetic algorithm fitness function
 
Support Vector machine
Support Vector machineSupport Vector machine
Support Vector machine
 
Anomaly detection Workshop slides
Anomaly detection Workshop slidesAnomaly detection Workshop slides
Anomaly detection Workshop slides
 
Support vector machine
Support vector machineSupport vector machine
Support vector machine
 
What is the Expectation Maximization (EM) Algorithm?
What is the Expectation Maximization (EM) Algorithm?What is the Expectation Maximization (EM) Algorithm?
What is the Expectation Maximization (EM) Algorithm?
 
Optimization in Deep Learning
Optimization in Deep LearningOptimization in Deep Learning
Optimization in Deep Learning
 
Intro to Feature Selection
Intro to Feature SelectionIntro to Feature Selection
Intro to Feature Selection
 
The world of loss function
The world of loss functionThe world of loss function
The world of loss function
 
Nature-inspired algorithms
Nature-inspired algorithmsNature-inspired algorithms
Nature-inspired algorithms
 
07 regularization
07 regularization07 regularization
07 regularization
 
Particle swarm optimization
Particle swarm optimizationParticle swarm optimization
Particle swarm optimization
 

Viewers also liked

Feature Selection using Complementary Particle Swarm Optimization for DNA Mic...
Feature Selection using Complementary Particle Swarm Optimization for DNA Mic...Feature Selection using Complementary Particle Swarm Optimization for DNA Mic...
Feature Selection using Complementary Particle Swarm Optimization for DNA Mic...
sky chang
 
Swarm intelligence pso and aco
Swarm intelligence pso and acoSwarm intelligence pso and aco
Swarm intelligence pso and aco
satish561
 
PSO and Its application in Engineering
PSO and Its application in EngineeringPSO and Its application in Engineering
PSO and Its application in Engineering
Prince Jain
 

Viewers also liked (20)

Particle Swarm Optimization: The Algorithm and Its Applications
Particle Swarm Optimization: The Algorithm and Its ApplicationsParticle Swarm Optimization: The Algorithm and Its Applications
Particle Swarm Optimization: The Algorithm and Its Applications
 
Practical Swarm Optimization (PSO)
Practical Swarm Optimization (PSO)Practical Swarm Optimization (PSO)
Practical Swarm Optimization (PSO)
 
PSOk-NN: A Particle Swarm Optimization Approach to Optimize k-Nearest Neighbo...
PSOk-NN: A Particle Swarm Optimization Approach to Optimize k-Nearest Neighbo...PSOk-NN: A Particle Swarm Optimization Approach to Optimize k-Nearest Neighbo...
PSOk-NN: A Particle Swarm Optimization Approach to Optimize k-Nearest Neighbo...
 
Particle swarm optimization
Particle swarm optimizationParticle swarm optimization
Particle swarm optimization
 
Particle Swarm Optimization and it's Applications in Electromagnetics
Particle Swarm Optimization and it's Applications in ElectromagneticsParticle Swarm Optimization and it's Applications in Electromagnetics
Particle Swarm Optimization and it's Applications in Electromagnetics
 
Feature Selection using Complementary Particle Swarm Optimization for DNA Mic...
Feature Selection using Complementary Particle Swarm Optimization for DNA Mic...Feature Selection using Complementary Particle Swarm Optimization for DNA Mic...
Feature Selection using Complementary Particle Swarm Optimization for DNA Mic...
 
Machine Learning Tools and Particle Swarm Optimization for Content-Based Sear...
Machine Learning Tools and Particle Swarm Optimization for Content-Based Sear...Machine Learning Tools and Particle Swarm Optimization for Content-Based Sear...
Machine Learning Tools and Particle Swarm Optimization for Content-Based Sear...
 
Swarm intelligence pso and aco
Swarm intelligence pso and acoSwarm intelligence pso and aco
Swarm intelligence pso and aco
 
Particles Swarm Optimization
Particles Swarm OptimizationParticles Swarm Optimization
Particles Swarm Optimization
 
Particle Swarm Optimization (A Circuit Optimization Problem)
Particle Swarm Optimization (A Circuit Optimization Problem)Particle Swarm Optimization (A Circuit Optimization Problem)
Particle Swarm Optimization (A Circuit Optimization Problem)
 
Particle Swarm Optimization Matlab code Using 50, 5000 Swarms
Particle Swarm Optimization Matlab code Using 50, 5000 SwarmsParticle Swarm Optimization Matlab code Using 50, 5000 Swarms
Particle Swarm Optimization Matlab code Using 50, 5000 Swarms
 
Data are the new oil: Big data, data mining and bio - inspiring techniques
Data are the new oil: Big data, data mining and bio - inspiring techniquesData are the new oil: Big data, data mining and bio - inspiring techniques
Data are the new oil: Big data, data mining and bio - inspiring techniques
 
Pso introduction
Pso introductionPso introduction
Pso introduction
 
PSO and Its application in Engineering
PSO and Its application in EngineeringPSO and Its application in Engineering
PSO and Its application in Engineering
 
Particle Swarm Optimization by Rajorshi Mukherjee
Particle Swarm Optimization by Rajorshi MukherjeeParticle Swarm Optimization by Rajorshi Mukherjee
Particle Swarm Optimization by Rajorshi Mukherjee
 
Solving travelling salesman problem using firefly algorithm
Solving travelling salesman problem using firefly algorithmSolving travelling salesman problem using firefly algorithm
Solving travelling salesman problem using firefly algorithm
 
A new hybrid particle swarm optimization with variable neighborhood search fo...
A new hybrid particle swarm optimization with variable neighborhood search fo...A new hybrid particle swarm optimization with variable neighborhood search fo...
A new hybrid particle swarm optimization with variable neighborhood search fo...
 
Firefly algorithm
Firefly algorithmFirefly algorithm
Firefly algorithm
 
Ant colony optimization
Ant colony optimizationAnt colony optimization
Ant colony optimization
 
PROPOSED FAULT DETECTION ON OVERHEAD TRANSMISSION LINE USING PARTICLE SWARM ...
PROPOSED FAULT DETECTION ON OVERHEAD TRANSMISSION LINE  USING PARTICLE SWARM ...PROPOSED FAULT DETECTION ON OVERHEAD TRANSMISSION LINE  USING PARTICLE SWARM ...
PROPOSED FAULT DETECTION ON OVERHEAD TRANSMISSION LINE USING PARTICLE SWARM ...
 

Similar to TEXT FEUTURE SELECTION USING PARTICLE SWARM OPTIMIZATION (PSO)

Particle Swarm Optimization Application In Power System
Particle Swarm Optimization Application In Power SystemParticle Swarm Optimization Application In Power System
Particle Swarm Optimization Application In Power System
Ministry of New & Renewable Energy, Govt of India
 
Particle swarm optimization (PSO) ppt presentation
Particle swarm optimization (PSO) ppt presentationParticle swarm optimization (PSO) ppt presentation
Particle swarm optimization (PSO) ppt presentation
LatestShorts
 

Similar to TEXT FEUTURE SELECTION USING PARTICLE SWARM OPTIMIZATION (PSO) (20)

Optimization and particle swarm optimization (O & PSO)
Optimization and particle swarm optimization (O & PSO) Optimization and particle swarm optimization (O & PSO)
Optimization and particle swarm optimization (O & PSO)
 
DriP PSO- A fast and inexpensive PSO for drifting problem spaces
DriP PSO- A fast and inexpensive PSO for drifting problem spacesDriP PSO- A fast and inexpensive PSO for drifting problem spaces
DriP PSO- A fast and inexpensive PSO for drifting problem spaces
 
Particle Swarm Optimization - PSO
Particle Swarm Optimization - PSOParticle Swarm Optimization - PSO
Particle Swarm Optimization - PSO
 
Particle swarm optimization
Particle swarm optimizationParticle swarm optimization
Particle swarm optimization
 
Pso notes
Pso notesPso notes
Pso notes
 
introduction pso.ppt
introduction pso.pptintroduction pso.ppt
introduction pso.ppt
 
Particle Swarm Optimization Application In Power System
Particle Swarm Optimization Application In Power SystemParticle Swarm Optimization Application In Power System
Particle Swarm Optimization Application In Power System
 
Bic pso
Bic psoBic pso
Bic pso
 
PSO
PSOPSO
PSO
 
PSO-ACO-Presentation.pptx
PSO-ACO-Presentation.pptxPSO-ACO-Presentation.pptx
PSO-ACO-Presentation.pptx
 
Particle Swarm Optimization.pptx
Particle Swarm Optimization.pptxParticle Swarm Optimization.pptx
Particle Swarm Optimization.pptx
 
PSO.ppsx
PSO.ppsxPSO.ppsx
PSO.ppsx
 
Particle swarm optimization (PSO) ppt presentation
Particle swarm optimization (PSO) ppt presentationParticle swarm optimization (PSO) ppt presentation
Particle swarm optimization (PSO) ppt presentation
 
swarm pso and gray wolf Optimization.pdf
swarm pso and gray wolf Optimization.pdfswarm pso and gray wolf Optimization.pdf
swarm pso and gray wolf Optimization.pdf
 
11-Optimization algorithm with swarm.pptx
11-Optimization algorithm with swarm.pptx11-Optimization algorithm with swarm.pptx
11-Optimization algorithm with swarm.pptx
 
PSO.pptx
PSO.pptxPSO.pptx
PSO.pptx
 
Soft computing
Soft computingSoft computing
Soft computing
 
Optimization Using Evolutionary Computing Techniques
Optimization Using Evolutionary Computing Techniques Optimization Using Evolutionary Computing Techniques
Optimization Using Evolutionary Computing Techniques
 
04 20259 real power loss
04 20259 real power loss04 20259 real power loss
04 20259 real power loss
 
Metaheuristics for software testing
Metaheuristics for software testingMetaheuristics for software testing
Metaheuristics for software testing
 

Recently uploaded

Architecting Cloud Native Applications
Architecting Cloud Native ApplicationsArchitecting Cloud Native Applications
Architecting Cloud Native Applications
WSO2
 
Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Finding Java's Hidden Performance Traps @ DevoxxUK 2024Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Victor Rentea
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Safe Software
 
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
Victor Rentea
 

Recently uploaded (20)

"I see eyes in my soup": How Delivery Hero implemented the safety system for ...
"I see eyes in my soup": How Delivery Hero implemented the safety system for ..."I see eyes in my soup": How Delivery Hero implemented the safety system for ...
"I see eyes in my soup": How Delivery Hero implemented the safety system for ...
 
Introduction to Multilingual Retrieval Augmented Generation (RAG)
Introduction to Multilingual Retrieval Augmented Generation (RAG)Introduction to Multilingual Retrieval Augmented Generation (RAG)
Introduction to Multilingual Retrieval Augmented Generation (RAG)
 
Elevate Developer Efficiency & build GenAI Application with Amazon Q​
Elevate Developer Efficiency & build GenAI Application with Amazon Q​Elevate Developer Efficiency & build GenAI Application with Amazon Q​
Elevate Developer Efficiency & build GenAI Application with Amazon Q​
 
AI in Action: Real World Use Cases by Anitaraj
AI in Action: Real World Use Cases by AnitarajAI in Action: Real World Use Cases by Anitaraj
AI in Action: Real World Use Cases by Anitaraj
 
Navigating Identity and Access Management in the Modern Enterprise
Navigating Identity and Access Management in the Modern EnterpriseNavigating Identity and Access Management in the Modern Enterprise
Navigating Identity and Access Management in the Modern Enterprise
 
Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...
Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...
Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...
 
Simplifying Mobile A11y Presentation.pptx
Simplifying Mobile A11y Presentation.pptxSimplifying Mobile A11y Presentation.pptx
Simplifying Mobile A11y Presentation.pptx
 
Vector Search -An Introduction in Oracle Database 23ai.pptx
Vector Search -An Introduction in Oracle Database 23ai.pptxVector Search -An Introduction in Oracle Database 23ai.pptx
Vector Search -An Introduction in Oracle Database 23ai.pptx
 
The Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and InsightThe Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and Insight
 
How to Check CNIC Information Online with Pakdata cf
How to Check CNIC Information Online with Pakdata cfHow to Check CNIC Information Online with Pakdata cf
How to Check CNIC Information Online with Pakdata cf
 
Corporate and higher education May webinar.pptx
Corporate and higher education May webinar.pptxCorporate and higher education May webinar.pptx
Corporate and higher education May webinar.pptx
 
Architecting Cloud Native Applications
Architecting Cloud Native ApplicationsArchitecting Cloud Native Applications
Architecting Cloud Native Applications
 
DBX First Quarter 2024 Investor Presentation
DBX First Quarter 2024 Investor PresentationDBX First Quarter 2024 Investor Presentation
DBX First Quarter 2024 Investor Presentation
 
Stronger Together: Developing an Organizational Strategy for Accessible Desig...
Stronger Together: Developing an Organizational Strategy for Accessible Desig...Stronger Together: Developing an Organizational Strategy for Accessible Desig...
Stronger Together: Developing an Organizational Strategy for Accessible Desig...
 
ChatGPT and Beyond - Elevating DevOps Productivity
ChatGPT and Beyond - Elevating DevOps ProductivityChatGPT and Beyond - Elevating DevOps Productivity
ChatGPT and Beyond - Elevating DevOps Productivity
 
Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Finding Java's Hidden Performance Traps @ DevoxxUK 2024Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Finding Java's Hidden Performance Traps @ DevoxxUK 2024
 
API Governance and Monetization - The evolution of API governance
API Governance and Monetization -  The evolution of API governanceAPI Governance and Monetization -  The evolution of API governance
API Governance and Monetization - The evolution of API governance
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
 
Platformless Horizons for Digital Adaptability
Platformless Horizons for Digital AdaptabilityPlatformless Horizons for Digital Adaptability
Platformless Horizons for Digital Adaptability
 
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
 

TEXT FEUTURE SELECTION USING PARTICLE SWARM OPTIMIZATION (PSO)

  • 1. YAHYE ABUKAR AHMED Supervisor: Yrd. Doç.Dr. MUSTAFA SERVET KIRAN FEUTURE SELECTION USING PARTICLE SWARM OPTIMIZATION (PSO) Date: 16 /12/ 2015
  • 2. Presentation outline 1.0 • Introduction 2.0 • Concept of PSO 3.0 • Literature Review 4.0 • PSO Algorithm-Parameter Selection 5.0 • DWC’s application 6.0 • Summary of reviewed paper7.0 • Implementing PSO on the Text
  • 3.  A group of birds are randomly searching food in an area. There is only one piece of food in the area being searched.  All the birds do not know where the food is. But they know how far the food is in each iteration.  So what's the best strategy to find the food?????????????????????  The effective one is to follow the bird which is nearest to the food. Problem Definition Suppose the following scenario:
  • 4. optimization of continuous nonlinear functions ↓ finding the best solution in problem space Problem Definition
  • 5. 5  Developed by James Kennedy((social-psychologist), Bureau of Labor Statistics, U.S. Department of Labor.  and Russ Eberhart (electrical engineer), Purdue University at 1995. Origins and Inspiration from Natural Systems INTRODCUTION
  • 6. • PSO is a robust stochastic optimization technique based on the movement and intelligence of swarms. • PSO applies the concept of social interaction to problem solving. • It uses a number of agents (particles) that constitute a swarm moving around in the search space looking for the best solution. • Each particle is treated as a point in a N-dimensional space which adjusts its “flying” according to its own flying experience as well as the flying experience of other particles. What is Particle Swarm Optimization (PSO)? INTRODCUTION
  • 7. • Each solution is considered as bird, called particle All the particles have a fitness value. • The fitness values can be calculated using objective function. • All the particles preserve their individual best performance They also know the best performance of their group They adjust their velocity considering their best performance and also considering the best performance of the best particle What is Particle Swarm Optimization (PSO)? INTRODCUTION
  • 8. • Each particle keeps track of its coordinates in the solution space which are associated with the best solution (fitness) that has achieved so far by that particle. This value is called personal best , pbest. • Another best value that is tracked by the PSO is the best value obtained so far by any particle in the neighborhood of that particle. This value is called gbest. • The basic concept of PSO lies in accelerating each particle toward its pbest and the gbest locations, with a random weighted accelaration at each time step as shown in Fig.1 INTRODCUTION What is Particle Swarm Optimization (PSO)?
  • 9. Fig.1 Concept of modification of a searching point by PSO sk : current searching point. sk+1: modified searching point. vk: current velocity. vk+1: modified velocity. vpbest : velocity based on pbest. vgbest : velocity based on gbest sk vk vpbest vgbest sk+1 vk+1 sk vk vpbest vgbest sk+1 vk+1 x y CONCEPT OF PSO
  • 10. • Each particle tries to modify its position using the following information:  the current positions,  the current velocities,  the distance between the current position and pbest,  the distance between the current position and the gbest. CONCEPT OF PSO
  • 11. • The modification of the particle’s position can be mathematically • modeled according the following equation : • Vi k+1 = wVi k +c1 rand1(…) x (pbesti-si k) + c2 rand2(…) x (gbest-si k) ….. (1) • where, vi k : velocity of agent i at iteration k, w: weighting function, cj : weighting factor, rand : uniformly distributed random number between 0 and 1, si k : current position of agent i at iteration k, pbesti : pbest of agent i, gbest: gbest of the group. CONCEPT OF PSO
  • 12. The following weighting function is usually utilized in (1) w = wMax-[(wMax-wMin) x iter]/maxIter (2) where wMax= initial weight, wMin = final weight, maxIter = maximum iteration number, iter = current iteration number. si k+1 = si k + Vi k+1 (3) CONCEPT OF PSO
  • 13. A large inertia weight (w) facilitates a global search while a small inertia weight facilitates a local search. By linearly decreasing the inertia weight from a relatively large value to a small value through the course of the PSO run gives the best PSO performance compared with fixed inertia weight settings. Larger w ----------- greater global search ability Smaller w ------------ greater local search ability. Inertial weight factor
  • 14. Flow chart depicting the General PSO Algorithm: Start Initialize particles with random position and velocity vectors. For each particle’s position (p) evaluate fitness If fitness(p) better than fitness(pbest) then pbest= p Loopuntilall particlesexhaust Set best of pBests as gBest Update particles velocity (eq. 1) and position (eq. 3) Loopuntilmaxiter Stop: giving gBest, optimal solution.
  • 15. Particles Adjust their positions according to a ``Psychosocial compromise’’ between what an individual is comfortable with, and what society reckons Here I am! The best perf. of my neighbours My best performance x pg pi v How It Works
  • 24.  Algorithm parameters – A : Population of agentspi : Position of agent ai in the solution space – pi : Position of agent ai in the solution space – f : Objective function – vi : Velocity of agent’s ai – V(ai) : Neighborhood of agent ai (fixed)  The neighborhood concept in PSO is not the same as the one used in other meta-heuristics search, since in PSO each particle’s neighborhood never changes (is fixed) Algorithm - Parameters
  • 25. [x*] = PSO() P = Particle_Initialization(); For i=1 to it_max For each particle p in P do fp = f(p); If fp is better than f(pBest) pBest = p; end end gBest = best p in P; For each particle p in P do v = v + c1*rand*(pBest – p) + c2*rand*(gBest – p); p = p + v; end end Algorithm - Parameters
  • 26.  Particle update rule p = p + v  with v = v + c1 * rand * (pBest – p) + c2 * rand * (gBest – p)  where • p: particle’s position • v: path direction • c1: weight of local information • c2: weight of global information • pBest: best position of the particle • gBest: best position of the swarm • rand: random variable Algorithm - Parameters
  • 28.  Number of particles usually between 10 and 50  C1 is the importance of personal best value  C2 is the importance of neighborhood best value  Usually C1 + C2 = 4 (empirically chosen value)  If velocity is too low → algorithm too slow  If velocity is too high → algorithm too unstable  Cognitive coefficient c1 and social coefficient c2 are constants known as acceleration coefficients, Algorithm - Parameters
  • 29. 1. Create a ‘population’ of agents (particles) uniformly distributed over X 2. Evaluate each particle’s position according to the objective function 3. If a particle’s current position is better than its previous best position, update it 4. Determine the best particle (according to the particle’s previous best positions) Algorithm - Parameters
  • 30. 5. Update particles’ velocities: 6. Move particles to their new positions: 7. Go to step 2 until stopping criteria are satisfied Algorithm - Parameters
  • 31. Particle’s velocity: • Makes the particle move in the same direction and with the same velocity 1. Inertia 2. Personal Influence 3. Social Influence • Improves the individual • Makes the particle return to a previous position, better than the current • Conservative • Makes the particle follow the best neighbors direction Algorithm - Parameters
  • 32.  Intensification: explores the previous solutions, finds the best solution of a given region  Diversification: searches new solutions, finds the regions with potentially the best solutions  In PSO: Algorithm - Parameters
  • 33. • Number of particles (swarmsize) • C1 (importance of personal best) • C2 (importance of neighbourhood best) • Vmax: limit on velocity • c1, c2 are learning factors DWC’s Parameters
  • 34. Play with DWC’s app for a while
  • 37. • Feature selection is used to identify a powerfully predictive subset of fields within the database and to reduce the number of fields presented to the mining process. • Affects several aspects of pattern classification: 1.The accuracy of classification algorithm learned 2.The time needed for learning a classification function 3.The number of examples needed for learning 4.The cost associated with feature Feature Selection
  • 38. • Filter • Wrapper Feature Generation Learning Algorithm Testing Learning Algorithm Good ? Phase 1 Phase 2 Subset Classifier Best Subset Training Data Training Data Testing Data Accuracy Yes No AccuracyFull Set Feature Selection
  • 39. 39 In text classification, we usually represent documents in a high-dimensional space, with each dimension corresponding to a term. In this lecture: axis = dimension = word = term = feature Many dimensions correspond to rare words. Rare words can mislead the classifier. Rare misleading features are called noise features. Eliminating noise features from the representation increases efficiency and effectiveness of text classification. Eliminating features is called feature selection. Feature Selection
  • 40. Types of Feature Document Frequency Gain Ratio (GR) Fisher Score Filter-basedWrapper based Embedded methods Feature Selection
  • 41. 41  A feature selection method is mainly defined by the feature utility measure it employs Feature utility measures: Frequency – select the most frequent terms Mutual information – select the terms with the highest mutual information Mutual information is also called information gain in this context. Chi-square Document Frequency (DF) Different feature selection methods
  • 42. Methods used in the reviewed papers The paper of Ferruh which about A New Feature Selection Method For Text Categorization Based On Information Gain And Particle Swarm Optimization
  • 43. Summary of reviewed parpers on particle swarm optimization algorithm
  • 44. Summary of reviewed parpers on particle swarm optimization algorithm