This talk is presented at Bio-inspiring and evolutionary computation: Trends, applications and open issues workshop, 7 Nov. 2015 Faculty of Computers and Information, Cairo University

1. AN INNOVATIVE APPROACH FOR FEATURE
SELECTION BASED ON CHICKEN SWARM
OPTIMIZATION
By
*Ahmed Hafez and Aboul Ella Hassanien
Bio-inspiring and evolutionary computation: Trends, applications and open issues workshop, 7 Nov. 2015 Faculty of
Computers and Information, Cairo University
www.egyptscience.
*Faculty of Computers and Information, Minia University and SRGE Member

2. Scientific Research Group in Egypt
www.egyptscience.net

3. Agenda
1. Introduction.
3. System Overview and Methods
4. Experimental Results.
5. Conclusions.
2. Motivation

4. Introduction
 Feature selection algorithm explores the data to
eliminate noisy, irrelevant, redundant data, and
simultaneously optimize the classification
performance.
 Feature selection is one of the most important stage in
data mining, multimedia information retrieval, pattern
classification, and machine learning applications,
which can influence the classification accuracy rate.
 In real world problems, feature selection is a must due
to the abundance of noisy, misleading or irrelevant
features

5. Introduction
Feature Selection in ML
 Naive theoretical view:
More features
=> More information
=> More discrimination power
 In practice:
many reasons why this is not the case!

6. Introduction
Feature Selection in ML
 Many explored domains have hundreds to tens of thousands
of variables/features with many irrelevant and redundant
ones.
 In domains with many features the underlying probability
distribution can be very complex and very hard to estimate
(e.g. dependencies between variables).
 Irrelevant and redundant features can “confuse” learners
(classifiers).
 Limited training data.
 Limited computational resources.

7. Agenda
1. Introduction.
3. System Overview and Methods
4. Experimental Results.
5. Conclusions.
2. Motivation.

8. Motivation
 Chicken Swarm Optimization is a new
evolutionary computation technique which
mimics the hierarchal order of a chicken
swarm and the behaviors of its individuals
chickens. in nature.
 The objective of feature selection:
1. Improving the prediction performance of the
predictors (classifiers).
2. Providing a faster and more cost-effective
predictors.
3. Providing a better understanding of the
underlying process that generated the data.

9. Motivation
 In this paper, a classification accuracy-based
fitness function is proposed by Chicken Swarm
Optimization to find optimal feature subset.
 The aim of the Chicken Swarm Optimization is
to find optimal regions of the complex search
space through the interaction of individuals in
the population. Compared with particle swarm
optimization (PSO) and Genetic Algorithms
(GA) over a set of UCI machine learning data
repository.

10. Agenda
1. Introduction.
3. System Overview and Methods
4. Experimental Results.
5. Conclusions.
2. Motivation.

11. Bio-inspired optimization
Introduction
 Inspired from the nature social behavior and dynamic
movements with communications of insects, birds, animals,
and fish.
 The term swarm (shoaling, swarming, or flocking) is applied to
fish, insects, birds, and describes a behavior of an aggregation of
animals of similar size and body orientation, generally cruising in
the same direction.

12. Bio-inspired optimization
Chicken Swarm Optimization (CSO)
 A chicken swarm is divided into several groups, each
group consists of one rooster and many hens and
chicks.
 Each type of chickens follow different laws of motions.
 A hierarchal order plays a significant role in the social
lives of chickens.
 The superior chickens in a flock will dominate the weak
ones.
 More dominant hens remain near to the head rooster

13. Bio-inspired optimization
Chicken Swarm Optimization (CSO)
 Mathematical model assumptions of CSO :
 The chicken swarm is divided into several groups. In each groups there is a
dominant rooster, followed by some hens and chicks.
 Hens follow their group-mate roosters to search for food.
 Chicks move around their mother to search for food.
 Fitness value outlines swarm hierarchy:
 individuals with best fitness value -> roosters ( group leader).
 individuals with the worst fitness values -> chicks
 others -> hens.
 The swarm hierarchy will remain unchanged only updated every several
(G) time steps..

14.  Rooster Movements:
 Hens Movements:
 Chicks Movements :
Bio-inspired optimization
Chicken Swarm Optimization (CSO)

15. Agenda
1. Introduction.
3. System Overview and Methods
4. Experimental Results.
5. Conclusions.
2. Motivation.

16. Experiment results
 We used 18 data sets for further
experiments. The data sets are drawn from
the UCI data repository. The data is divided
into 3 equal parts one for training, the
second part is for validation and the third
part is for testing.
 The CSO optimizer is evaluated against to
other algorithms : Genetic Algorithms (GA)
and Particle swarm optimization (PSO).
 The classification performance is measured
for each optimizer, compared to the
performance when using all features in the
each dataset without a feature selection
algorithm.
Data set No. of attributes No. of instances
Lymphography 18 148
WineEW 13 178
BreastEW 30 569
Breastcancer 9 699
CongressEW 16 435
Exactly 13 1000
Exactly2 13 1000
HeartEW 13 270
IonosphereEW 34 351
KrvskpEW 36 3196
M-of-n 13 1000
PenglungEW 325 73
SonarEW 60 208
SpectEW 22 267
Tic-tac-toe 9 958
Vote 16 300
WaveformEW 40 5000
Zoo 16 101

17.  Global parameter Values :
 Specific optimizer’s parameter :
Experiment results
Algorithms Parameter Value
CSO
r - The population size of roosters 0.15
h - The population size of hens 0.7
m - The population size of mother hens 0.5
PSO
w - value of the inertia factor 0.1
c - individual-best acceleration factor 0.1
GA
Crossover Fraction 0.8
Migration Fraction 0.2
Parameter Value
Fitness function constant 0.9999
The Number of iterations for optimization 70
Number of used search agents in the optimization 10
The number of times repeating the stochastic optimization 20

18. Experiment results
 Mean classification error on test data for each dataset compared to the data with all
features:
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
GA PSO CSO All Features

19. Experiment results
 Best fitness values for each dataset obtained by different optimizers :
0
0.05
0.1
0.15
0.2
0.25
0.3
GA PSO CSO All Features

20. Experiment results
 Worst fitness values for each dataset obtained by different optimizers :
0
0.1
0.2
0.3
0.4
0.5
GA PSO CSO All Features

21. Experiment results
 Mean and std fitness values obtained by different optimizers:
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
0.45
GA PSO CSO All Features

22. Experiment results
 Feature reduction on each dataset using different optimizers.
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
GA PSO CSO

23. Experiment results
0
0.05
0.1
0.15
0.2
0.25
Best fitness values Worst fitness values
Average Best and Worst values over
all datasets for each optimizer
GA PSO CSO All Features
0
0.05
0.1
0.15
0.2
0.25
GA PSO CSO All Features
Average – Mean fitness values over
all datasets for each optimizer

24. Experiment results
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
GA PSO CSO
Average - feature reduction over all
datasets for each optimizer
0
0.05
0.1
0.15
0.2
0.25
GA PSO CSO All Features
Average - Mean classification error
over all datasets for each optimizer

25. Agenda
1. Introduction.
3. System Overview and Methods
4. Experimental Results.
5. Conclusions.
2. Motivation.

26. Conclusions
 The objective of this paper was to propose a chicken swarm optimization
(CSO) algorithm for feature selection to choose minimal number of features
and to obtain comparable or even better classification accuracy from
utilizing all attributes.
 This study shows that CSO is an effective search algorithm for feature
selection problems.
 The used fitness function targets both the classification accuracy and
reduction size, which means we can obtain a set of minimum selected
features with maximum accuracy.
 The CSO proves an advance in both reduction size and classification
accuracy over PSO and GA.

27. Future Work
 We will work on the updating mechanisms in CSO to resolve feature
selection to further minimize the number of attributes, maximize the
classification accuracy.
 Also, we will examine the employ of chicken swarm optimization (CSO)
algorithm for feature selection on datasets with a large number of attributes.

28. Thank you
E-mail: ah.hafez@gmail.com