This explains the general algorithmic flow which goes into developing a Neural Network ensemble hybridized with evolutionary optimization schemes which are targeted in optimizing more than one cost function.

1. Colon Tumor Classification using
various Neural Network Models
coupled with Multi-Objective
Evolutionary Optimization Schemes
Anirudh Munnangi
Chandrasekar Venkatesh
Ahmed Sageer Cheriya Melat

2.  This project is an implementation of Neural Networks, with
hybrid evolutionary algorithm optimizing multiple
objectives for classification.
 Optimizing three objectives: Pareto non dominated sorting
genetic algorithm based optimization is done on norm of
the weights, mean norm square errors and complexity of
the network.
 The evolutionary algorithm is applied to Radial Basis
Function Networks (RBFNs) and on Multi-Layer
Perceptron networks (MLPs), these algorithms are applied
to classify real world two class colon tumor data.
Abstract

3.  Data is real world two class colon tumor data
 Data set consist of 62 data points
 Data points have feature space of 2000
 By principal component analysis method, the dimension of
data points is reduced to 47 feature space
Data processing

4. Important Concepts
• Radial Basis Function Networks
• Multi Layer Perceptron
• Pareto Optimality
• Genetic Algorithm

5. Radial Basis Function Networks

6. Radial Basis Function Networks
The following equation defines the process which is followed by the RBFN map.
𝜑𝑗 𝑥 = exp
− | 𝑥 − 𝑐𝑗 |
2𝜎𝑗
2
2
𝑦 =
𝑗=1
𝑃
𝑤𝑗 𝜑𝑗 𝑥
𝑌2
𝑌1
1
1
W20
W21
W22
Wm2
Wm1
W12
W10W11
∑
∑
X1
Xn
Input: 47
dimensional points
Output: 2 bits
representing class
Number of hidden layers: 1
Number of neurons in each layer: 20 or more

7. Multi Layer Perceptron

8. Multi Layer Perceptron
Xn
W2n
∑
∑
∑
𝑌2
𝑌1
W21
W1n
Wmn
Wm1
W11
∑
∑
X1
Input: 47
dimensional points
Output: 2 bits
representing class
Number of hidden layers: 1
Number of neurons in each layer: 20 or more

9. Pareto Optimality

10. Pareto Optimality
𝑀𝑖𝑛𝑖𝑚𝑖𝑧𝑒𝑀𝑎𝑥𝑖𝑚𝑖𝑧𝑒 𝑓𝑖 𝑥 𝑖 = 1,2, … . . , 𝑀
𝑆𝑢𝑏𝑗𝑒𝑐𝑡 𝑡𝑜:
𝑔𝑗 (𝑥) = 0 𝑗 = 1,2, … . , 𝑁
ℎ 𝑘 (𝑥) ≤ 0 𝑘 = 1,2, … . , 𝐾
The solution is pareto optimal if
∀𝑖 ∈ 1,2, … . , 𝑀 ∶ 𝑓𝑖 𝑥1 ≤ 𝑓𝑖 𝑥2
∃𝑗 ∈ 1,2, … . , 𝑀 ∶ 𝑓𝑗 𝑥1 < 𝑓𝑗 𝑥2
Objective functions considered in this project:
𝒇 𝟏 =
𝟏
𝑵 𝒋=𝟏
𝑵
||𝒚𝒋 − 𝒚𝒋|| 𝟐
- Mean norm square error
𝒇 𝟐 = 𝒉=𝟏
𝑵 𝒉
𝝆 𝒉 - Complexity map of the neural network
𝒇 𝟑 = | 𝒘 | - norm of all the weights in the network

11. Genetic Algorithm

12. Genetic Algorithm
• Population size = 20
• Each member represents Mean norm square error, complexity
of the network and norm of weights
• For each iteration
• Perform Non dominated sort (Pareto optimization)
• Choose fittest, crossover and form child population
• Combine parent and child population, perform non-
dominated sort
• Form new population

13. Program Flow

14. 2.6 2.8 3 3.2 3.4 3.6 3.8 4 4.2 4.4 4.6
0
10
20
30
40
50
60
f3=||w||
f1=MeannormSquareError
Convergence of Pareto Front
NSGA using RBFN
Optimized RBFN with the best
trade-off between objectives

15. 0 5 10 15 20 25 30 35 40 45
2
2.5
3
3.5
4
4.5
5
5.5
6
6.5
7
Convergence of Pareto Front
Objective function f2
Objectivefunctionf1
RBFN with NSGA
f1=MSE f2=Active hidden kernels
f1=MSE f2=||w||
f1=Active hidden kernels f2=||w||
Performance in different runs

16. 0 10 20 30 40 50 60 70
2
3
4
5
6
7
8
9
10
Convergence of Pareto Front
f3=||w||
MeanNormSquareError
RBFN NSGA
MLP NSGA
Comparative analysis of the
performance of RBFN Vs MLP

17. INFERENCE
• For the data set we used, optimized RBFN model seems to perform
better than MLP.
• Pareto optimization for multiple objectives works well with will all
objectives being achieved.
FURTHER WORK
• Can try more objectives, different parameters to optimize
• Try optimizing other network models apart from RBFN and MLP, maybe
try optimizing for a combination of networks.

18. REFERENCES
1. Sultan Noman Qasem, Siti Mariyam Shamsuddin, Azlan Mohd Zain “Multi-objective hybrid evolutionary algorithms for radial basis function
neural network design” Knowledge-Based Systems 27 (2012) 475–497 25 November 2011
2. Sultan Noman Qasema, Siti Mariyam Shamsuddina, “Memetic Elitist Pareto Differential Evolution algorithm based Radial basis function
networks for classification problems” Neurocomputing; Applied Soft Computing 11 (2011) 5565–5581- 6 May 2011
3. Aimin Zhou, Bo-Yang Qu, Hui Li, Shi Zheng Zhao, Ponnuthurai Nagaratnam Suganthan, Qingfu Zhang, “Multiobjective evolutionary algorithms:
A survey of the state of the art” Swarm and Evolutionary Computation 1 (2011) 32–49; 16 March 2011
4. Illya Kokshenev, Antonio Padua Braga, “An efficient multi-objective learning algorithm for RBF neural network. Neurocomputing
73(2010)2799–2808, 22August2010
5. Sultan Noman Qasem and Siti Mariyam Shamsuddin, Radial Basis Function Network based on time variant multiobjective particle swarm
optimization for medical disease analysis.
6. Jonathan E. Fieldsend and Sameer Singh, “Pareto Evolutionary Neural Networks” IEEE TRANSACTIONS ON NEURAL NETWORKS, VOL.
16, NO. 2, MARCH 2005
7. Kalyanmoy Deb, Amrit Pratap, Sameer Agarwal, and T. Meyarivan, “A Fast and Elitist Multiobjective Genetic Algorithm- NSGA-II” IEEE
TRANSACTIONS ON EVOLUTIONARY COMPUTATION, VOL. 6, NO. 2, APRIL 2002
8. David Lahoz and Pedro Mateo, “Neural Network Ensembles for Classification Problems using Multiobjective Genetic Algorithms”
9. U. Alon, et al. "Broad Patterns of Gene Expression Revealed by Clustering Analysis of Tumor and Normal Colon Tissues Probed by
Oligonucleotide Arrays", PNAS, 96:6745-6750, 1999