Hybrid PSO-SA algorithm for training a Neural Network for ClassificationIJCSEA Journal
In this work, we propose a Hybrid particle swarm optimization-Simulated annealing algorithm and present a comparison with i) Simulated annealing algorithm and ii) Back propagation algorithm for training neural networks. These neural networks were then tested on a classification task. In particle swarm optimization behaviour of a particle is influenced by the experiential knowledge of the particle as well as socially exchanged information. Particle swarm optimization follows a parallel search strategy. In simulated annealing uphill moves are made in the search space in a stochastic fashion in addition to the downhill moves. Simulated annealing therefore has better scope of escaping local minima and reach a global minimum in the search space. Thus simulated annealing gives a selective randomness to the search. Back propagation algorithm uses gradient descent approach search for minimizing the error. Our goal of global minima in the task being done here is to come to lowest energy state, where energy state is being modelled as the sum of the squares of the error between the target and observed output values for all the training samples. We compared the performance of the neural networks of identical architectures trained by the i) Hybrid particle swarm optimization-simulated annealing, ii) Simulated annealing and iii) Back propagation algorithms respectively on a classification task and noted the results obtained. Neural network trained by Hybrid particle swarm optimization-simulated annealing has given better results compared to the neural networks trained by the Simulated annealing and Back propagation algorithms in the tests conducted by us.
On The Application of Hyperbolic Activation Function in Computing the Acceler...iosrjce
Hyperbolic activation function is examined for its ability to accelerate the performance of doing data
mining by using a technique named as Reverse Analysis method. In this paper, we describe how Hopfield
network perform better with hyperbolic activation function and able to induce logical rules from large database
by using reverse analysis method: given the values of the connections of a network, we hope to determine what
logical rules are entrenched in the database. We limit our analysis to Horn clauses. The analysis for this study
was simulated using Microsoft Visual C++ software, 2010 Express.
Adaptive modified backpropagation algorithm based on differential errorsIJCSEA Journal
A new efficient modified back propagation algorithm with adaptive learning rate is proposed to increase the convergence speed and to minimize the error. The method eliminates initial fixing of learning rate through trial and error and replaces by adaptive learning rate. In each iteration, adaptive learning rate for output and hidden layer are determined by calculating differential linear and nonlinear errors of output layer and hidden layer separately. In this method, each layer has different learning rate in each iteration. The performance of the proposed algorithm is verified by the simulation results.
Simulation of Single and Multilayer of Artificial Neural Network using Verilogijsrd.com
Artificial neural network play an important role in VLSI circuit to find and diagnosis multiple fault in digital circuit. In this paper, the example of single layer and multi-layer neural network had been discussed secondly implement those structure by using verilog code and same idea must be implement in mat lab for getting number of iteration and verilog code gives us time taken to adjust the weight when error become almost equal to zero. The purposed aim at reducing resource requirement, without much compromises on the speed that neural network can be realized on single chip at lower cost.
MATLAB Code + Description : Very Simple Automatic English Optical Character R...Ahmed Gad
This file contains a simple description about what I have created about how to recognize characters using feed forward back propagation neural network as a pattern recognition project when being undergraduate student at 2013.
The MATLAB code of the system is also available in the document.
Find me on:
AFCIT
http://www.afcit.xyz
YouTube
https://www.youtube.com/channel/UCuewOYbBXH5gwhfOrQOZOdw
Google Plus
https://plus.google.com/u/0/+AhmedGadIT
SlideShare
https://www.slideshare.net/AhmedGadFCIT
LinkedIn
https://www.linkedin.com/in/ahmedfgad/
ResearchGate
https://www.researchgate.net/profile/Ahmed_Gad13
Academia
https://www.academia.edu/
Google Scholar
https://scholar.google.com.eg/citations?user=r07tjocAAAAJ&hl=en
Mendelay
https://www.mendeley.com/profiles/ahmed-gad12/
ORCID
https://orcid.org/0000-0003-1978-8574
StackOverFlow
http://stackoverflow.com/users/5426539/ahmed-gad
Twitter
https://twitter.com/ahmedfgad
Facebook
https://www.facebook.com/ahmed.f.gadd
Pinterest
https://www.pinterest.com/ahmedfgad/
Hybrid PSO-SA algorithm for training a Neural Network for ClassificationIJCSEA Journal
In this work, we propose a Hybrid particle swarm optimization-Simulated annealing algorithm and present a comparison with i) Simulated annealing algorithm and ii) Back propagation algorithm for training neural networks. These neural networks were then tested on a classification task. In particle swarm optimization behaviour of a particle is influenced by the experiential knowledge of the particle as well as socially exchanged information. Particle swarm optimization follows a parallel search strategy. In simulated annealing uphill moves are made in the search space in a stochastic fashion in addition to the downhill moves. Simulated annealing therefore has better scope of escaping local minima and reach a global minimum in the search space. Thus simulated annealing gives a selective randomness to the search. Back propagation algorithm uses gradient descent approach search for minimizing the error. Our goal of global minima in the task being done here is to come to lowest energy state, where energy state is being modelled as the sum of the squares of the error between the target and observed output values for all the training samples. We compared the performance of the neural networks of identical architectures trained by the i) Hybrid particle swarm optimization-simulated annealing, ii) Simulated annealing and iii) Back propagation algorithms respectively on a classification task and noted the results obtained. Neural network trained by Hybrid particle swarm optimization-simulated annealing has given better results compared to the neural networks trained by the Simulated annealing and Back propagation algorithms in the tests conducted by us.
On The Application of Hyperbolic Activation Function in Computing the Acceler...iosrjce
Hyperbolic activation function is examined for its ability to accelerate the performance of doing data
mining by using a technique named as Reverse Analysis method. In this paper, we describe how Hopfield
network perform better with hyperbolic activation function and able to induce logical rules from large database
by using reverse analysis method: given the values of the connections of a network, we hope to determine what
logical rules are entrenched in the database. We limit our analysis to Horn clauses. The analysis for this study
was simulated using Microsoft Visual C++ software, 2010 Express.
Adaptive modified backpropagation algorithm based on differential errorsIJCSEA Journal
A new efficient modified back propagation algorithm with adaptive learning rate is proposed to increase the convergence speed and to minimize the error. The method eliminates initial fixing of learning rate through trial and error and replaces by adaptive learning rate. In each iteration, adaptive learning rate for output and hidden layer are determined by calculating differential linear and nonlinear errors of output layer and hidden layer separately. In this method, each layer has different learning rate in each iteration. The performance of the proposed algorithm is verified by the simulation results.
Simulation of Single and Multilayer of Artificial Neural Network using Verilogijsrd.com
Artificial neural network play an important role in VLSI circuit to find and diagnosis multiple fault in digital circuit. In this paper, the example of single layer and multi-layer neural network had been discussed secondly implement those structure by using verilog code and same idea must be implement in mat lab for getting number of iteration and verilog code gives us time taken to adjust the weight when error become almost equal to zero. The purposed aim at reducing resource requirement, without much compromises on the speed that neural network can be realized on single chip at lower cost.
MATLAB Code + Description : Very Simple Automatic English Optical Character R...Ahmed Gad
This file contains a simple description about what I have created about how to recognize characters using feed forward back propagation neural network as a pattern recognition project when being undergraduate student at 2013.
The MATLAB code of the system is also available in the document.
Find me on:
AFCIT
http://www.afcit.xyz
YouTube
https://www.youtube.com/channel/UCuewOYbBXH5gwhfOrQOZOdw
Google Plus
https://plus.google.com/u/0/+AhmedGadIT
SlideShare
https://www.slideshare.net/AhmedGadFCIT
LinkedIn
https://www.linkedin.com/in/ahmedfgad/
ResearchGate
https://www.researchgate.net/profile/Ahmed_Gad13
Academia
https://www.academia.edu/
Google Scholar
https://scholar.google.com.eg/citations?user=r07tjocAAAAJ&hl=en
Mendelay
https://www.mendeley.com/profiles/ahmed-gad12/
ORCID
https://orcid.org/0000-0003-1978-8574
StackOverFlow
http://stackoverflow.com/users/5426539/ahmed-gad
Twitter
https://twitter.com/ahmedfgad
Facebook
https://www.facebook.com/ahmed.f.gadd
Pinterest
https://www.pinterest.com/ahmedfgad/
Character Recognition using Artificial Neural NetworksJaison Sabu
Mini Project, Computer Science Department, College of Engineering Chengannur 2003-2007, Affiliated to Cochin University of Science and Technology (CUSAT), Kerala, India
Link to code and webpage:
http://shashankg7.github.io/word2graph2vec/
Link to slides:
http://www.slideshare.net/nprateek/predictive-text-embedding-using-line
Link to report:
https://www.overleaf.com/read/sqhkzfvjhfkp
Classification of Electroencephalograph (EEG) Signals Using Quantum Neural Ne...CSCJournals
In this paper, quantum neural network (QNN), which is a class of feedforward neural networks (FFNN’s), is used to recognize (EEG) signals. For this purpose ,independent component analysis (ICA), wavelet transform (WT) and Fourier transform (FT) are used as a feature extraction after normalization of these signals. The architecture of (QNN’s) have inherently built in fuzzy. The hidden units of these networks develop quantized representations of the sample information provided by the training data set in various graded levels of certainty. Experimental results presented here show that (QNN’s) are capable of recognizing structures in data, a property that conventional (FFNN’s) with sigmoidal hidden units lack . Finally, (QNN) gave us kind of fast and realistic results compared with the (FFNN). Simulation results show that a total classification of 81.33% for (ICA), 76.67% for (WT) and 67.33% for (FT).
IJRET : International Journal of Research in Engineering and Technology is an international peer reviewed, online journal published by eSAT Publishing House for the enhancement of research in various disciplines of Engineering and Technology. The aim and scope of the journal is to provide an academic medium and an important reference for the advancement and dissemination of research results that support high-level learning, teaching and research in the fields of Engineering and Technology. We bring together Scientists, Academician, Field Engineers, Scholars and Students of related fields of Engineering and Technology
Genetic Algorithm Processor for Image Noise Filtering Using Evolvable HardwareCSCJournals
General-purpose image filters lack the flexibility and adaptability of un-modeled noise types. On the contrary, evolutionary algorithm based filter architectures seem to be very promising due to their capability of providing solutions to hard design problems. Through this novel approach, it is made possible to have an image filter that can employ a completely different design style that is performed by an evolutionary algorithm. In this context, an evolutionary algorithm based filter is designed in this paper with the kernel or the whole circuit for automatically evolved. The Evolvable Hard Ware architecture proposed in this paper can evolve filters without a priori information. The proposed filter architecture considers spatial domain approach and uses the overlapping window to filter the signal. The approach that is chosen in this work is based on functional level evolution whose architecture includes nonlinear functions and uses genetic algorithm for finding the best filter configuration.
Optimization of Number of Neurons in the Hidden Layer in Feed Forward Neural ...IJERA Editor
The architectures of Artificial Neural Networks (ANN) are based on the problem domain and it is applied during
the „training phase‟ of sample data and used to infer results for the remaining data in the testing phase.
Normally, the architecture consist of three layers as input, hidden, output layers with the number of nodes in the
input layer as number of known values on hand and the number of nodes as result to be computed out of the
values of input nodes and hidden nodes as the output layer. The number of nodes in the hidden layer is
heuristically decided so that the optimum value is obtained with reasonable number of iterations with other
parameters with its default values. This study mainly focuses on Cascade-Correlation Neural Networks (CCNN)
using Back-Propagation (BP) algorithm which finds the number of neurons during the training phase itself by
appending one from the previous iteration satisfying the error condition gives a promising result on the optimum
number of neurons in the hidden layer
Training artificial neural network using particle swarm optimization algorithmA. Roy
Abstract -
In this paper, the adaptation of network weights using Particle Swarm Optimization (PSO) was proposed as a mechanism to improve the performance of Artificial Neural Network (ANN) in classification of IRIS dataset. Classification is a machine learning technique used to predict group membership for data instances. To simplify the problem of classification neural networks are being introduced. This paper focuses on IRIS plant classification using Neural Network. The problem concerns the identification of IRIS plant species on the basis of plant attribute measurements. Classification of IRIS data set would be discovering patterns from examining petal and sepal size of the IRIS plant and how the prediction was made from analyzing the pattern to form the class of IRIS plant. By using this pattern and classification, in future upcoming years the unknown data can be predicted more precisely. Artificial neural networks have been successfully applied to problems in pattern classification, function approximations, optimization, and associative memories. In this work, Multilayer feed- forward networks are trained using back propagation learning algorithm.
Targeted Visual Content Recognition Using Multi-Layer Perceptron Neural Networkijceronline
Visual Content Recognition has become an attractive research oriented field of computer vision and machine learning for the last few decades. The focus of this work is monument recognition. Imagesof significant locations captured and maintainedas data bases can be used by the travelers before visiting the places. They can use images of a famous building to know the description of the building. In all these applications, the visual content recognition plays a key role. Humans can learn the contents of the images and quickly identify them by seeing again. In this paper we present a constructive training algorithm for Multi-Layer Perceptron Neural Network (MLPNN) applied to a set of targeted object recognition applications. The target set consists of famous monuments in India for travel guide applications. The training data set (TDS) consists 3000 images. The Gist features are extracted for the images. These are given to the neural network during training phase.The mean square error (MSE) on the training data is computed and used as metric to adjust the weights of the neural network,using back propagation algorithm. In the constructive learning, if the MSE is less than a predefined value, the number of hidden neurons is increased. Input patterns are trained incrementally until all patterns of TDS are presented and learned. The parameters or weights obtained during the training phase are used in the testing phase, in which new untrained images are given to the neural network for recognition. If the test image is recognized, the details of the image will also be displayed. The performance accuracy of this method is found to be 95%
LOG MESSAGE ANOMALY DETECTION WITH OVERSAMPLINGijaia
Imbalanced data is a significant challenge in classification with machine learning algorithms. This is particularly important with log message data as negative logs are sparse so this data is typically imbalanced. In this paper, a model to generate text log messages is proposed which employs a SeqGAN network. An Autoencoder is used for feature extraction and anomaly detection is done using a GRU network. The proposed model is evaluated with three imbalanced log data sets, namely BGL, OpenStack, and Thunderbird. Results are presented which show that appropriate oversampling and data balancing
improves anomaly detection accuracy.
A Comparison Study between Inferred State-Space and Neural Network Based Syst...Ahmed Momtaz Hosny, PhD
Abstract:
In this paper, system identifications of an unmanned aerial vehicle (UAV) based on inferred state space and multiple neural networks were presented. In this work an optimization approach was used to conclude an inferred state space and the multiple neural networks system identifications based on the genetic algorithms separately.
The presented work is focusing on an inferred state space based system identification which is a new approach seldom used, but it is also easier and more stable compared with the multi-network based system identification during the modeling of dynamic behavior of nonlinear systems.
A New Model for Credit Approval Problems: A Neuro-Genetic System with Quantum...Anderson Pinho
This paper presents a new model for neuro-evolutionary systems. It is a new quantum-inspired evolutionary algorithm with binary-real representation (QIEA-BR) for evolution of a neural network. The proposed model is an extension of the QIEA-R developed for numerical optimization. The Quantum-Inspired Neuro-Evolutionary Computation model (QINEA-BR) is able to completely configure a feed-forward neural network in terms of selecting the relevant input variables, number of neurons in the hidden layer and all existent synaptic weights. QINEA-BR is evaluated in a benchmark problem of financial credit evaluation. The results obtained demonstrate the effectiveness of this new model in comparison with other machine learning and statistical models, providing good accuracy in separating good from bad customers.
Character Recognition using Artificial Neural NetworksJaison Sabu
Mini Project, Computer Science Department, College of Engineering Chengannur 2003-2007, Affiliated to Cochin University of Science and Technology (CUSAT), Kerala, India
Link to code and webpage:
http://shashankg7.github.io/word2graph2vec/
Link to slides:
http://www.slideshare.net/nprateek/predictive-text-embedding-using-line
Link to report:
https://www.overleaf.com/read/sqhkzfvjhfkp
Classification of Electroencephalograph (EEG) Signals Using Quantum Neural Ne...CSCJournals
In this paper, quantum neural network (QNN), which is a class of feedforward neural networks (FFNN’s), is used to recognize (EEG) signals. For this purpose ,independent component analysis (ICA), wavelet transform (WT) and Fourier transform (FT) are used as a feature extraction after normalization of these signals. The architecture of (QNN’s) have inherently built in fuzzy. The hidden units of these networks develop quantized representations of the sample information provided by the training data set in various graded levels of certainty. Experimental results presented here show that (QNN’s) are capable of recognizing structures in data, a property that conventional (FFNN’s) with sigmoidal hidden units lack . Finally, (QNN) gave us kind of fast and realistic results compared with the (FFNN). Simulation results show that a total classification of 81.33% for (ICA), 76.67% for (WT) and 67.33% for (FT).
IJRET : International Journal of Research in Engineering and Technology is an international peer reviewed, online journal published by eSAT Publishing House for the enhancement of research in various disciplines of Engineering and Technology. The aim and scope of the journal is to provide an academic medium and an important reference for the advancement and dissemination of research results that support high-level learning, teaching and research in the fields of Engineering and Technology. We bring together Scientists, Academician, Field Engineers, Scholars and Students of related fields of Engineering and Technology
Genetic Algorithm Processor for Image Noise Filtering Using Evolvable HardwareCSCJournals
General-purpose image filters lack the flexibility and adaptability of un-modeled noise types. On the contrary, evolutionary algorithm based filter architectures seem to be very promising due to their capability of providing solutions to hard design problems. Through this novel approach, it is made possible to have an image filter that can employ a completely different design style that is performed by an evolutionary algorithm. In this context, an evolutionary algorithm based filter is designed in this paper with the kernel or the whole circuit for automatically evolved. The Evolvable Hard Ware architecture proposed in this paper can evolve filters without a priori information. The proposed filter architecture considers spatial domain approach and uses the overlapping window to filter the signal. The approach that is chosen in this work is based on functional level evolution whose architecture includes nonlinear functions and uses genetic algorithm for finding the best filter configuration.
Optimization of Number of Neurons in the Hidden Layer in Feed Forward Neural ...IJERA Editor
The architectures of Artificial Neural Networks (ANN) are based on the problem domain and it is applied during
the „training phase‟ of sample data and used to infer results for the remaining data in the testing phase.
Normally, the architecture consist of three layers as input, hidden, output layers with the number of nodes in the
input layer as number of known values on hand and the number of nodes as result to be computed out of the
values of input nodes and hidden nodes as the output layer. The number of nodes in the hidden layer is
heuristically decided so that the optimum value is obtained with reasonable number of iterations with other
parameters with its default values. This study mainly focuses on Cascade-Correlation Neural Networks (CCNN)
using Back-Propagation (BP) algorithm which finds the number of neurons during the training phase itself by
appending one from the previous iteration satisfying the error condition gives a promising result on the optimum
number of neurons in the hidden layer
Training artificial neural network using particle swarm optimization algorithmA. Roy
Abstract -
In this paper, the adaptation of network weights using Particle Swarm Optimization (PSO) was proposed as a mechanism to improve the performance of Artificial Neural Network (ANN) in classification of IRIS dataset. Classification is a machine learning technique used to predict group membership for data instances. To simplify the problem of classification neural networks are being introduced. This paper focuses on IRIS plant classification using Neural Network. The problem concerns the identification of IRIS plant species on the basis of plant attribute measurements. Classification of IRIS data set would be discovering patterns from examining petal and sepal size of the IRIS plant and how the prediction was made from analyzing the pattern to form the class of IRIS plant. By using this pattern and classification, in future upcoming years the unknown data can be predicted more precisely. Artificial neural networks have been successfully applied to problems in pattern classification, function approximations, optimization, and associative memories. In this work, Multilayer feed- forward networks are trained using back propagation learning algorithm.
Targeted Visual Content Recognition Using Multi-Layer Perceptron Neural Networkijceronline
Visual Content Recognition has become an attractive research oriented field of computer vision and machine learning for the last few decades. The focus of this work is monument recognition. Imagesof significant locations captured and maintainedas data bases can be used by the travelers before visiting the places. They can use images of a famous building to know the description of the building. In all these applications, the visual content recognition plays a key role. Humans can learn the contents of the images and quickly identify them by seeing again. In this paper we present a constructive training algorithm for Multi-Layer Perceptron Neural Network (MLPNN) applied to a set of targeted object recognition applications. The target set consists of famous monuments in India for travel guide applications. The training data set (TDS) consists 3000 images. The Gist features are extracted for the images. These are given to the neural network during training phase.The mean square error (MSE) on the training data is computed and used as metric to adjust the weights of the neural network,using back propagation algorithm. In the constructive learning, if the MSE is less than a predefined value, the number of hidden neurons is increased. Input patterns are trained incrementally until all patterns of TDS are presented and learned. The parameters or weights obtained during the training phase are used in the testing phase, in which new untrained images are given to the neural network for recognition. If the test image is recognized, the details of the image will also be displayed. The performance accuracy of this method is found to be 95%
LOG MESSAGE ANOMALY DETECTION WITH OVERSAMPLINGijaia
Imbalanced data is a significant challenge in classification with machine learning algorithms. This is particularly important with log message data as negative logs are sparse so this data is typically imbalanced. In this paper, a model to generate text log messages is proposed which employs a SeqGAN network. An Autoencoder is used for feature extraction and anomaly detection is done using a GRU network. The proposed model is evaluated with three imbalanced log data sets, namely BGL, OpenStack, and Thunderbird. Results are presented which show that appropriate oversampling and data balancing
improves anomaly detection accuracy.
A Comparison Study between Inferred State-Space and Neural Network Based Syst...Ahmed Momtaz Hosny, PhD
Abstract:
In this paper, system identifications of an unmanned aerial vehicle (UAV) based on inferred state space and multiple neural networks were presented. In this work an optimization approach was used to conclude an inferred state space and the multiple neural networks system identifications based on the genetic algorithms separately.
The presented work is focusing on an inferred state space based system identification which is a new approach seldom used, but it is also easier and more stable compared with the multi-network based system identification during the modeling of dynamic behavior of nonlinear systems.
A New Model for Credit Approval Problems: A Neuro-Genetic System with Quantum...Anderson Pinho
This paper presents a new model for neuro-evolutionary systems. It is a new quantum-inspired evolutionary algorithm with binary-real representation (QIEA-BR) for evolution of a neural network. The proposed model is an extension of the QIEA-R developed for numerical optimization. The Quantum-Inspired Neuro-Evolutionary Computation model (QINEA-BR) is able to completely configure a feed-forward neural network in terms of selecting the relevant input variables, number of neurons in the hidden layer and all existent synaptic weights. QINEA-BR is evaluated in a benchmark problem of financial credit evaluation. The results obtained demonstrate the effectiveness of this new model in comparison with other machine learning and statistical models, providing good accuracy in separating good from bad customers.
Large Scale Data Mining using Genetics-Based Machine LearningXavier Llorà
We are living in the peta-byte era.We have larger and larger data to analyze, process and transform into useful answers for the domain experts. Robust data mining tools, able to cope with petascale volumes and/or high dimensionality producing human-understandable solutions are key on several domain areas. Genetics-based machine learning (GBML) techniques are perfect candidates for this task, among others, due to the recent advances in representations, learning paradigms, and theoretical modeling. If evolutionary learning techniques aspire to be a relevant player in this context, they need to have the capacity of processing these vast amounts of data and they need to process this data within reasonable time. Moreover, massive computation cycles are getting cheaper and cheaper every day, allowing researchers to have access to unprecedented parallelization degrees. Several topics are interlaced in these two requirements: (1) having the proper learning paradigms and knowledge representations, (2) understanding them and knowing when are they suitable for the problem at hand, (3) using efficiency enhancement techniques, and (4) transforming and visualizing the produced solutions to give back as much insight as possible to the domain experts are few of them.
This tutorial will try to answer this question, following a roadmap that starts with the questions of what large means, and why large is a challenge for GBML methods. Afterwards, we will discuss different facets in which we can overcome this challenge: Efficiency enhancement techniques, representations able to cope with large dimensionality spaces, scalability of learning paradigms. We will also review a topic interlaced with all of them: how can we model the scalability of the components of our GBML systems to better engineer them to get the best performance out of them for large datasets. The roadmap continues with examples of real applications of GBML systems and finishes with an analysis of further directions.
its a presentation on stock market analysis using Genetic algorithm with Neural networks ,based on a scientific paper
,made in Cairo university under Supervision of prof.Dr. Magda
Neural network are systems modeled on the human brain which consist of number of neurons and connections between them. The neural networks weights are that what makes memory possible, i.e. acquiring certain knowledge, and they are modified through iterative learning process.In the process of learning, weight modifications are done by a learning algorithm and back-propagation (gradient descent) is the most famous one. However, the final result of back-propagation training is significantly dependent on initial weight values. Genetic algorithm is a stochastic search tool based on evolutive principles, which can be used as a learning algorithm without limitations. The scope of genetically trained networks is examined through the problem of credit risk assessment in banking, the research area known as credit scoring. Compared to back-propagation algorithms, experimental results on well known benchmark problems in this area (Australian and German credit data), show certain advantages of the genetic learning networks.
Neural network based numerical digits recognization using nnt in matlabijcses
Artificial neural networks are models inspired by human nervous system that is capable of learning. One of
the important applications of artificial neural network is character Recognition. Character Recognition
finds its application in number of areas, such as banking, security products, hospitals, in robotics also.
This paper is based on a system that recognizes a english numeral, given by the user, which is already
trained on the features of the numbers to be recognized using NNT (Neural network toolbox) .The system
has a neural network as its core, which is first trained on a database. The training of the neural network
extracts the features of the English numbers and stores in the database. The next phase of the system is to
recognize the number given by the user. The features of the number given by the user are extracted and
compared with the feature database and the recognized number is displayed.
Web spam classification using supervised artificial neural network algorithmsaciijournal
Due to the rapid growth in technology employed by the spammers, there is a need of classifiers that are more efficient, generic and highly adaptive. Neural Network based technologies have high ability of adaption as well as generalization. As per our knowledge, very little work has been done in this field using neural network. We present this paper to fill this gap. This paper evaluates performance of three supervised learning algorithms of artificial neural network by creating classifiers for the complex problem of latest web spam pattern classification. These algorithms are Conjugate Gradient algorithm, Resilient Backpropagation learning, and Levenberg-Marquardt algorithm.
Web Spam Classification Using Supervised Artificial Neural Network Algorithmsaciijournal
Due to the rapid growth in technology employed by the spammers, there is a need of classifiers that are
more efficient, generic and highly adaptive. Neural Network based technologies have high ability of
adaption as well as generalization. As per our knowledge, very little work has been done in this field using
neural network. We present this paper to fill this gap. This paper evaluates performance of three supervised
learning algorithms of artificial neural network by creating classifiers for the complex problem of latest
web spam pattern classification. These algorithms are Conjugate Gradient algorithm, Resilient Backpropagation learning, and Levenberg-Marquardt algorithm.
Classification Of Iris Plant Using Feedforward Neural Networkirjes
The classification and recognition of type on the basis of individual features and behaviors constitute
a preliminary measure and is an important target in the behavioral sciences. Current statistical methods do not
always yield satisfactory answers. A Feed Forward Artificial Neural Network is the computer model inspired by
the structure of the Human Brain. It views as in the set of artificial nerve cells that are interconnected with the
other neurons. The primary aim of this paper is to demonstrate the process of developing the Artificial Neural
network based classifier which classifies the Iris database. The problem concerns the identification of Iris plant
species on the basis of plant attribute measurements. This paper is related to the use of feed forward neural
networks towards the identification of iris plants on the basis of the following measurements: sepal length, sepal
width, petal length, and petal width. Using this data set a Neural Network (NN) is used for the classification of
iris data set. The EBPA is used for training of this ANN. The results of simulations illustrate the effectiveness of
the neural system in iris class identification.
Towards neuralprocessingofgeneralpurposeapproximateprogramsParidha Saxena
Did validation of one of the machine learning algorithms of neural networks,and compared the results for its implementation on hardware (FPGA) using xilinx, with that of a sequential code execution(using FANN).
Modeling of neural image compression using gradient decent technologytheijes
The International Journal of Engineering & Science is aimed at providing a platform for researchers, engineers, scientists, or educators to publish their original research results, to exchange new ideas, to disseminate information in innovative designs, engineering experiences and technological skills. It is also the Journal's objective to promote engineering and technology education. All papers submitted to the Journal will be blind peer-reviewed. Only original articles will be published.
The papers for publication in The International Journal of Engineering& Science are selected through rigorous peer reviews to ensure originality, timeliness, relevance, and readability.
Theoretical work submitted to the Journal should be original in its motivation or modeling structure. Empirical analysis should be based on a theoretical framework and should be capable of replication. It is expected that all materials required for replication (including computer programs and data sets) should be available upon request to the authors.
The International Journal of Engineering & Science would take much care in making your article published without much delay with your kind cooperation
Open CV Implementation of Object Recognition Using Artificial Neural Networksijceronline
International Journal of Computational Engineering Research (IJCER) is dedicated to protecting personal information and will make every reasonable effort to handle collected information appropriately. All information collected, as well as related requests, will be handled as carefully and efficiently as possible in accordance with IJCER standards for integrity and objectivity.
This work is proposed the feed forward neural network with symmetric table addition method to design the
neuron synapses algorithm of the sine function approximations, and according to the Taylor series
expansion. Matlab code and LabVIEW are used to build and create the neural network, which has been
designed and trained database set to improve its performance, and gets the best a global convergence with
small value of MSE errors and 97.22% accuracy.
Predicting rainfall using ensemble of ensemblesVarad Meru
The Paper was done in a group of three for the class project of CS 273: Introduction to Machine Learning at UC Irvine. The group members were Prolok Sundaresan, Varad Meru, and Prateek Jain.
Regression is an approach for modeling the relationship between data X and the dependent variable y. In this report, we present our experiments with multiple approaches, ranging from Ensemble of Learning to Deep Learning Networks on the weather modeling data to predict the rainfall. The competition was held on the online data science competition portal ‘Kaggle’. The results for weighted ensemble of learners gave us a top-10 ranking, with the testing root-mean-squared error being 0.5878.
Artificial Neural Networks (ANNS) For Prediction of California Bearing Ratio ...IJMER
The behaviour of soil at the location of the project and interactions of the earth materials during and after construction has a major influence on the success, economy and safety of the work. Another complexity associated with some geotechnical engineering materials, such as sand and gravel, is the difficulty in obtaining undisturbed samples and time consuming involving skilled
technician. Knowledge of California Bearing Ratio (C.B.R) is essential in finding the road thickness. To cope up with the difficulties involved, an attempt has been made to model C.B.R in terms of Fine Fraction, Liquid Limit, Plasticity Index, Maximum Dry density, and Optimum Moisture content. A multi-layer perceptron network with feed forward back propagation is used to model varying the
number of hidden layers. For this purposes 50 soils test data was collected from the laboratory test
results. Among the test data 30 soils data is used for training and remaining 20 soils for testing using
60-40 distribution. The architectures developed are 5-4-1, 5-5-1, and 5-6-1. Model with 5-6-1 architecture is found to be quite satisfactory in predicting C.B.R of soils. A graph is plotted between
the predicted values and observed values of outputs for training and testing process, from the graph it
is found that all the points are close to equality line, indicating predicted values are close to observed
values
Employing Neocognitron Neural Network Base Ensemble Classifiers To Enhance Ef...cscpconf
This paper presents an ensemble of neo-cognitron neural network base classifiers to enhance
the accuracy of the system, along the experimental results. The method offers lesser
computational preprocessing in comparison to other ensemble techniques as it ex-preempts
feature extraction process before feeding the data into base classifiers. This is achieved by the
basic nature of neo-cognitron, it is a multilayer feed-forward neural network. Ensemble of such
base classifiers gives class labels for each pattern that in turn is combined to give the final class
label for that pattern. The purpose of this paper is not only to exemplify learning behaviour of
neo-cognitron as base classifiers, but also to purport better fashion to combine neural network
based ensemble classifiers.
AN IMPROVED METHOD FOR IDENTIFYING WELL-TEST INTERPRETATION MODEL BASED ON AG...IAEME Publication
This paper presents an approach based on applying an aggregated predictor formed by multiple versions of a multilayer neural network with a back-propagation optimization algorithm for helping the engineer to get a list of the most appropriate well-test interpretation models for a given set of pressure/ production data. The proposed method consists of three stages: (1) data decorrelation through principal component analysis to reduce the covariance between the variables and the dimension of the input layer in the artificial neural network, (2) bootstrap replicates of the learning set where the data is repeatedly sampled with a random split of the data into train sets and using these as new learning sets, and (3) automatic reservoir model identification through aggregated predictor formed by a plurality vote when predicting a new class. This method is described in detail to ensure successful replication of results. The required training and test dataset were generated by using analytical solution models. In our case, there were used 600 samples: 300 for training, 100 for cross-validation, and 200 for testing. Different network structures were tested during this study to arrive at optimum network design. We notice that the single net methodology always brings about confusion in selecting the correct model even though the training results for the constructed networks are close to 1. We notice also that the principal component analysis is an effective strategy in reducing the number of input features, simplifying the network structure, and lowering the training time of the ANN. The results obtained show that the proposed model provides better performance when predicting new data with a coefficient of correlation approximately equal to 95% Compared to a previous approach 80%, the combination of the PCA and ANN is more stable and determine the more accurate results with lesser computational complexity than was feasible previously. Clearly, the aggregated predictor is more stable and shows less bad classes compared to the previous approach.
DevOps and Testing slides at DASA ConnectKari Kakkonen
My and Rik Marselis slides at 30.5.2024 DASA Connect conference. We discuss about what is testing, then what is agile testing and finally what is Testing in DevOps. Finally we had lovely workshop with the participants trying to find out different ways to think about quality and testing in different parts of the DevOps infinity loop.
Key Trends Shaping the Future of Infrastructure.pdfCheryl Hung
Keynote at DIGIT West Expo, Glasgow on 29 May 2024.
Cheryl Hung, ochery.com
Sr Director, Infrastructure Ecosystem, Arm.
The key trends across hardware, cloud and open-source; exploring how these areas are likely to mature and develop over the short and long-term, and then considering how organisations can position themselves to adapt and thrive.
Transcript: Selling digital books in 2024: Insights from industry leaders - T...BookNet Canada
The publishing industry has been selling digital audiobooks and ebooks for over a decade and has found its groove. What’s changed? What has stayed the same? Where do we go from here? Join a group of leading sales peers from across the industry for a conversation about the lessons learned since the popularization of digital books, best practices, digital book supply chain management, and more.
Link to video recording: https://bnctechforum.ca/sessions/selling-digital-books-in-2024-insights-from-industry-leaders/
Presented by BookNet Canada on May 28, 2024, with support from the Department of Canadian Heritage.
Let's dive deeper into the world of ODC! Ricardo Alves (OutSystems) will join us to tell all about the new Data Fabric. After that, Sezen de Bruijn (OutSystems) will get into the details on how to best design a sturdy architecture within ODC.
The Art of the Pitch: WordPress Relationships and SalesLaura Byrne
Clients don’t know what they don’t know. What web solutions are right for them? How does WordPress come into the picture? How do you make sure you understand scope and timeline? What do you do if sometime changes?
All these questions and more will be explored as we talk about matching clients’ needs with what your agency offers without pulling teeth or pulling your hair out. Practical tips, and strategies for successful relationship building that leads to closing the deal.
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...James Anderson
Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM
The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management.
The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM).
Speakers:
Bob Boule
Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle.
Gopinath Rebala
Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...Jeffrey Haguewood
Sidekick Solutions uses Bonterra Impact Management (fka Social Solutions Apricot) and automation solutions to integrate data for business workflows.
We believe integration and automation are essential to user experience and the promise of efficient work through technology. Automation is the critical ingredient to realizing that full vision. We develop integration products and services for Bonterra Case Management software to support the deployment of automations for a variety of use cases.
This video focuses on the notifications, alerts, and approval requests using Slack for Bonterra Impact Management. The solutions covered in this webinar can also be deployed for Microsoft Teams.
Interested in deploying notification automations for Bonterra Impact Management? Contact us at sales@sidekicksolutionsllc.com to discuss next steps.
GraphRAG is All You need? LLM & Knowledge GraphGuy Korland
Guy Korland, CEO and Co-founder of FalkorDB, will review two articles on the integration of language models with knowledge graphs.
1. Unifying Large Language Models and Knowledge Graphs: A Roadmap.
https://arxiv.org/abs/2306.08302
2. Microsoft Research's GraphRAG paper and a review paper on various uses of knowledge graphs:
https://www.microsoft.com/en-us/research/blog/graphrag-unlocking-llm-discovery-on-narrative-private-data/
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...UiPathCommunity
💥 Speed, accuracy, and scaling – discover the superpowers of GenAI in action with UiPath Document Understanding and Communications Mining™:
See how to accelerate model training and optimize model performance with active learning
Learn about the latest enhancements to out-of-the-box document processing – with little to no training required
Get an exclusive demo of the new family of UiPath LLMs – GenAI models specialized for processing different types of documents and messages
This is a hands-on session specifically designed for automation developers and AI enthusiasts seeking to enhance their knowledge in leveraging the latest intelligent document processing capabilities offered by UiPath.
Speakers:
👨🏫 Andras Palfi, Senior Product Manager, UiPath
👩🏫 Lenka Dulovicova, Product Program Manager, UiPath
"Impact of front-end architecture on development cost", Viktor TurskyiFwdays
I have heard many times that architecture is not important for the front-end. Also, many times I have seen how developers implement features on the front-end just following the standard rules for a framework and think that this is enough to successfully launch the project, and then the project fails. How to prevent this and what approach to choose? I have launched dozens of complex projects and during the talk we will analyze which approaches have worked for me and which have not.
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...Ramesh Iyer
In today's fast-changing business world, Companies that adapt and embrace new ideas often need help to keep up with the competition. However, fostering a culture of innovation takes much work. It takes vision, leadership and willingness to take risks in the right proportion. Sachin Dev Duggal, co-founder of Builder.ai, has perfected the art of this balance, creating a company culture where creativity and growth are nurtured at each stage.
From Siloed Products to Connected Ecosystem: Building a Sustainable and Scala...
Comparison of hybrid pso sa algorithm and genetic algorithm for classification
1. Computer Engineering and Intelligent Systems www.iiste.org
ISSN 2222-1719 (Paper) ISSN 2222-2863 (Online)
Vol 3, No.2, 2012
Comparison of Hybrid PSO-SA Algorithm and Genetic
Algorithm for Classification
S. G. Sanjeevi1* A. Naga Nikhila2 Thaseem Khan3 G. Sumathi4
6. Associate Professor, Dept. of Comp. Science & Engg., National Institute of Technology, Warangal,
A.P., India
7. Dept. of Comp. Science & Engg., National Institute of Technology, Warangal, A.P., India
8. Dept. of Comp. Science & Engg.,National Institute of Technology, Warangal, A.P., India
9. Dept. of Comp. Science & Engg., National Institute of Technology, Warangal, A.P., India
* E-mail of the corresponding author: sgsanjeevi@yahoo.com
Abstract
In this work, we propose and present a Hybrid particle swarm optimization-Simulated annealing algorithm
and compare it with a Genetic algorithm for training respectively neural networks of identical architectures.
These neural networks were then tested on a classification task. In particle swarm optimization, behavior of
a particle is influenced by the experiential knowledge of the particle as well as socially exchanged
information. Particle swarm optimization follows a parallel search strategy. In simulated annealing uphill
moves are made in the search space in a stochastic fashion in addition to the downhill moves. Simulated
annealing therefore has better scope of escaping local minima and reach a global minimum in the search
space. Thus simulated annealing gives a selective randomness to the search. Genetic algorithm performs
parallel and randomized search. The goal of training the neural network is to minimize the sum of the
squares of the error between the target and observed output values for all the training samples and to deliver
good test performance on the test inputs. We compared the performance of the neural networks of identical
architectures trained by the Hybrid particle swarm optimization-simulated annealing and Genetic
algorithm respectively on a classification task and noted the results obtained. Neural network trained by
Hybrid particle swarm optimization-simulated annealing has given better results compared to the neural
network trained by the Genetic algorithm in the tests conducted by us.
Keywords: Classification, Hybrid particle swarm optimization-Simulated annealing, Simulated
Annealing, Genetic algorithm, Neural Network etc.
1. Introduction
Classification is an important activity of machine learning. Various algorithms are conventionally used for
classification task namely, Decision tree learning using ID3[1], Concept learning using Candidate
elimination [2], Neural networks [3], Naïve Bayes classifier [4] are some of the traditional methods used
for classification. Ever since back propagation algorithm was invented and popularized by [3], [5] and [6]
neural networks were actively used for classification. However, since back-propagation method follows hill
climbing approach, it is susceptible to occurrence of local minima. Hence we examine the use of alternative
methods for training neural networks. We examine the use of i) Hybrid particle swarm optimization-
simulated annealing algorithm ii) Genetic algorithm to train the neural networks. We study and compare
the performance of the neural networks trained by these algorithms on a classification task.
2. Architecture of Neural Network
Neural network designed for the classification task has the following architecture. It has four input units,
three hidden units and three output units in the input layer, hidden layer and output layer respectively.
Sigmoid activation functions were used with hidden and output units. Figure 1 shows the architectural
diagram of the neural network. Neurons are connected in the feed forward fashion as shown. Neural
37
2. Computer Engineering and Intelligent Systems www.iiste.org
ISSN 2222-1719 (Paper) ISSN 2222-2863 (Online)
Vol 3, No.2, 2012
Network has 12 weights between input and hidden layer and 9 weights between hidden and output layer. So
the total number of weights in the network are 21.
Input Hidden Output
Figure 1 Architecture of Neural Network used
2.1 .Iris data
Neural Network shown in figure 1 is used to perform classification task on IRIS data. The data was taken
from the Univ. of California, Irvine (UCI), Machine learning repository. Iris data consists of 150 input-
output vector pairs. Each input vector consists of a 4 tuple having four attribute values corresponding to the
four input attributes respectively. Based on the input vector, output vector gives class to which it belongs.
Each output vector is a 3 tuple and will have a ‘1’ in first, second or third positions and zeros in rest two
positions, thereby indicating the class to which the input vector being considered belongs. Hence, we use 1-
of-n encoding on the output side for denoting the class value. The data of 150 input-output pairs is divided
randomly into two parts to create the training set and test set respectively. Data from training set is used to
train the neural network and data from the test set is used for test purposes. Few samples of IRIS data are
shown in table 1.
Table 1.Sample Of Iris Data Used.
S. No. Attr. 1 Attr. 2 Attr. 3 Attr. 4 Class 1 Class 2 Class 3
1 0.224 0.624 0.067 0.043 1 0 0
2 0.749 0.502 0.627 0.541 0 1 0
3 0.557 0.541 0.847 1 0 0 1
4 0.11 0.502 0.051 0.043 1 0 0
5 0.722 0.459 0.663 0.584 0 1 0
3. Genetic Algorithm to Train the Neural Network
Error back propagation algorithm is conventionally used to train the neural networks. Error-Back
propagation algorithm uses gradient descent search which is based on the concept of hill climbing. Main
disadvantage of neural network using back propagation algorithm is that since it uses hill climbing
approach it can get stuck at local minima.
Hence, here we explore the usage of alternative algorithms instead of conventional backpropagation
algorithm to optimize the performance of training a neural network and compare them.
The objective function required to be minimized for training the neural network is the error function E.
We define the error function E as follows. The error function E is given by
38
3. Computer Engineering and Intelligent Systems www.iiste.org
ISSN 2222-1719 (Paper) ISSN 2222-2863 (Online)
Vol 3, No.2, 2012
where x is a specific training example and X is the set of all training examples, denotes the target
output for the kth output neuron corresponding to training sample x, denotes the observed output for
the kth output neuron corresponding to training sample x. Hence, error function computed is a measure of
the sum of the squares of the error between target and observed output values for all the output neurons,
across all the training samples.
Genetic algorithms (GAs) were proposed and enunciated by John Holland at the University of Michigan
and popularized by David Goldberg [9, 10]. They have the characteristics of parallel search being
undertaken through potential solutions with a random exploration of the search space. They use the
evolutionary operators of selection, crossover and mutation on the population of the potential solutions.
Unlike backpropagation, GAs are stochastic and hence do not get into the problem of local minimum.
Genetic algorithms can be used for optimization. In the problem we have to optimize the error function
defined in equation (1) by minimizing it. Hence, the objective function we use is the error function in
equation (1) which needs to be minimized. To do this we define the fitness function f as A – E where A is a
positive number which is appropriately selected to be sufficiently large so that A – E is positive during the
search task undertaken through GA. Task of minimizing the error function is converted to maximizing the
value of A – E which is the fitness function f used with the genetic algorithm during the training of the
neural network.
We are using the genetic algorithm for the purpose of efficiently training the neural network. The idea of
efficiently training the neural network is achieved if the neural network predicts with higher accuracy the
test outputs corresponding to the test inputs given to the network. GAs use set of binary strings called
chromosomes to form a population of binary strings. Each chromosome which is a binary string is a
potential solution to the network. Hence, each chromosome needs to encode the set of weights present in a
neural network. The neural network shown in figure 1 has 21 weights in it. Hence each member of the
population (chromosome) needs to encode these complete set of 21 weights so that we can evaluate the
fitness function defined above. Below we describe the encoding of the weights for the neural network
shown in figure 1 for each chromosome.
3.1.Encoding of the Weights in the Neural Network
The Neural Network in figure 1 has 12 weights between input and hidden layer and 9 weights between
hidden and output layer. So the total number of weights in the network are 21. We coded these weights in
binary form. Each weight was assumed to vary between +12.75 and -12.75 with a precision of 0.05. This is
because weights learned by a typical neural network will be small positive or negative real numbers. Any
real number between -12.75 and +12.75 with a precision of 0.05 can be represented by a 9-bit binary
string. Thus, -12.75 was represented with a binary string ‘000000000’ and +12.75 was represented with
‘111111111’. For example, number 0.1 is represented by ‘100000010’. Each of the twenty one weights
present in the neural network was represented by a separate 9 bit binary string and the set of all the weights
present in the neural network is represented by their concatenation having 189 bits binary string. These 189
bits represent one possible assignment of values to the weights in the neural network.
In the initial population a set of 50 randomly generated binary strings were created, each binary string
having 189 bits. Different binary strings in population represent different potential solutions to the problem
of training the neural network. Each binary string represents one possible assignment of weights to the
neural network.
3.2. Genetic Algorithm
Here we describe the Genetic algorithm used for training the neural network.
1. A set of 50 binary strings, each string having 189 bits were randomly created to form the initial
39
4. Computer Engineering and Intelligent Systems www.iiste.org
ISSN 2222-1719 (Paper) ISSN 2222-2863 (Online)
Vol 3, No.2, 2012
population.
2. Evaluate the fitness function f for each of the binary strings in the present population P.
3. Elitism: Two of the binary strings with highest fitness values from the population are carried over
to the new population being created Pnew.
4. Selection: The selection operation selects the parent strings from the present population for the
crossover operation. This is done using the selection probability pi = fi /Σ fi . The roulette wheel
concept is used to make the selection of binary strings. The probability pi is determined as the
ratio of fitness of a given binary string with the sum of fitness values for all the 50 binary strings
in the population P.
5. Crossover: Pairs of chromosomes were selected from population P using selection probability
defined above and a single point crossover is performed on the pair to form two offspring.
Selection probabilities encourage the selection of high fitness individuals to be selected as a pair.
Crossover point was chosen at random and the two strings are interchanged at the point. This
process was repeated for each of the 24 pairs of strings selected. Selection of each pair was done
using selection probability calculations as explained above. Strings generated after crossover are
kept in the new population being created Pnew.
6. Mutation: Mutation involves flipping a randomly selected bit. Mutation was done with a
probability of 5% on the strings in Pnew . In the string selected for mutation, a randomly selected bit
is flipped.
7. On completing above step the new population Pnew is formed. Make the new population created
Pnew as the present population P for the next iteration of GA ie. Set P ←Pnew. Repeat steps 2 to 7
for sufficient number of iterations. Steps 2 to 7 were repeated for 15000 iterations in our work.
The string with the highest fitness among the population is returned as the solution and gives the
assignment of weights to the neural network after training through the Genetic Algorithm.
4. Hybrid PSO - SA Algorithm
In particle swarm optimization [11, 12] a swarm of particles are flown through a multidimensional search
space. Position of each particle represents a potential solution. Position of each particle is changed by
adding a velocity vector to it. Velocity vector is influenced by the experiential knowledge of the particle as
well as socially exchanged information. The experiential knowledge of a particle A describes the distance
of the particle A from its own best position since the particle A’s first time step. This best position of a
particle is referred to as the personal best of the particle. The global best position in a swarm at a time t is
the best position found in the swarm of particles at the time t. The socially exchanged information of a
particle A describes the distance of a particle A from the global best position in the swarm at time t. The
experiential knowledge and socially exchanged information are also referred to as cognitive and social
components respectively. We propose and present here the hybrid PSO-SA algorithm in table 2.
Table 2. Hybrid Particle Swarm Optimization-Simulated Annealing Algorithm
Create a nx dimensional swarm of ns particles;
repeat
for each particle i = 1, . . .,ns do
// yi denotes the personal best position of the particle i so far
// set the personal best position
if f (xi) <f (yi) then
40
5. Computer Engineering and Intelligent Systems www.iiste.org
ISSN 2222-1719 (Paper) ISSN 2222-2863 (Online)
Vol 3, No.2, 2012
yi = xi ;
end
// ŷ denotes the global best of the swarm so far
// set the global best position
if f (yi) <f ( ŷ ) then
ŷ = yi ;
end
end
for each particle i = 1, . . ., ns do
update the velocity vi of particle i using equation (2);
vi(t+1) = vi(t) + c1 r1(t)[yi(t) – xi(t) ] + c2 r2(t)[ŷ(t) – xi(t)] (2)
// where yi denotes the personal best position of the particle i
// and ŷ denotes the global best position of the swarm
// and xi denotes the present position vector of particle i.
update the position using equation (3);
bi(t) = xi (t) //storing present position
xi(t+1) = xi(t) + vi (t+1) (3)
// applying simulated annealing
compute ∆E = (f (xi(t) - f(xi(t+1)) (4)
// ∆E = (E value for the previous network before weight change) - (E value for the
present network with changed configuration of weights).
• if ∆E is positive then
// new value of E is smaller and therefore better than previous value. Then
// accept the new position and make it the current position
else accept the new position xi(t+1) (even though it has a higher E value) with a
probability p defined by
p= (5)
• Revise the temperature T as per schedule defined below.
The temperature schedule followed is defined as follows:
The starting temperature is taken as T =1050;
Current temp = T/log(iterations+1);
Current temperature was changed after every 100 iterations using above formula.
end
41
6. Computer Engineering and Intelligent Systems www.iiste.org
ISSN 2222-1719 (Paper) ISSN 2222-2863 (Online)
Vol 3, No.2, 2012
until stopping condition is true ;
In equation (2) of table 2, vi ,yi and ŷ are vectors of nx dimensions. vij denotes scalar component of vi in
dimension j. vij is calculated as shown in equation 6 where r1j and r2j are random values in the range [0,1]
and c1 and c2 are learning factors chosen as c1 = c2 = 2.
vij(t+1) = vij(t) + c1 r1j(t)[yij(t) – xij(t) ] + c2 r2j(t)[ŷj(t) – xij) (t)] (6)
4.1. Implementation details of Hybrid PSO-SA algorithm for training a neural network
We describe here the implementation details of hybrid PSO-SA algorithm for training the neural
network. There are 21 weights in the neural network shown in figure 1.
Hybrid PSO-SA algorithm combines particle swarm optimization algorithm with simulated annealing
approach in the context of training a neural network. The swarm is initialized with a population of 50
particles. Each particle has 21 weights. Each weight value corresponds to a position in a particular
dimension of the particle. Since there are 21 weights for each particle, there are therefore 21 dimensions.
Hence, position vector of each particle corresponds to a 21 dimensional weight vector. Position (weight) in
each dimension is modified by adding velocity value to it in that dimension. Fitness function
corresponding to a position xi , is f (xi) and it denotes the distance of position xi from the final solution. In
the hybrid PSO-SA algorithm this fitness function f (xi) is defined as error function E(w). Error function
E(w) is a function of the weights in the neural network as defined in equation (1). In the optimization
problem of training neural networks fitness function which is error function needs to be minimized.
Each particle’s velocity vector is updated by considering the personal best position of the particle, global
best position of the entire swarm and the present position vector of the particle as shown in equation (2) of
table 2. Velocity of a particle i in dimension j is calculated as shown in equation (6).
Hybrid PSO-SA algorithm combines pso algorithm with simulated annealing approach. Each of the 50
particles in the swarm is associated with 21 weights present in the neural network. The error function E(w)
which is a function of the weights in the neural network as defined in equation (1) is treated as the fitness
function. Error E(w) ( fitness function) needs to be minimized. For each of the 50 particles in the swarm,
the solution (position) given by the particle is accepted if the change in error function ∆E as defined in
equation 4 is positive, since this indicates error function is reducing in the present iteration compared to the
previous iteration value. If ∆Eis not positive then new position is accepted with a probability p given by
formula in equation (5). This is implemented by generating a random number between 0 and 1. If the
number generated is lesser than p then the new position is accepted, else the previous position of particle is
retained without changing. Each particle’s personal best position and the global best position of the swarm
are updated after each iteration. Hybrid PSO-SA algorithm was run for 15000 iterations. After 15000
iterations global best position gbest of the swarm is returned as the solution. Hence, stopping criterion for
the algorithm was chosen as completion of 15000 iterations.
Hybrid PSO-SA algorithm combines parallel search approach of PSO and selective random search and
global search properties of simulated annealing [7, 8] and hence combines the advantages of both the
approaches.
5. Experiments and Results
We have trained the neural network with architecture shown in figure 1 with genetic algorithm described in
section 3 on training set taken from the IRIS data. Training set was a subset of samples chosen randomly
from the IRIS data. Remaining samples from IRIS data were included in the test set. Performance was
observed on the test data set for predicting the class of each sample.
Table 3. Results Of Testing For Neural Network Using Genetic Algorithm
42
7. Computer Engineering and Intelligent Systems www.iiste.org
ISSN 2222-1719 (Paper) ISSN 2222-2863 (Online)
Vol 3, No.2, 2012
Sl. No Samples in Samples in Test Correct Misclassifications
Training Set Set classifications
1 100 50 44 6
2 95 55 45 10
3 85 65 53 12
4 75 75 61 14
We have chosen similarly, training sets of varying sizes from the IRIS data and included each time the
samples which were not selected for training set into test set. Neural network was trained by each of the
training sets and tested the performance of the network on corresponding test sets. Training was done for
15000 iterations with each of the training sets using the genetic algorithm. Results are shown in table 3.
We have also trained the neural network with architecture shown in figure 1 with Hybrid PSO-SA algorithm
described in section 4, with each of the training sets chosen above in table 3 and tested the performance of
the network on corresponding test sets. Training was done using Hybrid PSO-SA algorithm for 15000
iterations with each of the training sets. The results are shown in table 4.
Table 4. Results Of Testing For Neural Network Using Hybrid PSO-SA Algorithm
Sl.No Samples in Training Samples in Test Correct Misclassifications
Set Set Classifications
1 100 50 47 3
2 95 55 51 4
3 85 65 61 4
4 75 75 70 5
Two neural networks with same architecture as shown in figure 1 were used for training and testing with
the two algorithms of Hybrid PSO-SA algorithm and Genetic algorithm respectively. Training was
performed for same number of 15000 iterations on each neural network. Same training and test sets were
used for comparison of neural networks performance with both the algorithms. Results of experiments and
testing point out that neural network trained with Hybrid PSO-SA algorithm gives better performance over
neural network trained with the Genetic algorithm across the training and test sets used.
6. Conclusion
Our objective was to compare the performance of feed-forward neural network trained with Hybrid PSO-
SA algorithm with the neural network trained by Genetic algorithm. We have trained the neural networks
with identical architectures with the Hybrid PSO-SA algorithm and the Genetic algorithm respectively. The
task we have tested using neural networks trained separately using these two algorithms is the IRIS data
classification. We found that neural network trained with Hybrid PSO-SA algorithm has given better
43
8. Computer Engineering and Intelligent Systems www.iiste.org
ISSN 2222-1719 (Paper) ISSN 2222-2863 (Online)
Vol 3, No.2, 2012
classification performance among the two. Neural network trained with Hybrid PSO-SA algorithm
combines parallel search approach of PSO and selective random search and global search properties of
simulated annealing and hence combines the advantages of both the approaches. GA also uses parallel and
randomized search. Hence we compared these two global search approaches. Hybrid PSO-SA algorithm has
given better results for training a neural network than the Genetic algorithm.
References
A Quinlan, J.R.(1986). Induction of decision trees. Machine Learning, 1 (1), 81-106.
Mitchell, T.M.(1977). Version spaces: A candidate elimination approach to rule learning. Fifth International
Joint Conference on AI (pp.305-310). Cambridge, MA: MIT Press.
Rumelhart, D.E., & McClelland, J.L.(1986). Parallel distributed processing: exploration in the
microstructure of cognition(Vols.1 & 2). Cambridge, MA : MIT Press.
Duda, R.O., & Hart, P.E.(1973). Pattern classification and scene analysis New York: John Wiley & Sons.
Werbos, P. (1975). Beyond regression: New tools for prediction and analysis in the behavioral sciences
(Ph.D. dissertation). Harvard University.
Parker, D.(1985). Learning logic (MIT Technical Report TR-47).MIT Center for Research in
Computational Economics and Management Science.
Kirkpatrick, S., Gelatt,Jr., C.D., and M.P. Vecchi 1983. Optimization by simulated annealing.Science
220(4598).
Russel, S., &Norvig, P. (1995). Artificial intelligence: A modern approach. Englewood Cliffs, NJ:Prentice-
Hall.
Mitchell, T.M., 1997.Machine Learning, New York: McGraw-Hill.
K Goldberg, David E. (1989). Genetic Algorithms in Search Optimization and Machine Learning. Addison
Wesley. p. 41.ISBN 0201157675.
Kennedy, J., Eberhart, R., 1995. “Particle swarm optimization”, IEEE International Conference on Neural
Networks, Perth, WA , (pp.1942 – 1948), Australia.
Engelbrecht, A.P., 2007, Computational Intelligence, England: John Wiley & Sons.
Sriram G. Sanjeevi has acquired B.E. (Electronics and Communication Engineering) from Osmania
University, Hyderabad, India in 1981, M.Tech (Computer Science and Engineering) from M.I.T. Manipal,
Mangalore University, India in 1991 and Ph.D (Computer Science and Engineering) from IIT, Bombay,
India in 2009. He is currently Associate Professor & Head of Computer Science and Engineering
Department, National Institute of Technology, Warangal, India. His research interests are Neural networks,
Machine learning and Soft computing.
A. N. Nikhila has acquired B.Tech (Computer Science and Engineering) from NIT Warangal, India in
2011. Her research interests are Neural networks, Machine learning.
Thaseem Khan has acquired B.Tech (Computer Science and Engineering) from NIT Warangal, India in
2011. His research interests are Neural networks, Machine learning.
G. Sumathi has acquired B.Tech (Computer Science and Engineering) from NIT Warangal, India in 2011.
Her research interests are Neural networks, Machine learning.
44