In this work, we propose a Hybrid particle swarm optimization-Simulated annealing algorithm and present a comparison with i) Simulated annealing algorithm and ii) Back propagation algorithm for training neural networks. These neural networks were then tested on a classification task. In particle swarm optimization behaviour of a particle is influenced by the experiential knowledge of the particle as well as socially exchanged information. Particle swarm optimization follows a parallel search strategy. In simulated annealing uphill moves are made in the search space in a stochastic fashion in addition to the downhill moves. Simulated annealing therefore has better scope of escaping local minima and reach a global minimum in the search space. Thus simulated annealing gives a selective randomness to the search. Back propagation algorithm uses gradient descent approach search for minimizing the error. Our goal of global minima in the task being done here is to come to lowest energy state, where energy state is being modelled as the sum of the squares of the error between the target and observed output values for all the training samples. We compared the performance of the neural networks of identical architectures trained by the i) Hybrid particle swarm optimization-simulated annealing, ii) Simulated annealing and iii) Back propagation algorithms respectively on a classification task and noted the results obtained. Neural network trained by Hybrid particle swarm optimization-simulated annealing has given better results compared to the neural networks trained by the Simulated annealing and Back propagation algorithms in the tests conducted by us.
Simulation of Single and Multilayer of Artificial Neural Network using Verilogijsrd.com
This document discusses the simulation of single layer and multilayer artificial neural networks using Verilog. It begins with an introduction to artificial neural networks and their application in VLSI circuit fault diagnosis. It then provides details on the algorithm and design methodology for simulating a single layer neural network to model an AND gate, showing the calculation of error over iterations in Matlab and time taken using Verilog code. For a multilayer network modeling an XOR gate, it similarly discusses the backpropagation algorithm, showing error reduction over iterations in Matlab and time taken using Verilog. It concludes that neural networks can help minimize time to find faults in digital circuits.
11.comparison of hybrid pso sa algorithm and genetic algorithm for classifica...Alexander Decker
This document compares the performance of a hybrid particle swarm optimization-simulated annealing (PSO-SA) algorithm and genetic algorithm for training neural networks to perform classification. It describes the architecture of a neural network used for classification of iris data, with 4 input units, 3 hidden units, and 3 output units. The genetic algorithm and PSO-SA algorithm are used to train the neural network by minimizing an error function, and their ability to classify test data is compared. The neural network trained with the hybrid PSO-SA algorithm achieved better results than the genetic algorithm in tests conducted.
Comparison of hybrid pso sa algorithm and genetic algorithm for classificationAlexander Decker
This document compares the performance of a hybrid particle swarm optimization-simulated annealing (PSO-SA) algorithm and a genetic algorithm for training neural networks to perform classification. Both algorithms are compared to traditional backpropagation training. The PSO-SA algorithm combines particle swarm optimization with simulated annealing to avoid local minima. A genetic algorithm uses evolutionary operators like selection, crossover and mutation to perform stochastic parallel search of the solution space. Both algorithms are used to train a neural network with 4 inputs, 3 hidden units and 3 outputs on an iris flower classification task. The neural network trained with the PSO-SA algorithm achieved better classification results than the network trained with the genetic algorithm.
A Threshold Logic Unit (TLU) is a mathematical function conceived as a crude model, or abstraction of biological neurons. Threshold logic units are the constitutive units in an artificial neural network. In this paper a positive clock-edge triggered T flip-flop is designed using Perceptron Learning Algorithm, which is a basic design algorithm of threshold logic units. Then this T flip-flop is used to design a two-bit up-counter that goes through the states 0, 1, 2, 3, 0, 1… Ultimately, the goal is to show how to design simple logic units based on threshold logic based perceptron concepts.
Review: “Implementation of Feedforward and Feedback Neural Network for Signal...IJERA Editor
Main focus of project is on implementation of Neural Network Architecture (NNA) with on chip learning on
Analog VLSI Technology for signal processing application. In the proposed paper the analog components like
Gilbert Cell Multiplier (GCM), Neuron Activation Function (NAF) are used to implement artificial NNA.
Analog components used comprises of multiplier, adder and tan sigmoidal function circuit using MOS transistor.
This Neural Architecture is trained using Back Propagation (BP) Algorithm in analog domain with new
techniques of weight storage. Layout design and verification of above design is carried out using VLSI Backend
Microwind 3.1 software Tool. The technology used to design layout is 32 nm CMOS Technology.
Rainfall Prediction using Data-Core Based Fuzzy Min-Max Neural Network for Cl...IJERA Editor
This paper proposes the Rainfall Prediction System by using classification technique. The advanced and modified neural network called Data Core Based Fuzzy Min Max Neural Network (DCFMNN) is used for pattern classification. This classification method is applied to predict Rainfall. The neural network called fuzzy min max neural network (FMNN) that creates hyperboxes for classification and predication, has a problem of overlapping neurons that resoled in DCFMNN to give greater accuracy. This system is composed of forming of hyperboxes, and two kinds of neurons called as Overlapping Neurons and Classifying neurons, and classification used for prediction. For each kind of hyperbox its data core and geometric center of data is calculated. The advantage of this method is it gives high accuracy and strong robustness. According to evaluation results we can say that this system gives better prediction of rainfall and classification tool in real environment.
Iterative Determinant Method for Solving Eigenvalue Problemsijceronline
International Journal of Computational Engineering Research (IJCER) is dedicated to protecting personal information and will make every reasonable effort to handle collected information appropriately. All information collected, as well as related requests, will be handled as carefully and efficiently as possible in accordance with IJCER standards for integrity and objectivity.
Simulation of Single and Multilayer of Artificial Neural Network using Verilogijsrd.com
This document discusses the simulation of single layer and multilayer artificial neural networks using Verilog. It begins with an introduction to artificial neural networks and their application in VLSI circuit fault diagnosis. It then provides details on the algorithm and design methodology for simulating a single layer neural network to model an AND gate, showing the calculation of error over iterations in Matlab and time taken using Verilog code. For a multilayer network modeling an XOR gate, it similarly discusses the backpropagation algorithm, showing error reduction over iterations in Matlab and time taken using Verilog. It concludes that neural networks can help minimize time to find faults in digital circuits.
11.comparison of hybrid pso sa algorithm and genetic algorithm for classifica...Alexander Decker
This document compares the performance of a hybrid particle swarm optimization-simulated annealing (PSO-SA) algorithm and genetic algorithm for training neural networks to perform classification. It describes the architecture of a neural network used for classification of iris data, with 4 input units, 3 hidden units, and 3 output units. The genetic algorithm and PSO-SA algorithm are used to train the neural network by minimizing an error function, and their ability to classify test data is compared. The neural network trained with the hybrid PSO-SA algorithm achieved better results than the genetic algorithm in tests conducted.
Comparison of hybrid pso sa algorithm and genetic algorithm for classificationAlexander Decker
This document compares the performance of a hybrid particle swarm optimization-simulated annealing (PSO-SA) algorithm and a genetic algorithm for training neural networks to perform classification. Both algorithms are compared to traditional backpropagation training. The PSO-SA algorithm combines particle swarm optimization with simulated annealing to avoid local minima. A genetic algorithm uses evolutionary operators like selection, crossover and mutation to perform stochastic parallel search of the solution space. Both algorithms are used to train a neural network with 4 inputs, 3 hidden units and 3 outputs on an iris flower classification task. The neural network trained with the PSO-SA algorithm achieved better classification results than the network trained with the genetic algorithm.
A Threshold Logic Unit (TLU) is a mathematical function conceived as a crude model, or abstraction of biological neurons. Threshold logic units are the constitutive units in an artificial neural network. In this paper a positive clock-edge triggered T flip-flop is designed using Perceptron Learning Algorithm, which is a basic design algorithm of threshold logic units. Then this T flip-flop is used to design a two-bit up-counter that goes through the states 0, 1, 2, 3, 0, 1… Ultimately, the goal is to show how to design simple logic units based on threshold logic based perceptron concepts.
Review: “Implementation of Feedforward and Feedback Neural Network for Signal...IJERA Editor
Main focus of project is on implementation of Neural Network Architecture (NNA) with on chip learning on
Analog VLSI Technology for signal processing application. In the proposed paper the analog components like
Gilbert Cell Multiplier (GCM), Neuron Activation Function (NAF) are used to implement artificial NNA.
Analog components used comprises of multiplier, adder and tan sigmoidal function circuit using MOS transistor.
This Neural Architecture is trained using Back Propagation (BP) Algorithm in analog domain with new
techniques of weight storage. Layout design and verification of above design is carried out using VLSI Backend
Microwind 3.1 software Tool. The technology used to design layout is 32 nm CMOS Technology.
Rainfall Prediction using Data-Core Based Fuzzy Min-Max Neural Network for Cl...IJERA Editor
This paper proposes the Rainfall Prediction System by using classification technique. The advanced and modified neural network called Data Core Based Fuzzy Min Max Neural Network (DCFMNN) is used for pattern classification. This classification method is applied to predict Rainfall. The neural network called fuzzy min max neural network (FMNN) that creates hyperboxes for classification and predication, has a problem of overlapping neurons that resoled in DCFMNN to give greater accuracy. This system is composed of forming of hyperboxes, and two kinds of neurons called as Overlapping Neurons and Classifying neurons, and classification used for prediction. For each kind of hyperbox its data core and geometric center of data is calculated. The advantage of this method is it gives high accuracy and strong robustness. According to evaluation results we can say that this system gives better prediction of rainfall and classification tool in real environment.
Iterative Determinant Method for Solving Eigenvalue Problemsijceronline
International Journal of Computational Engineering Research (IJCER) is dedicated to protecting personal information and will make every reasonable effort to handle collected information appropriately. All information collected, as well as related requests, will be handled as carefully and efficiently as possible in accordance with IJCER standards for integrity and objectivity.
Classification of Electroencephalograph (EEG) Signals Using Quantum Neural Ne...CSCJournals
In this paper, quantum neural network (QNN), which is a class of feedforward neural networks (FFNN’s), is used to recognize (EEG) signals. For this purpose ,independent component analysis (ICA), wavelet transform (WT) and Fourier transform (FT) are used as a feature extraction after normalization of these signals. The architecture of (QNN’s) have inherently built in fuzzy. The hidden units of these networks develop quantized representations of the sample information provided by the training data set in various graded levels of certainty. Experimental results presented here show that (QNN’s) are capable of recognizing structures in data, a property that conventional (FFNN’s) with sigmoidal hidden units lack . Finally, (QNN) gave us kind of fast and realistic results compared with the (FFNN). Simulation results show that a total classification of 81.33% for (ICA), 76.67% for (WT) and 67.33% for (FT).
Modeling of neural image compression using gradient decent technologytheijes
The International Journal of Engineering & Science is aimed at providing a platform for researchers, engineers, scientists, or educators to publish their original research results, to exchange new ideas, to disseminate information in innovative designs, engineering experiences and technological skills. It is also the Journal's objective to promote engineering and technology education. All papers submitted to the Journal will be blind peer-reviewed. Only original articles will be published.
The papers for publication in The International Journal of Engineering& Science are selected through rigorous peer reviews to ensure originality, timeliness, relevance, and readability.
Theoretical work submitted to the Journal should be original in its motivation or modeling structure. Empirical analysis should be based on a theoretical framework and should be capable of replication. It is expected that all materials required for replication (including computer programs and data sets) should be available upon request to the authors.
The International Journal of Engineering & Science would take much care in making your article published without much delay with your kind cooperation
Enhancing Classification Accuracy of K-Nearest Neighbors Algorithm using Gain...IRJET Journal
1) The document proposes modifying the K-Nearest Neighbors (KNN) classification algorithm to improve its accuracy by incorporating the concept of attribute strength, as measured by entropy.
2) In the conventional KNN algorithm, the distance between data points is calculated without considering the strength of individual attributes. The proposed modification alters the distance calculation to take into account each attribute's entropy.
3) The accuracy of the modified KNN algorithm is evaluated on various datasets from the UCI Machine Learning Repository and compared to the accuracy of the conventional KNN algorithm.
Disease Classification using ECG Signal Based on PCA Feature along with GA & ...IRJET Journal
This document describes a method for classifying ECG signals to detect cardiovascular diseases using principal component analysis (PCA), genetic algorithms, and artificial neural networks. PCA is used to extract features from ECG signals. A genetic algorithm is then used to select optimal features and train an artificial neural network classifier. The method is tested on datasets from Physionet.org to classify ECG signals as normal or indicating conditions like bradycardia or tachycardia with high accuracy. The goal is to develop an automated system for ECG analysis and heart disease diagnosis.
Expert system design for elastic scattering neutrons optical model using bpnnijcsa
In present paper, a proposed expert system is designed to obtain a trained formulae for the optical model
parameters used in elastic scattering neutrons of light nuclei for (7Li), at energy range between [(1) to
(20)] MeV. A simple algorithm has used to design this expert system, while a multi-layer backwardpropagation
neural network (BPNN) is applied for training and testing the data used in this model. This
group of formulae may get a simple expert system occurring from governing formulae model, and predicts
the critical parameters usually resulted from the complicated computer coding methods. This expert system
may use in nuclear reactions yields in both fission and fusion nature who gives more closely results to the
real model.
This work is proposed the feed forward neural network with symmetric table addition method to design the
neuron synapses algorithm of the sine function approximations, and according to the Taylor series
expansion. Matlab code and LabVIEW are used to build and create the neural network, which has been
designed and trained database set to improve its performance, and gets the best a global convergence with
small value of MSE errors and 97.22% accuracy.
- The document proposes using a 3-layer convolutional neural network to learn to evaluate the winning chances in positions from the game Arimaa.
- It trains the CNN on a dataset of nearly 4 million positions extracted from expert Arimaa games and finds optimal hyperparameters through validation.
- The trained CNN achieves a mean squared error of 0.171 and classifies game outcomes correctly 73.5% of the time based on positional features alone.
Optimal neural network models for wind speed predictionIAEME Publication
The document presents two neural network models - multilayer perceptron (MLP) and radial basis function (RBF) networks - for wind speed prediction. It evaluates these models on real wind speed data collected over one year from wind farms in Coimbatore, India. The experimental results show that the RBF and MLP models can improve wind speed prediction accuracy compared to other approaches, according to statistical metrics like coefficient of determination, mean absolute error, root mean square error, and mean bias error.
ARTIFICIAL NEURAL NETWORK APPROACH TO MODELING OF POLYPROPYLENE REACTORijac123
This paper shows modeling of highly nonlinear polymerization process using the artificial neural network approach for the model predictive purposes. Polymerization occurs in a fluidized bed polypropylene reactor using Ziegler - Natta catalyst and the main objective was modeling of the reactor production rate.
The data set used for an identification of the model is a real process data received from an existing polypropylene plant and the identified model is a nonlinear autoregressive neural network with the exogenous input. Performance of a trained network has been verified using the real process data and the
ability of the production rate prediction is shown in the conclusion.
Adaptive modified backpropagation algorithm based on differential errorsIJCSEA Journal
A new efficient modified back propagation algorithm with adaptive learning rate is proposed to increase the convergence speed and to minimize the error. The method eliminates initial fixing of learning rate through trial and error and replaces by adaptive learning rate. In each iteration, adaptive learning rate for output and hidden layer are determined by calculating differential linear and nonlinear errors of output layer and hidden layer separately. In this method, each layer has different learning rate in each iteration. The performance of the proposed algorithm is verified by the simulation results.
The document summarizes five basic learning algorithms of artificial neural networks: Hebbian learning, memory-based learning, backpropagation, competitive learning, Adaline network, and Madaline network. It provides details on each algorithm, including mathematical formulas, steps involved, advantages and disadvantages, and applications.
Multilayer Backpropagation Neural Networks for Implementation of Logic GatesIJCSES Journal
ANN is a computational model that is composed of several processing elements (neurons) that tries to solve a specific problem. Like the human brain, it provides the ability to learn
from experiences without being explicitly programmed. This article is based on the implementation of artificial neural networks for logic gates. At first, the 3 layers Artificial Neural Network is
designed with 2 input neurons, 2 hidden neurons & 1 output neuron. after that model is trained
by using a backpropagation algorithm until the model satisfies the predefined error criteria (e)
which set 0.01 in this experiment. The learning rate (α) used for this experiment was 0.01. The
NN model produces correct output at iteration (p)= 20000 for AND, NAND & NOR gate. For
OR & XOR the correct output is predicted at iteration (p)=15000 & 80000 respectively
On The Application of Hyperbolic Activation Function in Computing the Acceler...iosrjce
1) The document discusses integrating a hyperbolic activation function into the Reverse Analysis method for data mining using a Hopfield neural network.
2) Reverse Analysis is a method that examines connection strengths in a neural network to deduce logical rules encoded in the database, while hyperbolic tangent is a smoother activation function compared to logistic.
3) Computer simulations showed the integrated method had faster processing times and was more efficient than the normal Reverse Analysis method as the network complexity increased with more neurons.
This document compares the Efficient Fuzzy Kohonen Clustering Network (EFKCN) algorithm to the original Fuzzy Kohonen Clustering Network (FKCN) algorithm to determine if EFKCN improves clustering results. Through empirical testing and simulations using expanded Fisher's Iris Data, the results showed that EFKCN did not achieve better accuracy than FKCN and is not yet a better clustering algorithm. While EFKCN had smaller errors in cluster center positions, FKCN consistently achieved higher correct clustering rates, even when the algorithms were run for the same number of iterations. Therefore, the document concludes that EFKCN does not actually improve upon the clustering performance of the original FKCN algorithm.
This document discusses using an artificial neural network to forecast electricity demand. It describes preprocessing data, creating a feed-forward neural network model with input, hidden and output layers, and training the model using backpropagation and incremental training. The model is trained on 80% of the data and tested on the remaining 20%. Mean square error is used to evaluate accuracy on both the training and test sets, with a lower error on the test set indicating better generalization of the model to new data. The goal is to accurately forecast future electricity demand based on input variables like population, GDP, price indexes, and past consumption data.
This document describes a study that used fuzzy kohonen clustering network (FKCN) and inverse distance weighting (IDW) to create a building damage hazard zonation for Banda Aceh city, Indonesia. FKCN was used to cluster physical parameter data (lithology, topography, peak ground acceleration) into 3 classes. IDW was then used to interpolate the data and divide the city into 3 zones - low, medium, and high building damage areas. The results showed most of the city is in the low to medium hazard zones, with only a small high hazard zone in the northwest near the coast. This differs from a previous study that found relatively more high hazard areas along the coast.
This document discusses using a fuzzy-neural network to forecast electricity demand. It proposes combining a neural network with fuzzy logic to overcome some limitations of only using artificial neural networks (ANNs). Specifically, it implements a fuzzy logic front-end processor to handle both numeric and fuzzy inputs before feeding them to a three-layer backpropagation neural network. This allows the neural network to capture unknown relationships between input variables like temperature, rain forecast, season and day type with the target output of electricity load. The strengths of this hybrid technique are its ability to incorporate both quantitative and qualitative knowledge and to produce more accurate forecasts.
Black-box modeling of nonlinear system using evolutionary neural NARX modelIJECEIAES
Nonlinear systems with uncertainty and disturbance are very difficult to model using mathematic approach. Therefore, a black-box modeling approach without any prior knowledge is necessary. There are some modeling approaches have been used to develop a black box model such as fuzzy logic, neural network, and evolution algorithms. In this paper, an evolutionary neural network by combining a neural network and a modified differential evolution algorithm is applied to model a nonlinear system. The feasibility and effectiveness of the proposed modeling are tested on a piezoelectric actuator SISO system and an experimental quadruple tank MIMO system.
Optimal neural network models for wind speed predictionIAEME Publication
The document describes using artificial neural networks for wind speed prediction. Specifically, it analyzes the performance of multilayer perceptron networks and radial basis function networks for wind speed forecasting using real-time data collected from wind farms in Coimbatore, India over one year. The models are trained on 3000 samples and tested on 1000 samples. Performance is evaluated using statistical metrics like coefficient of determination, mean absolute error, root mean square error, and mean bias error. Results show that the neural network models improve prediction accuracy compared to other approaches and the optimal model depends on factors like the number of hidden neurons and spread value.
Optimal neural network models for wind speed predictionIAEME Publication
The document presents two neural network models - multilayer perceptron (MLP) and radial basis function (RBF) networks - for wind speed prediction. It evaluates these models on real-time wind speed data collected from wind farms in Coimbatore, India over one year. The experimental results show that RBF and MLP networks can improve wind speed prediction accuracy compared to other approaches, according to statistical metrics like coefficient of determination, mean absolute error, root mean square error and mean bias error. The RBF and MLP models are able to handle the non-linear patterns in wind speed data, which conventional models struggle with, increasing prediction precision.
- The document presents a neural network model for recognizing handwritten digits. It uses a dataset of 20x20 pixel grayscale images of digits 0-9.
- The proposed neural network has an input layer of 400 nodes, a hidden layer of 25 nodes, and an output layer of 10 nodes. It is trained using backpropagation to classify images.
- The model achieves an accuracy of over 96.5% on test data after 200 iterations of training, outperforming a logistic regression model which achieved 91.5% accuracy. Future work could involve classifying more complex natural images.
Classification of Electroencephalograph (EEG) Signals Using Quantum Neural Ne...CSCJournals
In this paper, quantum neural network (QNN), which is a class of feedforward neural networks (FFNN’s), is used to recognize (EEG) signals. For this purpose ,independent component analysis (ICA), wavelet transform (WT) and Fourier transform (FT) are used as a feature extraction after normalization of these signals. The architecture of (QNN’s) have inherently built in fuzzy. The hidden units of these networks develop quantized representations of the sample information provided by the training data set in various graded levels of certainty. Experimental results presented here show that (QNN’s) are capable of recognizing structures in data, a property that conventional (FFNN’s) with sigmoidal hidden units lack . Finally, (QNN) gave us kind of fast and realistic results compared with the (FFNN). Simulation results show that a total classification of 81.33% for (ICA), 76.67% for (WT) and 67.33% for (FT).
Modeling of neural image compression using gradient decent technologytheijes
The International Journal of Engineering & Science is aimed at providing a platform for researchers, engineers, scientists, or educators to publish their original research results, to exchange new ideas, to disseminate information in innovative designs, engineering experiences and technological skills. It is also the Journal's objective to promote engineering and technology education. All papers submitted to the Journal will be blind peer-reviewed. Only original articles will be published.
The papers for publication in The International Journal of Engineering& Science are selected through rigorous peer reviews to ensure originality, timeliness, relevance, and readability.
Theoretical work submitted to the Journal should be original in its motivation or modeling structure. Empirical analysis should be based on a theoretical framework and should be capable of replication. It is expected that all materials required for replication (including computer programs and data sets) should be available upon request to the authors.
The International Journal of Engineering & Science would take much care in making your article published without much delay with your kind cooperation
Enhancing Classification Accuracy of K-Nearest Neighbors Algorithm using Gain...IRJET Journal
1) The document proposes modifying the K-Nearest Neighbors (KNN) classification algorithm to improve its accuracy by incorporating the concept of attribute strength, as measured by entropy.
2) In the conventional KNN algorithm, the distance between data points is calculated without considering the strength of individual attributes. The proposed modification alters the distance calculation to take into account each attribute's entropy.
3) The accuracy of the modified KNN algorithm is evaluated on various datasets from the UCI Machine Learning Repository and compared to the accuracy of the conventional KNN algorithm.
Disease Classification using ECG Signal Based on PCA Feature along with GA & ...IRJET Journal
This document describes a method for classifying ECG signals to detect cardiovascular diseases using principal component analysis (PCA), genetic algorithms, and artificial neural networks. PCA is used to extract features from ECG signals. A genetic algorithm is then used to select optimal features and train an artificial neural network classifier. The method is tested on datasets from Physionet.org to classify ECG signals as normal or indicating conditions like bradycardia or tachycardia with high accuracy. The goal is to develop an automated system for ECG analysis and heart disease diagnosis.
Expert system design for elastic scattering neutrons optical model using bpnnijcsa
In present paper, a proposed expert system is designed to obtain a trained formulae for the optical model
parameters used in elastic scattering neutrons of light nuclei for (7Li), at energy range between [(1) to
(20)] MeV. A simple algorithm has used to design this expert system, while a multi-layer backwardpropagation
neural network (BPNN) is applied for training and testing the data used in this model. This
group of formulae may get a simple expert system occurring from governing formulae model, and predicts
the critical parameters usually resulted from the complicated computer coding methods. This expert system
may use in nuclear reactions yields in both fission and fusion nature who gives more closely results to the
real model.
This work is proposed the feed forward neural network with symmetric table addition method to design the
neuron synapses algorithm of the sine function approximations, and according to the Taylor series
expansion. Matlab code and LabVIEW are used to build and create the neural network, which has been
designed and trained database set to improve its performance, and gets the best a global convergence with
small value of MSE errors and 97.22% accuracy.
- The document proposes using a 3-layer convolutional neural network to learn to evaluate the winning chances in positions from the game Arimaa.
- It trains the CNN on a dataset of nearly 4 million positions extracted from expert Arimaa games and finds optimal hyperparameters through validation.
- The trained CNN achieves a mean squared error of 0.171 and classifies game outcomes correctly 73.5% of the time based on positional features alone.
Optimal neural network models for wind speed predictionIAEME Publication
The document presents two neural network models - multilayer perceptron (MLP) and radial basis function (RBF) networks - for wind speed prediction. It evaluates these models on real wind speed data collected over one year from wind farms in Coimbatore, India. The experimental results show that the RBF and MLP models can improve wind speed prediction accuracy compared to other approaches, according to statistical metrics like coefficient of determination, mean absolute error, root mean square error, and mean bias error.
ARTIFICIAL NEURAL NETWORK APPROACH TO MODELING OF POLYPROPYLENE REACTORijac123
This paper shows modeling of highly nonlinear polymerization process using the artificial neural network approach for the model predictive purposes. Polymerization occurs in a fluidized bed polypropylene reactor using Ziegler - Natta catalyst and the main objective was modeling of the reactor production rate.
The data set used for an identification of the model is a real process data received from an existing polypropylene plant and the identified model is a nonlinear autoregressive neural network with the exogenous input. Performance of a trained network has been verified using the real process data and the
ability of the production rate prediction is shown in the conclusion.
Adaptive modified backpropagation algorithm based on differential errorsIJCSEA Journal
A new efficient modified back propagation algorithm with adaptive learning rate is proposed to increase the convergence speed and to minimize the error. The method eliminates initial fixing of learning rate through trial and error and replaces by adaptive learning rate. In each iteration, adaptive learning rate for output and hidden layer are determined by calculating differential linear and nonlinear errors of output layer and hidden layer separately. In this method, each layer has different learning rate in each iteration. The performance of the proposed algorithm is verified by the simulation results.
The document summarizes five basic learning algorithms of artificial neural networks: Hebbian learning, memory-based learning, backpropagation, competitive learning, Adaline network, and Madaline network. It provides details on each algorithm, including mathematical formulas, steps involved, advantages and disadvantages, and applications.
Multilayer Backpropagation Neural Networks for Implementation of Logic GatesIJCSES Journal
ANN is a computational model that is composed of several processing elements (neurons) that tries to solve a specific problem. Like the human brain, it provides the ability to learn
from experiences without being explicitly programmed. This article is based on the implementation of artificial neural networks for logic gates. At first, the 3 layers Artificial Neural Network is
designed with 2 input neurons, 2 hidden neurons & 1 output neuron. after that model is trained
by using a backpropagation algorithm until the model satisfies the predefined error criteria (e)
which set 0.01 in this experiment. The learning rate (α) used for this experiment was 0.01. The
NN model produces correct output at iteration (p)= 20000 for AND, NAND & NOR gate. For
OR & XOR the correct output is predicted at iteration (p)=15000 & 80000 respectively
On The Application of Hyperbolic Activation Function in Computing the Acceler...iosrjce
1) The document discusses integrating a hyperbolic activation function into the Reverse Analysis method for data mining using a Hopfield neural network.
2) Reverse Analysis is a method that examines connection strengths in a neural network to deduce logical rules encoded in the database, while hyperbolic tangent is a smoother activation function compared to logistic.
3) Computer simulations showed the integrated method had faster processing times and was more efficient than the normal Reverse Analysis method as the network complexity increased with more neurons.
This document compares the Efficient Fuzzy Kohonen Clustering Network (EFKCN) algorithm to the original Fuzzy Kohonen Clustering Network (FKCN) algorithm to determine if EFKCN improves clustering results. Through empirical testing and simulations using expanded Fisher's Iris Data, the results showed that EFKCN did not achieve better accuracy than FKCN and is not yet a better clustering algorithm. While EFKCN had smaller errors in cluster center positions, FKCN consistently achieved higher correct clustering rates, even when the algorithms were run for the same number of iterations. Therefore, the document concludes that EFKCN does not actually improve upon the clustering performance of the original FKCN algorithm.
This document discusses using an artificial neural network to forecast electricity demand. It describes preprocessing data, creating a feed-forward neural network model with input, hidden and output layers, and training the model using backpropagation and incremental training. The model is trained on 80% of the data and tested on the remaining 20%. Mean square error is used to evaluate accuracy on both the training and test sets, with a lower error on the test set indicating better generalization of the model to new data. The goal is to accurately forecast future electricity demand based on input variables like population, GDP, price indexes, and past consumption data.
This document describes a study that used fuzzy kohonen clustering network (FKCN) and inverse distance weighting (IDW) to create a building damage hazard zonation for Banda Aceh city, Indonesia. FKCN was used to cluster physical parameter data (lithology, topography, peak ground acceleration) into 3 classes. IDW was then used to interpolate the data and divide the city into 3 zones - low, medium, and high building damage areas. The results showed most of the city is in the low to medium hazard zones, with only a small high hazard zone in the northwest near the coast. This differs from a previous study that found relatively more high hazard areas along the coast.
This document discusses using a fuzzy-neural network to forecast electricity demand. It proposes combining a neural network with fuzzy logic to overcome some limitations of only using artificial neural networks (ANNs). Specifically, it implements a fuzzy logic front-end processor to handle both numeric and fuzzy inputs before feeding them to a three-layer backpropagation neural network. This allows the neural network to capture unknown relationships between input variables like temperature, rain forecast, season and day type with the target output of electricity load. The strengths of this hybrid technique are its ability to incorporate both quantitative and qualitative knowledge and to produce more accurate forecasts.
Black-box modeling of nonlinear system using evolutionary neural NARX modelIJECEIAES
Nonlinear systems with uncertainty and disturbance are very difficult to model using mathematic approach. Therefore, a black-box modeling approach without any prior knowledge is necessary. There are some modeling approaches have been used to develop a black box model such as fuzzy logic, neural network, and evolution algorithms. In this paper, an evolutionary neural network by combining a neural network and a modified differential evolution algorithm is applied to model a nonlinear system. The feasibility and effectiveness of the proposed modeling are tested on a piezoelectric actuator SISO system and an experimental quadruple tank MIMO system.
Optimal neural network models for wind speed predictionIAEME Publication
The document describes using artificial neural networks for wind speed prediction. Specifically, it analyzes the performance of multilayer perceptron networks and radial basis function networks for wind speed forecasting using real-time data collected from wind farms in Coimbatore, India over one year. The models are trained on 3000 samples and tested on 1000 samples. Performance is evaluated using statistical metrics like coefficient of determination, mean absolute error, root mean square error, and mean bias error. Results show that the neural network models improve prediction accuracy compared to other approaches and the optimal model depends on factors like the number of hidden neurons and spread value.
Optimal neural network models for wind speed predictionIAEME Publication
The document presents two neural network models - multilayer perceptron (MLP) and radial basis function (RBF) networks - for wind speed prediction. It evaluates these models on real-time wind speed data collected from wind farms in Coimbatore, India over one year. The experimental results show that RBF and MLP networks can improve wind speed prediction accuracy compared to other approaches, according to statistical metrics like coefficient of determination, mean absolute error, root mean square error and mean bias error. The RBF and MLP models are able to handle the non-linear patterns in wind speed data, which conventional models struggle with, increasing prediction precision.
- The document presents a neural network model for recognizing handwritten digits. It uses a dataset of 20x20 pixel grayscale images of digits 0-9.
- The proposed neural network has an input layer of 400 nodes, a hidden layer of 25 nodes, and an output layer of 10 nodes. It is trained using backpropagation to classify images.
- The model achieves an accuracy of over 96.5% on test data after 200 iterations of training, outperforming a logistic regression model which achieved 91.5% accuracy. Future work could involve classifying more complex natural images.
Web spam classification using supervised artificial neural network algorithmsaciijournal
Due to the rapid growth in technology employed by the spammers, there is a need of classifiers that are more efficient, generic and highly adaptive. Neural Network based technologies have high ability of adaption as well as generalization. As per our knowledge, very little work has been done in this field using neural network. We present this paper to fill this gap. This paper evaluates performance of three supervised learning algorithms of artificial neural network by creating classifiers for the complex problem of latest web spam pattern classification. These algorithms are Conjugate Gradient algorithm, Resilient Backpropagation learning, and Levenberg-Marquardt algorithm.
Predicting rainfall using ensemble of ensemblesVarad Meru
The Paper was done in a group of three for the class project of CS 273: Introduction to Machine Learning at UC Irvine. The group members were Prolok Sundaresan, Varad Meru, and Prateek Jain.
Regression is an approach for modeling the relationship between data X and the dependent variable y. In this report, we present our experiments with multiple approaches, ranging from Ensemble of Learning to Deep Learning Networks on the weather modeling data to predict the rainfall. The competition was held on the online data science competition portal ‘Kaggle’. The results for weighted ensemble of learners gave us a top-10 ranking, with the testing root-mean-squared error being 0.5878.
This document discusses using the Levenberg-Marquardt algorithm for forecasting stock exchange share rates on the Karachi Stock Exchange. It provides an overview of artificial neural networks and how they can be used for financial forecasting applications. The Levenberg-Marquardt algorithm is presented as an efficient method for training neural networks to minimize errors through gradient descent. The document applies this method to train a neural network to predict the direction of change in share prices on the Karachi Stock Exchange. The network is trained on historical stock price data and testing shows it can achieve the performance goal of forecasting next day price changes.
Efficiency of Neural Networks Study in the Design of TrussesIRJET Journal
The document examines the efficiency of different types of artificial neural networks (ANNs) in the design of trusses. It analyzes generalized regression, radial basis function, and linear layer neural networks using the MATLAB neural network tool. Various truss models are analyzed using the ANNs and STAAD Pro software. The ANNs are trained and tested for interpolation and extrapolation to calculate percentage errors. Parameters like spread constants, number of trainings, number of input/output variables are varied to study their effect on the ANN performance and efficiency. The study aims to determine the most suitable ANN type for truss design based on the percentage error results.
This document discusses deep neural networks and computational graphs. It begins by explaining key concepts like derivatives, partial derivatives, optimization, training sets, and activation functions. It then provides examples of applying the chain rule in deep learning, including forward and back propagation in a neural network. Specifically, it demonstrates forward propagation through a simple network and calculating the gradient using backpropagation and the chain rule. Finally, it works through an example applying these concepts to a neural network using sigmoid activation functions.
This document discusses supervised learning in artificial neural networks. It describes how artificial neural networks are modeled after biological neural networks, with nodes that perform functions like neurons. Weights are assigned to nodes and can be updated to optimize the network using techniques like backpropagation. The document specifically examines two neural network tools from Matlab's toolbox that use supervised learning: the fitting tool and pattern recognition tool. It also discusses factors involved in optimizing neural networks, including the basic principles of weights, backpropagation, and multi-layer perceptrons.
Design of airfoil using backpropagation training with mixed approachEditor Jacotech
Levenberg-Marquardt back-propagation training method has some limitations associated with over fitting and local optimum problems. Here, we proposed a new algorithm to increase the convergence speed of Backpropagation learning to design the airfoil. The aerodynamic force coefficients corresponding to series of airfoil are stored in a database along with the airfoil coordinates. A feedforward neural network is created with aerodynamic coefficient as input to produce the airfoil coordinates as output. In the proposed algorithm, for output layer, we used the cost function having linear & nonlinear error terms then for the hidden layer, we used steepest descent cost function. Results indicate that this mixed approach greatly enhances the training of artificial neural network and may accurately predict airfoil profile.
Design of airfoil using backpropagation training with mixed approachEditor Jacotech
The document describes a new algorithm for designing airfoils using neural networks. The algorithm uses a mixed training approach: it trains the output layer of the neural network using a cost function with linear and nonlinear error terms for faster convergence, while training the hidden layer using steepest descent. Results show the mixed approach converges much faster than traditional backpropagation or Levenberg-Marquardt algorithms alone. The algorithm more accurately predicts airfoil profiles with fewer training iterations.
Web Spam Classification Using Supervised Artificial Neural Network Algorithmsaciijournal
Due to the rapid growth in technology employed by the spammers, there is a need of classifiers that are
more efficient, generic and highly adaptive. Neural Network based technologies have high ability of
adaption as well as generalization. As per our knowledge, very little work has been done in this field using
neural network. We present this paper to fill this gap. This paper evaluates performance of three supervised
learning algorithms of artificial neural network by creating classifiers for the complex problem of latest
web spam pattern classification. These algorithms are Conjugate Gradient algorithm, Resilient Backpropagation learning, and Levenberg-Marquardt algorithm.
AN IMPROVED METHOD FOR IDENTIFYING WELL-TEST INTERPRETATION MODEL BASED ON AG...IAEME Publication
This paper presents an approach based on applying an aggregated predictor formed by multiple versions of a multilayer neural network with a back-propagation optimization algorithm for helping the engineer to get a list of the most appropriate well-test interpretation models for a given set of pressure/ production data. The proposed method consists of three stages: (1) data decorrelation through principal component analysis to reduce the covariance between the variables and the dimension of the input layer in the artificial neural network, (2) bootstrap replicates of the learning set where the data is repeatedly sampled with a random split of the data into train sets and using these as new learning sets, and (3) automatic reservoir model identification through aggregated predictor formed by a plurality vote when predicting a new class. This method is described in detail to ensure successful replication of results. The required training and test dataset were generated by using analytical solution models. In our case, there were used 600 samples: 300 for training, 100 for cross-validation, and 200 for testing. Different network structures were tested during this study to arrive at optimum network design. We notice that the single net methodology always brings about confusion in selecting the correct model even though the training results for the constructed networks are close to 1. We notice also that the principal component analysis is an effective strategy in reducing the number of input features, simplifying the network structure, and lowering the training time of the ANN. The results obtained show that the proposed model provides better performance when predicting new data with a coefficient of correlation approximately equal to 95% Compared to a previous approach 80%, the combination of the PCA and ANN is more stable and determine the more accurate results with lesser computational complexity than was feasible previously. Clearly, the aggregated predictor is more stable and shows less bad classes compared to the previous approach.
APPLYING NEURAL NETWORKS FOR SUPERVISED LEARNING OF MEDICAL DATAIJDKP
Constructing a classification model based on some given patterns is a form of learning from the environment perception. This modelling aims to discover new knowledge embedded in the input observations. Learning behaviour of the neural network model enhances the classification properties. This paper considers artificial neural networks for learning two different medical data sets in term of number of instances. The experiment results confirm that the back-propagation supervised learning algorithm has proved its efficiency for such non-linear classification issues.
Optimization of Number of Neurons in the Hidden Layer in Feed Forward Neural ...IJERA Editor
The architectures of Artificial Neural Networks (ANN) are based on the problem domain and it is applied during
the „training phase‟ of sample data and used to infer results for the remaining data in the testing phase.
Normally, the architecture consist of three layers as input, hidden, output layers with the number of nodes in the
input layer as number of known values on hand and the number of nodes as result to be computed out of the
values of input nodes and hidden nodes as the output layer. The number of nodes in the hidden layer is
heuristically decided so that the optimum value is obtained with reasonable number of iterations with other
parameters with its default values. This study mainly focuses on Cascade-Correlation Neural Networks (CCNN)
using Back-Propagation (BP) algorithm which finds the number of neurons during the training phase itself by
appending one from the previous iteration satisfying the error condition gives a promising result on the optimum
number of neurons in the hidden layer
Artificial Neural Networks (ANNS) For Prediction of California Bearing Ratio ...IJMER
The behaviour of soil at the location of the project and interactions of the earth materials during and after construction has a major influence on the success, economy and safety of the work. Another complexity associated with some geotechnical engineering materials, such as sand and gravel, is the difficulty in obtaining undisturbed samples and time consuming involving skilled
technician. Knowledge of California Bearing Ratio (C.B.R) is essential in finding the road thickness. To cope up with the difficulties involved, an attempt has been made to model C.B.R in terms of Fine Fraction, Liquid Limit, Plasticity Index, Maximum Dry density, and Optimum Moisture content. A multi-layer perceptron network with feed forward back propagation is used to model varying the
number of hidden layers. For this purposes 50 soils test data was collected from the laboratory test
results. Among the test data 30 soils data is used for training and remaining 20 soils for testing using
60-40 distribution. The architectures developed are 5-4-1, 5-5-1, and 5-6-1. Model with 5-6-1 architecture is found to be quite satisfactory in predicting C.B.R of soils. A graph is plotted between
the predicted values and observed values of outputs for training and testing process, from the graph it
is found that all the points are close to equality line, indicating predicted values are close to observed
values
Boundness of a neural network weights using the notion of a limit of a sequenceIJDKP
feed forward neural network with backpropagation le
arning algorithm is considered as a black box
learning classifier since there is no certain inter
pretation or anticipation of the behavior of a neur
al
network weights. The weights of a neural network ar
e considered as the learning tool of the classifier
, and
the learning task is performed by the repetition mo
dification of those weights. This modification is
performed using the delta rule which is mainly used
in the gradient descent technique. In this article
a
proof is provided that helps to understand and expl
ain the behavior of the weights in a feed forward n
eural
network with backpropagation learning algorithm. Al
so, it illustrates why a feed forward neural networ
k is
not always guaranteed to converge in a global minim
um. Moreover, the proof shows that the weights in t
he
neural network are upper bounded (i.e. they do not
approach infinity). Data Mining, Delta
Inroduction to Perceptron and how it is used in Machine Learning and Artificial Neural Network.
This presentation is prepared by Zaid Al-husseini, as a lectur for third stage of undergraduate students in Softwrae department - faculity of IT - University of Babylon, Iraq.
It is publicly availabe for the beginners to learn in theory and mathmatically how the Perceptron is working.
Notice: the slides are not detailed. And need a teacher to explain them deeply.
This document summarizes research on improving image classification results using neural networks. It compares common image classification methods like support vector machines (SVM) and K-nearest neighbors (KNN). It then evaluates the performance of multilayer perceptron (MLP) neural networks and radial basis function (RBF) neural networks on image classification. The document tests various configurations of MLP and RBF networks on a dataset containing 2310 images across 7 classes. It finds that a MLP network with two hidden layers of 10 neurons each achieves the best results, with an average accuracy of 98.84%. This is significantly higher than the 84.47% average accuracy of RBF networks and outperforms KNN classification as well. The research concludes that neural
Similar to Hybrid PSO-SA algorithm for training a Neural Network for Classification (19)
Blood finder application project report (1).pdfKamal Acharya
Blood Finder is an emergency time app where a user can search for the blood banks as
well as the registered blood donors around Mumbai. This application also provide an
opportunity for the user of this application to become a registered donor for this user have
to enroll for the donor request from the application itself. If the admin wish to make user
a registered donor, with some of the formalities with the organization it can be done.
Specialization of this application is that the user will not have to register on sign-in for
searching the blood banks and blood donors it can be just done by installing the
application to the mobile.
The purpose of making this application is to save the user’s time for searching blood of
needed blood group during the time of the emergency.
This is an android application developed in Java and XML with the connectivity of
SQLite database. This application will provide most of basic functionality required for an
emergency time application. All the details of Blood banks and Blood donors are stored
in the database i.e. SQLite.
This application allowed the user to get all the information regarding blood banks and
blood donors such as Name, Number, Address, Blood Group, rather than searching it on
the different websites and wasting the precious time. This application is effective and
user friendly.
Open Channel Flow: fluid flow with a free surfaceIndrajeet sahu
Open Channel Flow: This topic focuses on fluid flow with a free surface, such as in rivers, canals, and drainage ditches. Key concepts include the classification of flow types (steady vs. unsteady, uniform vs. non-uniform), hydraulic radius, flow resistance, Manning's equation, critical flow conditions, and energy and momentum principles. It also covers flow measurement techniques, gradually varied flow analysis, and the design of open channels. Understanding these principles is vital for effective water resource management and engineering applications.
Digital Twins Computer Networking Paper Presentation.pptxaryanpankaj78
A Digital Twin in computer networking is a virtual representation of a physical network, used to simulate, analyze, and optimize network performance and reliability. It leverages real-time data to enhance network management, predict issues, and improve decision-making processes.
A high-Speed Communication System is based on the Design of a Bi-NoC Router, ...DharmaBanothu
The Network on Chip (NoC) has emerged as an effective
solution for intercommunication infrastructure within System on
Chip (SoC) designs, overcoming the limitations of traditional
methods that face significant bottlenecks. However, the complexity
of NoC design presents numerous challenges related to
performance metrics such as scalability, latency, power
consumption, and signal integrity. This project addresses the
issues within the router's memory unit and proposes an enhanced
memory structure. To achieve efficient data transfer, FIFO buffers
are implemented in distributed RAM and virtual channels for
FPGA-based NoC. The project introduces advanced FIFO-based
memory units within the NoC router, assessing their performance
in a Bi-directional NoC (Bi-NoC) configuration. The primary
objective is to reduce the router's workload while enhancing the
FIFO internal structure. To further improve data transfer speed,
a Bi-NoC with a self-configurable intercommunication channel is
suggested. Simulation and synthesis results demonstrate
guaranteed throughput, predictable latency, and equitable
network access, showing significant improvement over previous
designs
Levelised Cost of Hydrogen (LCOH) Calculator ManualMassimo Talia
The aim of this manual is to explain the
methodology behind the Levelized Cost of
Hydrogen (LCOH) calculator. Moreover, this
manual also demonstrates how the calculator
can be used for estimating the expenses associated with hydrogen production in Europe
using low-temperature electrolysis considering different sources of electricity
Height and depth gauge linear metrology.pdfq30122000
Height gauges may also be used to measure the height of an object by using the underside of the scriber as the datum. The datum may be permanently fixed or the height gauge may have provision to adjust the scale, this is done by sliding the scale vertically along the body of the height gauge by turning a fine feed screw at the top of the gauge; then with the scriber set to the same level as the base, the scale can be matched to it. This adjustment allows different scribers or probes to be used, as well as adjusting for any errors in a damaged or resharpened probe.
Supermarket Management System Project Report.pdfKamal Acharya
Supermarket management is a stand-alone J2EE using Eclipse Juno program.
This project contains all the necessary required information about maintaining
the supermarket billing system.
The core idea of this project to minimize the paper work and centralize the
data. Here all the communication is taken in secure manner. That is, in this
application the information will be stored in client itself. For further security the
data base is stored in the back-end oracle and so no intruders can access it.
Accident detection system project report.pdfKamal Acharya
The Rapid growth of technology and infrastructure has made our lives easier. The
advent of technology has also increased the traffic hazards and the road accidents take place
frequently which causes huge loss of life and property because of the poor emergency facilities.
Many lives could have been saved if emergency service could get accident information and
reach in time. Our project will provide an optimum solution to this draw back. A piezo electric
sensor can be used as a crash or rollover detector of the vehicle during and after a crash. With
signals from a piezo electric sensor, a severe accident can be recognized. According to this
project when a vehicle meets with an accident immediately piezo electric sensor will detect the
signal or if a car rolls over. Then with the help of GSM module and GPS module, the location
will be sent to the emergency contact. Then after conforming the location necessary action will
be taken. If the person meets with a small accident or if there is no serious threat to anyone’s
life, then the alert message can be terminated by the driver by a switch provided in order to
avoid wasting the valuable time of the medical rescue team.
Hybrid PSO-SA algorithm for training a Neural Network for Classification
1. International Journal of Computer Science, Engineering and Applications (IJCSEA) Vol.1, No.6, December 2011
DOI : 10.5121/ijcsea.2011.1606 73
Hybrid PSO-SA algorithm for training a Neural
Network for Classification
Sriram G. Sanjeevi1
, A. Naga Nikhila2
,Thaseem Khan3
and G. Sumathi4
1
Associate Professor, Dept. of CSE, National Institute of Technology, Warangal, A.P.,
India
sgs@nitw.ac.in
2
Dept. of Comp. Science & Engg., National Institute of Technology, Warangal, A.P.,
India
108nikhila.an@gmail.com
3
Dept. of Comp. Science & Engg., National Institute of Technology, Warangal, A.P.,
India
thaseem7@gmail.com
4
Dept. of Comp. Science & Engg., National Institute of Technology, Warangal, A.P.,
India
sumathiguguloth@gmail.com
ABSTRACT
In this work, we propose a Hybrid particle swarm optimization-Simulated annealing algorithm and present
a comparison with i) Simulated annealing algorithm and ii) Back propagation algorithm for training
neural networks. These neural networks were then tested on a classification task. In particle swarm
optimization behaviour of a particle is influenced by the experiential knowledge of the particle as well as
socially exchanged information. Particle swarm optimization follows a parallel search strategy. In
simulated annealing uphill moves are made in the search space in a stochastic fashion in addition to the
downhill moves. Simulated annealing therefore has better scope of escaping local minima and reach a
global minimum in the search space. Thus simulated annealing gives a selective randomness to the search.
Back propagation algorithm uses gradient descent approach search for minimizing the error. Our goal of
global minima in the task being done here is to come to lowest energy state, where energy state is being
modelled as the sum of the squares of the error between the target and observed output values for all the
training samples. We compared the performance of the neural networks of identical architectures trained
by the i) Hybrid particle swarm optimization-simulated annealing, ii) Simulated annealing and iii) Back
propagation algorithms respectively on a classification task and noted the results obtained. Neural network
trained by Hybrid particle swarm optimization-simulated annealing has given better results compared to
the neural networks trained by the Simulated annealing and Back propagation algorithms in the tests
conducted by us.
KEYWORDS
Classification, Hybrid particle swarm optimization-Simulated annealing, Simulated Annealing, Gradient Descent
Search, Neural Network etc.
1. INTRODUCTION
Classification is an important activity of machine learning. Various algorithms are conventionally
used for classification task namely, Decision tree learning using ID3[1], Concept learning using
2. International Journal of Computer Science, Engineering and Applications (IJCSEA) Vol.1, No.6, December 2011
74
Candidate elimination [2], Neural networks [3], Naïve Bayes classifier [4] are some of the
traditional methods used for classification. Ever since back propagation algorithm was invented
and popularized by [3], [5] and [6] neural networks were actively used for classification.
However, since back-propagation method follows hill climbing approach, it is susceptible to
occurrence of local minima. We examine the use of i) Hybrid particle swarm optimization-
simulated annealing ii) simulated annealing and iii) backpropagation algorithms to train the
neural networks. We study and compare the performance of the neural networks trained by these
three algorithms on a classification task.
2. ARCHITECTURE OF NEURAL NETWORK
Neural network designed for the classification task has the following architecture. It has four
input units, three hidden units and three output units in the input layer, hidden layer and output
layer respectively. Sigmoid activation functions were used with hidden and output units. Figure 1
shows the architectural diagram of the neural network. Neurons are connected in the feed forward
fashion as shown. Neural Network has 12 weights between input and hidden layer and 9 weights
between hidden and output layer. So the total number of weights in the network are 21.
Fig. 1 Architecture of Neural Network used
2.1. Iris data
Neural Network shown in figure 1 is used to perform classification task on IRIS data. The data
was taken from the Univ. of California, Irvine (UCI), Machine learning repository. Iris data
consists of 150 input-output vector pairs. Each input vector consists of a 4 tuple having four
attribute values corresponding to the four input attributes respectively. Based on the input vector,
output vector gives class to which it belongs. Each output vector is a 3 tuple and will have a ‘1’ in
first, second or third positions and zeros in rest two positions, thereby indicating the class to
which the input vector being considered belongs. Hence, we use 1-of-n encoding on the output
side for denoting the class value. The data of 150 input-output pairs is divided randomly into two
parts to create the training set and test set respectively. Data from training set is used to train the
neural network and data from the test set is used for test purposes. Few samples of IRIS data are
shown in table 1.
3. SIMULATED ANNEALING TO TRAIN THE NEURAL NETWORK
Simulated annealing [7], [8] was originally proposed as a process which mimics the physical
process of annealing wherein molten metal is cooled gradually from high energy state attained at
high temperatures to low temperature by following an annealing schedule. Annealing schedule
defines how the temperature is gradually decreased. The goal is to make the metal reach a
minimum attainable energy state. In the simulated annealing lower energy states are produced in
Input Hidden Output
3. International Journal of Computer Science, Engineering and Applications (IJCSEA) Vol.1, No.6, December 2011
75
succeeding transitions. However, a higher energy state is also allowed to be a successor state
from a given current state with a certain probability. Hence, movement to higher energy states is
also allowed. The probability of these possible uphill moves decreases as the temperature is
decreased.
Table 1. Sample of IRIS data used.
S. No. Attr. 1 Attr. 2 Attr. 3 Attr. 4 Class 1 Class 2 Class 3
1 0.224 0.624 0.067 0.043 1 0 0
2 0.749 0.502 0.627 0.541 0 1 0
3 0.557 0.541 0.847 1 0 0 1
4 0.11 0.502 0.051 0.043 1 0 0
5 0.722 0.459 0.663 0.584 0 1 0
While error back propagation algorithm follows the concept of hill climbing, Simulated annealing
allows some down hill movements to be made stochastically, in addition to making the uphill
moves. In the context of a neural network, we actually need to minimize our objective function.
Simulated annealing tries to produce the minimal energy state. It mimics the physical annealing
process wherein molten metal at high temperatures is cooled according to an annealing schedule
and brought down to a low temperature with minimal energy. Simulated annealing can be used to
solve an optimization problem. The objective function required to be minimized for training the
neural network is the error function E.
We define the error function E as follows. The error function E is given by
where x is a specific training example and X is the set of all training examples, denotes the
target output for the kth output neuron corresponding to training sample x, denotes the
observed output for the kth output neuron corresponding to training sample x. Hence, error
function computed is a measure of the sum of the squares of the error between target and
observed output values for all the output neurons, across all the training samples.
3.1. Simulated Annealing Algorithm
Here we describe the Simulated Annealing algorithm used for training the neural network.
1.Initialize the weights of the neural network shown in figure 1 to small random values.
2.Evaluate the objective function E shown in equation (1) for the neural network with the
present configuration of weights used in the neural network.
Follow a temperature annealing schedule with the algorithm.
3.Change one randomly selected weight of the neural network with a random perturbation.
Obtain the objective function E for the neural network with the changed configuration of weights.
Also compute ∆E = (E value for the previous network before weight change) - (E value for the
present network with changed configuration of weights).
4. International Journal of Computer Science, Engineering and Applications (IJCSEA) Vol.1, No.6, December 2011
76
4.If ∆E is positive then new value of E is smaller and therefore better than previous value. Then
accept the new state and make it the current state. else accept the new state (even though it has a
higher E value) with a probability p defined by
p = ( 2)
5.Revise the temperature T as per schedule defined.
6.Go to step 3 if E value obtained is not yet satisfactory else return solution. Solution is the
present configuration of the weights in the neural network.
3.2 Encoding of the Weights in the Neural Network
The Neural Network in figure 1 has 12 weights between input and hidden layer and 9 weights
between hidden and output layer. So the total number of weights in the network are 21. We coded
these weights in binary form. Each weight was assumed to vary between +12.75 and -12.75 with
a precision of 0.05. This is because weights learned by a typical neural network will be small
positive or negative real numbers. Any real number between -12.75 and +12.75 with a precision
of 0.05 can be represented by a 9-bit binary string. Thus, -12.75 was represented with a binary
string ‘000000000’ and +12.75 was represented with ‘111111111’. For example, number 0.1 is
represented by ‘100000010’. Each of the twenty one weights present in the neural network was
represented by a separate 9 bit binary string and the set of all the weights present in the neural
network is represented by their concatenation having 189 bits. These 189 bits represent one
possible assignment of values to the weights in the neural network.
Initially, a random binary string having 189 bits was generated to make the assignment of weights
randomly for the network. We obtain the value of objective function E for this assignment of
weights to the neural network. Simulated Annealing requires a random perturbation to be given to
the network. We do this by selecting randomly one of the 189 binary bits and flipping it. If the
selected bit was 1 it is changed to 0 and if it was 0 it is changed to 1. For the new configuration of
network with weight change objective function E is calculated. Then, ∆E was calculated as (E
value for the previous network before weight change) - (E value for the present network with
changed configuration of weights). Using this approach, simulated annealing algorithm shown in
section 3.1 was executed following a temperature schedule.
The temperature schedule followed is defined as follows:
The starting temperature is taken as T =1050;
Current temp = T/log(iterations+1);
Current temperature was changed after every 100 iterations using above formula.
4. ERROR BACK-PROPAGATION
Error-Back propagation algorithm uses gradient descent search. Gradient descent approach
searches for the weight vector that minimizes error E. Towards this, it starts from an arbitrary
initial weight vector and then changes it in small steps. At each of these steps the weight is altered
in the direction which produces steepest descent along the error surface. Hence, gradient descent
approach follows the hill climbing approach.
The direction of the steepest descent along the error surface is found by computing the derivative
of E with respect to the weight vector w.
Gradient of E with respect to w is = (3)
5. International Journal of Computer Science, Engineering and Applications (IJCSEA) Vol.1, No.6, December 2011
77
4.1 Algorithm for Back-propagation
Since the back propagation algorithm has been introduced [3], [5] it has been used widely as the
training algorithm for training the weights of the feed forward neural network. It was able to
overcome the limitation faced by the perceptron since it can be used for classification for non-
linearly separable problems unlike perceptron. Perceptron can only solve linearly separable
problems. Main disadvantage of neural network using back propagation algorithm is that since it
uses hill climbing approach it can get stuck at local minima. Below we present in table 2 the back
propagation algorithm [9] for the feed forward neural network. It describes the Back propagation
algorithm using batch update wherein weight updates are computed for each weight in the
network after considering all the training samples in a single iteration or epoch. It takes many
iterations or epochs to complete the training of the neural network.
Table 2. Back propagation Algorithm
Back propagation algorithm
Each training sample is a pair of the form ( ), where x is the input vector, and t is the
target vector.
µ is the learning rate, m is the number of network inputs to the neural network, y is the
number of units in the hidden layer , z is the number of output units.
The input from unit i into unit j is denoted by xji , and the weight from unit i into unit j is
denoted by wji .
● Create a feed-forward network with m inputs, y hidden units, and z output units.
● Initialize all network weights to small random numbers.
● Until the termination condition is satisfied, do
● For each in training samples , do
Propagate the input vector forward through the network:
1. Input the instance to the network and compute the output ou of
every unit u in the network.
Propagate the errors backward through the network:
2. For each network output unit k , with target output calculate its
error term
3. For each hidden unit h, calculate its error term
4. For each network weight , do
● For each network weight , do
6. International Journal of Computer Science, Engineering and Applications (IJCSEA) Vol.1, No.6, December 2011
78
5. HYBRID PSO-SA ALGORITHM
In particle swarm optimization [11, 12] a swarm of particles are flown through a
multidimensional search space. Position of each particle represents a potential solution. Position
of each particle is changed by adding a velocity vector to it. Velocity vector is influenced by the
experiential knowledge of the particle as well as socially exchanged information. The
experiential knowledge of a particle A describes the distance of the particle A from its own best
position since the particle A’s first time step. This best position of a particle is referred to as the
personal best of the particle. The global best position in a swarm at a time t is the best position
found in the swarm of particles at the time t. The socially exchanged information of a particle A
describes the distance of a particle A from the global best position in the swarm at time t. The
experiential knowledge and socially exchanged information are also referred to as cognitive and
social components respectively. We propose here the hybrid PSO-SA algorithm in table 3.
Table 3. Hybrid Particle swarm optimization-Simulated annealing algorithm
Create a nx dimensional swarm of ns particles;
repeat
for each particle i = 1, . . .,ns do
// yi denotes the personal best position of the particle i so far
// set the personal best position
if f (xi) < f (yi) then
yi = xi ;
end
// ŷ denotes the global best of the swarm so far
// set the global best position
if f (yi) < f ( ŷ ) then
ŷ = yi ;
end
end
for each particle i = 1, . . ., ns do
update the velocity vi of particle i using equation (4);
vi (t+1) = vi (t) + c1 r1 (t)[yi (t) – xi(t) ] + c2 r2 (t)[ŷ (t) – xi(t)] (4)
// where yi denotes the personal best position of the particle i
// and ŷ denotes the global best position of the swarm
// and xi denotes the present position vector of particle i.
update the position using equation (5);
bi(t) = xi (t) //storing present position
xi (t+1) = xi (t) + vi (t+1) (5)
// applying simulated annealing
compute ∆E = (f (xi) - f(xi (t+1)) (6)
// ∆E = (E value for the previous network before weight change) - (E value
7. International Journal of Computer Science, Engineering and Applications (IJCSEA) Vol.1, No.6, December 2011
79
for the present network with changed configuration of weights).
• if ∆E is positive then
// new value of E is smaller and therefore better than previous value. Then
// accept the new position and make it the current position
else accept the new position xi (t+1) (even though it has a higher E value)
with a probability p defined by
p = (7)
• Revise the temperature T as per schedule defined below.
The temperature schedule followed is defined as follows:
The starting temperature is taken as T =1050;
Current temp = T/log(iterations+1);
Current temperature was changed after every 100 iterations using
above formula.
end
until stopping condition is true ;
In equation (4) of table 3, vi , yi and ŷ are vectors of nx dimensions. vij denotes scalar component
of vi in dimension j. vij is calculated as shown in equation 7 where r1j and r2j are random values in
the range [0,1] and c1 and c2 are learning factors chosen as c1 = c2 = 2.
vij (t+1) = vij (t) + c1 r1j (t)[yij (t) – xij(t) ] + c2 r2j (t)[ŷj (t) – xij) (t)] (8)
5.1 Implementation details of Hybrid PSO-SA algorithm for training a neural
network
We describe here the implementation details of hybrid PSO-SA algorithm for training the neural
network. There are 21 weights in the neural network shown in figure 1.
Hybrid PSO-SA algorithm combines particle swarm optimization algorithm with simulated
annealing approach in the context of training a neural network. The swarm is initialized with a
population of 50 particles. Each particle has 21 weights. Each weight value corresponds to a
position in a particular dimension of the particle. Since there are 21 weights for each particle,
there are therefore 21 dimensions. Hence, position vector of each particle corresponds to a 21
dimensional weight vector. Position (weight) in each dimension is modified by adding velocity
value to it in that dimension.
Each particle’s velocity vector is updated by considering the personal best position of the
particle, global best position of the entire swarm and the present position vector of the particle as
shown in equation (4) of table 3. Velocity of a particle i in dimension j is calculated as shown in
equation (8).
Hybrid PSO-SA algorithm combines pso algorithm with simulated annealing approach. Each of
the 50 particles in the swarm is associated with 21 weights present in the neural network. The
error function E(w) which is a function of the weights in the neural network as defined in
equation (1) is treated as the fitness function. Error E(w) ( fitness function) needs to be
minimized. For each of the 50 particles in the swarm, the solution (position) given by the
8. International Journal of Computer Science, Engineering and Applications (IJCSEA) Vol.1, No.6, December 2011
80
particle is accepted if the change in error function ∆E as defined in equation 6 is positive, since
this indicates error function is reducing in the present iteration compared to the previous iteration
value. If ∆E is not positive then new position is accepted with a probability p given by formula in
equation (7). This is implemented by generating a random number between 0 and 1. If the
number generated is lesser than p then the new position is accepted, else the previous position of
particle is retained without changing. Each particle’s personal best position and the global best
position of the swarm are updated after each iteration. Hybrid PSO-SA algorithm was run for
15000 iterations. After 15000 iterations global best position gbest of the swarm is returned as the
solution. Hence, stopping criterion for the algorithm was chosen as completion of 15000
iterations.
Hybrid PSO-SA algorithm combines parallel search approach of PSO and selective random search
and global search properties of simulated annealing and hence combines the advantages of both
the approaches.
6. EXPERIMENTS AND RESULTS
We have trained the neural network with architecture shown in figure 1 with back propagation
algorithm on training set taken from the IRIS data. Training set was a subset of samples chosen
randomly from the IRIS data. Remaining samples from IRIS data were included in the test set.
Performance was observed on the test data set for predicting the class of each sample.
Table 4. Results of testing for Neural Network using Back propagation
Sl. No Samples in
Training Set
Samples in
Test Set
Correct
classifications
Misclassifications
1 100 50 40 10
2 95 55 37 18
3 85 65 42 23
4 75 75 49 26
We have chosen similarly, training sets of varying sizes from the IRIS data and included each
time the samples which were not selected for training set into test set. Neural network was trained
by each of the training sets and tested the performance of the network on corresponding test sets.
Training was done for 15000 iterations with each of the training sets. The value of learning rate µ
used was 0.2. Results are shown in table 4.
9. International Journal of Computer Science, Engineering and Applications (IJCSEA) Vol.1, No.6, December 2011
81
Table 5. Results of testing for Neural Network using Simulated annealing algorithm
Sl. No Samples in
Training Set
Samples in
Test Set
Correct classifications Misclassifications
1 100 50 44 6
2 95 55 40 15
3 85 65 50 15
4 75 75 59 16
We have also trained the neural network with architecture shown in figure 1 with simulated
annealing algorithm described in section 3, with each of the training sets chosen above in table 4
and tested the performance of the network on corresponding test sets. Training was done using
simulated annealing algorithm for 15000 iterations with each of the training sets. The results are
shown in table 5.
We have also trained the neural network with architecture shown in figure 1 with Hybrid PSO-SA
algorithm described in section 5, with each of the training sets chosen above in table 4 and tested
the performance of the network on corresponding test sets. Training was done using Hybrid PSO-
SA algorithm for 15000 iterations with each of the training sets. The results are shown in table 6.
Table 6. Results of testing for Neural Network using Hybrid PSO-SA algorithm
Three neural networks with same architecture as shown in figure 1 were used for training and
testing with the three algorithms of Hybrid PSO-SA algorithm, simulated annealing algorithm and
back propagation algorithm respectively. Training was performed for same number of 15000
iterations on each neural network. Same training and test sets were used for comparison of neural
networks performance with all the three algorithms. Results of experiments and testing point out
that neural network trained with Hybrid PSO-SA algorithm gives better performance over neural
networks trained with the simulated annealing and back propagation algorithms respectively
across the training and test sets used.
Sl.No Samples in
Training Set
Samples in
Test Set
Correct
Classifications
Misclassifications
1 100 50 47 3
2 95 55 51 4
3 85 65 61 4
4 75 75 70 5
10. International Journal of Computer Science, Engineering and Applications (IJCSEA) Vol.1, No.6, December 2011
82
7. CONCLUSIONS
Our objective was to compare the performance of feed-forward neural network trained with
Hybrid PSO-SA algorithm with the neural networks trained by simulated annealing and back
propagation algorithm respectively. The task we have tested using neural networks trained
separately using these three algorithms is the IRIS data classification. We found that neural
network trained with Hybrid PSO-SA algorithm has given better classification performance
among the three. Neural network trained with Hybrid PSO-SA algorithm combines parallel
search approach of PSO and selective random search and global search properties of simulated
annealing and hence combines the advantages of both the approaches. Hybrid PSO-SA algorithm
has performed better than simulated annealing algorithm using its parallel search strategy with a
swarm of particles. It was able to avoid the local minima by stochastically accepting uphill moves
in addition to the making the normal downhill moves across the error surface. It has given better
classification performance over neural network trained by back-propagation. In general, neural
networks trained by back propagation algorithm are susceptible for falling in local minima.
Hence, Hybrid PSO-SA algorithm has given better results for training a neural network and is a
better alternative to both the Simulated annealing and Back propagation algorithms.
REFERENCES
[1] Quinlan, J.R.(1986). Induction of decision trees. Machine Learning, 1 (1), 81-106.
[2] Mitchell, T.M.(1977). Version spaces: A candidate elimination approach to rule learning. Fifth
International Joint Conference on AI (pp.305-310). Cambridge, MA: MIT Press.
[3] Rumelhart, D.E., & McClelland, J.L.(1986). Parallel distributed processing: exploration in the
microstructure of cognition(Vols.1 & 2). Cambridge, MA : MIT Press.
[4] Duda, R.O., & Hart, P.E.(1973). Pattern classification and scene analysis New York: John Wiley &
Sons.
[5] Werbos, P. (1975). Beyond regression: New tools for prediction and analysis in the behavioral
sciences (Ph.D. dissertation). Harvard University.
[6] Parker, D.(1985). Learning logic (MIT Technical Report TR-47). MIT Center for Research in
Computational Economics and Management Science.
[7] Kirkpatrick, S., Gelatt,Jr., C.D., and M.P. Vecchi 1983. Optimization by simulated annealing. Science
220(4598).
[8] Russel, S., & Norvig, P. (1995). Artificial intelligence: A modern approach. Englewood Cliffs,
NJ:Prentice-Hall.
[9] Mitchell, T.M., 1997.Machine Learning, New York: McGraw-Hill.
[10] Kirkpatrick, S., 1984 “Optimization by simulated annealing: Quantitative Studies,” Journal of
Statistical Physics, vol.34,pp.975-986.
[11] Kennedy, J., Eberhart, R., 1995. “Particle swarm optimization”, IEEE International Conference on
Neural Networks, Perth, WA , (pp.1942 – 1948), Australia.
[12] Engelbrecht, A.P., 2007, Computational Intelligence, England: John Wiley & Sons.
11. International Journal of Computer Science, Engineering and Applications (IJCSEA) Vol.1, No.6, December 2011
83
Authors
Sriram G. Sanjeevi has B.E. from Osmania University, M.Tech from M.I.T.
Manipal, Mangalore University and PhD from IIT, Bombay. He is currently
Associate Professor & Head of Computer Science & Engineering Dept., NIT
Warangal. His research interests are Neural networks, Machine learning and Soft
computing.
A. N. Nikhila has B.Tech(CSE) from NIT Warangal. Her research interests are Neural networks, Machine
learning.
Thaseem Khan has B.Tech(CSE) from NIT Warangal. His research interests are Neural networks, Machine
learning.
G. Sumathi has B.Tech(CSE) from NIT Warangal. Her research interests are Neural networks, Machine
learning.