This document summarizes research on improving image classification results using neural networks. It compares common image classification methods like support vector machines (SVM) and K-nearest neighbors (KNN). It then evaluates the performance of multilayer perceptron (MLP) neural networks and radial basis function (RBF) neural networks on image classification. The document tests various configurations of MLP and RBF networks on a dataset containing 2310 images across 7 classes. It finds that a MLP network with two hidden layers of 10 neurons each achieves the best results, with an average accuracy of 98.84%. This is significantly higher than the 84.47% average accuracy of RBF networks and outperforms KNN classification as well. The research concludes that neural
With the increase in Internet users the number of malicious users are also growing day-by-day posing a
serious problem in distinguishing between normal and abnormal behavior of users in the network. This
has led to the research area of intrusion detection which essentially analyzes the network traffic and tries
to determine normal and abnormal patterns of behavior.In this paper, we have analyzed the standard
NSL-KDD intrusion dataset using some neural network based techniques for predicting possible
intrusions. Four most effective classification methods, namely, Radial Basis Function Network, Self-
Organizing Map, Sequential Minimal Optimization, and Projective Adaptive Resonance Theory have been
applied. In order to enhance the performance of the classifiers, three entropy based feature selection
methods have been applied as preprocessing of data. Performances of different combinations of classifiers
and attribute reduction methods have also been compared.
In recent machine learning community, there is a trend of constructing a linear logarithm version of
nonlinear version through the ‘kernel method’ for example kernel principal component analysis, kernel
fisher discriminant analysis, support Vector Machines (SVMs), and the current kernel clustering
algorithms. Typically, in unsupervised methods of clustering algorithms utilizing kernel method, a
nonlinear mapping is operated initially in order to map the data into a much higher space feature, and then
clustering is executed. A hitch of these kernel clustering algorithms is that the clustering prototype resides
in increased features specs of dimensions and therefore lack intuitive and clear descriptions without
utilizing added approximation of projection from the specs to the data as executed in the literature
presented. This paper aims to utilize the ‘kernel method’, a novel clustering algorithm, founded on the
conventional fuzzy clustering algorithm (FCM) is anticipated and known as kernel fuzzy c-means algorithm
(KFCM). This method embraces a novel kernel-induced metric in the space of data in order to interchange
the novel Euclidean matric norm in cluster prototype and fuzzy clustering algorithm still reside in the space
of data so that the results of clustering could be interpreted and reformulated in the spaces which are
original. This property is used for clustering incomplete data. Execution on supposed data illustrate that
KFCM has improved performance of clustering and stout as compare to other transformations of FCM for
clustering incomplete data.
International Journal of Computational Engineering Research (IJCER) is dedicated to protecting personal information and will make every reasonable effort to handle collected information appropriately. All information collected, as well as related requests, will be handled as carefully and efficiently as possible in accordance with IJCER standards for integrity and objectivity
Investigations on Hybrid Learning in ANFISIJERA Editor
Neural networks have attractiveness to several researchers due to their great closeness to the structure of the brain, their characteristics not shared by many traditional systems. An Artificial Neural Network (ANN) is a network of interconnected artificial processing elements (called neurons) that co-operate with one another in order to solve specific issues. ANNs are inspired by the structure and functional aspects of biological nervous systems. Neural networks, which recognize patterns and adopt themselves to cope with changing environments. Fuzzy inference system incorporates human knowledge and performs inferencing and decision making. The integration of these two complementary approaches together with certain derivative free optimization techniques, results in a novel discipline called Neuro Fuzzy. In Neuro fuzzy development a specific approach is called Adaptive Neuro Fuzzy Inference System (ANFIS), which has shown significant results in modeling nonlinear functions. The basic idea behind the paper is to design a system that uses a fuzzy system to represent knowledge in an interpretable manner and have the learning ability derived from a Runge-Kutta learning method (RKLM) to adjust its membership functions and parameters in order to enhance the system performance. The problem of finding appropriate membership functions and fuzzy rules is often a tiring process of trial and error. It requires users to understand the data before training, which is usually difficult to achieve when the database is relatively large. To overcome these problems, a hybrid of Back Propagation Neural network (BPN) and RKLM can combine the advantages of two systems and avoid their disadvantages.
IJRET : International Journal of Research in Engineering and Technology is an international peer reviewed, online journal published by eSAT Publishing House for the enhancement of research in various disciplines of Engineering and Technology. The aim and scope of the journal is to provide an academic medium and an important reference for the advancement and dissemination of research results that support high-level learning, teaching and research in the fields of Engineering and Technology. We bring together Scientists, Academician, Field Engineers, Scholars and Students of related fields of Engineering and Technology
Architecture neural network deep optimizing based on self organizing feature ...journalBEEI
Forward neural network (FNN) execution relying on the algorithm of training and architecture selection. Different parameters using for nip out the architecture of FNN such as the connections number among strata, neurons hidden number in each strata hidden and hidden strata number. Feature architectural combinations exponential could be uncontrollable manually so specific architecture can be design automatically by using special algorithm which build system with ability generalization better. Determination of architecture FNN can be done by using the algorithm of optimization numerous. In this paper methodology new proposes achievement where FNN neurons respective with hidden layers estimation work where in this work collect algorithm training self organizing feature map (SOFM) with advantages to explain how the best architectural selected automatically by SOFM from criteria error testing based on architecture populated. Different size of dataset benchmark of 4 classifications tested for approach proposed.
Comparison Between Levenberg-Marquardt And Scaled Conjugate Gradient Training...CSCJournals
The Internet paved way for information sharing all over the world decades ago and its popularity for distribution of data has spread like a wildfire ever since. Data in the form of images, sounds, animations and videos is gaining users’ preference in comparison to plain text all across the globe. Despite unprecedented progress in the fields of data storage, computing speed and data transmission speed, the demands of available data and its size (due to the increase in both, quality and quantity) continue to overpower the supply of resources. One of the reasons for this may be how the uncompressed data is compressed in order to send it across the network. This paper compares the two most widely used training algorithms for multilayer perceptron (MLP) image compression – the Levenberg-Marquardt algorithm and the Scaled Conjugate Gradient algorithm. We test the performance of the two training algorithms by compressing the standard test image (Lena or Lenna) in terms of accuracy and speed. Based on our results, we conclude that both algorithms were comparable in terms of speed and accuracy. However, the Levenberg- Marquardt algorithm has shown slightly better performance in terms of accuracy (as found in the average training accuracy and mean squared error), whereas the Scaled Conjugate Gradient algorithm faired better in terms of speed (as found in the average training iteration) on a simple MLP structure (2 hidden layers).
With the increase in Internet users the number of malicious users are also growing day-by-day posing a
serious problem in distinguishing between normal and abnormal behavior of users in the network. This
has led to the research area of intrusion detection which essentially analyzes the network traffic and tries
to determine normal and abnormal patterns of behavior.In this paper, we have analyzed the standard
NSL-KDD intrusion dataset using some neural network based techniques for predicting possible
intrusions. Four most effective classification methods, namely, Radial Basis Function Network, Self-
Organizing Map, Sequential Minimal Optimization, and Projective Adaptive Resonance Theory have been
applied. In order to enhance the performance of the classifiers, three entropy based feature selection
methods have been applied as preprocessing of data. Performances of different combinations of classifiers
and attribute reduction methods have also been compared.
In recent machine learning community, there is a trend of constructing a linear logarithm version of
nonlinear version through the ‘kernel method’ for example kernel principal component analysis, kernel
fisher discriminant analysis, support Vector Machines (SVMs), and the current kernel clustering
algorithms. Typically, in unsupervised methods of clustering algorithms utilizing kernel method, a
nonlinear mapping is operated initially in order to map the data into a much higher space feature, and then
clustering is executed. A hitch of these kernel clustering algorithms is that the clustering prototype resides
in increased features specs of dimensions and therefore lack intuitive and clear descriptions without
utilizing added approximation of projection from the specs to the data as executed in the literature
presented. This paper aims to utilize the ‘kernel method’, a novel clustering algorithm, founded on the
conventional fuzzy clustering algorithm (FCM) is anticipated and known as kernel fuzzy c-means algorithm
(KFCM). This method embraces a novel kernel-induced metric in the space of data in order to interchange
the novel Euclidean matric norm in cluster prototype and fuzzy clustering algorithm still reside in the space
of data so that the results of clustering could be interpreted and reformulated in the spaces which are
original. This property is used for clustering incomplete data. Execution on supposed data illustrate that
KFCM has improved performance of clustering and stout as compare to other transformations of FCM for
clustering incomplete data.
International Journal of Computational Engineering Research (IJCER) is dedicated to protecting personal information and will make every reasonable effort to handle collected information appropriately. All information collected, as well as related requests, will be handled as carefully and efficiently as possible in accordance with IJCER standards for integrity and objectivity
Investigations on Hybrid Learning in ANFISIJERA Editor
Neural networks have attractiveness to several researchers due to their great closeness to the structure of the brain, their characteristics not shared by many traditional systems. An Artificial Neural Network (ANN) is a network of interconnected artificial processing elements (called neurons) that co-operate with one another in order to solve specific issues. ANNs are inspired by the structure and functional aspects of biological nervous systems. Neural networks, which recognize patterns and adopt themselves to cope with changing environments. Fuzzy inference system incorporates human knowledge and performs inferencing and decision making. The integration of these two complementary approaches together with certain derivative free optimization techniques, results in a novel discipline called Neuro Fuzzy. In Neuro fuzzy development a specific approach is called Adaptive Neuro Fuzzy Inference System (ANFIS), which has shown significant results in modeling nonlinear functions. The basic idea behind the paper is to design a system that uses a fuzzy system to represent knowledge in an interpretable manner and have the learning ability derived from a Runge-Kutta learning method (RKLM) to adjust its membership functions and parameters in order to enhance the system performance. The problem of finding appropriate membership functions and fuzzy rules is often a tiring process of trial and error. It requires users to understand the data before training, which is usually difficult to achieve when the database is relatively large. To overcome these problems, a hybrid of Back Propagation Neural network (BPN) and RKLM can combine the advantages of two systems and avoid their disadvantages.
IJRET : International Journal of Research in Engineering and Technology is an international peer reviewed, online journal published by eSAT Publishing House for the enhancement of research in various disciplines of Engineering and Technology. The aim and scope of the journal is to provide an academic medium and an important reference for the advancement and dissemination of research results that support high-level learning, teaching and research in the fields of Engineering and Technology. We bring together Scientists, Academician, Field Engineers, Scholars and Students of related fields of Engineering and Technology
Architecture neural network deep optimizing based on self organizing feature ...journalBEEI
Forward neural network (FNN) execution relying on the algorithm of training and architecture selection. Different parameters using for nip out the architecture of FNN such as the connections number among strata, neurons hidden number in each strata hidden and hidden strata number. Feature architectural combinations exponential could be uncontrollable manually so specific architecture can be design automatically by using special algorithm which build system with ability generalization better. Determination of architecture FNN can be done by using the algorithm of optimization numerous. In this paper methodology new proposes achievement where FNN neurons respective with hidden layers estimation work where in this work collect algorithm training self organizing feature map (SOFM) with advantages to explain how the best architectural selected automatically by SOFM from criteria error testing based on architecture populated. Different size of dataset benchmark of 4 classifications tested for approach proposed.
Comparison Between Levenberg-Marquardt And Scaled Conjugate Gradient Training...CSCJournals
The Internet paved way for information sharing all over the world decades ago and its popularity for distribution of data has spread like a wildfire ever since. Data in the form of images, sounds, animations and videos is gaining users’ preference in comparison to plain text all across the globe. Despite unprecedented progress in the fields of data storage, computing speed and data transmission speed, the demands of available data and its size (due to the increase in both, quality and quantity) continue to overpower the supply of resources. One of the reasons for this may be how the uncompressed data is compressed in order to send it across the network. This paper compares the two most widely used training algorithms for multilayer perceptron (MLP) image compression – the Levenberg-Marquardt algorithm and the Scaled Conjugate Gradient algorithm. We test the performance of the two training algorithms by compressing the standard test image (Lena or Lenna) in terms of accuracy and speed. Based on our results, we conclude that both algorithms were comparable in terms of speed and accuracy. However, the Levenberg- Marquardt algorithm has shown slightly better performance in terms of accuracy (as found in the average training accuracy and mean squared error), whereas the Scaled Conjugate Gradient algorithm faired better in terms of speed (as found in the average training iteration) on a simple MLP structure (2 hidden layers).
A Time Series ANN Approach for Weather Forecastingijctcm
Weather forecasting is most challenging problem around the world. There are various reason because of its experimented values in meteorology, but it is also a typical unbiased time series forecasting problem in scientific research. A lots of methods proposed by various scientists. The motive behind research is to predict more accurate. This paper contribute the same using artificial neural network (ANN) and simulated in MATLAB to predict two important weather parameters i.e. maximum and minimum temperature. The model has been trained using past 60 years of real data collected from(1901-1960) and tested over 40 years to forecast maximum and minimum temperature. The results based on mean square error function (MSE) confirm, this model which is based on multilayer perceptron has the potential to successful application to weather forecasting
Targeted Visual Content Recognition Using Multi-Layer Perceptron Neural Networkijceronline
Visual Content Recognition has become an attractive research oriented field of computer vision and machine learning for the last few decades. The focus of this work is monument recognition. Imagesof significant locations captured and maintainedas data bases can be used by the travelers before visiting the places. They can use images of a famous building to know the description of the building. In all these applications, the visual content recognition plays a key role. Humans can learn the contents of the images and quickly identify them by seeing again. In this paper we present a constructive training algorithm for Multi-Layer Perceptron Neural Network (MLPNN) applied to a set of targeted object recognition applications. The target set consists of famous monuments in India for travel guide applications. The training data set (TDS) consists 3000 images. The Gist features are extracted for the images. These are given to the neural network during training phase.The mean square error (MSE) on the training data is computed and used as metric to adjust the weights of the neural network,using back propagation algorithm. In the constructive learning, if the MSE is less than a predefined value, the number of hidden neurons is increased. Input patterns are trained incrementally until all patterns of TDS are presented and learned. The parameters or weights obtained during the training phase are used in the testing phase, in which new untrained images are given to the neural network for recognition. If the test image is recognized, the details of the image will also be displayed. The performance accuracy of this method is found to be 95%
Black-box modeling of nonlinear system using evolutionary neural NARX modelIJECEIAES
Nonlinear systems with uncertainty and disturbance are very difficult to model using mathematic approach. Therefore, a black-box modeling approach without any prior knowledge is necessary. There are some modeling approaches have been used to develop a black box model such as fuzzy logic, neural network, and evolution algorithms. In this paper, an evolutionary neural network by combining a neural network and a modified differential evolution algorithm is applied to model a nonlinear system. The feasibility and effectiveness of the proposed modeling are tested on a piezoelectric actuator SISO system and an experimental quadruple tank MIMO system.
Fuzzy modeling require two main steps which are structure identification and parameter optimization, the
first one determines the numbers of membership functions and fuzzy if-then rules, while the second
identifies a feasible set of parameters under the given structure. However, the increase of input dimension,
rule numbers will have an exponential growth and there will cause problem of “rule disaster”. In this
paper, we have applied adaptive network fuzzy inference system ANFIS for phonemes recognition. The
appropriate learning algorithm is performed on TIMIT speech database supervised type, a pre-processing
of the acoustic signal and extracting the coefficients MFCCs parameters relevant to the recognition system.
First learning of the network structure by subtractive clustering, in order to define an optimal structure and
obtain small number of rules, then learning of parameters network by hybrid learning which combine the
gradient decent and least square estimation LSE to find a feasible set of antecedents and consequents
parameters. The results obtained show the effectiveness of the method in terms of recognition rate and
number of fuzzy rules generated.
Image Segmentation Using Two Weighted Variable Fuzzy K MeansEditor IJCATR
Image segmentation is the first step in image analysis and pattern recognition. Image segmentation is the process of dividing an image into different regions such that each region is homogeneous. The accurate and effective algorithm for segmenting image is very useful in many fields, especially in medical image. This paper presents a new approach for image segmentation by applying k-means algorithm with two level variable weighting. In image segmentation, clustering algorithms are very popular as they are intuitive and are also easy to implement. The K-means and Fuzzy k-means clustering algorithm is one of the most widely used algorithms in the literature, and many authors successfully compare their new proposal with the results achieved by the k-Means and Fuzzy k-Means. This paper proposes a new clustering algorithm called TW-fuzzy k-means, an automated two-level variable weighting clustering algorithm for segmenting object. In this algorithm, a variable weight is also assigned to each variable on the current partition of data. This could be applied on general images and/or specific images (i.e., medical and microscopic images). The proposed TW-Fuzzy k-means algorithm in terms of providing a better segmentation performance for various type of images. Based on the results obtained, the proposed algorithm gives better visual quality as compared to several other clustering methods.
Hyper-parameter optimization of convolutional neural network based on particl...journalBEEI
Deep neural networks have accomplished enormous progress in tackling many problems. More specifically, convolutional neural network (CNN) is a category of deep networks that have been a dominant technique in computer vision tasks. Despite that these deep neural networks are highly effective; the ideal structure is still an issue that needs a lot of investigation. Deep Convolutional Neural Network model is usually designed manually by trials and repeated tests which enormously constrain its application. Many hyper-parameters of the CNN can affect the model performance. These parameters are depth of the network, numbers of convolutional layers, and numbers of kernels with their sizes. Therefore, it may be a huge challenge to design an appropriate CNN model that uses optimized hyper-parameters and reduces the reliance on manual involvement and domain expertise. In this paper, a design architecture method for CNNs is proposed by utilization of particle swarm optimization (PSO) algorithm to learn the optimal CNN hyper-parameters values. In the experiment, we used Modified National Institute of Standards and Technology (MNIST) database of handwritten digit recognition. The experiments showed that our proposed approach can find an architecture that is competitive to the state-of-the-art models with a testing error of 0.87%.
In this paper, a fruit image data set is used to compare the efficiency and accuracy of two widely used Convolutional Neural Network, namely the ResNet and the DenseNet, for the recognition of 50 different kinds of fruits. In the experiment, the structure of ResNet-34 and DenseNet_BC-121 (with bottleneck layer) are used. The mathematic principle, experiment detail and the experiment result will be explained through comparison.
Optimization as a model for few shot learningKaty Lee
paper presentation of "Optimization as a model for few shot learning" at ICLR 2017 by Sachin Ravi and Hugo Larochelle
highly related to "learning to learn by gradient descent by gradient descent"
Expert system design for elastic scattering neutrons optical model using bpnnijcsa
In present paper, a proposed expert system is designed to obtain a trained formulae for the optical model
parameters used in elastic scattering neutrons of light nuclei for (7Li), at energy range between [(1) to
(20)] MeV. A simple algorithm has used to design this expert system, while a multi-layer backwardpropagation
neural network (BPNN) is applied for training and testing the data used in this model. This
group of formulae may get a simple expert system occurring from governing formulae model, and predicts
the critical parameters usually resulted from the complicated computer coding methods. This expert system
may use in nuclear reactions yields in both fission and fusion nature who gives more closely results to the
real model.
A Time Series ANN Approach for Weather Forecastingijctcm
Weather forecasting is most challenging problem around the world. There are various reason because of its experimented values in meteorology, but it is also a typical unbiased time series forecasting problem in scientific research. A lots of methods proposed by various scientists. The motive behind research is to predict more accurate. This paper contribute the same using artificial neural network (ANN) and simulated in MATLAB to predict two important weather parameters i.e. maximum and minimum temperature. The model has been trained using past 60 years of real data collected from(1901-1960) and tested over 40 years to forecast maximum and minimum temperature. The results based on mean square error function (MSE) confirm, this model which is based on multilayer perceptron has the potential to successful application to weather forecasting
Targeted Visual Content Recognition Using Multi-Layer Perceptron Neural Networkijceronline
Visual Content Recognition has become an attractive research oriented field of computer vision and machine learning for the last few decades. The focus of this work is monument recognition. Imagesof significant locations captured and maintainedas data bases can be used by the travelers before visiting the places. They can use images of a famous building to know the description of the building. In all these applications, the visual content recognition plays a key role. Humans can learn the contents of the images and quickly identify them by seeing again. In this paper we present a constructive training algorithm for Multi-Layer Perceptron Neural Network (MLPNN) applied to a set of targeted object recognition applications. The target set consists of famous monuments in India for travel guide applications. The training data set (TDS) consists 3000 images. The Gist features are extracted for the images. These are given to the neural network during training phase.The mean square error (MSE) on the training data is computed and used as metric to adjust the weights of the neural network,using back propagation algorithm. In the constructive learning, if the MSE is less than a predefined value, the number of hidden neurons is increased. Input patterns are trained incrementally until all patterns of TDS are presented and learned. The parameters or weights obtained during the training phase are used in the testing phase, in which new untrained images are given to the neural network for recognition. If the test image is recognized, the details of the image will also be displayed. The performance accuracy of this method is found to be 95%
Black-box modeling of nonlinear system using evolutionary neural NARX modelIJECEIAES
Nonlinear systems with uncertainty and disturbance are very difficult to model using mathematic approach. Therefore, a black-box modeling approach without any prior knowledge is necessary. There are some modeling approaches have been used to develop a black box model such as fuzzy logic, neural network, and evolution algorithms. In this paper, an evolutionary neural network by combining a neural network and a modified differential evolution algorithm is applied to model a nonlinear system. The feasibility and effectiveness of the proposed modeling are tested on a piezoelectric actuator SISO system and an experimental quadruple tank MIMO system.
Fuzzy modeling require two main steps which are structure identification and parameter optimization, the
first one determines the numbers of membership functions and fuzzy if-then rules, while the second
identifies a feasible set of parameters under the given structure. However, the increase of input dimension,
rule numbers will have an exponential growth and there will cause problem of “rule disaster”. In this
paper, we have applied adaptive network fuzzy inference system ANFIS for phonemes recognition. The
appropriate learning algorithm is performed on TIMIT speech database supervised type, a pre-processing
of the acoustic signal and extracting the coefficients MFCCs parameters relevant to the recognition system.
First learning of the network structure by subtractive clustering, in order to define an optimal structure and
obtain small number of rules, then learning of parameters network by hybrid learning which combine the
gradient decent and least square estimation LSE to find a feasible set of antecedents and consequents
parameters. The results obtained show the effectiveness of the method in terms of recognition rate and
number of fuzzy rules generated.
Image Segmentation Using Two Weighted Variable Fuzzy K MeansEditor IJCATR
Image segmentation is the first step in image analysis and pattern recognition. Image segmentation is the process of dividing an image into different regions such that each region is homogeneous. The accurate and effective algorithm for segmenting image is very useful in many fields, especially in medical image. This paper presents a new approach for image segmentation by applying k-means algorithm with two level variable weighting. In image segmentation, clustering algorithms are very popular as they are intuitive and are also easy to implement. The K-means and Fuzzy k-means clustering algorithm is one of the most widely used algorithms in the literature, and many authors successfully compare their new proposal with the results achieved by the k-Means and Fuzzy k-Means. This paper proposes a new clustering algorithm called TW-fuzzy k-means, an automated two-level variable weighting clustering algorithm for segmenting object. In this algorithm, a variable weight is also assigned to each variable on the current partition of data. This could be applied on general images and/or specific images (i.e., medical and microscopic images). The proposed TW-Fuzzy k-means algorithm in terms of providing a better segmentation performance for various type of images. Based on the results obtained, the proposed algorithm gives better visual quality as compared to several other clustering methods.
Hyper-parameter optimization of convolutional neural network based on particl...journalBEEI
Deep neural networks have accomplished enormous progress in tackling many problems. More specifically, convolutional neural network (CNN) is a category of deep networks that have been a dominant technique in computer vision tasks. Despite that these deep neural networks are highly effective; the ideal structure is still an issue that needs a lot of investigation. Deep Convolutional Neural Network model is usually designed manually by trials and repeated tests which enormously constrain its application. Many hyper-parameters of the CNN can affect the model performance. These parameters are depth of the network, numbers of convolutional layers, and numbers of kernels with their sizes. Therefore, it may be a huge challenge to design an appropriate CNN model that uses optimized hyper-parameters and reduces the reliance on manual involvement and domain expertise. In this paper, a design architecture method for CNNs is proposed by utilization of particle swarm optimization (PSO) algorithm to learn the optimal CNN hyper-parameters values. In the experiment, we used Modified National Institute of Standards and Technology (MNIST) database of handwritten digit recognition. The experiments showed that our proposed approach can find an architecture that is competitive to the state-of-the-art models with a testing error of 0.87%.
In this paper, a fruit image data set is used to compare the efficiency and accuracy of two widely used Convolutional Neural Network, namely the ResNet and the DenseNet, for the recognition of 50 different kinds of fruits. In the experiment, the structure of ResNet-34 and DenseNet_BC-121 (with bottleneck layer) are used. The mathematic principle, experiment detail and the experiment result will be explained through comparison.
Optimization as a model for few shot learningKaty Lee
paper presentation of "Optimization as a model for few shot learning" at ICLR 2017 by Sachin Ravi and Hugo Larochelle
highly related to "learning to learn by gradient descent by gradient descent"
Expert system design for elastic scattering neutrons optical model using bpnnijcsa
In present paper, a proposed expert system is designed to obtain a trained formulae for the optical model
parameters used in elastic scattering neutrons of light nuclei for (7Li), at energy range between [(1) to
(20)] MeV. A simple algorithm has used to design this expert system, while a multi-layer backwardpropagation
neural network (BPNN) is applied for training and testing the data used in this model. This
group of formulae may get a simple expert system occurring from governing formulae model, and predicts
the critical parameters usually resulted from the complicated computer coding methods. This expert system
may use in nuclear reactions yields in both fission and fusion nature who gives more closely results to the
real model.
Fault detection and diagnosis of high speed switching devices in power invertereSAT Journals
Abstract
Power electronic based inverters are the major components in industry. A fault diagnostics framework composed of a pattern recognition system, having machine learning technology as its integral part is utilized for failure detection of different switches and tracing multiple types of faults in an inverter. Hardware point of view power electronics inverter can be considered to be the weakest link. Hence, this work is carried on detecting faults and classifies which switches in the inverter cause the fault. Diagnosis can help to avoid unplanned breakdown, to make possible to run an emergency operation in case of a fault. On the basis of theoretical foundations of electronic power inverter a simulation model has been developed to simulate the healthy condition and all single-switch open circuit faults. The generated signal is processed using Discrete Wavelet Transform (DWT) and Fuzzy Inference Logic (FIL). A smart and accurate classification of faults is obtained using simulation results, which are tested on a wide operation domain and various load conditions.
Keywords: Fault Diagnosis, DWT, Fuzzy Logic, Artificial Intelligence (AI).
89 jan-roger linna - 7032576 - capillary heating control and fault detectio...Mello_Patent_Registry
Jan-Roger Linna, John Paul Mello - Capillary Heating Control and Fault Detection System and Methodology for Fuel System in an Internal Combustion Engine
Neural network based numerical digits recognization using nnt in matlabijcses
Artificial neural networks are models inspired by human nervous system that is capable of learning. One of
the important applications of artificial neural network is character Recognition. Character Recognition
finds its application in number of areas, such as banking, security products, hospitals, in robotics also.
This paper is based on a system that recognizes a english numeral, given by the user, which is already
trained on the features of the numbers to be recognized using NNT (Neural network toolbox) .The system
has a neural network as its core, which is first trained on a database. The training of the neural network
extracts the features of the English numbers and stores in the database. The next phase of the system is to
recognize the number given by the user. The features of the number given by the user are extracted and
compared with the feature database and the recognized number is displayed.
Comparison of Neural Network Training Functions for Hematoma Classification i...IOSR Journals
Classification is one of the most important task in application areas of artificial neural networks
(ANN).Training neural networks is a complex task in the supervised learning field of research. The main
difficulty in adopting ANN is to find the most appropriate combination of learning, transfer and training
function for the classification task. We compared the performances of three types of training algorithms in feed
forward neural network for brain hematoma classification. In this work we have selected Gradient Descent
based backpropagation, Gradient Descent with momentum, Resilence backpropogation algorithms. Under
conjugate based algorithms, Scaled Conjugate back propagation, Conjugate Gradient backpropagation with
Polak-Riebreupdates(CGP) and Conjugate Gradient backpropagation with Fletcher-Reeves updates (CGF).The
last category is Quasi Newton based algorithm, under this BFGS, Levenberg-Marquardt algorithms are
selected. Proposed work compared training algorithm on the basis of mean square error, accuracy, rate of
convergence and correctness of the classification. Our conclusion about the training functions is based on the
simulation results
AN IMPROVED METHOD FOR IDENTIFYING WELL-TEST INTERPRETATION MODEL BASED ON AG...IAEME Publication
This paper presents an approach based on applying an aggregated predictor formed by multiple versions of a multilayer neural network with a back-propagation optimization algorithm for helping the engineer to get a list of the most appropriate well-test interpretation models for a given set of pressure/ production data. The proposed method consists of three stages: (1) data decorrelation through principal component analysis to reduce the covariance between the variables and the dimension of the input layer in the artificial neural network, (2) bootstrap replicates of the learning set where the data is repeatedly sampled with a random split of the data into train sets and using these as new learning sets, and (3) automatic reservoir model identification through aggregated predictor formed by a plurality vote when predicting a new class. This method is described in detail to ensure successful replication of results. The required training and test dataset were generated by using analytical solution models. In our case, there were used 600 samples: 300 for training, 100 for cross-validation, and 200 for testing. Different network structures were tested during this study to arrive at optimum network design. We notice that the single net methodology always brings about confusion in selecting the correct model even though the training results for the constructed networks are close to 1. We notice also that the principal component analysis is an effective strategy in reducing the number of input features, simplifying the network structure, and lowering the training time of the ANN. The results obtained show that the proposed model provides better performance when predicting new data with a coefficient of correlation approximately equal to 95% Compared to a previous approach 80%, the combination of the PCA and ANN is more stable and determine the more accurate results with lesser computational complexity than was feasible previously. Clearly, the aggregated predictor is more stable and shows less bad classes compared to the previous approach.
AN ANN APPROACH FOR NETWORK INTRUSION DETECTION USING ENTROPY BASED FEATURE S...IJNSA Journal
With the increase in Internet users the number of malicious users are also growing day-by-day posing a serious problem in distinguishing between normal and abnormal behavior of users in the network. This has led to the research area of intrusion detection which essentially analyzes the network traffic and tries to determine normal and abnormal patterns of behavior.In this paper, we have analyzed the standard NSL-KDD intrusion dataset using some neural network based techniques for predicting possible intrusions. Four most effective classification methods, namely, Radial Basis Function Network, SelfOrganizing Map, Sequential Minimal Optimization, and Projective Adaptive Resonance Theory have been applied. In order to enhance the performance of the classifiers, three entropy based feature selection methods have been applied as preprocessing of data. Performances of different combinations of classifiers and attribute reduction methods have also been compared.
Web Spam Classification Using Supervised Artificial Neural Network Algorithmsaciijournal
Due to the rapid growth in technology employed by the spammers, there is a need of classifiers that are
more efficient, generic and highly adaptive. Neural Network based technologies have high ability of
adaption as well as generalization. As per our knowledge, very little work has been done in this field using
neural network. We present this paper to fill this gap. This paper evaluates performance of three supervised
learning algorithms of artificial neural network by creating classifiers for the complex problem of latest
web spam pattern classification. These algorithms are Conjugate Gradient algorithm, Resilient Backpropagation learning, and Levenberg-Marquardt algorithm.
Web spam classification using supervised artificial neural network algorithmsaciijournal
Due to the rapid growth in technology employed by the spammers, there is a need of classifiers that are more efficient, generic and highly adaptive. Neural Network based technologies have high ability of adaption as well as generalization. As per our knowledge, very little work has been done in this field using neural network. We present this paper to fill this gap. This paper evaluates performance of three supervised learning algorithms of artificial neural network by creating classifiers for the complex problem of latest web spam pattern classification. These algorithms are Conjugate Gradient algorithm, Resilient Backpropagation learning, and Levenberg-Marquardt algorithm.
X-TREPAN: A MULTI CLASS REGRESSION AND ADAPTED EXTRACTION OF COMPREHENSIBLE D...cscpconf
In this work, the TREPAN algorithm is enhanced and extended for extracting decision trees from neural networks. We empirically evaluated the performance of the algorithm on a set of databases from real world events. This benchmark enhancement was achieved by adapting Single-test TREPAN and C4.5 decision tree induction algorithms to analyze the datasets. The models are then compared with X-TREPAN for comprehensibility and classification accuracy. Furthermore, we validate the experimentations by applying statistical methods. Finally, the modified algorithm is extended to work with multi-class regression problems and the ability to comprehend generalized feed forward networks is achieved.
X-TREPAN : A Multi Class Regression and Adapted Extraction of Comprehensible ...csandit
In this work, the TREPAN algorithm is enhanced and extended for extracting decision trees
from neural networks. We empirically evaluated the performance of the algorithm on a set of
databases from real world events. This benchmark enhancement was achieved by adapting
Single-test TREPAN and C4.5 decision tree induction algorithms to analyze the datasets. The
models are then compared with X-TREPAN for comprehensibility and classification accuracy.
Furthermore, we validate the experimentations by applying statistical methods. Finally, the
modified algorithm is extended to work with multi-class regression problems and the ability to
comprehend generalized feed forward networks is achieved.
Improving face recognition by artificial neural network using principal compo...TELKOMNIKA JOURNAL
The face-recognition system is among the most effective pattern recognition and image analysis techniques. This technique has met great attention from academic and industrial fields because of its extensive use in detecting the identity of individuals for monitoring systems, security and many other practical fields. In this paper, an effective method of face recognition was proposed. Ten person's faces images were selected from ORL dataset, for each person (42) image with total of (420) images as dataset. Features are extracted using principle component analysis PCA to reduce the dimensionality of the face images. Four models where created, the first one was trained using feed forward back propagation learning (FFBBL) with 40 features, the second was trained using 50 features with FFBBL, the third was trained using the same features but using Elman Neural Network. For each person (24) image used as training set for the neural networks, while the remaining images used as testing set. The results showed that the proposed method was effective and highly accurate. FFBBL give accuracy of (98.33,97.14) with (40, 50) features respectively, while Elman gives (98.33, 98.80) for with (40, 50) features respectively.
Q UANTUM C LUSTERING -B ASED F EATURE SUBSET S ELECTION FOR MAMMOGRAPHIC I...ijcsit
In this paper, we present an algorithm for feature selection. This algorithm labeled QC-FS: Quantum
Clustering for Feature Selection performs the selection in two steps. Partitioning the original features
space in order to group similar features is performed using the Quantum Clustering algorithm. Then the
selection of a representative for each cluster is carried out. It uses similarity measures such as correlation
coefficient (CC) and the mutual information (MI). The feature which maximizes this information is chosen
by the algorithm
Image classification is perhaps the most important part of digital image analysis. In this paper, we compare the most widely used model CNN Convolutional Neural Network , and MLP Multilayer Perceptron . We aim to show how both models differ and how both models approach towards the final goal, which is image classification. Souvik Banerjee | Dr. A Rengarajan "Hand-Written Digit Classification" Published in International Journal of Trend in Scientific Research and Development (ijtsrd), ISSN: 2456-6470, Volume-5 | Issue-4 , June 2021, URL: https://www.ijtsrd.compapers/ijtsrd42444.pdf Paper URL: https://www.ijtsrd.comcomputer-science/artificial-intelligence/42444/handwritten-digit-classification/souvik-banerjee
Comparison Between Clustering Algorithms for Microarray Data AnalysisIOSR Journals
Currently, there are two techniques used for large-scale gene-expression profiling; microarray and
RNA-Sequence (RNA-Seq).This paper is intended to study and compare different clustering algorithms that used
in microarray data analysis. Microarray is a DNA molecules array which allows multiple hybridization
experiments to be carried out simultaneously and trace expression levels of thousands of genes. It is a highthroughput
technology for gene expression analysis and becomes an effective tool for biomedical research.
Microarray analysis aims to interpret the data produced from experiments on DNA, RNA, and protein
microarrays, which enable researchers to investigate the expression state of a large number of genes. Data
clustering represents the first and main process in microarray data analysis. The k-means, fuzzy c-mean, selforganizing
map, and hierarchical clustering algorithms are under investigation in this paper. These algorithms
are compared based on their clustering model.
Transcript: Selling digital books in 2024: Insights from industry leaders - T...BookNet Canada
The publishing industry has been selling digital audiobooks and ebooks for over a decade and has found its groove. What’s changed? What has stayed the same? Where do we go from here? Join a group of leading sales peers from across the industry for a conversation about the lessons learned since the popularization of digital books, best practices, digital book supply chain management, and more.
Link to video recording: https://bnctechforum.ca/sessions/selling-digital-books-in-2024-insights-from-industry-leaders/
Presented by BookNet Canada on May 28, 2024, with support from the Department of Canadian Heritage.
Neuro-symbolic is not enough, we need neuro-*semantic*Frank van Harmelen
Neuro-symbolic (NeSy) AI is on the rise. However, simply machine learning on just any symbolic structure is not sufficient to really harvest the gains of NeSy. These will only be gained when the symbolic structures have an actual semantics. I give an operational definition of semantics as “predictable inference”.
All of this illustrated with link prediction over knowledge graphs, but the argument is general.
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...Jeffrey Haguewood
Sidekick Solutions uses Bonterra Impact Management (fka Social Solutions Apricot) and automation solutions to integrate data for business workflows.
We believe integration and automation are essential to user experience and the promise of efficient work through technology. Automation is the critical ingredient to realizing that full vision. We develop integration products and services for Bonterra Case Management software to support the deployment of automations for a variety of use cases.
This video focuses on the notifications, alerts, and approval requests using Slack for Bonterra Impact Management. The solutions covered in this webinar can also be deployed for Microsoft Teams.
Interested in deploying notification automations for Bonterra Impact Management? Contact us at sales@sidekicksolutionsllc.com to discuss next steps.
Key Trends Shaping the Future of Infrastructure.pdfCheryl Hung
Keynote at DIGIT West Expo, Glasgow on 29 May 2024.
Cheryl Hung, ochery.com
Sr Director, Infrastructure Ecosystem, Arm.
The key trends across hardware, cloud and open-source; exploring how these areas are likely to mature and develop over the short and long-term, and then considering how organisations can position themselves to adapt and thrive.
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...UiPathCommunity
💥 Speed, accuracy, and scaling – discover the superpowers of GenAI in action with UiPath Document Understanding and Communications Mining™:
See how to accelerate model training and optimize model performance with active learning
Learn about the latest enhancements to out-of-the-box document processing – with little to no training required
Get an exclusive demo of the new family of UiPath LLMs – GenAI models specialized for processing different types of documents and messages
This is a hands-on session specifically designed for automation developers and AI enthusiasts seeking to enhance their knowledge in leveraging the latest intelligent document processing capabilities offered by UiPath.
Speakers:
👨🏫 Andras Palfi, Senior Product Manager, UiPath
👩🏫 Lenka Dulovicova, Product Program Manager, UiPath
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...DanBrown980551
Do you want to learn how to model and simulate an electrical network from scratch in under an hour?
Then welcome to this PowSyBl workshop, hosted by Rte, the French Transmission System Operator (TSO)!
During the webinar, you will discover the PowSyBl ecosystem as well as handle and study an electrical network through an interactive Python notebook.
PowSyBl is an open source project hosted by LF Energy, which offers a comprehensive set of features for electrical grid modelling and simulation. Among other advanced features, PowSyBl provides:
- A fully editable and extendable library for grid component modelling;
- Visualization tools to display your network;
- Grid simulation tools, such as power flows, security analyses (with or without remedial actions) and sensitivity analyses;
The framework is mostly written in Java, with a Python binding so that Python developers can access PowSyBl functionalities as well.
What you will learn during the webinar:
- For beginners: discover PowSyBl's functionalities through a quick general presentation and the notebook, without needing any expert coding skills;
- For advanced developers: master the skills to efficiently apply PowSyBl functionalities to your real-world scenarios.
DevOps and Testing slides at DASA ConnectKari Kakkonen
My and Rik Marselis slides at 30.5.2024 DASA Connect conference. We discuss about what is testing, then what is agile testing and finally what is Testing in DevOps. Finally we had lovely workshop with the participants trying to find out different ways to think about quality and testing in different parts of the DevOps infinity loop.
State of ICS and IoT Cyber Threat Landscape Report 2024 previewPrayukth K V
The IoT and OT threat landscape report has been prepared by the Threat Research Team at Sectrio using data from Sectrio, cyber threat intelligence farming facilities spread across over 85 cities around the world. In addition, Sectrio also runs AI-based advanced threat and payload engagement facilities that serve as sinks to attract and engage sophisticated threat actors, and newer malware including new variants and latent threats that are at an earlier stage of development.
The latest edition of the OT/ICS and IoT security Threat Landscape Report 2024 also covers:
State of global ICS asset and network exposure
Sectoral targets and attacks as well as the cost of ransom
Global APT activity, AI usage, actor and tactic profiles, and implications
Rise in volumes of AI-powered cyberattacks
Major cyber events in 2024
Malware and malicious payload trends
Cyberattack types and targets
Vulnerability exploit attempts on CVEs
Attacks on counties – USA
Expansion of bot farms – how, where, and why
In-depth analysis of the cyber threat landscape across North America, South America, Europe, APAC, and the Middle East
Why are attacks on smart factories rising?
Cyber risk predictions
Axis of attacks – Europe
Systemic attacks in the Middle East
Download the full report from here:
https://sectrio.com/resources/ot-threat-landscape-reports/sectrio-releases-ot-ics-and-iot-security-threat-landscape-report-2024/
Essentials of Automations: Optimizing FME Workflows with ParametersSafe Software
Are you looking to streamline your workflows and boost your projects’ efficiency? Do you find yourself searching for ways to add flexibility and control over your FME workflows? If so, you’re in the right place.
Join us for an insightful dive into the world of FME parameters, a critical element in optimizing workflow efficiency. This webinar marks the beginning of our three-part “Essentials of Automation” series. This first webinar is designed to equip you with the knowledge and skills to utilize parameters effectively: enhancing the flexibility, maintainability, and user control of your FME projects.
Here’s what you’ll gain:
- Essentials of FME Parameters: Understand the pivotal role of parameters, including Reader/Writer, Transformer, User, and FME Flow categories. Discover how they are the key to unlocking automation and optimization within your workflows.
- Practical Applications in FME Form: Delve into key user parameter types including choice, connections, and file URLs. Allow users to control how a workflow runs, making your workflows more reusable. Learn to import values and deliver the best user experience for your workflows while enhancing accuracy.
- Optimization Strategies in FME Flow: Explore the creation and strategic deployment of parameters in FME Flow, including the use of deployment and geometry parameters, to maximize workflow efficiency.
- Pro Tips for Success: Gain insights on parameterizing connections and leveraging new features like Conditional Visibility for clarity and simplicity.
We’ll wrap up with a glimpse into future webinars, followed by a Q&A session to address your specific questions surrounding this topic.
Don’t miss this opportunity to elevate your FME expertise and drive your projects to new heights of efficiency.
Essentials of Automations: Optimizing FME Workflows with Parameters
F017533540
1. IOSR Journal of Computer Engineering (IOSR-JCE)
e-ISSN: 2278-0661,p-ISSN: 2278-8727, Volume 17, Issue 5, Ver. III (Sep. – Oct. 2015), PP 35-40
www.iosrjournals.org
DOI: 10.9790/0661-17533540 www.iosrjournals.org 35 | Page
Improving Image Classification Result using
Neural Networks
1
Seyed Peyman Zolnouri, 2
Fardad Farokhi, 3
Mehdi Nadiri Andabili
1,2,3
(Department of Electrical Engineering, Central Tehran Branch, Islamic Azad University, Tehran, Iran)
Abstract: Image classification is among the complex processes which depend on several factors. In this paper,
the most common existing methods for image classification are compared first and different structures of
multilayer Perceptron (MLP) and Radial Basis neural networks (RBF) are then reviewed for classification.
Finally, they are compared to non-neural algorithms. The function obtained by using the proposed method
enjoys better results as compared to other methods and it has the best result among other methods with respect
to accuracy with an average accuracy of 98.84%.
Keywords: Image Classification, MLP Neural Network, RBF, KNN
I. Introduction
In the recent years, several methods have been presented for image classification. Some of these
methods include minimum distance, maximum similarity, and support vector machine (SVM). The main steps in
image classification involve determining the appropriate classification system, choosing the appropriate
approach for classification, post-classification processing and accuracy assessment [1].
A successful classification requires an appropriate classification system and sufficient training samples.
Another method which is widely used in image classification is the use of a neural network which includes error
back-propagation networks as well as RBF. In this paper, we modify classification accuracy by MLP. The
results show that the proposed method has the best answer among the referred algorithms. Order of the contents
of paper is such that SVM method which has been already used for classification in previous articles is
discussed in section II. The proposed algorithm which is based on back-propagation neural network is presented
in section III. Simulation and comparison of the obtained results are described in sections VI- VII. Finally, the
conclusion obtained from the paper is presented in section VIII.
II. Image Classification Methods
1. Support Vector Machine (SVM)
SVM is a multivariable machine learning system developed by Vapink [2]. An appropriate method for
data classification includes SVM which uses learning methods with supervisor for data classification. SVM is a
means for estimation of classification which uses machine learning theory, in the manner that the main idea of
SVM is to maximize the margin between the two classes and to minimize the error. In this method, the two
classes are separated by using a linear boundary. A number of learning points with the minimum distance to
decision border can be considered as a subset to define decision boarders and as a support vector. Learning set is
in terms of
N
iii
yx 1
)},{(
with any input for 1i
y . SVM algorithm is calculated as (1) for 0i
.
minimize
N
i
i
cw
1
2
2
1
(1)
Where C is a parameter which is adjusted by user and i
measures the difference between ),( wxa i
and i
y [3].
2. K NEAREST NEIGHBOR (KNN)
Classification K nearest neighbor is the most common and most appropriate method for classification
due to a high understanding and lack of any need to making any hypothesis on the data. In this classification, the
sample existing in the test set belongs to a class with the highest number of votes among k of its nearest
neighbors. Euclidean distance is used to calculate the nearest neighbor of a sample which is in terms of the
following relation.
),(),(
1
txdtxd
m
i
i
eucleucl
(2)
If the amounts are continuous, the amount of eucl
d will be calculated as follows:
2. Improving Image Classification Result using Neural Networks
DOI: 10.9790/0661-17533540 www.iosrjournals.org 36 | Page
2
))()((),( taxatxd ii
i
eucl
(3)
Two major disadvantages of this method are as follows where Determination of k amount by user
which is one of the important points of KNN method and All the nearest neighbors of test sample are
considered with the same degree of importance [4].
3. Neural Network Classification
The book ―Classification Algorithm‖ which was written in 2003 by Landgrebe describes different
image classification algorithms. In the recent years, advanced classification approaches such as artificial neural
networks, fuzzy sets and expert systems which are highly used in image classification have been introduced.
Classification approaches are classified in terms of with and without supervisor, parametric and non-parametric,
soft and hard (fuzzy) classification and pre-pixel approaches. Neural networks are in terms of pre-pixel
algorithms [1].
Neural network is an important means for classification. Researches show that neural networks can be
a substitute alternative for traditional classification methods. Neural networks are used for classification in
industry, trade and science. In most papers, multilayer perceptron neural networks (MLP) are used for
classification. A comparison between neural networks and traditional classification methods which are generally
in terms of statistical methods shows that neural networks for classification are in terms of linear methods and
free models while statistical methods are linear and based on the model [5].
III. Implementation
As we know, type of inputting data and order of data are very important for neural networks so that
they will have a close relation with the final results. For this purpose, a series of processes shall first be made on
the database so that finally the data is prepared to be applied to the network in two groups, namely learning data
and test data. The following steps were taken in order to prepare the aforesaid data.
First step: To make the data uniform, at first total data is changed into numerical data by removing the
character data in Classes section of database and replacing that with numbers. The number attributed to each
class is presented in table(1).
Second step: As we know, if a feature has a constant amount in all instances, it will have no effect on
the network performance. As a result, the third feature called region-pixel-count with a constant amount of 9
was removed at this stage aiming at preprocess of data. In this case, number of features has decreased to 18
features which are in turn useful for the network.
Table 1. Replacing Characters with numbers
Class Assigned Number
BRICKFACE 1
SKY 2
FOLIAGE 3
CEMENT 4
WINDOW 5
PATH 6
GRASS 7
Third step: Then, by surveying each column of features and by finding maximum and minimum
amounts, all the data of that column are normalized by passing through a linear function (4), in the manner that
after completion of this stage, all the amounts will be placed in the interval of {-1,1}.
1
minmax
min
2
x
f (4)
Fourth step: Finally, one-third of all data in which there are equal numbers of each class is
considered as learning data and the rest is considered as test data. Moreover, all learning data are rearranged
randomly to remove network sensitivity to order of instances.
3. Improving Image Classification Result using Neural Networks
DOI: 10.9790/0661-17533540 www.iosrjournals.org 37 | Page
IV. MLP Network With Back Propagation Algorithm
MLP neural networks are among the most widely used and most common methods for classification
of inputs so that Several algorithms have been provided to adjust the parameters of this type of network and
Back Propagation algorithm is one of them which modifies the weighs of network by calculating the error in
terms of returning and calculation of error of each neuron. As shown in Fig.1, MLP networks are constituted of
several sections: inputting section which is directly connected to data, middle layers which are also called
Hidden Layers, output layers which are equal to the number of classes in view of number of neurons, neurons
which are constituted of Activation Functions and weighs which indicate the degree of importance of one input.
Figure 1. A scheme of MLP network
Moreover, BP algorithm adds another parameter to the aforesaid parameters called Learning rate.
Since this type of network and algorithm have equal and predefined principles, the difference seen in the results
is due to the difference in the amounts of network parameters and adjusting them by error will result in
questioning the type of network. Considering the aforesaid issues, the amounts of parameters are explained.
Input: number of inputs is always equal to the number of features and here it is 18.
Number of internal layers: In this paper, number of hidden layers starts from one layer and will finally end
in two layers.
Number of neurons of external layer is equal to the number of classes which is 7 for this database.
Number of neurons of hidden layers is 5 and 10 neurons for each hidden layer, respectively.
Moreover, in case activator function is Antisymmetric (symmetric against the origin, odd function);
learning rate with BP algorithm will be higher. As we know, the standard logistic function has not this form
while hyperbolic tangent function with a=1.7159 and b=2/3 coefficients enjoys these conditions [6]. First of all,
some amounts should be considered for the weighs to start program performance. It is better that these amounts
have zero averages and variance amounts of weighs are equal to 2
1
mw
(m: number of synoptic relations of
a neuron). Here, initial amounts of weighs are selected within an interval of [-0.5 +0.5].
In addition to the above items, network learning method can also play an important role in the final
results. Basically, Sequential learning method in which weighs are modified after providing each example, will
offer better results in this algorithm. Further to this, training is preferred to sequential method in view of
operations because it requires less local storage of weighs for each connection. Moreover, searching for a weight
space is made randomly by providing instances to the network on a natural basis. This in turn lessens the
probability of network to be trapped in the local minimums. This is while in Batch Learning method, weighs are
modified after providing all the examples. However, learning speed in both methods depends on the parameter
of learning rate. So that learning speed decreases by having a low learning rate and vice versa. Here, learning
rate starts from 0.01 and then decreases in the middle of training.
V. Radial Basis Function Network
In fact, this type of network is a special type of MLP network with only one hidden layer which is
shown in Fig.2. Moreover, in contrast with BP algorithm, modification of network parameters starts from input
towards output. As it is clear from its name, the main kernel of the neurons of this network is developed based
on radial functions such as Gaussian Functions (3). Since the number of hidden layers is limited to one layer, the
most important parameters of this network are therefore limited to Number of neurons or internal layer kernels,
type of activator function in each neuron, Radius and centers of activator functions and weighs.
4. Improving Image Classification Result using Neural Networks
DOI: 10.9790/0661-17533540 www.iosrjournals.org 38 | Page
Figure 2. RBF Network
In this paper, RBF networks are tested with 10, 15 and 20 neurons all of which use Gaussian
Functions. Moreover, with the initial amounts, the centers are randomly selected from several examples based
on which the initial amounts of variance or radius of each neuron are calculated considering (5)
M
d
2
(5)
where d is the space between selected centers and M is the number of neurons. Furthermore, initial
amounts of weighs are similar to MLP-BP network. Since three parameters are modified during learning
process, three parameters are required to determine learning rate. According to the obtained results, the less the
learning rate is for the centers, the better the results of the network will be. Here, three initial amounts are
considered for the three mentioned parameters according to table(2).
Table 2. RBP Learning rates
ValueLearning Rate
0.001Weight rate
0.0001Center rate
0.001Variance rate
VI. K-Nearest Neighbors Algorithm
As it was said at the beginning, this algorithm is outside of neural networks. In fact, this method acts
based on Euclidean distance between the samples. Despite neural networks which have two processes, namely
training and testing, this method performs all stages within just one stage. The goal of using this algorithm is to
establish a basis for the networks used in this paper. In addition, different nature of this method results in
application of a series of different processes in establishing the inputting data. Since the best results obtained by
this algorithm were examined in testing the studied database based on K=3 neighborhood, this amount will be
considered as a basis for comparison.
VII. Results
1. Image classification database
UCI image classification database is used to check the results of the proposed method. The database has
2310 instances and 7 output classes. Each instance is in terms of a 3*3 area with 19 features [3] .
2. Simulation
Different structures of networks are tested, in the manner that the aforesaid database is used for all of
them. Finally, after comparing the results of RBF and MLP-BP networks, the best results of which are compared
to the results of KNN algorithm. First of all, accuracy results are provided for 4 different MLP-BP neural
networks, so that the first one has one hidden layer with 5 neurons and the second one has the same layer with
10 neurons. This is while by adding another layer to the first and second network with the same number of
neurons, the third and fourth networks are obtained, respectively. As seen in Fig.3, the network has offered the
Best result among other configurations with two hidden layers each with 10 neurons. However, from practical
viewpoint, considering the results of the second network and since it has less layers and less neurons and as a
result, it needs a smaller memory, it is preferred to other configurations for implementation in the embedded
systems.
5. Improving Image Classification Result using Neural Networks
DOI: 10.9790/0661-17533540 www.iosrjournals.org 39 | Page
Figure 3. MLP Network Results
The results for three different configurations of RBF network are then described, in the manner that
the first one has 10 neurons and the second and third ones have 15 and 20 neurons, respectively. As seen in
Fig.4, the third configuration is better in four classes as compared to other configurations and it is a little
different with the first and second networks among others, but generally, RBF network with an average
accuracy of 84.47% is significantly different with MLP-BP network with an average accuracy of 98.84%, in the
manner that in order to prove the function of MLP-BP network, its results are provided in table(3) beside the
results of KNN algorithm. By examining the results in table(3), it is concluded that MLP-BP network has had a
better function even better than KNN algorithm which is based on Euclidean distance, in the manner that it
almost has 100% accuracy in four classes and their minimum for other classes is 96.7%. Moreover, by
comparing the results in [3], it can be seen that our proposed method has a better function compared to that
method.
Figure 4. RBF Network Results
Table 3. MLP versus KNN results
Classes
7 6 5 4 3 2 1
MLP
Sensitivity 0.985 1.00 0.845 0.94 0.95 1.00 0.995
Specificity 0.996 1.00 0.997 0.988 0.97 0.999 1.00
Accuracy 0.995 1.00 0.975 0.981 0.969 0.999 0.999
KNN
Sensitivity 0.965 1.00 0.86 0.810 0.930 1.00 0.995
Specificity 0.994 1.00 0.986 0.992 0.961 0.991 1.00
Accuracy 0.990 1.00 0.968 0.966 0.957 0.992 0.999
6. Improving Image Classification Result using Neural Networks
DOI: 10.9790/0661-17533540 www.iosrjournals.org 40 | Page
VIII. Conclusion
In this paper, a new look is given to image classification using neural networks. Furthermore, by
using different networks with variable configurations and also by comparing the results With non-neural
algorithms, it is shown that MLP-BP algorithm has a better function as compared to other aforesaid networks.
The importance of this issue is further shown by describing data preparation method for the network. In
addition, as the future work, we first want to completely apply the preprocessed methods on the database to
decrease the number of features if there is any dependency between the features. We also want to use
evolutionary neural networks such as genetic algorithms to increase the space for searching for number of layers
and number neurons in MLP-BP networks.
References
[1] D. LU,Q. WENG, A survey of image classification methods and techniques for improving classification performance, International
Journal of Remote Sensing 2007, 823–870.
[2] M.He, F. M.Imran, B. Belkacem, and Sh.Mei , IMPROVING HYPERSPECTRAL IMAGE CLASSIFICATION ACCURACY
USING ITERATIVE SVM WITH SPATIAL-SPECTRAL INFORMATION, IEEE conference, 2013.
[3] J.Tin,Y.Kwok, Moderating the Outputs of Support Vector Machine Classifiers, IEEE Conference,1999.
[4] N.Verbiest, C.Cornelis, and R.Jensen, Fuzzy rough Positive region based Nearest Neighbour Classificaton , IEEE World congress
on Computational Intelligence, 2012.
[5] G.P.Zhang, Neural Networks for Classification: A Survey ,IEEE TRANSACTIONS ON SYSTEMS, MAN, AND
CYBERNETICS—PART C: APPLICATIONS AND REVIEWS, 2000.
[6] Simon Haykin, Multilayer Perceptrons (in Neural Networks, 2th ed. New Jersey, Prentice Hall, 1999),156-178.