This document compares the performance of two neural network architectures, multi-layer perceptron (MLP) and radial basis function (RBF) networks, on a face recognition system. It trains MLP networks using different variants of the backpropagation algorithm and compares the results to RBF networks. The document finds that RBF networks provide better generalization performance compared to backpropagation algorithms and have faster training times, making them more suitable for face recognition.
International Journal of Engineering Research and Applications (IJERA) is an open access online peer reviewed international journal that publishes research and review articles in the fields of Computer Science, Neural Networks, Electrical Engineering, Software Engineering, Information Technology, Mechanical Engineering, Chemical Engineering, Plastic Engineering, Food Technology, Textile Engineering, Nano Technology & science, Power Electronics, Electronics & Communication Engineering, Computational mathematics, Image processing, Civil Engineering, Structural Engineering, Environmental Engineering, VLSI Testing & Low Power VLSI Design etc.
A Study of BFLOAT16 for Deep Learning TrainingSubhajit Sahu
Highlighted notes of:
A Study of BFLOAT16 for Deep Learning Training
This paper presents the first comprehensive empirical study demonstrating the efficacy of the Brain Floating Point (BFLOAT16) half-precision format for DeepLearning training across image classification, speech recognition, language model-ing, generative networks, and industrial recommendation systems. BFLOAT16 is attractive for Deep Learning training for two reasons: the range of values it can represent is the same as that of IEEE 754 floating-point format (FP32) and conversion to/from FP32 is simple. Maintaining the same range as FP32 is important to ensure that no hyper-parameter tuning is required for convergence; e.g., IEEE 754compliant half-precision floating point (FP16) requires hyper-parameter tuning. In this paper, we discuss the flow of tensors and various key operations in mixed-precision training and delve into details of operations, such as the rounding modes for converting FP32 tensors to BFLOAT16. We have implemented a method to emulate BFLOAT16 operations in Tensorflow, Caffe2, IntelCaffe, and Neon for our experiments. Our results show that deep learning training using BFLOAT16tensors achieves the same state-of-the-art (SOTA) results across domains as FP32tensors in the same number of iterations and with no changes to hyper-parameters.
1. The document presents a hybrid algorithm that combines Kernelized Fuzzy C-Means (KFCM), Hybrid Ant Colony Optimization (HACO), and Fuzzy Adaptive Particle Swarm Optimization (FAPSO) to improve clustering of electrocardiogram (ECG) beat data.
2. The algorithm maps data into a higher dimensional space using kernel functions to make clusters more linearly separable, addresses issues with KFCM being sensitive to initialization and prone to local minima.
3. It uses HACO to optimize cluster centers and membership degrees, and FAPSO to evaluate fitness values and optimize weight vectors, forming usable clusters for applications like ECG classification.
Investigations on Hybrid Learning in ANFISIJERA Editor
Neural networks have attractiveness to several researchers due to their great closeness to the structure of the brain, their characteristics not shared by many traditional systems. An Artificial Neural Network (ANN) is a network of interconnected artificial processing elements (called neurons) that co-operate with one another in order to solve specific issues. ANNs are inspired by the structure and functional aspects of biological nervous systems. Neural networks, which recognize patterns and adopt themselves to cope with changing environments. Fuzzy inference system incorporates human knowledge and performs inferencing and decision making. The integration of these two complementary approaches together with certain derivative free optimization techniques, results in a novel discipline called Neuro Fuzzy. In Neuro fuzzy development a specific approach is called Adaptive Neuro Fuzzy Inference System (ANFIS), which has shown significant results in modeling nonlinear functions. The basic idea behind the paper is to design a system that uses a fuzzy system to represent knowledge in an interpretable manner and have the learning ability derived from a Runge-Kutta learning method (RKLM) to adjust its membership functions and parameters in order to enhance the system performance. The problem of finding appropriate membership functions and fuzzy rules is often a tiring process of trial and error. It requires users to understand the data before training, which is usually difficult to achieve when the database is relatively large. To overcome these problems, a hybrid of Back Propagation Neural network (BPN) and RKLM can combine the advantages of two systems and avoid their disadvantages.
Efficient design of feedforward network for pattern classificationIOSR Journals
This document compares the performance of radial basis function (RBF) networks and multi-layer perceptron (MLP) networks for pattern classification tasks. It analyzes the training time of RBF and MLP networks on two datasets: a below poverty line (BPL) dataset with 293 samples and 13 features, and a breast cancer dataset with 699 samples and 9 features. For both datasets, RBF networks trained significantly faster than MLP networks using the same number of hidden neurons, without affecting classification performance. The document concludes that RBF networks perform training faster than MLP networks for these pattern classification problems.
This document discusses using the Levenberg-Marquardt algorithm for forecasting stock exchange share rates on the Karachi Stock Exchange. It provides an overview of artificial neural networks and how they can be used for financial forecasting applications. The Levenberg-Marquardt algorithm is presented as an efficient method for training neural networks to minimize errors through gradient descent. The document applies this method to train a neural network to predict the direction of change in share prices on the Karachi Stock Exchange. The network is trained on historical stock price data and testing shows it can achieve the performance goal of forecasting next day price changes.
BACKPROPAGATION LEARNING ALGORITHM BASED ON LEVENBERG MARQUARDT ALGORITHMcscpconf
Data Mining aims at discovering knowledge out of data and presenting it in a form that is easily
compressible to humans. Data Mining represents a process developed to examine large amounts
of data routinely collected. The term also refers to a collection of tools used to perform the
process. One of the useful applications in the field of medicine is the incurable chronic disease
diabetes. Data Mining algorithm is used for testing the accuracy in predicting diabetic status.
Fuzzy Systems are been used for solving a wide range of problems in different application
domain and Genetic Algorithm for designing. Fuzzy systems allows in introducing the learning
and adaptation capabilities. Neural Networks are efficiently used for learning membership
functions. Diabetes occurs throughout the world, but Type 2 is more common in the most
developed countries. The greater increase in prevalence is however expected in Asia and Africa
where most patients will likely be found by 2030. This paper is proposed on the Levenberg –
Marquardt algorithm which is specifically designed to minimize sum-of-square error functions.
Levernberg-Marquardt algorithm gives the best performance in the prediction of diabetes compared to any other backpropogation algorithm
Black-box modeling of nonlinear system using evolutionary neural NARX modelIJECEIAES
Nonlinear systems with uncertainty and disturbance are very difficult to model using mathematic approach. Therefore, a black-box modeling approach without any prior knowledge is necessary. There are some modeling approaches have been used to develop a black box model such as fuzzy logic, neural network, and evolution algorithms. In this paper, an evolutionary neural network by combining a neural network and a modified differential evolution algorithm is applied to model a nonlinear system. The feasibility and effectiveness of the proposed modeling are tested on a piezoelectric actuator SISO system and an experimental quadruple tank MIMO system.
International Journal of Engineering Research and Applications (IJERA) is an open access online peer reviewed international journal that publishes research and review articles in the fields of Computer Science, Neural Networks, Electrical Engineering, Software Engineering, Information Technology, Mechanical Engineering, Chemical Engineering, Plastic Engineering, Food Technology, Textile Engineering, Nano Technology & science, Power Electronics, Electronics & Communication Engineering, Computational mathematics, Image processing, Civil Engineering, Structural Engineering, Environmental Engineering, VLSI Testing & Low Power VLSI Design etc.
A Study of BFLOAT16 for Deep Learning TrainingSubhajit Sahu
Highlighted notes of:
A Study of BFLOAT16 for Deep Learning Training
This paper presents the first comprehensive empirical study demonstrating the efficacy of the Brain Floating Point (BFLOAT16) half-precision format for DeepLearning training across image classification, speech recognition, language model-ing, generative networks, and industrial recommendation systems. BFLOAT16 is attractive for Deep Learning training for two reasons: the range of values it can represent is the same as that of IEEE 754 floating-point format (FP32) and conversion to/from FP32 is simple. Maintaining the same range as FP32 is important to ensure that no hyper-parameter tuning is required for convergence; e.g., IEEE 754compliant half-precision floating point (FP16) requires hyper-parameter tuning. In this paper, we discuss the flow of tensors and various key operations in mixed-precision training and delve into details of operations, such as the rounding modes for converting FP32 tensors to BFLOAT16. We have implemented a method to emulate BFLOAT16 operations in Tensorflow, Caffe2, IntelCaffe, and Neon for our experiments. Our results show that deep learning training using BFLOAT16tensors achieves the same state-of-the-art (SOTA) results across domains as FP32tensors in the same number of iterations and with no changes to hyper-parameters.
1. The document presents a hybrid algorithm that combines Kernelized Fuzzy C-Means (KFCM), Hybrid Ant Colony Optimization (HACO), and Fuzzy Adaptive Particle Swarm Optimization (FAPSO) to improve clustering of electrocardiogram (ECG) beat data.
2. The algorithm maps data into a higher dimensional space using kernel functions to make clusters more linearly separable, addresses issues with KFCM being sensitive to initialization and prone to local minima.
3. It uses HACO to optimize cluster centers and membership degrees, and FAPSO to evaluate fitness values and optimize weight vectors, forming usable clusters for applications like ECG classification.
Investigations on Hybrid Learning in ANFISIJERA Editor
Neural networks have attractiveness to several researchers due to their great closeness to the structure of the brain, their characteristics not shared by many traditional systems. An Artificial Neural Network (ANN) is a network of interconnected artificial processing elements (called neurons) that co-operate with one another in order to solve specific issues. ANNs are inspired by the structure and functional aspects of biological nervous systems. Neural networks, which recognize patterns and adopt themselves to cope with changing environments. Fuzzy inference system incorporates human knowledge and performs inferencing and decision making. The integration of these two complementary approaches together with certain derivative free optimization techniques, results in a novel discipline called Neuro Fuzzy. In Neuro fuzzy development a specific approach is called Adaptive Neuro Fuzzy Inference System (ANFIS), which has shown significant results in modeling nonlinear functions. The basic idea behind the paper is to design a system that uses a fuzzy system to represent knowledge in an interpretable manner and have the learning ability derived from a Runge-Kutta learning method (RKLM) to adjust its membership functions and parameters in order to enhance the system performance. The problem of finding appropriate membership functions and fuzzy rules is often a tiring process of trial and error. It requires users to understand the data before training, which is usually difficult to achieve when the database is relatively large. To overcome these problems, a hybrid of Back Propagation Neural network (BPN) and RKLM can combine the advantages of two systems and avoid their disadvantages.
Efficient design of feedforward network for pattern classificationIOSR Journals
This document compares the performance of radial basis function (RBF) networks and multi-layer perceptron (MLP) networks for pattern classification tasks. It analyzes the training time of RBF and MLP networks on two datasets: a below poverty line (BPL) dataset with 293 samples and 13 features, and a breast cancer dataset with 699 samples and 9 features. For both datasets, RBF networks trained significantly faster than MLP networks using the same number of hidden neurons, without affecting classification performance. The document concludes that RBF networks perform training faster than MLP networks for these pattern classification problems.
This document discusses using the Levenberg-Marquardt algorithm for forecasting stock exchange share rates on the Karachi Stock Exchange. It provides an overview of artificial neural networks and how they can be used for financial forecasting applications. The Levenberg-Marquardt algorithm is presented as an efficient method for training neural networks to minimize errors through gradient descent. The document applies this method to train a neural network to predict the direction of change in share prices on the Karachi Stock Exchange. The network is trained on historical stock price data and testing shows it can achieve the performance goal of forecasting next day price changes.
BACKPROPAGATION LEARNING ALGORITHM BASED ON LEVENBERG MARQUARDT ALGORITHMcscpconf
Data Mining aims at discovering knowledge out of data and presenting it in a form that is easily
compressible to humans. Data Mining represents a process developed to examine large amounts
of data routinely collected. The term also refers to a collection of tools used to perform the
process. One of the useful applications in the field of medicine is the incurable chronic disease
diabetes. Data Mining algorithm is used for testing the accuracy in predicting diabetic status.
Fuzzy Systems are been used for solving a wide range of problems in different application
domain and Genetic Algorithm for designing. Fuzzy systems allows in introducing the learning
and adaptation capabilities. Neural Networks are efficiently used for learning membership
functions. Diabetes occurs throughout the world, but Type 2 is more common in the most
developed countries. The greater increase in prevalence is however expected in Asia and Africa
where most patients will likely be found by 2030. This paper is proposed on the Levenberg –
Marquardt algorithm which is specifically designed to minimize sum-of-square error functions.
Levernberg-Marquardt algorithm gives the best performance in the prediction of diabetes compared to any other backpropogation algorithm
Black-box modeling of nonlinear system using evolutionary neural NARX modelIJECEIAES
Nonlinear systems with uncertainty and disturbance are very difficult to model using mathematic approach. Therefore, a black-box modeling approach without any prior knowledge is necessary. There are some modeling approaches have been used to develop a black box model such as fuzzy logic, neural network, and evolution algorithms. In this paper, an evolutionary neural network by combining a neural network and a modified differential evolution algorithm is applied to model a nonlinear system. The feasibility and effectiveness of the proposed modeling are tested on a piezoelectric actuator SISO system and an experimental quadruple tank MIMO system.
Implementation Of Back-Propagation Neural Network For Isolated Bangla Speech ...ijistjournal
This paper is concerned with the development of Back-propagation Neural Network for Bangla Speech Recognition. In this paper, ten bangla digits were recorded from ten speakers and have been recognized. The features of these speech digits were extracted by the method of Mel Frequency Cepstral Coefficient (MFCC) analysis. The mfcc features of five speakers were used to train the network with Back propagation algorithm. The mfcc features of ten bangla digit speeches, from 0 to 9, of another five speakers were used to test the system. All the methods and algorithms used in this research were implemented using the features of Turbo C and C++ languages. From our investigation it is seen that the developed system can successfully encode and analyze the mfcc features of the speech signal to recognition. The developed system achieved recognition rate about 96.332% for known speakers (i.e., speaker dependent) and 92% for unknown speakers (i.e., speaker independent).
Implementation of Fuzzy Logic for the High-Resolution Remote Sensing Images w...IOSR Journals
This document describes an implementation of fuzzy logic for high-resolution remote sensing image classification with improved accuracy. It discusses using an object-based approach with fuzzy rules to classify urban land covers in a satellite image. The approach involves image segmentation using k-means clustering or ISODATA clustering. Features are then extracted from the image objects and fuzzy logic is applied to classify the objects based on membership functions. The method was tested on different sensor and resolution images in MATLAB and showed improved classification accuracy over other techniques, achieving lower entropy in results. Future work planned includes designing an unsupervised classification model combining k-means clustering and fuzzy-based object orientation.
Architecture neural network deep optimizing based on self organizing feature ...journalBEEI
Forward neural network (FNN) execution relying on the algorithm of training and architecture selection. Different parameters using for nip out the architecture of FNN such as the connections number among strata, neurons hidden number in each strata hidden and hidden strata number. Feature architectural combinations exponential could be uncontrollable manually so specific architecture can be design automatically by using special algorithm which build system with ability generalization better. Determination of architecture FNN can be done by using the algorithm of optimization numerous. In this paper methodology new proposes achievement where FNN neurons respective with hidden layers estimation work where in this work collect algorithm training self organizing feature map (SOFM) with advantages to explain how the best architectural selected automatically by SOFM from criteria error testing based on architecture populated. Different size of dataset benchmark of 4 classifications tested for approach proposed.
Hybrid multi objectives genetic algorithms and immigrants scheme for dynamic ...khalil IBRAHIM
the main concept of intelligent optimization techniques, artificial neural networks, and new genetic algorithms to solve the multi-objective multicast routing problems with shortest path (SP) problem that used in the addresses networks and improve all processes addressing in the wireless communications based on multi-objective optimization. The most important characteristics in mobile wireless networks is the topology dynamics and the network topology changes over time, the routing problem (SPRP) in mobile ad hoc networks (MANETs) turns out to be a dynamic optimization problem[13], the hybrid immigrants multiple-objective genetic algorithm (HIMOGAs) in the real- world are dynamic in nature, that has objective functions, constraints, and parameters, the dynamic optimization problems (DOPs) are a big challenges to evolutionary multi-objective, since any environmental change may affect the objective vector, constraints, and parameters, HIMOGA for the optimization goal is to track the moving of parameters and get a sequence of approximations solutions over time. The quantity of services (QoS) is supporting guarantee for all data traffic and getting the maximizing utilization for network, the QoS based on multicast routing offers significant challenges, and increases to use an efficient multicast routing protocol that will be able to check multicast routing and satisfying QoS constraints, The author propose to use HIMOGAs and SP algorithm to solve multicast problem that produces new generation wireless networks with immigrants schema to get high-quality solutions after each change and satisfying all objectives.
This document summarizes an FPGA implementation of a trained neural network. It describes implementing a 3-2-1 multilayer perceptron network on an FPGA for a fault identification application. The key modules implemented include multiply-accumulate, truncation, sigmoid and linear activation functions. Resource utilization is low, with the entire integrated network using only 2.2% of FPGA slices. Simulation results match manual calculations, demonstrating the network accurately classifies faults.
FPGA Optimized Fuzzy Controller Design for Magnetic Ball Levitation using Gen...IDES Editor
This paper presents an optimum approach for
designing of fuzzy controller for nonlinear system using
FPGA technology with Genetic Algorithms (GA) optimization
tool. A magnetic levitation system is considered as a case study
and the fuzzy controller is designed to keep a magnetic object
suspended in the air counteracting the weight of the object.
Fuzzy controller will be implemented using FPGA chip.
Genetic Algorithm (GA) is used in this paper as optimization
method that optimizes the membership, output gain and inputs
gains of the fuzzy controllers. The design will use a highlevel
programming language HDL for implementing the fuzzy
logic controller using the Xfuzzy tools to implement the fuzzy
logic controller into HDL code. This paper, advocates a novel
approach to implement the fuzzy logic controller for magnetic
ball levitation system by using FPGA with GA.
IJRET : International Journal of Research in Engineering and Technology is an international peer reviewed, online journal published by eSAT Publishing House for the enhancement of research in various disciplines of Engineering and Technology. The aim and scope of the journal is to provide an academic medium and an important reference for the advancement and dissemination of research results that support high-level learning, teaching and research in the fields of Engineering and Technology. We bring together Scientists, Academician, Field Engineers, Scholars and Students of related fields of Engineering and Technology
This document discusses parallelizing graph algorithms on GPUs for optimization. It summarizes previous work on parallel Breadth-First Search (BFS), All Pair Shortest Path (APSP), and Traveling Salesman Problem (TSP) algorithms. It then proposes implementing BFS, APSP, and TSP on GPUs using optimization techniques like reducing data transfers between CPU and GPU and modifying the algorithms to maximize GPU computing power and memory usage. The paper claims this will improve performance and speedup over CPU implementations. It focuses on optimizing graph algorithms for parallel GPU processing to accelerate applications involving large graph analysis and optimization problems.
Comparison Between Levenberg-Marquardt And Scaled Conjugate Gradient Training...CSCJournals
The document compares the Levenberg-Marquardt and Scaled Conjugate Gradient algorithms for training a multilayer perceptron neural network for image compression. It finds that while both algorithms performed comparably in terms of accuracy and speed, the Levenberg-Marquardt algorithm achieved slightly better accuracy as measured by average training accuracy and mean squared error, while the Scaled Conjugate Gradient algorithm was faster as measured by average training iterations. The document compresses a standard test image called Lena using both algorithms and analyzes the results.
Adapted Branch-and-Bound Algorithm Using SVM With Model SelectionIJECEIAES
Branch-and-Bound algorithm is the basis for the majority of solving methods in mixed integer linear programming. It has been proving its efficiency in different fields. In fact, it creates little by little a tree of nodes by adopting two strategies. These strategies are variable selection strategy and node selection strategy. In our previous work, we experienced a methodology of learning branch-and-bound strategies using regression-based support vector machine twice. That methodology allowed firstly to exploit information from previous executions of Branch-and-Bound algorithm on other instances. Secondly, it created information channel between node selection strategy and variable branching strategy. And thirdly, it gave good results in term of running time comparing to standard Branch-and-Bound algorithm. In this work, we will focus on increasing SVM performance by using cross validation coupled with model selection.
Parallel random projection using R high performance computing for planted mot...TELKOMNIKA JOURNAL
Motif discovery in DNA sequences is one of the most important issues in bioinformatics. Thus, algorithms for dealing with the problem accurately and quickly have always been the goal of research in bioinformatics. Therefore, this study is intended to modify the random projection algorithm to be implemented on R high performance computing (i.e., the R package pbdMPI). Some steps are needed to achieve this objective, ie preprocessing data, splitting data according to number of batches, modifying and implementing random projection in the pbdMPI package, and then aggregating the results. To validate the proposed approach, some experiments have been conducted. Several benchmarking data were used in this study by sensitivity analysis on number of cores and batches. Experimental results show that computational cost can be reduced, which is that the computation cost of 6 cores is faster around 34 times compared with the standalone mode. Thus, the proposed approach can be used for motif discovery effectively and efficiently.
A Novel Penalized and Compensated Constraints Based Modified Fuzzy Possibilis...ijsrd.com
A cluster is a group of objects which are similar to each other within a cluster and are dissimilar to the objects of other clusters. The similarity is typically calculated on the basis of distance between two objects or clusters. Two or more objects present inside a cluster and only if those objects are close to each other based on the distance between them.The major objective of clustering is to discover collection of comparable objects based on similarity metric. Fuzzy Possibilistic C-Means (FPCM) is the effective clustering algorithm available to cluster unlabeled data that produces both membership and typicality values during clustering process. In this approach, the efficiency of the Fuzzy Possibilistic C-means clustering approach is enhanced by using the penalized and compensated constraints based FPCM (PCFPCM). The proposed PCFPCM approach differ from the conventional clustering techniques by imposing the possibilistic reasoning strategy on fuzzy clustering with penalized and compensated constraints for updating the grades of membership and typicality. The performance of the proposed approaches is evaluated on the University of California, Irvine (UCI) machine repository datasets such as Iris, Wine, Lung Cancer and Lymphograma. The parameters used for the evaluation is Clustering accuracy, Mean Squared Error (MSE), Execution Time and Convergence behavior.
Integration of a Predictive, Continuous Time Neural Network into Securities M...Chris Kirk, PhD, FIAP
This paper describes recent development and test implementation of a continuous time recurrent neural network that has been configured to predict rates of change in securities. It presents outcomes in the
context of popular technical analysis indicators and highlights the potential impact of continuous predictive capability on securities
market trading operations.
Machine learning in Dynamic Adaptive Streaming over HTTP (DASH)Eswar Publications
Recently machine learning has been introduced into the area of adaptive video streaming. This paper explores a novel taxonomy that includes six state of the art techniques of machine learning that have been applied to Dynamic Adaptive Streaming over HTTP (DASH): (1) Q-learning, (2) Reinforcement learning, (3) Regression, (4) Classification, (5) Decision Tree learning, and (6) Neural networks.
CHANNEL ESTIMATION FOR THE ISDBT B FBMC SYSTEM USING NEURAL NETWORKS: A PROPO...csandit
Due to the evolution of technology and the diffusion of digital television, many researchers have
studied more efficient transmission and reception methods. This fact occurs because of the
demand of transmitting videos with better quality using new standards such 8K SUPER Hi-
VISION. In this scenario, modulation techniques such as Filter Bank Multi Carrier, associated
with advanced coding and synchronization methods, are being applied, aiming to achieve the
desired data rate to support ultra-high definition videos. Simultaneously, it is also important to
investigate ways of channel estimation that enable a better reception of the transmitted signal.
This task is not always trivial, depending of the characteristics of the channel. Thus, the use of
artificial intelligence can contribute to estimate the channel frequency response, from the
transmitted pilots. A classical algorithm called Back-propagation Training can be applied to
find the channel equalizer coefficients, making possible the correct reception of TV signals.
Therefore, this work presents a method of channel estimation that uses neural network
techniques to obtain the channel response in the Brazilian Digital System Television, called
ISDB-TB, using Filter Bank Multi Carrier.
International Journal of Engineering Research and Applications (IJERA) is an open access online peer reviewed international journal that publishes research and review articles in the fields of Computer Science, Neural Networks, Electrical Engineering, Software Engineering, Information Technology, Mechanical Engineering, Chemical Engineering, Plastic Engineering, Food Technology, Textile Engineering, Nano Technology & science, Power Electronics, Electronics & Communication Engineering, Computational mathematics, Image processing, Civil Engineering, Structural Engineering, Environmental Engineering, VLSI Testing & Low Power VLSI Design etc.
Image Steganography Using Wavelet Transform And Genetic AlgorithmAM Publications
This paper presents the application of Wavelet Transform and Genetic Algorithm in a novel
steganography scheme. We employ a genetic algorithm based mapping function to embed data in Discrete Wavelet
Transform coefficients in 4x4 blocks on the cover image. The optimal pixel adjustment process is applied after
embedding the message. We utilize the frequency domain to improve the robustness of steganography and, we
implement Genetic Algorithm and Optimal Pixel Adjustment Process to obtain an optimal mapping function to
reduce the difference error between the cover and the stego-image, therefore improving the hiding capacity with
low distortions. Our Simulation results reveal that the novel scheme outperforms adaptive steganography technique
based on wavelet transform in terms of peak signal to noise ratio and capacity, 39.94 dB and 50% respectively.
Scalability has been an essential factor for any kind of computational algorithm while considering its performance. In this Big Data era, gathering of large amounts of data is becoming easy. Data analysis on Big Data is not feasible using the existing Machine Learning (ML) algorithms and it perceives them to perform poorly. This is due to the fact that the computational logic for these algorithms is previously designed in sequential way. MapReduce becomes the solution for handling billions of data efficiently. In this report we discuss the basic building block for the computations behind ML algorithms, two different attempts to parallelize machine learning algorithms using MapReduce and a brief description on the overhead in parallelization of ML algorithms.
Local Binary Fitting Segmentation by Cooperative Quantum Particle OptimizationTELKOMNIKA JOURNAL
Recently, sophisticated segmentation techniques, such as level set method, which using valid
numerical calculation methods to process the evolution of the curve by solving linear or nonlinear elliptic
equations to divide the image availably, has become being more popular and effective. In Local Binary
Fitting (LBF) algorithm, a simple contour is initialized in an image and then the steepest-descent algorithm
is employed to constrain it to minimize the fitting energy functional. Hence, the initial position of the contour
is difficult or impossible to be well chosen for the final performance. To overcoming this drawback, this
work treats the energy fitting problem as a meta-heuristic optimization algorithm and imports a varietal
particle swarm optimization (PSO) method into the inner optimization process. The experimental results of
segmentations on medical images show that the proposed method is not only effective to both simple and
complex medical images with adequate stochastic effects, but also shows the accuracy and high
efficiency.
Objective Evaluation of a Deep Neural Network Approach for Single-Channel Spe...csandit
Single-channel speech intelligibility enhancement is much more difficult than multi-channel
intelligibility enhancement. It has recently been reported that machine learning training-based
single-channel speech intelligibility enhancement algorithms perform better than traditional
algorithms. In this paper, the performance of a deep neural network method using a multiresolution
cochlea-gram feature set recently proposed to perform single-channel speech
intelligibility enhancement processing is evaluated. Various conditions such as different
speakers for training and testing as well as different noise conditions are tested. Simulations
and objective test results show that the method performs better than another deep neural
networks setup recently proposed for the same task, and leads to a more robust convergence
compared to a recently proposed Gaussian mixture model approach.
Comparison of Neural Network Training Functions for Hematoma Classification i...IOSR Journals
Classification is one of the most important task in application areas of artificial neural networks
(ANN).Training neural networks is a complex task in the supervised learning field of research. The main
difficulty in adopting ANN is to find the most appropriate combination of learning, transfer and training
function for the classification task. We compared the performances of three types of training algorithms in feed
forward neural network for brain hematoma classification. In this work we have selected Gradient Descent
based backpropagation, Gradient Descent with momentum, Resilence backpropogation algorithms. Under
conjugate based algorithms, Scaled Conjugate back propagation, Conjugate Gradient backpropagation with
Polak-Riebreupdates(CGP) and Conjugate Gradient backpropagation with Fletcher-Reeves updates (CGF).The
last category is Quasi Newton based algorithm, under this BFGS, Levenberg-Marquardt algorithms are
selected. Proposed work compared training algorithm on the basis of mean square error, accuracy, rate of
convergence and correctness of the classification. Our conclusion about the training functions is based on the
simulation results
This document summarizes research on improving image classification results using neural networks. It compares common image classification methods like support vector machines (SVM) and K-nearest neighbors (KNN). It then evaluates the performance of multilayer perceptron (MLP) neural networks and radial basis function (RBF) neural networks on image classification. The document tests various configurations of MLP and RBF networks on a dataset containing 2310 images across 7 classes. It finds that a MLP network with two hidden layers of 10 neurons each achieves the best results, with an average accuracy of 98.84%. This is significantly higher than the 84.47% average accuracy of RBF networks and outperforms KNN classification as well. The research concludes that neural
Implementation Of Back-Propagation Neural Network For Isolated Bangla Speech ...ijistjournal
This paper is concerned with the development of Back-propagation Neural Network for Bangla Speech Recognition. In this paper, ten bangla digits were recorded from ten speakers and have been recognized. The features of these speech digits were extracted by the method of Mel Frequency Cepstral Coefficient (MFCC) analysis. The mfcc features of five speakers were used to train the network with Back propagation algorithm. The mfcc features of ten bangla digit speeches, from 0 to 9, of another five speakers were used to test the system. All the methods and algorithms used in this research were implemented using the features of Turbo C and C++ languages. From our investigation it is seen that the developed system can successfully encode and analyze the mfcc features of the speech signal to recognition. The developed system achieved recognition rate about 96.332% for known speakers (i.e., speaker dependent) and 92% for unknown speakers (i.e., speaker independent).
Implementation of Fuzzy Logic for the High-Resolution Remote Sensing Images w...IOSR Journals
This document describes an implementation of fuzzy logic for high-resolution remote sensing image classification with improved accuracy. It discusses using an object-based approach with fuzzy rules to classify urban land covers in a satellite image. The approach involves image segmentation using k-means clustering or ISODATA clustering. Features are then extracted from the image objects and fuzzy logic is applied to classify the objects based on membership functions. The method was tested on different sensor and resolution images in MATLAB and showed improved classification accuracy over other techniques, achieving lower entropy in results. Future work planned includes designing an unsupervised classification model combining k-means clustering and fuzzy-based object orientation.
Architecture neural network deep optimizing based on self organizing feature ...journalBEEI
Forward neural network (FNN) execution relying on the algorithm of training and architecture selection. Different parameters using for nip out the architecture of FNN such as the connections number among strata, neurons hidden number in each strata hidden and hidden strata number. Feature architectural combinations exponential could be uncontrollable manually so specific architecture can be design automatically by using special algorithm which build system with ability generalization better. Determination of architecture FNN can be done by using the algorithm of optimization numerous. In this paper methodology new proposes achievement where FNN neurons respective with hidden layers estimation work where in this work collect algorithm training self organizing feature map (SOFM) with advantages to explain how the best architectural selected automatically by SOFM from criteria error testing based on architecture populated. Different size of dataset benchmark of 4 classifications tested for approach proposed.
Hybrid multi objectives genetic algorithms and immigrants scheme for dynamic ...khalil IBRAHIM
the main concept of intelligent optimization techniques, artificial neural networks, and new genetic algorithms to solve the multi-objective multicast routing problems with shortest path (SP) problem that used in the addresses networks and improve all processes addressing in the wireless communications based on multi-objective optimization. The most important characteristics in mobile wireless networks is the topology dynamics and the network topology changes over time, the routing problem (SPRP) in mobile ad hoc networks (MANETs) turns out to be a dynamic optimization problem[13], the hybrid immigrants multiple-objective genetic algorithm (HIMOGAs) in the real- world are dynamic in nature, that has objective functions, constraints, and parameters, the dynamic optimization problems (DOPs) are a big challenges to evolutionary multi-objective, since any environmental change may affect the objective vector, constraints, and parameters, HIMOGA for the optimization goal is to track the moving of parameters and get a sequence of approximations solutions over time. The quantity of services (QoS) is supporting guarantee for all data traffic and getting the maximizing utilization for network, the QoS based on multicast routing offers significant challenges, and increases to use an efficient multicast routing protocol that will be able to check multicast routing and satisfying QoS constraints, The author propose to use HIMOGAs and SP algorithm to solve multicast problem that produces new generation wireless networks with immigrants schema to get high-quality solutions after each change and satisfying all objectives.
This document summarizes an FPGA implementation of a trained neural network. It describes implementing a 3-2-1 multilayer perceptron network on an FPGA for a fault identification application. The key modules implemented include multiply-accumulate, truncation, sigmoid and linear activation functions. Resource utilization is low, with the entire integrated network using only 2.2% of FPGA slices. Simulation results match manual calculations, demonstrating the network accurately classifies faults.
FPGA Optimized Fuzzy Controller Design for Magnetic Ball Levitation using Gen...IDES Editor
This paper presents an optimum approach for
designing of fuzzy controller for nonlinear system using
FPGA technology with Genetic Algorithms (GA) optimization
tool. A magnetic levitation system is considered as a case study
and the fuzzy controller is designed to keep a magnetic object
suspended in the air counteracting the weight of the object.
Fuzzy controller will be implemented using FPGA chip.
Genetic Algorithm (GA) is used in this paper as optimization
method that optimizes the membership, output gain and inputs
gains of the fuzzy controllers. The design will use a highlevel
programming language HDL for implementing the fuzzy
logic controller using the Xfuzzy tools to implement the fuzzy
logic controller into HDL code. This paper, advocates a novel
approach to implement the fuzzy logic controller for magnetic
ball levitation system by using FPGA with GA.
IJRET : International Journal of Research in Engineering and Technology is an international peer reviewed, online journal published by eSAT Publishing House for the enhancement of research in various disciplines of Engineering and Technology. The aim and scope of the journal is to provide an academic medium and an important reference for the advancement and dissemination of research results that support high-level learning, teaching and research in the fields of Engineering and Technology. We bring together Scientists, Academician, Field Engineers, Scholars and Students of related fields of Engineering and Technology
This document discusses parallelizing graph algorithms on GPUs for optimization. It summarizes previous work on parallel Breadth-First Search (BFS), All Pair Shortest Path (APSP), and Traveling Salesman Problem (TSP) algorithms. It then proposes implementing BFS, APSP, and TSP on GPUs using optimization techniques like reducing data transfers between CPU and GPU and modifying the algorithms to maximize GPU computing power and memory usage. The paper claims this will improve performance and speedup over CPU implementations. It focuses on optimizing graph algorithms for parallel GPU processing to accelerate applications involving large graph analysis and optimization problems.
Comparison Between Levenberg-Marquardt And Scaled Conjugate Gradient Training...CSCJournals
The document compares the Levenberg-Marquardt and Scaled Conjugate Gradient algorithms for training a multilayer perceptron neural network for image compression. It finds that while both algorithms performed comparably in terms of accuracy and speed, the Levenberg-Marquardt algorithm achieved slightly better accuracy as measured by average training accuracy and mean squared error, while the Scaled Conjugate Gradient algorithm was faster as measured by average training iterations. The document compresses a standard test image called Lena using both algorithms and analyzes the results.
Adapted Branch-and-Bound Algorithm Using SVM With Model SelectionIJECEIAES
Branch-and-Bound algorithm is the basis for the majority of solving methods in mixed integer linear programming. It has been proving its efficiency in different fields. In fact, it creates little by little a tree of nodes by adopting two strategies. These strategies are variable selection strategy and node selection strategy. In our previous work, we experienced a methodology of learning branch-and-bound strategies using regression-based support vector machine twice. That methodology allowed firstly to exploit information from previous executions of Branch-and-Bound algorithm on other instances. Secondly, it created information channel between node selection strategy and variable branching strategy. And thirdly, it gave good results in term of running time comparing to standard Branch-and-Bound algorithm. In this work, we will focus on increasing SVM performance by using cross validation coupled with model selection.
Parallel random projection using R high performance computing for planted mot...TELKOMNIKA JOURNAL
Motif discovery in DNA sequences is one of the most important issues in bioinformatics. Thus, algorithms for dealing with the problem accurately and quickly have always been the goal of research in bioinformatics. Therefore, this study is intended to modify the random projection algorithm to be implemented on R high performance computing (i.e., the R package pbdMPI). Some steps are needed to achieve this objective, ie preprocessing data, splitting data according to number of batches, modifying and implementing random projection in the pbdMPI package, and then aggregating the results. To validate the proposed approach, some experiments have been conducted. Several benchmarking data were used in this study by sensitivity analysis on number of cores and batches. Experimental results show that computational cost can be reduced, which is that the computation cost of 6 cores is faster around 34 times compared with the standalone mode. Thus, the proposed approach can be used for motif discovery effectively and efficiently.
A Novel Penalized and Compensated Constraints Based Modified Fuzzy Possibilis...ijsrd.com
A cluster is a group of objects which are similar to each other within a cluster and are dissimilar to the objects of other clusters. The similarity is typically calculated on the basis of distance between two objects or clusters. Two or more objects present inside a cluster and only if those objects are close to each other based on the distance between them.The major objective of clustering is to discover collection of comparable objects based on similarity metric. Fuzzy Possibilistic C-Means (FPCM) is the effective clustering algorithm available to cluster unlabeled data that produces both membership and typicality values during clustering process. In this approach, the efficiency of the Fuzzy Possibilistic C-means clustering approach is enhanced by using the penalized and compensated constraints based FPCM (PCFPCM). The proposed PCFPCM approach differ from the conventional clustering techniques by imposing the possibilistic reasoning strategy on fuzzy clustering with penalized and compensated constraints for updating the grades of membership and typicality. The performance of the proposed approaches is evaluated on the University of California, Irvine (UCI) machine repository datasets such as Iris, Wine, Lung Cancer and Lymphograma. The parameters used for the evaluation is Clustering accuracy, Mean Squared Error (MSE), Execution Time and Convergence behavior.
Integration of a Predictive, Continuous Time Neural Network into Securities M...Chris Kirk, PhD, FIAP
This paper describes recent development and test implementation of a continuous time recurrent neural network that has been configured to predict rates of change in securities. It presents outcomes in the
context of popular technical analysis indicators and highlights the potential impact of continuous predictive capability on securities
market trading operations.
Machine learning in Dynamic Adaptive Streaming over HTTP (DASH)Eswar Publications
Recently machine learning has been introduced into the area of adaptive video streaming. This paper explores a novel taxonomy that includes six state of the art techniques of machine learning that have been applied to Dynamic Adaptive Streaming over HTTP (DASH): (1) Q-learning, (2) Reinforcement learning, (3) Regression, (4) Classification, (5) Decision Tree learning, and (6) Neural networks.
CHANNEL ESTIMATION FOR THE ISDBT B FBMC SYSTEM USING NEURAL NETWORKS: A PROPO...csandit
Due to the evolution of technology and the diffusion of digital television, many researchers have
studied more efficient transmission and reception methods. This fact occurs because of the
demand of transmitting videos with better quality using new standards such 8K SUPER Hi-
VISION. In this scenario, modulation techniques such as Filter Bank Multi Carrier, associated
with advanced coding and synchronization methods, are being applied, aiming to achieve the
desired data rate to support ultra-high definition videos. Simultaneously, it is also important to
investigate ways of channel estimation that enable a better reception of the transmitted signal.
This task is not always trivial, depending of the characteristics of the channel. Thus, the use of
artificial intelligence can contribute to estimate the channel frequency response, from the
transmitted pilots. A classical algorithm called Back-propagation Training can be applied to
find the channel equalizer coefficients, making possible the correct reception of TV signals.
Therefore, this work presents a method of channel estimation that uses neural network
techniques to obtain the channel response in the Brazilian Digital System Television, called
ISDB-TB, using Filter Bank Multi Carrier.
International Journal of Engineering Research and Applications (IJERA) is an open access online peer reviewed international journal that publishes research and review articles in the fields of Computer Science, Neural Networks, Electrical Engineering, Software Engineering, Information Technology, Mechanical Engineering, Chemical Engineering, Plastic Engineering, Food Technology, Textile Engineering, Nano Technology & science, Power Electronics, Electronics & Communication Engineering, Computational mathematics, Image processing, Civil Engineering, Structural Engineering, Environmental Engineering, VLSI Testing & Low Power VLSI Design etc.
Image Steganography Using Wavelet Transform And Genetic AlgorithmAM Publications
This paper presents the application of Wavelet Transform and Genetic Algorithm in a novel
steganography scheme. We employ a genetic algorithm based mapping function to embed data in Discrete Wavelet
Transform coefficients in 4x4 blocks on the cover image. The optimal pixel adjustment process is applied after
embedding the message. We utilize the frequency domain to improve the robustness of steganography and, we
implement Genetic Algorithm and Optimal Pixel Adjustment Process to obtain an optimal mapping function to
reduce the difference error between the cover and the stego-image, therefore improving the hiding capacity with
low distortions. Our Simulation results reveal that the novel scheme outperforms adaptive steganography technique
based on wavelet transform in terms of peak signal to noise ratio and capacity, 39.94 dB and 50% respectively.
Scalability has been an essential factor for any kind of computational algorithm while considering its performance. In this Big Data era, gathering of large amounts of data is becoming easy. Data analysis on Big Data is not feasible using the existing Machine Learning (ML) algorithms and it perceives them to perform poorly. This is due to the fact that the computational logic for these algorithms is previously designed in sequential way. MapReduce becomes the solution for handling billions of data efficiently. In this report we discuss the basic building block for the computations behind ML algorithms, two different attempts to parallelize machine learning algorithms using MapReduce and a brief description on the overhead in parallelization of ML algorithms.
Local Binary Fitting Segmentation by Cooperative Quantum Particle OptimizationTELKOMNIKA JOURNAL
Recently, sophisticated segmentation techniques, such as level set method, which using valid
numerical calculation methods to process the evolution of the curve by solving linear or nonlinear elliptic
equations to divide the image availably, has become being more popular and effective. In Local Binary
Fitting (LBF) algorithm, a simple contour is initialized in an image and then the steepest-descent algorithm
is employed to constrain it to minimize the fitting energy functional. Hence, the initial position of the contour
is difficult or impossible to be well chosen for the final performance. To overcoming this drawback, this
work treats the energy fitting problem as a meta-heuristic optimization algorithm and imports a varietal
particle swarm optimization (PSO) method into the inner optimization process. The experimental results of
segmentations on medical images show that the proposed method is not only effective to both simple and
complex medical images with adequate stochastic effects, but also shows the accuracy and high
efficiency.
Objective Evaluation of a Deep Neural Network Approach for Single-Channel Spe...csandit
Single-channel speech intelligibility enhancement is much more difficult than multi-channel
intelligibility enhancement. It has recently been reported that machine learning training-based
single-channel speech intelligibility enhancement algorithms perform better than traditional
algorithms. In this paper, the performance of a deep neural network method using a multiresolution
cochlea-gram feature set recently proposed to perform single-channel speech
intelligibility enhancement processing is evaluated. Various conditions such as different
speakers for training and testing as well as different noise conditions are tested. Simulations
and objective test results show that the method performs better than another deep neural
networks setup recently proposed for the same task, and leads to a more robust convergence
compared to a recently proposed Gaussian mixture model approach.
Comparison of Neural Network Training Functions for Hematoma Classification i...IOSR Journals
Classification is one of the most important task in application areas of artificial neural networks
(ANN).Training neural networks is a complex task in the supervised learning field of research. The main
difficulty in adopting ANN is to find the most appropriate combination of learning, transfer and training
function for the classification task. We compared the performances of three types of training algorithms in feed
forward neural network for brain hematoma classification. In this work we have selected Gradient Descent
based backpropagation, Gradient Descent with momentum, Resilence backpropogation algorithms. Under
conjugate based algorithms, Scaled Conjugate back propagation, Conjugate Gradient backpropagation with
Polak-Riebreupdates(CGP) and Conjugate Gradient backpropagation with Fletcher-Reeves updates (CGF).The
last category is Quasi Newton based algorithm, under this BFGS, Levenberg-Marquardt algorithms are
selected. Proposed work compared training algorithm on the basis of mean square error, accuracy, rate of
convergence and correctness of the classification. Our conclusion about the training functions is based on the
simulation results
This document summarizes research on improving image classification results using neural networks. It compares common image classification methods like support vector machines (SVM) and K-nearest neighbors (KNN). It then evaluates the performance of multilayer perceptron (MLP) neural networks and radial basis function (RBF) neural networks on image classification. The document tests various configurations of MLP and RBF networks on a dataset containing 2310 images across 7 classes. It finds that a MLP network with two hidden layers of 10 neurons each achieves the best results, with an average accuracy of 98.84%. This is significantly higher than the 84.47% average accuracy of RBF networks and outperforms KNN classification as well. The research concludes that neural
PSO-based Training, Pruning, and Ensembling of Extreme Learning Machine RBF N...ijceronline
International Journal of Computational Engineering Research (IJCER) is dedicated to protecting personal information and will make every reasonable effort to handle collected information appropriately. All information collected, as well as related requests, will be handled as carefully and efficiently as possible in accordance with IJCER standards for integrity and objectivity.
This document summarizes a research paper that implemented Levenberg-Marquardt artificial neural network training using graphics processing unit (GPU) hardware acceleration. The key points are:
1) This appears to be the first description of implementing artificial neural networks using the Levenberg-Marquardt training method on a GPU.
2) The paper describes their approach for implementing the Levenberg-Marquardt algorithm on a GPU, which involves solving the matrix inversion operation that is typically computationally expensive.
3) Results show that training networks using the GPU implementation can be up to 10 times faster than using a CPU-only implementation on the same hardware.
This paper proposes using the Levenberg-Marquardt algorithm for training a neural network to predict diabetes. The Levenberg-Marquardt algorithm is designed to minimize error functions and combines gradient descent and the Gauss-Newton method. The neural network is trained on a dataset from an Indian hospital to classify patients as diabetic or non-diabetic. Experimental results show that the Levenberg-Marquardt algorithm achieves the best validation performance of 0.00012359 with perfect correlation between outputs and targets, outperforming other backpropagation algorithms for this diabetes prediction task.
Application of support vector machines for prediction of anti hiv activity of...Alexander Decker
This document describes a study that used support vector machines (SVM) to develop a quantitative structure-activity relationship (QSAR) model to predict the anti-HIV activity of TIBO derivatives. The SVM model achieved high correlation (q2=0.96) and low error (RMSE=0.212), outperforming artificial neural networks and multiple linear regression models developed on the same data set. The results indicate that SVM is a valuable tool for QSAR modeling and predicting anti-HIV activity of chemical compounds.
Using Multi-layered Feed-forward Neural Network (MLFNN) Architecture as Bidir...IOSR Journals
This document presents a method for using a multi-layered feed-forward neural network (MLFNN) architecture as a bidirectional associative memory (BAM) for function approximation. It proposes applying the backpropagation algorithm in two phases - first in the forward direction, then in the backward direction - which allows the MLFNN to work like a BAM. Simulation results show that this two-phase backpropagation algorithm achieves convergence faster than standard backpropagation when approximating the sine function, demonstrating that the MLFNN architecture is better suited for function approximation when trained this way.
A COMPARATIVE STUDY OF BACKPROPAGATION ALGORITHMS IN FINANCIAL PREDICTIONIJCSEA Journal
Stock market price index prediction is a challenging task for investors and scholars. Artificial neural networks have been widely employed to predict financial stock market levels thanks to their ability to model nonlinear functions. The accuracy of backpropagation neural networks trained with different heuristic and numerical algorithms is measured for comparison purpose. It is found that numerical algorithm outperform heuristic techniques.
Implementation Of Back-Propagation Neural Network For Isolated Bangla Speech ...ijistjournal
This document describes the implementation of a back-propagation neural network for isolated Bangla speech recognition. The network was trained on Mel Frequency Cepstral Coefficient (MFCC) features extracted from recordings of 10 Bangla digits spoken by 10 speakers. The network architecture included an input layer of 250 neurons, a hidden layer of 16 neurons, and an output layer of 10 neurons. The network was trained using backpropagation and achieved a recognition rate of 96.3% for known speakers and 92% for unknown speakers. The system demonstrates the potential for developing speaker-independent isolated digit speech recognition in Bangla.
The document presents research on using neural networks to predict Earth Orientation Parameters (EOP) such as UT1-TAI. Three neural network models were tested:
1) Network 1 varied the number of neurons proportionally with increasing training sample size.
2) Network 2 kept the number of neurons constant while increasing sample size.
3) Network 3 used daily training data with 2 neurons and sample sizes of 4, 10, 20, and 365 days.
The goal was to minimize prediction error (RMSE) for horizons of 5-25 days by adjusting sample size and neurons. Results showed the best balance was needed between these factors, and that short-term prediction was possible within 10 days using
Evolutionary Algorithm for Optimal Connection Weights in Artificial Neural Ne...CSCJournals
A neural network may be considered as an adaptive system that progressively self-organizes in order to approximate the solution, making the problem solver free from the need to accurately and unambiguously specify the steps towards the solution. Moreover, Evolutionary Artificial Neural Networks (EANNs) have the ability to progressively improve their performance on a given task by executing learning. An evolutionary computation gives adaptability for connection weights using feed forward architecture. In this paper, the use of evolutionary computation for feed-forward neural network learning is discussed. To check the validation of proposed method, XOR benchmark problem has been used. The accuracy of the proposed model is more satisfactory as compared to gradient method.
Memory Polynomial Based Adaptive Digital PredistorterIJERA Editor
Digital predistortion (DPD) is a baseband signal processing technique that corrects for impairments in RF
power amplifiers (PAs). These impairments cause out-of-band emissions or spectral regrowth and in-band
distortion, which correlate with an increased bit error rate (BER). Wideband signals with a high peak-to-average
ratio, are more susceptible to these unwanted effects. So to reduce these impairments, this paper proposes the
modeling of the digital predistortion for the power amplifier using GSA algorithm.
Neural network based numerical digits recognization using nnt in matlabijcses
Artificial neural networks are models inspired by human nervous system that is capable of learning. One of
the important applications of artificial neural network is character Recognition. Character Recognition
finds its application in number of areas, such as banking, security products, hospitals, in robotics also.
This paper is based on a system that recognizes a english numeral, given by the user, which is already
trained on the features of the numbers to be recognized using NNT (Neural network toolbox) .The system
has a neural network as its core, which is first trained on a database. The training of the neural network
extracts the features of the English numbers and stores in the database. The next phase of the system is to
recognize the number given by the user. The features of the number given by the user are extracted and
compared with the feature database and the recognized number is displayed.
We propose an algorithm for training Multi Layer Preceptrons for classification problems, that we named Hidden Layer Learning Vector Quantization (H-LVQ). It consists of applying Learning Vector Quantization to the last hidden layer of a MLP and it gave very successful results on problems containing a large number of correlated inputs. It was applied with excellent results on classification of Rurtherford
backscattering spectra and on a benchmark problem of image recognition. It may also be used for efficient feature extraction.
Artificial Neural Network and Multi-Response Optimization in Reliability Meas...inventionjournals
Neural network is an important tool for reliability analysis, including estimation of reliability or utility function which are too complicated to be analytical expressed for large or complex system. It has been demonstrated the neural network has significant improvement in the parameter estimation accuracy over the traditional chi-square test. There are many parameters of a neural network that should be determined while training the dataset, since different setups of algorithm parameters affect the estimation performance in either accuracy or computation efficiency. In this paper, neural network training is used to estimate the utility function for the parallel-series redundancy allocation problem, and weighted principal component based multi-response optimization method is applied to find the optimal setting of neural network parameters so that the simultaneous minimizations of training error and computing time are achieved.
Short Term Electrical Load Forecasting by Artificial Neural NetworkIJERA Editor
This paper presents an application of artificial neural networks for short-term times series electrical load
forecasting. An adaptive learning algorithm is derived from system stability to ensure the convergence of
training process. Historical data of hourly power load as well as hourly wind power generation are sourced from
European Open Power System Platform. The simulation demonstrates that errors steadily decrease in training
with the adaptive learning factor starting at different initial value and errors behave volatile with constant
learning factors with different values
International Journal of Engineering Research and Applications (IJERA) is an open access online peer reviewed international journal that publishes research and review articles in the fields of Computer Science, Neural Networks, Electrical Engineering, Software Engineering, Information Technology, Mechanical Engineering, Chemical Engineering, Plastic Engineering, Food Technology, Textile Engineering, Nano Technology & science, Power Electronics, Electronics & Communication Engineering, Computational mathematics, Image processing, Civil Engineering, Structural Engineering, Environmental Engineering, VLSI Testing & Low Power VLSI Design etc.
A comprehensive review on hybrid network traffic prediction model IJECEIAES
Network traffic is a typical nonlinear time series. As such, traditional linear and nonlinear models are inadequate to describe the multi-scale characteristics of traffic, thus compromising the prediction accuracy. Therefore, the research to date has tended to focus on hybrid models rather than the traditional linear and non-linear ones. Generally, a hybrid model adopts two or more methods as combined modelling to analyze and then predict the network traffic. Against this backdrop, this paper will review past research conducted on hybrid network traffic prediction models. The review concludes with a summary of the strengths and limitations of existing hybrid network prediction models which use optimization and decomposition techniques, respectively. These two techniques have been identified as major contributing factors in constructing a more accurate and fast response hybrid network traffic prediction.
Combination of Immune Genetic Particle Swarm Optimization algorithm with BP a...paperpublications3
Abstract:In this paper, merging Immune Genetic Particle Swarm Optimization algorithm (IGPSO) with BP algorithm to optimize BP Neural Network parameter i.e., BPIGPSO amalgamation to solve optimal reactive power dispatch algorithm. The basic perception is that first training BP neural network with IGPSO to find out a comparatively optimal solution, then take the network parameter at this time as the preliminary parameter of BP algorithm to carry out the training, finally searching the optimal solution. The proposed BPIGPSO has been tested on standard IEEE 57 bus test system and simulation results show clearly the better performance of the proposed algorithm in reducing the real power loss.
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAUpanagenda
Webinar Recording: https://www.panagenda.com/webinars/hcl-notes-und-domino-lizenzkostenreduzierung-in-der-welt-von-dlau/
DLAU und die Lizenzen nach dem CCB- und CCX-Modell sind für viele in der HCL-Community seit letztem Jahr ein heißes Thema. Als Notes- oder Domino-Kunde haben Sie vielleicht mit unerwartet hohen Benutzerzahlen und Lizenzgebühren zu kämpfen. Sie fragen sich vielleicht, wie diese neue Art der Lizenzierung funktioniert und welchen Nutzen sie Ihnen bringt. Vor allem wollen Sie sicherlich Ihr Budget einhalten und Kosten sparen, wo immer möglich. Das verstehen wir und wir möchten Ihnen dabei helfen!
Wir erklären Ihnen, wie Sie häufige Konfigurationsprobleme lösen können, die dazu führen können, dass mehr Benutzer gezählt werden als nötig, und wie Sie überflüssige oder ungenutzte Konten identifizieren und entfernen können, um Geld zu sparen. Es gibt auch einige Ansätze, die zu unnötigen Ausgaben führen können, z. B. wenn ein Personendokument anstelle eines Mail-Ins für geteilte Mailboxen verwendet wird. Wir zeigen Ihnen solche Fälle und deren Lösungen. Und natürlich erklären wir Ihnen das neue Lizenzmodell.
Nehmen Sie an diesem Webinar teil, bei dem HCL-Ambassador Marc Thomas und Gastredner Franz Walder Ihnen diese neue Welt näherbringen. Es vermittelt Ihnen die Tools und das Know-how, um den Überblick zu bewahren. Sie werden in der Lage sein, Ihre Kosten durch eine optimierte Domino-Konfiguration zu reduzieren und auch in Zukunft gering zu halten.
Diese Themen werden behandelt
- Reduzierung der Lizenzkosten durch Auffinden und Beheben von Fehlkonfigurationen und überflüssigen Konten
- Wie funktionieren CCB- und CCX-Lizenzen wirklich?
- Verstehen des DLAU-Tools und wie man es am besten nutzt
- Tipps für häufige Problembereiche, wie z. B. Team-Postfächer, Funktions-/Testbenutzer usw.
- Praxisbeispiele und Best Practices zum sofortigen Umsetzen
Letter and Document Automation for Bonterra Impact Management (fka Social Sol...Jeffrey Haguewood
Sidekick Solutions uses Bonterra Impact Management (fka Social Solutions Apricot) and automation solutions to integrate data for business workflows.
We believe integration and automation are essential to user experience and the promise of efficient work through technology. Automation is the critical ingredient to realizing that full vision. We develop integration products and services for Bonterra Case Management software to support the deployment of automations for a variety of use cases.
This video focuses on automated letter generation for Bonterra Impact Management using Google Workspace or Microsoft 365.
Interested in deploying letter generation automations for Bonterra Impact Management? Contact us at sales@sidekicksolutionsllc.com to discuss next steps.
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdfMalak Abu Hammad
Discover how MongoDB Atlas and vector search technology can revolutionize your application's search capabilities. This comprehensive presentation covers:
* What is Vector Search?
* Importance and benefits of vector search
* Practical use cases across various industries
* Step-by-step implementation guide
* Live demos with code snippets
* Enhancing LLM capabilities with vector search
* Best practices and optimization strategies
Perfect for developers, AI enthusiasts, and tech leaders. Learn how to leverage MongoDB Atlas to deliver highly relevant, context-aware search results, transforming your data retrieval process. Stay ahead in tech innovation and maximize the potential of your applications.
#MongoDB #VectorSearch #AI #SemanticSearch #TechInnovation #DataScience #LLM #MachineLearning #SearchTechnology
5th LF Energy Power Grid Model Meet-up SlidesDanBrown980551
5th Power Grid Model Meet-up
It is with great pleasure that we extend to you an invitation to the 5th Power Grid Model Meet-up, scheduled for 6th June 2024. This event will adopt a hybrid format, allowing participants to join us either through an online Mircosoft Teams session or in person at TU/e located at Den Dolech 2, Eindhoven, Netherlands. The meet-up will be hosted by Eindhoven University of Technology (TU/e), a research university specializing in engineering science & technology.
Power Grid Model
The global energy transition is placing new and unprecedented demands on Distribution System Operators (DSOs). Alongside upgrades to grid capacity, processes such as digitization, capacity optimization, and congestion management are becoming vital for delivering reliable services.
Power Grid Model is an open source project from Linux Foundation Energy and provides a calculation engine that is increasingly essential for DSOs. It offers a standards-based foundation enabling real-time power systems analysis, simulations of electrical power grids, and sophisticated what-if analysis. In addition, it enables in-depth studies and analysis of the electrical power grid’s behavior and performance. This comprehensive model incorporates essential factors such as power generation capacity, electrical losses, voltage levels, power flows, and system stability.
Power Grid Model is currently being applied in a wide variety of use cases, including grid planning, expansion, reliability, and congestion studies. It can also help in analyzing the impact of renewable energy integration, assessing the effects of disturbances or faults, and developing strategies for grid control and optimization.
What to expect
For the upcoming meetup we are organizing, we have an exciting lineup of activities planned:
-Insightful presentations covering two practical applications of the Power Grid Model.
-An update on the latest advancements in Power Grid -Model technology during the first and second quarters of 2024.
-An interactive brainstorming session to discuss and propose new feature requests.
-An opportunity to connect with fellow Power Grid Model enthusiasts and users.
Driving Business Innovation: Latest Generative AI Advancements & Success StorySafe Software
Are you ready to revolutionize how you handle data? Join us for a webinar where we’ll bring you up to speed with the latest advancements in Generative AI technology and discover how leveraging FME with tools from giants like Google Gemini, Amazon, and Microsoft OpenAI can supercharge your workflow efficiency.
During the hour, we’ll take you through:
Guest Speaker Segment with Hannah Barrington: Dive into the world of dynamic real estate marketing with Hannah, the Marketing Manager at Workspace Group. Hear firsthand how their team generates engaging descriptions for thousands of office units by integrating diverse data sources—from PDF floorplans to web pages—using FME transformers, like OpenAIVisionConnector and AnthropicVisionConnector. This use case will show you how GenAI can streamline content creation for marketing across the board.
Ollama Use Case: Learn how Scenario Specialist Dmitri Bagh has utilized Ollama within FME to input data, create custom models, and enhance security protocols. This segment will include demos to illustrate the full capabilities of FME in AI-driven processes.
Custom AI Models: Discover how to leverage FME to build personalized AI models using your data. Whether it’s populating a model with local data for added security or integrating public AI tools, find out how FME facilitates a versatile and secure approach to AI.
We’ll wrap up with a live Q&A session where you can engage with our experts on your specific use cases, and learn more about optimizing your data workflows with AI.
This webinar is ideal for professionals seeking to harness the power of AI within their data management systems while ensuring high levels of customization and security. Whether you're a novice or an expert, gain actionable insights and strategies to elevate your data processes. Join us to see how FME and AI can revolutionize how you work with data!
TrustArc Webinar - 2024 Global Privacy SurveyTrustArc
How does your privacy program stack up against your peers? What challenges are privacy teams tackling and prioritizing in 2024?
In the fifth annual Global Privacy Benchmarks Survey, we asked over 1,800 global privacy professionals and business executives to share their perspectives on the current state of privacy inside and outside of their organizations. This year’s report focused on emerging areas of importance for privacy and compliance professionals, including considerations and implications of Artificial Intelligence (AI) technologies, building brand trust, and different approaches for achieving higher privacy competence scores.
See how organizational priorities and strategic approaches to data security and privacy are evolving around the globe.
This webinar will review:
- The top 10 privacy insights from the fifth annual Global Privacy Benchmarks Survey
- The top challenges for privacy leaders, practitioners, and organizations in 2024
- Key themes to consider in developing and maintaining your privacy program
Introduction of Cybersecurity with OSS at Code Europe 2024Hiroshi SHIBATA
I develop the Ruby programming language, RubyGems, and Bundler, which are package managers for Ruby. Today, I will introduce how to enhance the security of your application using open-source software (OSS) examples from Ruby and RubyGems.
The first topic is CVE (Common Vulnerabilities and Exposures). I have published CVEs many times. But what exactly is a CVE? I'll provide a basic understanding of CVEs and explain how to detect and handle vulnerabilities in OSS.
Next, let's discuss package managers. Package managers play a critical role in the OSS ecosystem. I'll explain how to manage library dependencies in your application.
I'll share insights into how the Ruby and RubyGems core team works to keep our ecosystem safe. By the end of this talk, you'll have a better understanding of how to safeguard your code.
Ivanti’s Patch Tuesday breakdown goes beyond patching your applications and brings you the intelligence and guidance needed to prioritize where to focus your attention first. Catch early analysis on our Ivanti blog, then join industry expert Chris Goettl for the Patch Tuesday Webinar Event. There we’ll do a deep dive into each of the bulletins and give guidance on the risks associated with the newly-identified vulnerabilities.
Dive into the realm of operating systems (OS) with Pravash Chandra Das, a seasoned Digital Forensic Analyst, as your guide. 🚀 This comprehensive presentation illuminates the core concepts, types, and evolution of OS, essential for understanding modern computing landscapes.
Beginning with the foundational definition, Das clarifies the pivotal role of OS as system software orchestrating hardware resources, software applications, and user interactions. Through succinct descriptions, he delineates the diverse types of OS, from single-user, single-task environments like early MS-DOS iterations, to multi-user, multi-tasking systems exemplified by modern Linux distributions.
Crucial components like the kernel and shell are dissected, highlighting their indispensable functions in resource management and user interface interaction. Das elucidates how the kernel acts as the central nervous system, orchestrating process scheduling, memory allocation, and device management. Meanwhile, the shell serves as the gateway for user commands, bridging the gap between human input and machine execution. 💻
The narrative then shifts to a captivating exploration of prominent desktop OSs, Windows, macOS, and Linux. Windows, with its globally ubiquitous presence and user-friendly interface, emerges as a cornerstone in personal computing history. macOS, lauded for its sleek design and seamless integration with Apple's ecosystem, stands as a beacon of stability and creativity. Linux, an open-source marvel, offers unparalleled flexibility and security, revolutionizing the computing landscape. 🖥️
Moving to the realm of mobile devices, Das unravels the dominance of Android and iOS. Android's open-source ethos fosters a vibrant ecosystem of customization and innovation, while iOS boasts a seamless user experience and robust security infrastructure. Meanwhile, discontinued platforms like Symbian and Palm OS evoke nostalgia for their pioneering roles in the smartphone revolution.
The journey concludes with a reflection on the ever-evolving landscape of OS, underscored by the emergence of real-time operating systems (RTOS) and the persistent quest for innovation and efficiency. As technology continues to shape our world, understanding the foundations and evolution of operating systems remains paramount. Join Pravash Chandra Das on this illuminating journey through the heart of computing. 🌟
Generating privacy-protected synthetic data using Secludy and MilvusZilliz
During this demo, the founders of Secludy will demonstrate how their system utilizes Milvus to store and manipulate embeddings for generating privacy-protected synthetic data. Their approach not only maintains the confidentiality of the original data but also enhances the utility and scalability of LLMs under privacy constraints. Attendees, including machine learning engineers, data scientists, and data managers, will witness first-hand how Secludy's integration with Milvus empowers organizations to harness the power of LLMs securely and efficiently.
Salesforce Integration for Bonterra Impact Management (fka Social Solutions A...Jeffrey Haguewood
Sidekick Solutions uses Bonterra Impact Management (fka Social Solutions Apricot) and automation solutions to integrate data for business workflows.
We believe integration and automation are essential to user experience and the promise of efficient work through technology. Automation is the critical ingredient to realizing that full vision. We develop integration products and services for Bonterra Case Management software to support the deployment of automations for a variety of use cases.
This video focuses on integration of Salesforce with Bonterra Impact Management.
Interested in deploying an integration with Salesforce for Bonterra Impact Management? Contact us at sales@sidekicksolutionsllc.com to discuss next steps.
Monitoring and Managing Anomaly Detection on OpenShift.pdfTosin Akinosho
Monitoring and Managing Anomaly Detection on OpenShift
Overview
Dive into the world of anomaly detection on edge devices with our comprehensive hands-on tutorial. This SlideShare presentation will guide you through the entire process, from data collection and model training to edge deployment and real-time monitoring. Perfect for those looking to implement robust anomaly detection systems on resource-constrained IoT/edge devices.
Key Topics Covered
1. Introduction to Anomaly Detection
- Understand the fundamentals of anomaly detection and its importance in identifying unusual behavior or failures in systems.
2. Understanding Edge (IoT)
- Learn about edge computing and IoT, and how they enable real-time data processing and decision-making at the source.
3. What is ArgoCD?
- Discover ArgoCD, a declarative, GitOps continuous delivery tool for Kubernetes, and its role in deploying applications on edge devices.
4. Deployment Using ArgoCD for Edge Devices
- Step-by-step guide on deploying anomaly detection models on edge devices using ArgoCD.
5. Introduction to Apache Kafka and S3
- Explore Apache Kafka for real-time data streaming and Amazon S3 for scalable storage solutions.
6. Viewing Kafka Messages in the Data Lake
- Learn how to view and analyze Kafka messages stored in a data lake for better insights.
7. What is Prometheus?
- Get to know Prometheus, an open-source monitoring and alerting toolkit, and its application in monitoring edge devices.
8. Monitoring Application Metrics with Prometheus
- Detailed instructions on setting up Prometheus to monitor the performance and health of your anomaly detection system.
9. What is Camel K?
- Introduction to Camel K, a lightweight integration framework built on Apache Camel, designed for Kubernetes.
10. Configuring Camel K Integrations for Data Pipelines
- Learn how to configure Camel K for seamless data pipeline integrations in your anomaly detection workflow.
11. What is a Jupyter Notebook?
- Overview of Jupyter Notebooks, an open-source web application for creating and sharing documents with live code, equations, visualizations, and narrative text.
12. Jupyter Notebooks with Code Examples
- Hands-on examples and code snippets in Jupyter Notebooks to help you implement and test anomaly detection models.
This presentation provides valuable insights into effective cost-saving techniques on AWS. Learn how to optimize your AWS resources by rightsizing, increasing elasticity, picking the right storage class, and choosing the best pricing model. Additionally, discover essential governance mechanisms to ensure continuous cost efficiency. Whether you are new to AWS or an experienced user, this presentation provides clear and practical tips to help you reduce your cloud costs and get the most out of your budget.
1. Kiran Arya et al Int. Journal of Engineering Research and Application
ISSN : 2248-9622, Vol. 3, Issue 5, Sep-Oct 2013, pp.1588-1595
RESEARCH ARTICLE
www.ijera.com
OPEN ACCESS
Performance Comparison on Face Recognition System Using
Different Variants of Back-Propagation Algorithm with Radial
Basis Function Neural Networks
Kiran Arya and Virendra P. Vishwakarma
Department of Computer Science and Engineering, Inderprastha Engineering College (Affiliated to Mahamaya
Technical University, UP, India), Ghaziabad 201010, UP, India
ABSTRACT
This paper presents the performance comparison of two architectures of neural networks: multi-layer perceptron
(MLP) neural networks and radial basis function (RBF) neural networks on face recognition system (FRS). We
are training MLP using different variants of back-propagation (BP) algorithm. AT&T database has been used
for performance comparison. The BP is gradient descent based iterative algorithm which takes larger training
time for high dimensional pattern recognition problem. Local minimum, improper learning rate and over-fitting
are some of the other issues. To overcome these issues, we used RBF based FRS that is robust than other
conventional methods like BP algorithm and has better performance of recognition rate. The training results
show that in all the situations, RBF provides better generalization performance in compared of BP
General Terms: Artificial Neural Network, BP, Radial Basis Function
Keywords: Face recognition, Performance Comparison
I.
INTRODUCTION
It seems to be easy for human being to
recognize the faces but for the machine based system it
is still not easy to achieve completely reliable and
robust performance of FRS. These problems occurred
due to different visual variations of images like
illumination, head size, orientation, pose, expression,
age, facial hair, glasses, background, and change in
environment conditions etc. MLP is widely used to
solve complex problems in pattern classification. One
of the characteristics of the MLP is its learning (or
training) ability. By training, the neural network can
give correct answers not only for learned examples, but
also for the models similar to the learned examples,
showing its strong associative ability and rational
ability which are suitable for solving large, nonlinear,
and complex classification and function approximation
problems [1].
Gradient based methods are one of the most
widely used error minimization methods used to train
BP networks. The back-propagation (BP) training
algorithm is a supervised learning method for multilayered feed-forward neural networks [2].
R.A. Finan, A.T. Sapelu, and R.I. Damper [3]
trained the MLP using BP and RBF for text-dependent
speaker recognition and compared the training results.
They had shown that an RBF system can provide even
better results with a suitable training set. For a good
training set, a significant improvement would be
expected for an RBF network relative to an MLP,
whereas a poor training set will not show much
improvement.
www.ijera.com
Haddadnia and Ahmadi [4] used a hybrid N
feature (RBF) neural network method for face
recognition system, which extracts a set of different
kind of features from face images with RBF networks.
These are combined together for classification purpose
through the majority rule. They have used three
different feature domains for features extraction from
input images.
Vu N.P. Dao and Rao Vemuri [5] used five
different methods of BP which are the gradient descent
BP, the gradient descent BP with momentum, the
variable learning rate Gradient descent BP, the
conjugate gradient BP, and the quasi-Newton method,
for training the neural network. They applied these
methods to define the problem of computer network
intrusion detection as a classification problem and
showed that the performance of the BP methods
depends on the neural networks topology but it was not
as reliable as RBF.
N. M. Nawi, R. S. Ransing and M. R. Ransing
[6] proposed a new computationally efficient algorithm
CGFR/AG for training of MLP, in which the conjugate
gradient optimization algorithm is combined with the
modified BP algorithm. This algorithm is generic and
easy to implement in all commonly used gradient
based optimization processes. It is robust and has a
potential to significantly enhance the computational
efficiency of the training process.
In [7] it’s shown that the performance of
training algorithm depends on many factors, including
the complexity of the problem, the no of data points in
the training set, the no. of weights and biases in the
1588 | P a g e
2. Kiran Arya et al Int. Journal of Engineering Research and Application
ISSN : 2248-9622, Vol. 3, Issue 5, Sep-Oct 2013, pp.1588-1595
network. So it is difficult to know which methods or
algorithm performs better than the other.
Performance Comparison between the architectures of
BP and RBF neural network shown in [8], which
based on four classification and function
approximation problem and comparative study of BP
methods is shown in[9].
V. P. Vishwakarma and M. N. Gupta[11]
proposed a new learning Algorithm SLFN_BVOI for
the training of single hidden layer feed-forward neural
network (SLFN) which perform effectively on high
dimension and high variations problems of Face
recognition. They showed that this algorithm learns on
an average 734.2 times faster than BP and 9.2 times
faster than ELM for different sizes of training set on
AT&T face database.
In this paper, MLP and RBF networks are
studied and compared based on their generalization
capability and learning speed. All methods applied to
the FRS system and used AT&T face database for
recognition purpose.
This paper is organized as follows. The
description of RBF along with BP is presented in
Section 2. Section 3 describes the details of database
used for our purpose. In Section 4, the experimental
results and discussions on the data set are presented.
Finally, conclusions are drawn in Section 5.
II.
Radial Basis Function Network and
Back Propagation Algorithm
2-1 BP Algorithm:
The BP algorithm is a technique used in
training MLP in a supervised manner. BP also known
as the error BP algorithm is based on the errorcorrection learning rule [14]-[16]. BP algorithm is a
stochastic algorithm based on the steepest decent
principle, wherein the weights of the neural network
are updated along the negative gradient direction in the
weight space. The simplest implementation of BP
learning updates the network weights and biases in the
direction in which the performance function decreases
most rapidly i.e. the negative of the gradient [15].
There are a number of variations of the BP algorithm
that are based on other standard optimization
techniques. This chapter explains how to use each of
these variations and discusses the advantages and
disadvantages of each.
Fig. 1: Multi-layered neural network.
www.ijera.com
www.ijera.com
The MLP consists of three types of layers[8]. The first
layer is the input layer and corresponds to the problem
input variables with one node for each input variable.
The second layer is the hidden layer used to capture
non-linear relationships among variables. The third
layer is the output layer used to provide predicted
values.
Computations related with the single neuron include:
i) Input for hidden layer is given by:
netj=
(1)
jwji
ii) The units of output vector of hidden layer after
passing through the activation function are given by:
hj =
(2)
In same manner, input for output layer is given by
netk =
mwki
(3)
and the units of output vector of output layer are given
by
ok=
(4)
For updating the weights, we need to calculate the
error. This can be done by
ξ(k)=
i-ti)
2
(5)
The input weights and biases for the next iteration are
given as:
W(k+1)=w(k)-η
(6)
b(k+1)=b(k)-η
(7)
Similarly, the output weight and biases are updated as:
β(k+1)= β(k)-η
(8)
α(k+1)=α(k)-η
(9)
Here η-is the learning rate. The performance
of the algorithm is very sensitive to the proper setting
of learning rate. If learning rate is set to high, the
algorithm may oscillate and become unstable. If the
learning rate is too small, the algorithm will take too
long to converge. There are different variants of BP
algorithm. With standard steepest descent BP, the
learning rate is held constant throughout training. It is
not practical to determine the optimal setting for the
learning rate before training, and, in fact, the optimal
learning rate changes during the training process, as
the algorithm moves across the performance surface
[8]-[10].
Some variants of BP:
There are two BP training algorithms:
gradient descent and gradient descent with momentum
1589 | P a g e
3. Kiran Arya et al Int. Journal of Engineering Research and Application
ISSN : 2248-9622, Vol. 3, Issue 5, Sep-Oct 2013, pp.1588-1595
are often too slow for practical problems. So we
discuss several high performance algorithms that can
converge from ten to one hundred times faster than
these algorithms [15].
Gradient Descent BP (GD): This method updates the
network weights and biases in the direction of the
performance function that decreases most rapidly, i.e.
the negative of the gradient. The new weight vector
wk+1 is adjusted according to:
wk+1=wk - αgk
(10)
The parameter α is the learning rate and gk is the
gradient of the error with respect to the weight vector.
The negative sign indicates that the new weight vector
wk+1 is moving in a direction opposite to that of the
gradient.
Gradient Descent BP with Momentum (GDM):
Momentum allows a network to respond not
only to the local gradient, but also to recent trends in
the error surface. Momentum allows the network to
ignore small features in the error surface. Without
momentum a network may get stuck in a shallow local
minimum. With momentum a network can slide
through such a minimum [14]-[16]. Momentum can be
added to BP method learning by making weight
changes equal to the sum of a fraction of the last
weight change and the new change suggested by the
gradient descent BP rule. The magnitude of the effect
that the last weight change is allowed to have is
mediated by a momentum constant, μ, which can be
any number between 0 and 1. When the momentum
constant is 0 a weight change is based solely on the
gradient. When the momentum constant is 1 the new
weight change is set to equal the last weight change
and the gradient is simply ignored. The new weight
vector wk+1 is adjusted as:
wk+1=wk - αgk+ µwk-1
(11)
Gradient Descent with Adaptive learning rate
(GDA): With standard steepest descent, the learning
rate is held constant throughout training. The
performance of the algorithm is very sensitive to the
proper setting of the learning rate. If the learning rate is
set too high, the algorithm may oscillate and become
unstable. If the learning rate is too small, the algorithm
will take too long to converge. It is not practical to
determine the optimal setting for the learning rate
before training, and, in fact, the optimal learning rate
changes during the training process, as the algorithm
moves across the performance surface.
The performance of the steepest descent
algorithm can be improved if we allow the learning
rate to change during the training process. An adaptive
learning rate will attempt to keep the learning step size
as large as possible while keeping learning stable. The
learning rate is made responsive to the complexity of
the local error surface.
www.ijera.com
www.ijera.com
An adaptive learning rate requires some
changes in the training procedure used by traingd.
First, the initial network output and error are
calculated. At each epoch new weights and biases are
calculated using the current learning rate. New outputs
and errors are then calculated.
This procedure increases the learning rate, but
only to the extent that the network can learn without
large error increases. When a larger learning rate could
result in stable learning, the learning rate is increased.
When the learning rate is too high to guarantee a
decrease in error, it gets decreased until stable learning
resumes. BP training with an adaptive learning rate is
implemented with the function traingda.
Variable Learning Rate BP with Momentum
(GDX):
The learning rate parameter is used to
determine how fast the BP method converges to the
minimum solution. The larger the learning rate, the
bigger the step and the faster the convergence.
However, if the learning rate is made too large the
algorithm will become unstable. On the other hand, if
the learning rate is set to too small, the algorithm will
take a long time to converge. To speed up the
convergence time, the variable learning rate gradient
descent BP utilizes larger learning rate α when the
neural network model is far from the solution and
smaller learning rate α when the neural net is near the
solution. The new weight vector wk+1 is adjusted the
same as in the gradient descent with momentum above
but with a varying αk. Typically, the new weight
vector wk+1 is defined as:
wk+1=wk - αk+1 gk + µwk-1
(12)
αk+1= α αk
(13)
Resilient BP (Trainrp): The purpose of the resilient
BP (Rprop) training algorithm is to eliminate these
harmful effects of the magnitudes of the partial
derivatives. Only the sign of the derivative is used to
determine the direction of the weight update; the
magnitude of the derivative has no effect on the weight
update. The size of the weight change is determined by
a separate update value. The update value for each
weight and bias is increased by a factor delt_inc
whenever the derivative of the performance function
with respect to that weight has the same sign for two
successive iterations. The update value is decreased by
a factor delt_dec whenever the derivative with respect
that weight changes sign from the previous iteration. If
the derivative is zero, then the update value remains
the same. Whenever the weights are oscillating the
weight change will be reduced. If the weight continues
to change in the same direction for several iterations,
then the magnitude of the weight change will be
increased.
2.2 Radial Basis Function Network (RBFN)
1590 | P a g e
4. Kiran Arya et al Int. Journal of Engineering Research and Application
ISSN : 2248-9622, Vol. 3, Issue 5, Sep-Oct 2013, pp.1588-1595
The back-propagation algorithm of a multilayer feed-forward ANN is a gradient descent
algorithm that may terminate at a local optimum, in
addition to its long training time. We used of number
of different methods for training MLP neural networks
but no one gives faster speed of training neural
network. This problem is overcome in RBF networks
by incorporating the non-linearity in the activation
functions of the nodes of the hidden layer [10].
An RBF neural network describe by Fu [11],
The structure of RBF is similar to a traditional feedforward neural network which is shown in the Fig. 2.
The construction of the RBF neural network involves
three different layers with feed-forward architecture.
The input layer of this network is a set of n units,
which accept the elements of an n-dimensional input
feature vector. The input units are fully connected to
the hidden layer with r hidden units. The goal of the
hidden layer is to cluster the data and reduce its
dimensionality. In this structure hidden layer is named
RBF units. The RBF units are also fully connected to
the output layer. The output layer supplies the response
of neural network to the activation pattern applied to
the input layer. The transformation from the input
space to the RBF-unit space is nonlinear Gaussian
function is used as a non-linear function, whereas the
transformation from the RBF unit space to the output
space is linear.
It should be noted that x is an n-dimensional
input feature vector, ci is an n-dimensional vector
called the center of the RBF unit, σi is the width of
RBF unit and r is the number of the RBF units.
Typically the activation function of the RBF units is
chosen as a Gaussian function with mean vector ci and
variance vector σi as follows:
Φj(x)=exp
),1≤j≤n
(14)
Fig. 2: RBF Neural Network
The output units are linear and therefore the
response of the j-th output unit for input x is given as:
yj(x)=b(j)+
αj(x)w2(i,j)
(15)
Where w2(i,j) is the weight of the link between the ith
hidden layer neuron and the jth output layer neuron, b(j)
is the bi jth output later neuron.
www.ijera.com
www.ijera.com
Functions of RBF Neural Network: Radial basis
function network can be designed with the functions
newrbe and newrb.
Newrbe:
This function can produce a network with
zero error on training vectors. It is called in the
following way.
net = newrbe (P, T, SPREAD)
The function newrbe takes matrices of input
vectors P and target vectors T, and a spread constant
SPREAD for the radial basis layer, and returns a
network with weights and biases such that the outputs
are exactly T when the inputs are P.
The drawback to newrbe is that it produces a
network with as many hidden neurons as there are
input vectors. For this reason, newrbe does not return
an acceptable solution when many input vectors are
needed to properly define a network, as is typically the
case.
Newrb:
The function newrb iteratively creates a radial
basis network one neuron at a time. Neurons are added
to the network until the sum-squared error falls
beneath an error goal or a maximum number of
neurons have been reached. The call for this function
is:
net = newrb (P, T, GOAL, SPREAD)
The function newrb takes matrices of input
and target vectors, P and T, and design parameters
GOAL and, SPREAD, and returns the desired network.
The design method of newrb is similar to that of
newrbe. The difference is that newrb creates neurons
one at a time. At each iteration the input vector that
results in lowering the network error the most, is used
to create a radbas neuron. The error of the new
network is checked, and if low enough newrb is
finished. Otherwise the next neuron is added. This
procedure is repeated until the error goal is met, or the
maximum number of neurons is reached.
Probabilistic Neural Networks
Probabilistic neural networks can be used for
classification problems. When an input is presented,
the first layer computes distances from the input vector
to the training input vectors, and produces a vector
whose elements indicate how close the input is to a
training input. The second layer sums these
contributions for each class of inputs to produce as its
net output a vector of probabilities. Finally, a compete
transfer function on the output of the second layer
picks the maximum of these probabilities, and
produces a 1 for that class and a 0 for the other classes.
2.3 Comparison between MLP and RBFN:In this work we present the properties of two
types of neural networks: traditional neural networks
and RBF networks, both of which are considered as
1591 | P a g e
5. Kiran Arya et al Int. Journal of Engineering Research and Application
ISSN : 2248-9622, Vol. 3, Issue 5, Sep-Oct 2013, pp.1588-1595
universal approximators. The advantages and
disadvantages of the two types of neural network
architectures are analyzed and compared based on FRS
[7].
An RBFN has a single hidden layer, whereas an
MLP may have one or more hidden layers.
Typically the computation nodes of an MLP,
located in a hidden layer or an output layer, share
a common neuronal model. On the other hand, the
computation nodes in the hidden layer of RBFN
are quite different and serve a different purpose
from those in the output layer of the network.
Both are universal approximations. Thus, a RBF
network exists for every MLP, and vice versa.
The activation functions of the hidden nodes of an
RBF network is based on the Euclidean norm of
the input with respect to a center, while that of an
MLP is based on the inner product of input and
weights.
MLPs construct global approximations to
nonlinear input-output mapping. This is a
consequence of the global activation function
(sigmoid) used in MLPs
RBF networks construct local approximations to
input-output data. This is a consequence of the
local Gaussian functions.
III.
Database Used AT&T Face Database
The AT&T database [12] contains 400 grayscale images of 40 persons. Each person has 10
different images of own, each having a resolution of
112×92, and 256 gray levels. Images of the individuals
have been taken varying light intensity, facial
expressions (open/closed eyes, smiling/not smiling)
and facial details (glasses/no glasses). All the images
were taken against a dark homogeneous background,
with tilt and rotation up to 20o sample of images shown
in the Fig. 3.
Fig. 3: sample of images from AT&T database
IV.
EXPERIMENTAL RESULTS AND
DISCUSSIONS
www.ijera.com
For evaluating the proposed work, the
experiments have been performed on AT&T [12] face
database. To establish the improvement in
generalization capability and learning speed of the
neural network, these parameters of generalization
performance have been compared with those of BP and
RBF. The error rate variations for BP learning
algorithm along with RBF have
been evaluated and compared on AT&T database to
show the generalization capability of neural network.
The training time has been measured for these methods
to establish the improvement in the learning speed of
the present algorithm. The experiments have been
performed on a laptop pc with windows 8, 2.0 GHz
core i3, Intel processor using MATLAB 7.0.1.
The experiments have been performed with different
size of training set. The size of training set and test set
is varying based on the number of images per subject,
used for training. For example, if we take one image
per subject for training, the training set size is 40 for
AT & T face database. Similarly for two images per
subject; the training set size is 80 and so on. The
remaining images of the database are used for testing.
The images are taken sequentially from database to
build training set and test set, i.e. if number of images
per subject for training is four, then the first four
images per subject are used in training set and
remaining six images are used for testing. The images
of the database which have been used for training are
not used for testing.
4.1 Error rate on AT&T face database: The
percentage error rate variations on AT&T face
database for different variants of BP algorithm as well
as for RBF with respect to different training set size
has been shown in Fig.4 and Fig. 5 respectively. Table
1 lists the percentage error rate for different variants of
BP algorithm. The percentage error rate decreases as
the number of images per subject for training,
increases. The experimental results of three variants of
BP show that Bp with adaptive learning rate
(TRAINGDA)
obtained
better
generalization
capability rather than other methods like TRAINGDX
and TRAINRP.
Table 2 lists the percentage error rate for different
methods of RBF neural network. There are three
training methods i.e. NEWRB, NEWRBE and
NEWPNN used for trained the RBF neural network, in
which NEWRBE function provides the significant
reduction in percentage error rates in compare of other
two.
Table 1. Comparison of percentage error rate on AT&T face database (percentage error rate using three different
variants of BP i.e. TRAINGDA, TRAINRP, TRAINGDX.
%Error Rate
Number of images per subject used for training
Training Algorithm
1
www.ijera.com
2
3
4
5
6
7
8
1592 | P a g e
9
6. Kiran Arya et al Int. Journal of Engineering Research and Application
ISSN : 2248-9622, Vol. 3, Issue 5, Sep-Oct 2013, pp.1588-1595
%Error rate using
adaptive learning BP
%Error rate using
resilient BP
%Error rate using
adaptive learning with
momentum BP
www.ijera.com
34.72
20
16.62
11.06
10.05
5
5
5
2.5
54.44
53.43
41.78
37.07
36.50
18.75
18.33
15
15
38.61
20.62
16.42
11.07
10.0
6.87
6.6
5
2.5
Performance comparision on ATT database
Performance comparision on ATT database
60
35
Error rate by adaptive learning bp
Error rate by adaptive learning with momentum bp
Error rate by resilient bp
50
Error rate by RBFNN with NEWRB function
Error rate by RBFNN with NEWRBE function
Error rate by RBFNN with NEWPNN function
30
Percentage error rate
Percentage error rate
25
40
30
20
20
15
10
10
0
5
1
2
3
4
5
6
7
No. of images per subject used for training
8
0
9
1
2
3
4
5
6
7
No. of images per subject used for training
8
9
Fig. 4 Comparison of percentage error rate for
Fig. 5 Comparison of percentage error rate different
variants of BP, learning algorithms
for different variants of RBF, method on AT&T face database
AT&T face database.
Table 2. Comparison of percentage error rate on AT&T face database (percentage error rate using three different
variants of RBF i.e. NEWRB, NEWRBE, NEWPNN).
%Error Rate
Training Algorithm
%Error rate using
rbfnn (newrb)
%Error rate using
rbfnn (newrbe)
%Error rate using
rbfnn (newpnn)
Number of images per subject used for training
1
35
2
34.6
3
23.5
4
19.2
5
12.5
6
7.5
7
12.5
6.2
5
31.38
16.87
15.35
9.57
11.0
4.37
4.16
2.5
2.5
30.55
24.68
22.50
21.25
20.0
18.75
15.3
21.25
25.0
4.2 Learning speed on AT&T face database:
Fig. 6 and Fig. 7 show the learning speed of
neural network using different variants of BP and RBF,
which show that the RBF is much faster than the BP
Algorithm in all the situations. Table 3 shows the
comparison of training time obtained using three
different variants of BP algorithm i.e. (TRAINGDA,
TRAINGDX and TRAINRP). The training time
decreases as the number of images per subject for
training, increases. Experimental results show that the
TRAINRP method is faster than the other two
8
9
methods. Some other faster method of BP like trainlm
(Levenberg- Marquardt BP) due to high dimensional
training set (Out of memory error is generated when
using trainlm) so it is not used for our purpose.
Table 4 shows the comparison of training time
obtained using three different methods of RBF neural
network i.e. (NEWRB, NEWRBE and NEWPNN).
The training time decreases as the number of images
per subject for training, increases. Experimental results
show that the NEWRBE method is faster than the other
two methods.
Table 3. Comparison of Training Time (in seconds) on AT&T face database (percentage error rate using different
variants of BP training algorithm).
Training Time(Sec.) &
Ratio
Number of images per subject used for training
Training Algorithm
1
2
3
4
5
6
7
8
9
Training Time using BP
with TRAINGDA
www.ijera.com
75.2
5
123.5
7
115.3
5
210.85
173.96
251.29
254.65
343.50
316.70
1593 | P a g e
7. Kiran Arya et al Int. Journal of Engineering Research and Application
ISSN : 2248-9622, Vol. 3, Issue 5, Sep-Oct 2013, pp.1588-1595
Training Time using BP
with TRAINRP
Training Time using BP
with TRAINGDX
www.ijera.com
7.85
34.10
32.70
72.35
93.10
103.23
176.09
145.85
220.93
73.6
7
119.7
9
188.5
6
306.21
211.9
250.65
340.40
495.10
360.35
Comparision of training time(in seconds) on ATT database
Comparision of training time(in seconds) on ATT database
500
30
Training time by adaptive bp
Training time by adaptive learning with momentum bp
Training time by resilient bp
450
400
Training time by RBFNN with NEWRB
Training time by RBFNN with NEWRBE
Training time by NEWPNN
25
350
Training time
Training time
20
300
250
200
15
10
150
100
5
50
0
1
2
3
4
5
6
7
No. of images per subject used for training
8
Fig. 6 Comparison of training time for
different variants of BP, learning algorithms
database
AT&T
0
9
1
2
3
4
5
6
7
No. of images per subject used for training
8
9
Figure 7 Comparison of training time for
different variants of RBF, method on on AT&T face
face database.
Table 4. Comparison of Training Time (in seconds) on AT&T face database using different variants of RBFNN
training method.
Training Time(Sec.) &
Number of images per subject used for training
Ratio
1
2
3
4
5
6
7
8
9
Training Algorithm
Training Time using
1.14
4.42
5.8
8.8
10.6
13.8
17.0
21.6
25.3
rbfnn with NEWRB
Training Time using
0.43
0.32
0.46
0.64
1.14
0.95
1.32
1.59
1.96
rbfnn with NEWRBE
Training Time using
rbfnn with NEWPNN
0.64
0.29
0.34
0.50
4.3 Error rate with respect to different variations of
Spread vales
RBF neural network is trained using
NEWRBE function; training set used 4 images per
subjects for training and remaining six used for the
testing purpose. We used different spread values and
find out the error rate with respect to these values and
plotted the graph between these two parameters (i.e.
0.32
0.29
0.53
0.35
0.36
spread values and error rate). In Fig. 8 it is shown that
the error rate
is minimum at the point where the spread value is 25.
So a spread value 25 is taken as
an optimal value and used for training purpose.
Error Rate with respect to different SPREAD values for newrbe
70
65
Percentage Error Rate
60
55
50
45
40
35
30
25
20
0
10
20
30
40
50
60
70
Didderent Variations Of Spread Values
80
90
100
Fig. 8
4.4. Performance Comparison
www.ijera.com
The five BP methods that we used are the
gradient descent (GD), the gradient descent with
1594 | P a g e