In real-world scenarios, a system's continual updating of learning knowledge becomes more critical as the data grows faster, producing vast volumes of data. Moreover, the learning process becomes complex when the data features become varied due to the addition or deletion of classes. In such cases, the generated model should learn effectively. Incremental learning refers to the learning of data which constantly arrives over time. This learning requires continuous model adaptation but with limited memory resources without sacrificing model accuracy. In this paper, we proposed a straightforward knowledge transfer algorithm (convolutional auto-encoded extreme learning machine (CAE-ELM)) implemented through the incremental learning methodology for the task of supervised classification using an extreme learning machine (ELM). Incremental learning is achieved by creating an individual train model for each set of homogeneous data and incorporating the knowledge transfer among the models without sacrificing accuracy with minimal memory resources. In CAE-ELM, convolutional neural network (CNN) extracts the features, stacked autoencoder (SAE) reduces the size, and ELM learns and classifies the images. Our proposed algorithm is implemented and experimented on various standard datasets: MNIST, ORL, JAFFE, FERET and Caltech. The results show a positive sign of the correctness of the proposed algorithm.
Effective Multi-Stage Training Model for Edge Computing Devices in Intrusion ...IJCNCJournal
Intrusion detection poses a significant challenge within expansive and persistently interconnected environments. As malicious code continues to advance and sophisticated attack methodologies proliferate, various advanced deep learning-based detection approaches have been proposed. Nevertheless, the complexity and accuracy of intrusion detection models still need further enhancement to render them more adaptable to diverse system categories, particularly within resource-constrained devices, such as those embedded in edge computing systems. This research introduces a three-stage training paradigm, augmented by an enhanced pruning methodology and model compression techniques. The objective is to elevate the system's effectiveness, concurrently maintaining a high level of accuracy for intrusion detection. Empirical assessments conducted on the UNSW-NB15 dataset evince that this solution notably reduces the model's dimensions, while upholding accuracy levels equivalent to similar proposals.
Effective Multi-Stage Training Model for Edge Computing Devices in Intrusion ...IJCNCJournal
Intrusion detection poses a significant challenge within expansive and persistently interconnected environments. As malicious code continues to advance and sophisticated attack methodologies proliferate, various advanced deep learning-based detection approaches have been proposed. Nevertheless, the complexity and accuracy of intrusion detection models still need further enhancement to render them more adaptable to diverse system categories, particularly within resource-constrained devices, such as those embedded in edge computing systems. This research introduces a three-stage training paradigm, augmented by an enhanced pruning methodology and model compression techniques. The objective is to elevate the system's effectiveness, concurrently maintaining a high level of accuracy for intrusion detection. Empirical assessments conducted on the UNSW-NB15 dataset evince that this solution notably reduces the model's dimensions, while upholding accuracy levels equivalent to similar proposals.
Enhanced transformer long short-term memory framework for datastream predictionIJECEIAES
In machine learning, datastream prediction is a challenging issue, particularly when dealing with enormous amounts of continuous data. The dynamic nature of data makes it difficult for traditional models to handle and sustain real-time prediction accuracy. This research uses a multi-processor long short-term memory (MPLSTM) architecture to present a unique framework for datastream regression. By employing several central processing units (CPUs) to divide the datastream into multiple parallel chunks, the MPLSTM framework illustrates the intrinsic parallelism of long short-term memory (LSTM) networks. The MPLSTM framework ensures accurate predictions by skillfully learning and adapting to changing data distributions. Extensive experimental assessments on real-world datasets have demonstrated the clear superiority of the MPLSTM architecture over previous methods. This study uses the transformer, the most recent deep learning breakthrough technology, to demonstrate how well it can handle challenging tasks and emphasizes its critical role as a cutting-edge approach to raising the bar for machine learning.
A new approachto image classification based on adeep multiclass AdaBoosting e...IJECEIAES
In recent years, deep learning methods have been developed in order to solve the problems. These methods were effective in solving complex problems. Convolution is one of the learning methods. This method is applied in classifying and processing of images as well. Hybrid methods are another multi-component machine learning method. These methods are categorized into independent and dependent types. Ada-Boosting algorithm is one of these methods. Today, the classification of images has many applications. So far, several algorithms have been presented for binary and multi-class classification. Most of the above-mentioned methods have a high dependence on the data. The present study intends to use a combination of deep learning methods and associated hybrid methods to classify the images. It is presumed that this method is able to reduce the error rate in images classification. The proposed algorithm consists of the Ada-Boosting hybrid method and bi-layer convolutional learning method. The proposed method was analyzed after it was implemented on a multi-class Mnist data set and displayed the result of the error rate reduction. The results of this study indicate that the error rate of the proposed method is less than Ada-Boosting and convolution methods. Also, the network has more stability compared to the other methods.
Analysis of machine learning algorithms for character recognition: a case stu...nooriasukmaningtyas
This paper covers the work done in handwritten digit recognition and the
various classifiers that have been developed. Methods like MLP, SVM,
Bayesian networks, and Random forests were discussed with their accuracy
and are empirically evaluated. Boosted LetNet 4, an ensemble of various
classifiers, has shown maximum efficiency among these methods.
Quantum machine learning, an important element of quantum computing, recently has gained research attention around the world. In this paper, we have proposed a quantum machine learning model to classify images using a quantum classifier. We exhibit the results of a comprehensive quantum classifier with transfer learning applied to image datasets in particular. The work uses hybrid transfer learning technique along with the classical pre-trained network and variational quantum circuits as their final layers on a small scale of dataset. The implementation is carried out in a quantum processor of a chosen set of highly informative functions using PennyLane a cross-platform software package for using quantum computers to evaluate the high-resolution image classifier. The performance of the model proved to be more accurate than its counterpart and outperforms all other existing classical models in terms of time and competence.
A new model for iris data set classification based on linear support vector m...IJECEIAES
1. The authors propose a new model for classifying the iris data set using a linear support vector machine (SVM) classifier with genetic algorithm optimization of the SVM's C and gamma parameters.
2. Principal component analysis was used to reduce the iris data set features from four to three before classification.
3. The genetic algorithm was shown to optimize the SVM parameters, achieving 98.7% accuracy on the iris data set classification compared to 95.3% accuracy without parameter optimization.
A Time Series ANN Approach for Weather Forecastingijctcm
Weather forecasting is most challenging problem around the world. There are various reason because of its experimented values in meteorology, but it is also a typical unbiased time series forecasting problem in scientific research. A lots of methods proposed by various scientists. The motive behind research is to predict more accurate. This paper contribute the same using artificial neural network (ANN) and simulated in MATLAB to predict two important weather parameters i.e. maximum and minimum temperature. The model has been trained using past 60 years of real data collected from(1901-1960) and tested over 40 years to forecast maximum and minimum temperature. The results based on mean square error function (MSE) confirm, this model which is based on multilayer perceptron has the potential to successful application to weather forecasting
Effective Multi-Stage Training Model for Edge Computing Devices in Intrusion ...IJCNCJournal
Intrusion detection poses a significant challenge within expansive and persistently interconnected environments. As malicious code continues to advance and sophisticated attack methodologies proliferate, various advanced deep learning-based detection approaches have been proposed. Nevertheless, the complexity and accuracy of intrusion detection models still need further enhancement to render them more adaptable to diverse system categories, particularly within resource-constrained devices, such as those embedded in edge computing systems. This research introduces a three-stage training paradigm, augmented by an enhanced pruning methodology and model compression techniques. The objective is to elevate the system's effectiveness, concurrently maintaining a high level of accuracy for intrusion detection. Empirical assessments conducted on the UNSW-NB15 dataset evince that this solution notably reduces the model's dimensions, while upholding accuracy levels equivalent to similar proposals.
Effective Multi-Stage Training Model for Edge Computing Devices in Intrusion ...IJCNCJournal
Intrusion detection poses a significant challenge within expansive and persistently interconnected environments. As malicious code continues to advance and sophisticated attack methodologies proliferate, various advanced deep learning-based detection approaches have been proposed. Nevertheless, the complexity and accuracy of intrusion detection models still need further enhancement to render them more adaptable to diverse system categories, particularly within resource-constrained devices, such as those embedded in edge computing systems. This research introduces a three-stage training paradigm, augmented by an enhanced pruning methodology and model compression techniques. The objective is to elevate the system's effectiveness, concurrently maintaining a high level of accuracy for intrusion detection. Empirical assessments conducted on the UNSW-NB15 dataset evince that this solution notably reduces the model's dimensions, while upholding accuracy levels equivalent to similar proposals.
Enhanced transformer long short-term memory framework for datastream predictionIJECEIAES
In machine learning, datastream prediction is a challenging issue, particularly when dealing with enormous amounts of continuous data. The dynamic nature of data makes it difficult for traditional models to handle and sustain real-time prediction accuracy. This research uses a multi-processor long short-term memory (MPLSTM) architecture to present a unique framework for datastream regression. By employing several central processing units (CPUs) to divide the datastream into multiple parallel chunks, the MPLSTM framework illustrates the intrinsic parallelism of long short-term memory (LSTM) networks. The MPLSTM framework ensures accurate predictions by skillfully learning and adapting to changing data distributions. Extensive experimental assessments on real-world datasets have demonstrated the clear superiority of the MPLSTM architecture over previous methods. This study uses the transformer, the most recent deep learning breakthrough technology, to demonstrate how well it can handle challenging tasks and emphasizes its critical role as a cutting-edge approach to raising the bar for machine learning.
A new approachto image classification based on adeep multiclass AdaBoosting e...IJECEIAES
In recent years, deep learning methods have been developed in order to solve the problems. These methods were effective in solving complex problems. Convolution is one of the learning methods. This method is applied in classifying and processing of images as well. Hybrid methods are another multi-component machine learning method. These methods are categorized into independent and dependent types. Ada-Boosting algorithm is one of these methods. Today, the classification of images has many applications. So far, several algorithms have been presented for binary and multi-class classification. Most of the above-mentioned methods have a high dependence on the data. The present study intends to use a combination of deep learning methods and associated hybrid methods to classify the images. It is presumed that this method is able to reduce the error rate in images classification. The proposed algorithm consists of the Ada-Boosting hybrid method and bi-layer convolutional learning method. The proposed method was analyzed after it was implemented on a multi-class Mnist data set and displayed the result of the error rate reduction. The results of this study indicate that the error rate of the proposed method is less than Ada-Boosting and convolution methods. Also, the network has more stability compared to the other methods.
Analysis of machine learning algorithms for character recognition: a case stu...nooriasukmaningtyas
This paper covers the work done in handwritten digit recognition and the
various classifiers that have been developed. Methods like MLP, SVM,
Bayesian networks, and Random forests were discussed with their accuracy
and are empirically evaluated. Boosted LetNet 4, an ensemble of various
classifiers, has shown maximum efficiency among these methods.
Quantum machine learning, an important element of quantum computing, recently has gained research attention around the world. In this paper, we have proposed a quantum machine learning model to classify images using a quantum classifier. We exhibit the results of a comprehensive quantum classifier with transfer learning applied to image datasets in particular. The work uses hybrid transfer learning technique along with the classical pre-trained network and variational quantum circuits as their final layers on a small scale of dataset. The implementation is carried out in a quantum processor of a chosen set of highly informative functions using PennyLane a cross-platform software package for using quantum computers to evaluate the high-resolution image classifier. The performance of the model proved to be more accurate than its counterpart and outperforms all other existing classical models in terms of time and competence.
A new model for iris data set classification based on linear support vector m...IJECEIAES
1. The authors propose a new model for classifying the iris data set using a linear support vector machine (SVM) classifier with genetic algorithm optimization of the SVM's C and gamma parameters.
2. Principal component analysis was used to reduce the iris data set features from four to three before classification.
3. The genetic algorithm was shown to optimize the SVM parameters, achieving 98.7% accuracy on the iris data set classification compared to 95.3% accuracy without parameter optimization.
A Time Series ANN Approach for Weather Forecastingijctcm
Weather forecasting is most challenging problem around the world. There are various reason because of its experimented values in meteorology, but it is also a typical unbiased time series forecasting problem in scientific research. A lots of methods proposed by various scientists. The motive behind research is to predict more accurate. This paper contribute the same using artificial neural network (ANN) and simulated in MATLAB to predict two important weather parameters i.e. maximum and minimum temperature. The model has been trained using past 60 years of real data collected from(1901-1960) and tested over 40 years to forecast maximum and minimum temperature. The results based on mean square error function (MSE) confirm, this model which is based on multilayer perceptron has the potential to successful application to weather forecasting
Adversarial attack driven data augmentation for medical imagesIJECEIAES
An important stage in medical image analysis is segmentation, which aids in focusing on the required area of an image and speeds up findings. Fortunately, deep learning models have taken over with their high-performing capabilities, making this process simpler. The deep learning model’s reliance on vast data, however, makes it difficult to utilize for medical image analysis due to the scarcity of data samples. Too far, a number of data augmentations techniques have been employed to address the issue of data unavailability. Here, we present a novel method of augmentation that enabled the UNet model to segment the input dataset with about 90% accuracy in just 30 epochs. We describe the us- age of fast gradient sign method (FGSM) as an augmentation tool for adversarial machine learning attack methods. Besides, we have developed the method of Inverse FGSM, which im- proves performance by operating in the opposite way from FGSM adversarial attacks. In comparison to the conventional FGSM methodology, our strategy boosted performance up to 6% to 7% on average. The model became more resilient to hostile attacks because to these two strategies. An innovative implementation of adversarial machine learning and resilience augmentation is revealed by the overall analysis of this study.
Transfer learning with multiple pre-trained network for fundus classificationTELKOMNIKA JOURNAL
Transfer learning (TL) is a technique of reuse and modify a pre-trained network.
It reuses feature extraction layer at a pre-trained network. A target domain in TL
obtains the features knowledge from the source domain. TL modified
classification layer at a pre-trained network. The target domain can do new tasks
according to a purpose. In this article, the target domain is fundus image
classification includes normal and neovascularization. Data consist of
100 patches. The comparison of training and validation data was 70:30.
The selection of training and validation data is done randomly. Steps of TL i.e
load pre-trained networks, replace final layers, train the network, and assess
network accuracy. First, the pre-trained network is a layer configuration of
the convolutional neural network architecture. Pre-trained network used are
AlexNet, VGG16, VGG19, ResNet50, ResNet101, GoogLeNet, Inception-V3,
InceptionResNetV2, and squeezenet. Second, replace the final layer is to replace
the last three layers. They are fully connected layer, softmax, and output layer.
The layer is replaced with a fully connected layer that classifies according to
number of classes. Furthermore, it's followed by a softmax and output layer that
matches with the target domain. Third, we trained the network. Networks were
trained to produce optimal accuracy. In this section, we use gradient descent
algorithm optimization. Fourth, assess network accuracy. The experiment results
show a testing accuracy between 80% and 100%.
A novel ensemble modeling for intrusion detection system IJECEIAES
Vast increase in data through internet services has made computer systems more vulnerable and difficult to protect from malicious attacks. Intrusion detection systems (IDSs) must be more potent in monitoring intrusions. Therefore an effectual Intrusion Detection system architecture is built which employs a facile classification model and generates low false alarm rates and high accuracy. Noticeably, IDS endure enormous amounts of data traffic that contain redundant and irrelevant features, which affect the performance of the IDS negatively. Despite good feature selection approaches leads to a reduction of unrelated and redundant features and attain better classification accuracy in IDS. This paper proposes a novel ensemble model for IDS based on two algorithms Fuzzy Ensemble Feature selection (FEFS) and Fusion of Multiple Classifier (FMC). FEFS is a unification of five feature scores. These scores are obtained by using feature-class distance functions. Aggregation is done using fuzzy union operation. On the other hand, the FMC is the fusion of three classifiers. It works based on Ensemble decisive function. Experiments were made on KDD cup 99 data set have shown that our proposed system works superior to well-known methods such as Support Vector Machines (SVMs), K-Nearest Neighbor (KNN) and Artificial Neural Networks (ANNs). Our examinations ensured clearly the prominence of using ensemble methodology for modeling IDSs, and hence our system is robust and efficient.
Enhancing the stability of the deep neural network using a non-constant lear...IJECEIAES
The data stream is considered the backbone of many real-world applications. These applications are most effective when using modern techniques of machine learning like deep neural networks (DNNs). DNNs are very sensitive to set parameters, the most prominent one is the learning rate. Choosing an appropriate learning rate value is critical because it is able to control the overall network performance. This paper presents a new developing DNN model using a multi-layer perceptron (MLP) structure that includes network training based on the optimal learning rate. Thereupon, this model consists of three hidden layers and does not adopt the stability of the learning rate but has a non-constant value (varying over time) to obtain the optimal learning rate which is able to reduce the error in each iteration and increase the model accuracy. This is done by deriving a new parameter that is added to and subtracted from the learning rate. The proposed model is evaluated by three streaming datasets: electricity, network security layer-knowledge discovery in database (NSL-KDD), and human gait database (HuGaDB) datasets. The results proved that the proposed model achieves better results than the constant model and outperforms previous models in terms of accuracy, where it achieved 88.16%, 98.67%, and 97.63% respectively.
The document describes a machine learning toolbox developed using Python that implements and compares several supervised machine learning algorithms, including Naive Bayes, K-nearest neighbors, decision trees, SVM, and neural networks. The toolbox allows users to test algorithms on various datasets, including Iris and diabetes data, and compare the accuracy results. Testing on these datasets showed Naive Bayes and K-nearest neighbors had the highest average accuracy rates, while neural networks and decision trees showed more variable performance depending on parameters and dataset splits. The toolbox is intended to help users evaluate which algorithms best fit their datasets.
DETECTION OF DIFFERENT TYPES OF SKIN DISEASES USING RASPBERRY PIIRJET Journal
This document presents a study on detecting different types of skin diseases using a Raspberry Pi. The study uses a convolutional neural network model for image classification of skin diseases. Images of different skin diseases are collected and preprocessed. A transfer learning technique is applied using pretrained models like ResNet50, InceptionV3, and MobileNet. The models are trained on dermatological images classified into diseases like melanoma, vitiligo and others. The trained models achieve over 90% accuracy. A Raspberry Pi with camera is used to capture skin images, detect the disease using the model and display the result on an IoT page for easy diagnosis. The system provides a low-cost solution for identifying common skin diseases through smartphone-based analysis.
DETECTION OF DIFFERENT TYPES OF SKIN DISEASES USING RASPBERRY PIIRJET Journal
This document presents a study on detecting different types of skin diseases using a Raspberry Pi. The study uses a convolutional neural network model for image classification. Skin images are collected from datasets and preprocessed by removing noise and performing image enhancement. Various machine learning techniques like transfer learning and data augmentation are used to train models like ResNet50, InceptionV3, and MobileNet for classification. The trained models can classify images of common skin diseases like melanoma, vitiligo, and vascular tumors. The models achieve high accuracy, precision, and recall. The best performing model is integrated with a Raspberry Pi camera to detect skin diseases in real-time and display the results on an IoT webpage for easy diagnosis.
A systematic review on sequence-to-sequence learning with neural network and ...IJECEIAES
We develop a precise writing survey on sequence-to-sequence learning with neural network and its models. The primary aim of this report is to enhance the knowledge of the sequence-to-sequence neural network and to locate the best way to deal with executing it. Three models are mostly used in sequence-to-sequence neural network applications, namely: recurrent neural networks (RNN), connectionist temporal classification (CTC), and attention model. The evidence we adopted in conducting this survey included utilizing the examination inquiries or research questions to determine keywords, which were used to search for bits of peer-reviewed papers, articles, or books at scholastic directories. Through introductory hunts, 790 papers, and scholarly works were found, and with the assistance of choice criteria and PRISMA methodology, the number of papers reviewed decreased to 16. Every one of the 16 articles was categorized by their contribution to each examination question, and they were broken down. At last, the examination papers experienced a quality appraisal where the subsequent range was from 83.3% to 100%. The proposed systematic review enabled us to collect, evaluate, analyze, and explore different approaches of implementing sequence-to-sequence neural network models and pointed out the most common use in machine learning. We followed a methodology that shows the potential of applying these models to real-world applications.
Novel Ensemble Tree for Fast Prediction on Data StreamsIJERA Editor
Data Streams are sequential set of data records. When data appears at highest speed and constantly, so predicting
the class accordingly to the time is very essential. Currently Ensemble modeling techniques are growing
speedily in Classification of Data Stream. Ensemble learning will be accepted since its benefit to manage huge
amount of data stream, means it will manage the data in a large size and also it will be able to manage concept
drifting. Prior learning, mostly focused on accuracy of ensemble model, prediction efficiency has not considered
much since existing ensemble model predicts in linear time, which is enough for small applications and
accessible models workings on integrating some of the classifier. Although real time application has huge
amount of data stream so we required base classifier to recognize dissimilar model and make a high grade
ensemble model. To fix these challenges we developed Ensemble tree which is height balanced tree indexing
structure of base classifier for quick prediction on data streams by ensemble modeling techniques. Ensemble
Tree manages ensembles as geodatabases and it utilizes R tree similar to structure to achieve sub linear time
complexity
Comparative Study of Neural Networks Algorithms for Cloud Computing CPU Sched...IJECEIAES
Cloud Computing is the most powerful computing model of our time. While the major IT providers and consumers are competing to exploit the benefits of this computing model in order to thrive their profits, most of the cloud computing platforms are still built on operating systems that uses basic CPU (Core Processing Unit) scheduling algorithms that lacks the intelligence needed for such innovative computing model. Correspdondingly, this paper presents the benefits of applying Artificial Neural Networks algorithms in regards to enhancing CPU scheduling for Cloud Computing model. Furthermore, a set of characteristics and theoretical metrics are proposed for the sake of comparing the different Artificial Neural Networks algorithms and finding the most accurate algorithm for Cloud Computing CPU Scheduling.
A fuzzy clustering algorithm for high dimensional streaming dataAlexander Decker
This document summarizes a research paper that proposes a new dimension-reduced weighted fuzzy clustering algorithm (sWFCM-HD) for high-dimensional streaming data. The algorithm can cluster datasets that have both high dimensionality and a streaming (continuously arriving) nature. It combines previous work on clustering algorithms for streaming data and high-dimensional data. The paper introduces the algorithm and compares it experimentally to show improvements in memory usage and runtime over other approaches for these types of datasets.
Text classification based on gated recurrent unit combines with support vecto...IJECEIAES
As the amount of unstructured text data that humanity produce largely and a lot of texts are grows on the Internet, so the one of the intelligent technique is require processing it and extracting different types of knowledge from it. Gated recurrent unit (GRU) and support vector machine (SVM) have been successfully used to Natural Language Processing (NLP) systems with comparative, remarkable results. GRU networks perform well in sequential learning tasks and overcome the issues of “vanishing and explosion of gradients in standard recurrent neural networks (RNNs) when captureing long-term dependencies. In this paper, we proposed a text classification model based on improved approaches to this norm by presenting a linear support vector machine (SVM) as the replacement of Softmax in the final output layer of a GRU model. Furthermore, the cross-entropy function shall be replaced with a margin-based function. Empirical results present that the proposed GRU-SVM model achieved comparatively better results than the baseline approaches BLSTM-C, DABN.
End-to-end deep auto-encoder for segmenting a moving object with limited tra...IJECEIAES
The document proposes two end-to-end deep auto-encoder approaches for segmenting moving objects from surveillance videos when limited training data is available. The first approach uses transfer learning with a pre-trained VGG-16 model as the encoder and its transposed architecture as the decoder. The second approach uses a multi-depth auto-encoder with convolutional and upsampling layers. Both approaches apply data augmentation techniques like PCA and traditional methods to increase the training data size. The models are trained and evaluated on the CDnet2014 dataset, achieving better performance than other models trained with limited data.
TOWARDS MORE ACCURATE CLUSTERING METHOD BY USING DYNAMIC TIME WARPINGijdkp
An intrinsic problem of classifiers based on machine learning (ML) methods is that their learning time
grows as the size and complexity of the training dataset increases. For this reason, it is important to have
efficient computational methods and algorithms that can be applied on large datasets, such that it is still
possible to complete the machine learning tasks in reasonable time. In this context, we present in this paper
a more accurate simple process to speed up ML methods. An unsupervised clustering algorithm is
combined with Expectation, Maximization (EM) algorithm to develop an efficient Hidden Markov Model
(HMM) training. The idea of the proposed process consists of two steps. In the first step, training instances
with similar inputs are clustered and a weight factor which represents the frequency of these instances is
assigned to each representative cluster. Dynamic Time Warping technique is used as a dissimilarity
function to cluster similar examples. In the second step, all formulas in the classical HMM training
algorithm (EM) associated with the number of training instances are modified to include the weight factor
in appropriate terms. This process significantly accelerates HMM training while maintaining the same
initial, transition and emission probabilities matrixes as those obtained with the classical HMM training
algorithm. Accordingly, the classification accuracy is preserved. Depending on the size of the training set,
speedups of up to 2200 times is possible when the size is about 100.000 instances. The proposed approach
is not limited to training HMMs, but it can be employed for a large variety of MLs methods.
A Novel GA-SVM Model For Vehicles And Pedestrial Classification In Videosijtsrd
The paper presents a novel algorithm for object classification in videos based on improved support vector machine (SVM) and genetic algorithm. One of the problems of support vector machine is selection of the appropriate parameters for the kernel. This has affected the accuracy of the SVM over the years. This research aims at optimizing the SVM Radial Basis kernel parameters using the genetic algorithm. Moving object classification is a requirement in smart visual surveillance systems as it allows the system to know the kind of object in the scene and be able to recognize the actions the object can perform. This paper presents an GA-SVM machine learning approach for real time object classification in videos. Radial distance signal features are extracted from the silhouettes of object detected in videos. The radial distance signals features are then normalized and fed into the GA-SVM model. The classification rate of 99.39% is achieved with the genetically trained SVM algorithm while 99.1% classification accuracy is achieved with the normal SVM. A comparison of this classifier with some other classifiers in terms of classification accuracy shows a better performance than other classifiers such as the normal SVM, Artificial neural network (ANN), Genetic Artificial neural network (GANN), K-nearest neighbor (K-NN) and K-Means classifiers. Akintola Kolawole G."A Novel GA-SVM Model For Vehicles And Pedestrial Classification In Videos" Published in International Journal of Trend in Scientific Research and Development (ijtsrd), ISSN: 2456-6470, Volume-1 | Issue-4 , June 2017, URL: http://www.ijtsrd.com/papers/ijtsrd109.pdf http://www.ijtsrd.com/computer-science/artificial-intelligence/109/a-novel-ga-svm-model-for-vehicles-and-pedestrial-classification-in-videos/akintola-kolawole-g
A SIMPLE PROCESS TO SPEED UP MACHINE LEARNING METHODS: APPLICATION TO HIDDEN ...cscpconf
An intrinsic problem of classifiers based on machine learning (ML) methods is that their
learning time grows as the size and complexity of the training dataset increases. For this reason, it is important to have efficient computational methods and algorithms that can be applied on large datasets, such that it is still possible to complete the machine learning tasks in reasonable time. In this context, we present in this paper a simple process to speed up ML methods. An unsupervised clustering algorithm is combined with Expectation, Maximization (EM) algorithm to develop an efficient Hidden Markov Model (HMM) training. The idea of the proposed process consists of two steps. The first one involves a preprocessing step to reduce the number of training instances with any information loss. In this step, training instances with similar inputs are clustered and a weight factor which represents the frequency of these instances is assigned to each representative cluster. In the second step, all formulas in theclassical HMM training algorithm (EM) associated with the number of training instances are modified to include the weight factor in appropriate terms. This process significantly accelerates HMM training while maintaining the same initial, transition and emission probabilities matrixes as those obtained with the classical HMM training algorithm. Accordingly, the classification accuracy is preserved. Depending on the size of the training set, speedups of up to 2200 times is possible when the size is about 100.000 instances. The proposed approach is not limited to training HMMs, but it can be employed for a large variety of MLs methods.
IMPROVING SUPERVISED CLASSIFICATION OF DAILY ACTIVITIES LIVING USING NEW COST...cscpconf
The growing population of elders in the society calls for a new approach in care giving. By inferring what activities elderly are performing in their houses it is possible to determine their
physical and cognitive capabilities. In this paper we show the potential of important discriminative classifiers namely the Soft-Support Vector Machines (C-SVM), Conditional Random Fields (CRF) and k-Nearest Neighbors (k-NN) for recognizing activities from sensor patterns in a smart home environment. We address also the class imbalance problem in activity recognition field which has been known to hinder the learning performance of classifiers. Cost sensitive learning is attractive under most imbalanced circumstances, but it is difficult to determine the precise misclassification costs in practice. We introduce a new criterion for selecting the suitable cost parameter C of the C-SVM method. Through our evaluation on four real world imbalanced activity datasets, we demonstrate that C-SVM based on our proposed criterion outperforms the state-of-the-art discriminative methods in activity recognition.
A chi-square-SVM based pedagogical rule extraction method for microarray data...IJAAS Team
Support Vector Machine (SVM) is currently an efficient classification technique due to its ability to capture nonlinearities in diagnostic systems, but it does not reveal the knowledge learnt during training. It is important to understand of how a decision is reached in the machine learning technology, such as bioinformatics. On the other hand, a decision tree has good comprehensibility; the process of converting such incomprehensible models into an understandable model is often regarded as rule extraction. In this paper we proposed an approach for extracting rules from SVM for microarray dataset by combining the merits of both the SVM and decision tree. The proposed approach consists of three steps; the SVM-CHI-SQUARE is employed to reduce the feature set. Dataset with reduced features is used to obtain SVM model and synthetic data is generated. Classification and Regression Tree (CART) is used to generate Rules as the Last phase. We use breast masses dataset from UCI repository where comprehensibility is a key requirement. From the result of the experiment as the reduced feature dataset is used, the proposed approach extracts smaller length rules, thereby improving the comprehensibility of the system. We obtained accuracy of 93.53%, sensitivity of 89.58%, specificity of 96.70%, and training time of 3.195 seconds. A comparative analysis is carried out done with other algorithms.
Using Multi-layered Feed-forward Neural Network (MLFNN) Architecture as Bidir...IOSR Journals
This document presents a method for using a multi-layered feed-forward neural network (MLFNN) architecture as a bidirectional associative memory (BAM) for function approximation. It proposes applying the backpropagation algorithm in two phases - first in the forward direction, then in the backward direction - which allows the MLFNN to work like a BAM. Simulation results show that this two-phase backpropagation algorithm achieves convergence faster than standard backpropagation when approximating the sine function, demonstrating that the MLFNN architecture is better suited for function approximation when trained this way.
AN IMPROVED METHOD FOR IDENTIFYING WELL-TEST INTERPRETATION MODEL BASED ON AG...IAEME Publication
This paper presents an approach based on applying an aggregated predictor formed by multiple versions of a multilayer neural network with a back-propagation optimization algorithm for helping the engineer to get a list of the most appropriate well-test interpretation models for a given set of pressure/ production data. The proposed method consists of three stages: (1) data decorrelation through principal component analysis to reduce the covariance between the variables and the dimension of the input layer in the artificial neural network, (2) bootstrap replicates of the learning set where the data is repeatedly sampled with a random split of the data into train sets and using these as new learning sets, and (3) automatic reservoir model identification through aggregated predictor formed by a plurality vote when predicting a new class. This method is described in detail to ensure successful replication of results. The required training and test dataset were generated by using analytical solution models. In our case, there were used 600 samples: 300 for training, 100 for cross-validation, and 200 for testing. Different network structures were tested during this study to arrive at optimum network design. We notice that the single net methodology always brings about confusion in selecting the correct model even though the training results for the constructed networks are close to 1. We notice also that the principal component analysis is an effective strategy in reducing the number of input features, simplifying the network structure, and lowering the training time of the ANN. The results obtained show that the proposed model provides better performance when predicting new data with a coefficient of correlation approximately equal to 95% Compared to a previous approach 80%, the combination of the PCA and ANN is more stable and determine the more accurate results with lesser computational complexity than was feasible previously. Clearly, the aggregated predictor is more stable and shows less bad classes compared to the previous approach.
Redefining brain tumor segmentation: a cutting-edge convolutional neural netw...IJECEIAES
Medical image analysis has witnessed significant advancements with deep learning techniques. In the domain of brain tumor segmentation, the ability to
precisely delineate tumor boundaries from magnetic resonance imaging (MRI)
scans holds profound implications for diagnosis. This study presents an ensemble convolutional neural network (CNN) with transfer learning, integrating
the state-of-the-art Deeplabv3+ architecture with the ResNet18 backbone. The
model is rigorously trained and evaluated, exhibiting remarkable performance
metrics, including an impressive global accuracy of 99.286%, a high-class accuracy of 82.191%, a mean intersection over union (IoU) of 79.900%, a weighted
IoU of 98.620%, and a Boundary F1 (BF) score of 83.303%. Notably, a detailed comparative analysis with existing methods showcases the superiority of
our proposed model. These findings underscore the model’s competence in precise brain tumor localization, underscoring its potential to revolutionize medical
image analysis and enhance healthcare outcomes. This research paves the way
for future exploration and optimization of advanced CNN models in medical
imaging, emphasizing addressing false positives and resource efficiency.
Embedded machine learning-based road conditions and driving behavior monitoringIJECEIAES
Car accident rates have increased in recent years, resulting in losses in human lives, properties, and other financial costs. An embedded machine learning-based system is developed to address this critical issue. The system can monitor road conditions, detect driving patterns, and identify aggressive driving behaviors. The system is based on neural networks trained on a comprehensive dataset of driving events, driving styles, and road conditions. The system effectively detects potential risks and helps mitigate the frequency and impact of accidents. The primary goal is to ensure the safety of drivers and vehicles. Collecting data involved gathering information on three key road events: normal street and normal drive, speed bumps, circular yellow speed bumps, and three aggressive driving actions: sudden start, sudden stop, and sudden entry. The gathered data is processed and analyzed using a machine learning system designed for limited power and memory devices. The developed system resulted in 91.9% accuracy, 93.6% precision, and 92% recall. The achieved inference time on an Arduino Nano 33 BLE Sense with a 32-bit CPU running at 64 MHz is 34 ms and requires 2.6 kB peak RAM and 139.9 kB program flash memory, making it suitable for resource-constrained embedded systems.
More Related Content
Similar to Convolutional auto-encoded extreme learning machine for incremental learning of heterogeneous images
Adversarial attack driven data augmentation for medical imagesIJECEIAES
An important stage in medical image analysis is segmentation, which aids in focusing on the required area of an image and speeds up findings. Fortunately, deep learning models have taken over with their high-performing capabilities, making this process simpler. The deep learning model’s reliance on vast data, however, makes it difficult to utilize for medical image analysis due to the scarcity of data samples. Too far, a number of data augmentations techniques have been employed to address the issue of data unavailability. Here, we present a novel method of augmentation that enabled the UNet model to segment the input dataset with about 90% accuracy in just 30 epochs. We describe the us- age of fast gradient sign method (FGSM) as an augmentation tool for adversarial machine learning attack methods. Besides, we have developed the method of Inverse FGSM, which im- proves performance by operating in the opposite way from FGSM adversarial attacks. In comparison to the conventional FGSM methodology, our strategy boosted performance up to 6% to 7% on average. The model became more resilient to hostile attacks because to these two strategies. An innovative implementation of adversarial machine learning and resilience augmentation is revealed by the overall analysis of this study.
Transfer learning with multiple pre-trained network for fundus classificationTELKOMNIKA JOURNAL
Transfer learning (TL) is a technique of reuse and modify a pre-trained network.
It reuses feature extraction layer at a pre-trained network. A target domain in TL
obtains the features knowledge from the source domain. TL modified
classification layer at a pre-trained network. The target domain can do new tasks
according to a purpose. In this article, the target domain is fundus image
classification includes normal and neovascularization. Data consist of
100 patches. The comparison of training and validation data was 70:30.
The selection of training and validation data is done randomly. Steps of TL i.e
load pre-trained networks, replace final layers, train the network, and assess
network accuracy. First, the pre-trained network is a layer configuration of
the convolutional neural network architecture. Pre-trained network used are
AlexNet, VGG16, VGG19, ResNet50, ResNet101, GoogLeNet, Inception-V3,
InceptionResNetV2, and squeezenet. Second, replace the final layer is to replace
the last three layers. They are fully connected layer, softmax, and output layer.
The layer is replaced with a fully connected layer that classifies according to
number of classes. Furthermore, it's followed by a softmax and output layer that
matches with the target domain. Third, we trained the network. Networks were
trained to produce optimal accuracy. In this section, we use gradient descent
algorithm optimization. Fourth, assess network accuracy. The experiment results
show a testing accuracy between 80% and 100%.
A novel ensemble modeling for intrusion detection system IJECEIAES
Vast increase in data through internet services has made computer systems more vulnerable and difficult to protect from malicious attacks. Intrusion detection systems (IDSs) must be more potent in monitoring intrusions. Therefore an effectual Intrusion Detection system architecture is built which employs a facile classification model and generates low false alarm rates and high accuracy. Noticeably, IDS endure enormous amounts of data traffic that contain redundant and irrelevant features, which affect the performance of the IDS negatively. Despite good feature selection approaches leads to a reduction of unrelated and redundant features and attain better classification accuracy in IDS. This paper proposes a novel ensemble model for IDS based on two algorithms Fuzzy Ensemble Feature selection (FEFS) and Fusion of Multiple Classifier (FMC). FEFS is a unification of five feature scores. These scores are obtained by using feature-class distance functions. Aggregation is done using fuzzy union operation. On the other hand, the FMC is the fusion of three classifiers. It works based on Ensemble decisive function. Experiments were made on KDD cup 99 data set have shown that our proposed system works superior to well-known methods such as Support Vector Machines (SVMs), K-Nearest Neighbor (KNN) and Artificial Neural Networks (ANNs). Our examinations ensured clearly the prominence of using ensemble methodology for modeling IDSs, and hence our system is robust and efficient.
Enhancing the stability of the deep neural network using a non-constant lear...IJECEIAES
The data stream is considered the backbone of many real-world applications. These applications are most effective when using modern techniques of machine learning like deep neural networks (DNNs). DNNs are very sensitive to set parameters, the most prominent one is the learning rate. Choosing an appropriate learning rate value is critical because it is able to control the overall network performance. This paper presents a new developing DNN model using a multi-layer perceptron (MLP) structure that includes network training based on the optimal learning rate. Thereupon, this model consists of three hidden layers and does not adopt the stability of the learning rate but has a non-constant value (varying over time) to obtain the optimal learning rate which is able to reduce the error in each iteration and increase the model accuracy. This is done by deriving a new parameter that is added to and subtracted from the learning rate. The proposed model is evaluated by three streaming datasets: electricity, network security layer-knowledge discovery in database (NSL-KDD), and human gait database (HuGaDB) datasets. The results proved that the proposed model achieves better results than the constant model and outperforms previous models in terms of accuracy, where it achieved 88.16%, 98.67%, and 97.63% respectively.
The document describes a machine learning toolbox developed using Python that implements and compares several supervised machine learning algorithms, including Naive Bayes, K-nearest neighbors, decision trees, SVM, and neural networks. The toolbox allows users to test algorithms on various datasets, including Iris and diabetes data, and compare the accuracy results. Testing on these datasets showed Naive Bayes and K-nearest neighbors had the highest average accuracy rates, while neural networks and decision trees showed more variable performance depending on parameters and dataset splits. The toolbox is intended to help users evaluate which algorithms best fit their datasets.
DETECTION OF DIFFERENT TYPES OF SKIN DISEASES USING RASPBERRY PIIRJET Journal
This document presents a study on detecting different types of skin diseases using a Raspberry Pi. The study uses a convolutional neural network model for image classification of skin diseases. Images of different skin diseases are collected and preprocessed. A transfer learning technique is applied using pretrained models like ResNet50, InceptionV3, and MobileNet. The models are trained on dermatological images classified into diseases like melanoma, vitiligo and others. The trained models achieve over 90% accuracy. A Raspberry Pi with camera is used to capture skin images, detect the disease using the model and display the result on an IoT page for easy diagnosis. The system provides a low-cost solution for identifying common skin diseases through smartphone-based analysis.
DETECTION OF DIFFERENT TYPES OF SKIN DISEASES USING RASPBERRY PIIRJET Journal
This document presents a study on detecting different types of skin diseases using a Raspberry Pi. The study uses a convolutional neural network model for image classification. Skin images are collected from datasets and preprocessed by removing noise and performing image enhancement. Various machine learning techniques like transfer learning and data augmentation are used to train models like ResNet50, InceptionV3, and MobileNet for classification. The trained models can classify images of common skin diseases like melanoma, vitiligo, and vascular tumors. The models achieve high accuracy, precision, and recall. The best performing model is integrated with a Raspberry Pi camera to detect skin diseases in real-time and display the results on an IoT webpage for easy diagnosis.
A systematic review on sequence-to-sequence learning with neural network and ...IJECEIAES
We develop a precise writing survey on sequence-to-sequence learning with neural network and its models. The primary aim of this report is to enhance the knowledge of the sequence-to-sequence neural network and to locate the best way to deal with executing it. Three models are mostly used in sequence-to-sequence neural network applications, namely: recurrent neural networks (RNN), connectionist temporal classification (CTC), and attention model. The evidence we adopted in conducting this survey included utilizing the examination inquiries or research questions to determine keywords, which were used to search for bits of peer-reviewed papers, articles, or books at scholastic directories. Through introductory hunts, 790 papers, and scholarly works were found, and with the assistance of choice criteria and PRISMA methodology, the number of papers reviewed decreased to 16. Every one of the 16 articles was categorized by their contribution to each examination question, and they were broken down. At last, the examination papers experienced a quality appraisal where the subsequent range was from 83.3% to 100%. The proposed systematic review enabled us to collect, evaluate, analyze, and explore different approaches of implementing sequence-to-sequence neural network models and pointed out the most common use in machine learning. We followed a methodology that shows the potential of applying these models to real-world applications.
Novel Ensemble Tree for Fast Prediction on Data StreamsIJERA Editor
Data Streams are sequential set of data records. When data appears at highest speed and constantly, so predicting
the class accordingly to the time is very essential. Currently Ensemble modeling techniques are growing
speedily in Classification of Data Stream. Ensemble learning will be accepted since its benefit to manage huge
amount of data stream, means it will manage the data in a large size and also it will be able to manage concept
drifting. Prior learning, mostly focused on accuracy of ensemble model, prediction efficiency has not considered
much since existing ensemble model predicts in linear time, which is enough for small applications and
accessible models workings on integrating some of the classifier. Although real time application has huge
amount of data stream so we required base classifier to recognize dissimilar model and make a high grade
ensemble model. To fix these challenges we developed Ensemble tree which is height balanced tree indexing
structure of base classifier for quick prediction on data streams by ensemble modeling techniques. Ensemble
Tree manages ensembles as geodatabases and it utilizes R tree similar to structure to achieve sub linear time
complexity
Comparative Study of Neural Networks Algorithms for Cloud Computing CPU Sched...IJECEIAES
Cloud Computing is the most powerful computing model of our time. While the major IT providers and consumers are competing to exploit the benefits of this computing model in order to thrive their profits, most of the cloud computing platforms are still built on operating systems that uses basic CPU (Core Processing Unit) scheduling algorithms that lacks the intelligence needed for such innovative computing model. Correspdondingly, this paper presents the benefits of applying Artificial Neural Networks algorithms in regards to enhancing CPU scheduling for Cloud Computing model. Furthermore, a set of characteristics and theoretical metrics are proposed for the sake of comparing the different Artificial Neural Networks algorithms and finding the most accurate algorithm for Cloud Computing CPU Scheduling.
A fuzzy clustering algorithm for high dimensional streaming dataAlexander Decker
This document summarizes a research paper that proposes a new dimension-reduced weighted fuzzy clustering algorithm (sWFCM-HD) for high-dimensional streaming data. The algorithm can cluster datasets that have both high dimensionality and a streaming (continuously arriving) nature. It combines previous work on clustering algorithms for streaming data and high-dimensional data. The paper introduces the algorithm and compares it experimentally to show improvements in memory usage and runtime over other approaches for these types of datasets.
Text classification based on gated recurrent unit combines with support vecto...IJECEIAES
As the amount of unstructured text data that humanity produce largely and a lot of texts are grows on the Internet, so the one of the intelligent technique is require processing it and extracting different types of knowledge from it. Gated recurrent unit (GRU) and support vector machine (SVM) have been successfully used to Natural Language Processing (NLP) systems with comparative, remarkable results. GRU networks perform well in sequential learning tasks and overcome the issues of “vanishing and explosion of gradients in standard recurrent neural networks (RNNs) when captureing long-term dependencies. In this paper, we proposed a text classification model based on improved approaches to this norm by presenting a linear support vector machine (SVM) as the replacement of Softmax in the final output layer of a GRU model. Furthermore, the cross-entropy function shall be replaced with a margin-based function. Empirical results present that the proposed GRU-SVM model achieved comparatively better results than the baseline approaches BLSTM-C, DABN.
End-to-end deep auto-encoder for segmenting a moving object with limited tra...IJECEIAES
The document proposes two end-to-end deep auto-encoder approaches for segmenting moving objects from surveillance videos when limited training data is available. The first approach uses transfer learning with a pre-trained VGG-16 model as the encoder and its transposed architecture as the decoder. The second approach uses a multi-depth auto-encoder with convolutional and upsampling layers. Both approaches apply data augmentation techniques like PCA and traditional methods to increase the training data size. The models are trained and evaluated on the CDnet2014 dataset, achieving better performance than other models trained with limited data.
TOWARDS MORE ACCURATE CLUSTERING METHOD BY USING DYNAMIC TIME WARPINGijdkp
An intrinsic problem of classifiers based on machine learning (ML) methods is that their learning time
grows as the size and complexity of the training dataset increases. For this reason, it is important to have
efficient computational methods and algorithms that can be applied on large datasets, such that it is still
possible to complete the machine learning tasks in reasonable time. In this context, we present in this paper
a more accurate simple process to speed up ML methods. An unsupervised clustering algorithm is
combined with Expectation, Maximization (EM) algorithm to develop an efficient Hidden Markov Model
(HMM) training. The idea of the proposed process consists of two steps. In the first step, training instances
with similar inputs are clustered and a weight factor which represents the frequency of these instances is
assigned to each representative cluster. Dynamic Time Warping technique is used as a dissimilarity
function to cluster similar examples. In the second step, all formulas in the classical HMM training
algorithm (EM) associated with the number of training instances are modified to include the weight factor
in appropriate terms. This process significantly accelerates HMM training while maintaining the same
initial, transition and emission probabilities matrixes as those obtained with the classical HMM training
algorithm. Accordingly, the classification accuracy is preserved. Depending on the size of the training set,
speedups of up to 2200 times is possible when the size is about 100.000 instances. The proposed approach
is not limited to training HMMs, but it can be employed for a large variety of MLs methods.
A Novel GA-SVM Model For Vehicles And Pedestrial Classification In Videosijtsrd
The paper presents a novel algorithm for object classification in videos based on improved support vector machine (SVM) and genetic algorithm. One of the problems of support vector machine is selection of the appropriate parameters for the kernel. This has affected the accuracy of the SVM over the years. This research aims at optimizing the SVM Radial Basis kernel parameters using the genetic algorithm. Moving object classification is a requirement in smart visual surveillance systems as it allows the system to know the kind of object in the scene and be able to recognize the actions the object can perform. This paper presents an GA-SVM machine learning approach for real time object classification in videos. Radial distance signal features are extracted from the silhouettes of object detected in videos. The radial distance signals features are then normalized and fed into the GA-SVM model. The classification rate of 99.39% is achieved with the genetically trained SVM algorithm while 99.1% classification accuracy is achieved with the normal SVM. A comparison of this classifier with some other classifiers in terms of classification accuracy shows a better performance than other classifiers such as the normal SVM, Artificial neural network (ANN), Genetic Artificial neural network (GANN), K-nearest neighbor (K-NN) and K-Means classifiers. Akintola Kolawole G."A Novel GA-SVM Model For Vehicles And Pedestrial Classification In Videos" Published in International Journal of Trend in Scientific Research and Development (ijtsrd), ISSN: 2456-6470, Volume-1 | Issue-4 , June 2017, URL: http://www.ijtsrd.com/papers/ijtsrd109.pdf http://www.ijtsrd.com/computer-science/artificial-intelligence/109/a-novel-ga-svm-model-for-vehicles-and-pedestrial-classification-in-videos/akintola-kolawole-g
A SIMPLE PROCESS TO SPEED UP MACHINE LEARNING METHODS: APPLICATION TO HIDDEN ...cscpconf
An intrinsic problem of classifiers based on machine learning (ML) methods is that their
learning time grows as the size and complexity of the training dataset increases. For this reason, it is important to have efficient computational methods and algorithms that can be applied on large datasets, such that it is still possible to complete the machine learning tasks in reasonable time. In this context, we present in this paper a simple process to speed up ML methods. An unsupervised clustering algorithm is combined with Expectation, Maximization (EM) algorithm to develop an efficient Hidden Markov Model (HMM) training. The idea of the proposed process consists of two steps. The first one involves a preprocessing step to reduce the number of training instances with any information loss. In this step, training instances with similar inputs are clustered and a weight factor which represents the frequency of these instances is assigned to each representative cluster. In the second step, all formulas in theclassical HMM training algorithm (EM) associated with the number of training instances are modified to include the weight factor in appropriate terms. This process significantly accelerates HMM training while maintaining the same initial, transition and emission probabilities matrixes as those obtained with the classical HMM training algorithm. Accordingly, the classification accuracy is preserved. Depending on the size of the training set, speedups of up to 2200 times is possible when the size is about 100.000 instances. The proposed approach is not limited to training HMMs, but it can be employed for a large variety of MLs methods.
IMPROVING SUPERVISED CLASSIFICATION OF DAILY ACTIVITIES LIVING USING NEW COST...cscpconf
The growing population of elders in the society calls for a new approach in care giving. By inferring what activities elderly are performing in their houses it is possible to determine their
physical and cognitive capabilities. In this paper we show the potential of important discriminative classifiers namely the Soft-Support Vector Machines (C-SVM), Conditional Random Fields (CRF) and k-Nearest Neighbors (k-NN) for recognizing activities from sensor patterns in a smart home environment. We address also the class imbalance problem in activity recognition field which has been known to hinder the learning performance of classifiers. Cost sensitive learning is attractive under most imbalanced circumstances, but it is difficult to determine the precise misclassification costs in practice. We introduce a new criterion for selecting the suitable cost parameter C of the C-SVM method. Through our evaluation on four real world imbalanced activity datasets, we demonstrate that C-SVM based on our proposed criterion outperforms the state-of-the-art discriminative methods in activity recognition.
A chi-square-SVM based pedagogical rule extraction method for microarray data...IJAAS Team
Support Vector Machine (SVM) is currently an efficient classification technique due to its ability to capture nonlinearities in diagnostic systems, but it does not reveal the knowledge learnt during training. It is important to understand of how a decision is reached in the machine learning technology, such as bioinformatics. On the other hand, a decision tree has good comprehensibility; the process of converting such incomprehensible models into an understandable model is often regarded as rule extraction. In this paper we proposed an approach for extracting rules from SVM for microarray dataset by combining the merits of both the SVM and decision tree. The proposed approach consists of three steps; the SVM-CHI-SQUARE is employed to reduce the feature set. Dataset with reduced features is used to obtain SVM model and synthetic data is generated. Classification and Regression Tree (CART) is used to generate Rules as the Last phase. We use breast masses dataset from UCI repository where comprehensibility is a key requirement. From the result of the experiment as the reduced feature dataset is used, the proposed approach extracts smaller length rules, thereby improving the comprehensibility of the system. We obtained accuracy of 93.53%, sensitivity of 89.58%, specificity of 96.70%, and training time of 3.195 seconds. A comparative analysis is carried out done with other algorithms.
Using Multi-layered Feed-forward Neural Network (MLFNN) Architecture as Bidir...IOSR Journals
This document presents a method for using a multi-layered feed-forward neural network (MLFNN) architecture as a bidirectional associative memory (BAM) for function approximation. It proposes applying the backpropagation algorithm in two phases - first in the forward direction, then in the backward direction - which allows the MLFNN to work like a BAM. Simulation results show that this two-phase backpropagation algorithm achieves convergence faster than standard backpropagation when approximating the sine function, demonstrating that the MLFNN architecture is better suited for function approximation when trained this way.
AN IMPROVED METHOD FOR IDENTIFYING WELL-TEST INTERPRETATION MODEL BASED ON AG...IAEME Publication
This paper presents an approach based on applying an aggregated predictor formed by multiple versions of a multilayer neural network with a back-propagation optimization algorithm for helping the engineer to get a list of the most appropriate well-test interpretation models for a given set of pressure/ production data. The proposed method consists of three stages: (1) data decorrelation through principal component analysis to reduce the covariance between the variables and the dimension of the input layer in the artificial neural network, (2) bootstrap replicates of the learning set where the data is repeatedly sampled with a random split of the data into train sets and using these as new learning sets, and (3) automatic reservoir model identification through aggregated predictor formed by a plurality vote when predicting a new class. This method is described in detail to ensure successful replication of results. The required training and test dataset were generated by using analytical solution models. In our case, there were used 600 samples: 300 for training, 100 for cross-validation, and 200 for testing. Different network structures were tested during this study to arrive at optimum network design. We notice that the single net methodology always brings about confusion in selecting the correct model even though the training results for the constructed networks are close to 1. We notice also that the principal component analysis is an effective strategy in reducing the number of input features, simplifying the network structure, and lowering the training time of the ANN. The results obtained show that the proposed model provides better performance when predicting new data with a coefficient of correlation approximately equal to 95% Compared to a previous approach 80%, the combination of the PCA and ANN is more stable and determine the more accurate results with lesser computational complexity than was feasible previously. Clearly, the aggregated predictor is more stable and shows less bad classes compared to the previous approach.
Similar to Convolutional auto-encoded extreme learning machine for incremental learning of heterogeneous images (20)
Redefining brain tumor segmentation: a cutting-edge convolutional neural netw...IJECEIAES
Medical image analysis has witnessed significant advancements with deep learning techniques. In the domain of brain tumor segmentation, the ability to
precisely delineate tumor boundaries from magnetic resonance imaging (MRI)
scans holds profound implications for diagnosis. This study presents an ensemble convolutional neural network (CNN) with transfer learning, integrating
the state-of-the-art Deeplabv3+ architecture with the ResNet18 backbone. The
model is rigorously trained and evaluated, exhibiting remarkable performance
metrics, including an impressive global accuracy of 99.286%, a high-class accuracy of 82.191%, a mean intersection over union (IoU) of 79.900%, a weighted
IoU of 98.620%, and a Boundary F1 (BF) score of 83.303%. Notably, a detailed comparative analysis with existing methods showcases the superiority of
our proposed model. These findings underscore the model’s competence in precise brain tumor localization, underscoring its potential to revolutionize medical
image analysis and enhance healthcare outcomes. This research paves the way
for future exploration and optimization of advanced CNN models in medical
imaging, emphasizing addressing false positives and resource efficiency.
Embedded machine learning-based road conditions and driving behavior monitoringIJECEIAES
Car accident rates have increased in recent years, resulting in losses in human lives, properties, and other financial costs. An embedded machine learning-based system is developed to address this critical issue. The system can monitor road conditions, detect driving patterns, and identify aggressive driving behaviors. The system is based on neural networks trained on a comprehensive dataset of driving events, driving styles, and road conditions. The system effectively detects potential risks and helps mitigate the frequency and impact of accidents. The primary goal is to ensure the safety of drivers and vehicles. Collecting data involved gathering information on three key road events: normal street and normal drive, speed bumps, circular yellow speed bumps, and three aggressive driving actions: sudden start, sudden stop, and sudden entry. The gathered data is processed and analyzed using a machine learning system designed for limited power and memory devices. The developed system resulted in 91.9% accuracy, 93.6% precision, and 92% recall. The achieved inference time on an Arduino Nano 33 BLE Sense with a 32-bit CPU running at 64 MHz is 34 ms and requires 2.6 kB peak RAM and 139.9 kB program flash memory, making it suitable for resource-constrained embedded systems.
Advanced control scheme of doubly fed induction generator for wind turbine us...IJECEIAES
This paper describes a speed control device for generating electrical energy on an electricity network based on the doubly fed induction generator (DFIG) used for wind power conversion systems. At first, a double-fed induction generator model was constructed. A control law is formulated to govern the flow of energy between the stator of a DFIG and the energy network using three types of controllers: proportional integral (PI), sliding mode controller (SMC) and second order sliding mode controller (SOSMC). Their different results in terms of power reference tracking, reaction to unexpected speed fluctuations, sensitivity to perturbations, and resilience against machine parameter alterations are compared. MATLAB/Simulink was used to conduct the simulations for the preceding study. Multiple simulations have shown very satisfying results, and the investigations demonstrate the efficacy and power-enhancing capabilities of the suggested control system.
Neural network optimizer of proportional-integral-differential controller par...IJECEIAES
Wide application of proportional-integral-differential (PID)-regulator in industry requires constant improvement of methods of its parameters adjustment. The paper deals with the issues of optimization of PID-regulator parameters with the use of neural network technology methods. A methodology for choosing the architecture (structure) of neural network optimizer is proposed, which consists in determining the number of layers, the number of neurons in each layer, as well as the form and type of activation function. Algorithms of neural network training based on the application of the method of minimizing the mismatch between the regulated value and the target value are developed. The method of back propagation of gradients is proposed to select the optimal training rate of neurons of the neural network. The neural network optimizer, which is a superstructure of the linear PID controller, allows increasing the regulation accuracy from 0.23 to 0.09, thus reducing the power consumption from 65% to 53%. The results of the conducted experiments allow us to conclude that the created neural superstructure may well become a prototype of an automatic voltage regulator (AVR)-type industrial controller for tuning the parameters of the PID controller.
An improved modulation technique suitable for a three level flying capacitor ...IJECEIAES
This research paper introduces an innovative modulation technique for controlling a 3-level flying capacitor multilevel inverter (FCMLI), aiming to streamline the modulation process in contrast to conventional methods. The proposed
simplified modulation technique paves the way for more straightforward and
efficient control of multilevel inverters, enabling their widespread adoption and
integration into modern power electronic systems. Through the amalgamation of
sinusoidal pulse width modulation (SPWM) with a high-frequency square wave
pulse, this controlling technique attains energy equilibrium across the coupling
capacitor. The modulation scheme incorporates a simplified switching pattern
and a decreased count of voltage references, thereby simplifying the control
algorithm.
A review on features and methods of potential fishing zoneIJECEIAES
This review focuses on the importance of identifying potential fishing zones in seawater for sustainable fishing practices. It explores features like sea surface temperature (SST) and sea surface height (SSH), along with classification methods such as classifiers. The features like SST, SSH, and different classifiers used to classify the data, have been figured out in this review study. This study underscores the importance of examining potential fishing zones using advanced analytical techniques. It thoroughly explores the methodologies employed by researchers, covering both past and current approaches. The examination centers on data characteristics and the application of classification algorithms for classification of potential fishing zones. Furthermore, the prediction of potential fishing zones relies significantly on the effectiveness of classification algorithms. Previous research has assessed the performance of models like support vector machines, naïve Bayes, and artificial neural networks (ANN). In the previous result, the results of support vector machine (SVM) were 97.6% more accurate than naive Bayes's 94.2% to classify test data for fisheries classification. By considering the recent works in this area, several recommendations for future works are presented to further improve the performance of the potential fishing zone models, which is important to the fisheries community.
Electrical signal interference minimization using appropriate core material f...IJECEIAES
As demand for smaller, quicker, and more powerful devices rises, Moore's law is strictly followed. The industry has worked hard to make little devices that boost productivity. The goal is to optimize device density. Scientists are reducing connection delays to improve circuit performance. This helped them understand three-dimensional integrated circuit (3D IC) concepts, which stack active devices and create vertical connections to diminish latency and lower interconnects. Electrical involvement is a big worry with 3D integrates circuits. Researchers have developed and tested through silicon via (TSV) and substrates to decrease electrical wave involvement. This study illustrates a novel noise coupling reduction method using several electrical involvement models. A 22% drop in electrical involvement from wave-carrying to victim TSVs introduces this new paradigm and improves system performance even at higher THz frequencies.
Electric vehicle and photovoltaic advanced roles in enhancing the financial p...IJECEIAES
Climate change's impact on the planet forced the United Nations and governments to promote green energies and electric transportation. The deployments of photovoltaic (PV) and electric vehicle (EV) systems gained stronger momentum due to their numerous advantages over fossil fuel types. The advantages go beyond sustainability to reach financial support and stability. The work in this paper introduces the hybrid system between PV and EV to support industrial and commercial plants. This paper covers the theoretical framework of the proposed hybrid system including the required equation to complete the cost analysis when PV and EV are present. In addition, the proposed design diagram which sets the priorities and requirements of the system is presented. The proposed approach allows setup to advance their power stability, especially during power outages. The presented information supports researchers and plant owners to complete the necessary analysis while promoting the deployment of clean energy. The result of a case study that represents a dairy milk farmer supports the theoretical works and highlights its advanced benefits to existing plants. The short return on investment of the proposed approach supports the paper's novelty approach for the sustainable electrical system. In addition, the proposed system allows for an isolated power setup without the need for a transmission line which enhances the safety of the electrical network
Bibliometric analysis highlighting the role of women in addressing climate ch...IJECEIAES
Fossil fuel consumption increased quickly, contributing to climate change
that is evident in unusual flooding and draughts, and global warming. Over
the past ten years, women's involvement in society has grown dramatically,
and they succeeded in playing a noticeable role in reducing climate change.
A bibliometric analysis of data from the last ten years has been carried out to
examine the role of women in addressing the climate change. The analysis's
findings discussed the relevant to the sustainable development goals (SDGs),
particularly SDG 7 and SDG 13. The results considered contributions made
by women in the various sectors while taking geographic dispersion into
account. The bibliometric analysis delves into topics including women's
leadership in environmental groups, their involvement in policymaking, their
contributions to sustainable development projects, and the influence of
gender diversity on attempts to mitigate climate change. This study's results
highlight how women have influenced policies and actions related to climate
change, point out areas of research deficiency and recommendations on how
to increase role of the women in addressing the climate change and
achieving sustainability. To achieve more successful results, this initiative
aims to highlight the significance of gender equality and encourage
inclusivity in climate change decision-making processes.
Voltage and frequency control of microgrid in presence of micro-turbine inter...IJECEIAES
The active and reactive load changes have a significant impact on voltage
and frequency. In this paper, in order to stabilize the microgrid (MG) against
load variations in islanding mode, the active and reactive power of all
distributed generators (DGs), including energy storage (battery), diesel
generator, and micro-turbine, are controlled. The micro-turbine generator is
connected to MG through a three-phase to three-phase matrix converter, and
the droop control method is applied for controlling the voltage and
frequency of MG. In addition, a method is introduced for voltage and
frequency control of micro-turbines in the transition state from gridconnected mode to islanding mode. A novel switching strategy of the matrix
converter is used for converting the high-frequency output voltage of the
micro-turbine to the grid-side frequency of the utility system. Moreover,
using the switching strategy, the low-order harmonics in the output current
and voltage are not produced, and consequently, the size of the output filter
would be reduced. In fact, the suggested control strategy is load-independent
and has no frequency conversion restrictions. The proposed approach for
voltage and frequency regulation demonstrates exceptional performance and
favorable response across various load alteration scenarios. The suggested
strategy is examined in several scenarios in the MG test systems, and the
simulation results are addressed.
Enhancing battery system identification: nonlinear autoregressive modeling fo...IJECEIAES
Precisely characterizing Li-ion batteries is essential for optimizing their
performance, enhancing safety, and prolonging their lifespan across various
applications, such as electric vehicles and renewable energy systems. This
article introduces an innovative nonlinear methodology for system
identification of a Li-ion battery, employing a nonlinear autoregressive with
exogenous inputs (NARX) model. The proposed approach integrates the
benefits of nonlinear modeling with the adaptability of the NARX structure,
facilitating a more comprehensive representation of the intricate
electrochemical processes within the battery. Experimental data collected
from a Li-ion battery operating under diverse scenarios are employed to
validate the effectiveness of the proposed methodology. The identified
NARX model exhibits superior accuracy in predicting the battery's behavior
compared to traditional linear models. This study underscores the
importance of accounting for nonlinearities in battery modeling, providing
insights into the intricate relationships between state-of-charge, voltage, and
current under dynamic conditions.
Smart grid deployment: from a bibliometric analysis to a surveyIJECEIAES
Smart grids are one of the last decades' innovations in electrical energy.
They bring relevant advantages compared to the traditional grid and
significant interest from the research community. Assessing the field's
evolution is essential to propose guidelines for facing new and future smart
grid challenges. In addition, knowing the main technologies involved in the
deployment of smart grids (SGs) is important to highlight possible
shortcomings that can be mitigated by developing new tools. This paper
contributes to the research trends mentioned above by focusing on two
objectives. First, a bibliometric analysis is presented to give an overview of
the current research level about smart grid deployment. Second, a survey of
the main technological approaches used for smart grid implementation and
their contributions are highlighted. To that effect, we searched the Web of
Science (WoS), and the Scopus databases. We obtained 5,663 documents
from WoS and 7,215 from Scopus on smart grid implementation or
deployment. With the extraction limitation in the Scopus database, 5,872 of
the 7,215 documents were extracted using a multi-step process. These two
datasets have been analyzed using a bibliometric tool called bibliometrix.
The main outputs are presented with some recommendations for future
research.
Use of analytical hierarchy process for selecting and prioritizing islanding ...IJECEIAES
One of the problems that are associated to power systems is islanding
condition, which must be rapidly and properly detected to prevent any
negative consequences on the system's protection, stability, and security.
This paper offers a thorough overview of several islanding detection
strategies, which are divided into two categories: classic approaches,
including local and remote approaches, and modern techniques, including
techniques based on signal processing and computational intelligence.
Additionally, each approach is compared and assessed based on several
factors, including implementation costs, non-detected zones, declining
power quality, and response times using the analytical hierarchy process
(AHP). The multi-criteria decision-making analysis shows that the overall
weight of passive methods (24.7%), active methods (7.8%), hybrid methods
(5.6%), remote methods (14.5%), signal processing-based methods (26.6%),
and computational intelligent-based methods (20.8%) based on the
comparison of all criteria together. Thus, it can be seen from the total weight
that hybrid approaches are the least suitable to be chosen, while signal
processing-based methods are the most appropriate islanding detection
method to be selected and implemented in power system with respect to the
aforementioned factors. Using Expert Choice software, the proposed
hierarchy model is studied and examined.
Enhancing of single-stage grid-connected photovoltaic system using fuzzy logi...IJECEIAES
The power generated by photovoltaic (PV) systems is influenced by
environmental factors. This variability hampers the control and utilization of
solar cells' peak output. In this study, a single-stage grid-connected PV
system is designed to enhance power quality. Our approach employs fuzzy
logic in the direct power control (DPC) of a three-phase voltage source
inverter (VSI), enabling seamless integration of the PV connected to the
grid. Additionally, a fuzzy logic-based maximum power point tracking
(MPPT) controller is adopted, which outperforms traditional methods like
incremental conductance (INC) in enhancing solar cell efficiency and
minimizing the response time. Moreover, the inverter's real-time active and
reactive power is directly managed to achieve a unity power factor (UPF).
The system's performance is assessed through MATLAB/Simulink
implementation, showing marked improvement over conventional methods,
particularly in steady-state and varying weather conditions. For solar
irradiances of 500 and 1,000 W/m2
, the results show that the proposed
method reduces the total harmonic distortion (THD) of the injected current
to the grid by approximately 46% and 38% compared to conventional
methods, respectively. Furthermore, we compare the simulation results with
IEEE standards to evaluate the system's grid compatibility.
Enhancing photovoltaic system maximum power point tracking with fuzzy logic-b...IJECEIAES
Photovoltaic systems have emerged as a promising energy resource that
caters to the future needs of society, owing to their renewable, inexhaustible,
and cost-free nature. The power output of these systems relies on solar cell
radiation and temperature. In order to mitigate the dependence on
atmospheric conditions and enhance power tracking, a conventional
approach has been improved by integrating various methods. To optimize
the generation of electricity from solar systems, the maximum power point
tracking (MPPT) technique is employed. To overcome limitations such as
steady-state voltage oscillations and improve transient response, two
traditional MPPT methods, namely fuzzy logic controller (FLC) and perturb
and observe (P&O), have been modified. This research paper aims to
simulate and validate the step size of the proposed modified P&O and FLC
techniques within the MPPT algorithm using MATLAB/Simulink for
efficient power tracking in photovoltaic systems.
Adaptive synchronous sliding control for a robot manipulator based on neural ...IJECEIAES
Robot manipulators have become important equipment in production lines, medical fields, and transportation. Improving the quality of trajectory tracking for
robot hands is always an attractive topic in the research community. This is a
challenging problem because robot manipulators are complex nonlinear systems
and are often subject to fluctuations in loads and external disturbances. This
article proposes an adaptive synchronous sliding control scheme to improve trajectory tracking performance for a robot manipulator. The proposed controller
ensures that the positions of the joints track the desired trajectory, synchronize
the errors, and significantly reduces chattering. First, the synchronous tracking
errors and synchronous sliding surfaces are presented. Second, the synchronous
tracking error dynamics are determined. Third, a robust adaptive control law is
designed,the unknown components of the model are estimated online by the neural network, and the parameters of the switching elements are selected by fuzzy
logic. The built algorithm ensures that the tracking and approximation errors
are ultimately uniformly bounded (UUB). Finally, the effectiveness of the constructed algorithm is demonstrated through simulation and experimental results.
Simulation and experimental results show that the proposed controller is effective with small synchronous tracking errors, and the chattering phenomenon is
significantly reduced.
Remote field-programmable gate array laboratory for signal acquisition and de...IJECEIAES
A remote laboratory utilizing field-programmable gate array (FPGA) technologies enhances students’ learning experience anywhere and anytime in embedded system design. Existing remote laboratories prioritize hardware access and visual feedback for observing board behavior after programming, neglecting comprehensive debugging tools to resolve errors that require internal signal acquisition. This paper proposes a novel remote embeddedsystem design approach targeting FPGA technologies that are fully interactive via a web-based platform. Our solution provides FPGA board access and debugging capabilities beyond the visual feedback provided by existing remote laboratories. We implemented a lab module that allows users to seamlessly incorporate into their FPGA design. The module minimizes hardware resource utilization while enabling the acquisition of a large number of data samples from the signal during the experiments by adaptively compressing the signal prior to data transmission. The results demonstrate an average compression ratio of 2.90 across three benchmark signals, indicating efficient signal acquisition and effective debugging and analysis. This method allows users to acquire more data samples than conventional methods. The proposed lab allows students to remotely test and debug their designs, bridging the gap between theory and practice in embedded system design.
Detecting and resolving feature envy through automated machine learning and m...IJECEIAES
Efficiently identifying and resolving code smells enhances software project quality. This paper presents a novel solution, utilizing automated machine learning (AutoML) techniques, to detect code smells and apply move method refactoring. By evaluating code metrics before and after refactoring, we assessed its impact on coupling, complexity, and cohesion. Key contributions of this research include a unique dataset for code smell classification and the development of models using AutoGluon for optimal performance. Furthermore, the study identifies the top 20 influential features in classifying feature envy, a well-known code smell, stemming from excessive reliance on external classes. We also explored how move method refactoring addresses feature envy, revealing reduced coupling and complexity, and improved cohesion, ultimately enhancing code quality. In summary, this research offers an empirical, data-driven approach, integrating AutoML and move method refactoring to optimize software project quality. Insights gained shed light on the benefits of refactoring on code quality and the significance of specific features in detecting feature envy. Future research can expand to explore additional refactoring techniques and a broader range of code metrics, advancing software engineering practices and standards.
Smart monitoring technique for solar cell systems using internet of things ba...IJECEIAES
Rapidly and remotely monitoring and receiving the solar cell systems status parameters, solar irradiance, temperature, and humidity, are critical issues in enhancement their efficiency. Hence, in the present article an improved smart prototype of internet of things (IoT) technique based on embedded system through NodeMCU ESP8266 (ESP-12E) was carried out experimentally. Three different regions at Egypt; Luxor, Cairo, and El-Beheira cities were chosen to study their solar irradiance profile, temperature, and humidity by the proposed IoT system. The monitoring data of solar irradiance, temperature, and humidity were live visualized directly by Ubidots through hypertext transfer protocol (HTTP) protocol. The measured solar power radiation in Luxor, Cairo, and El-Beheira ranged between 216-1000, 245-958, and 187-692 W/m 2 respectively during the solar day. The accuracy and rapidity of obtaining monitoring results using the proposed IoT system made it a strong candidate for application in monitoring solar cell systems. On the other hand, the obtained solar power radiation results of the three considered regions strongly candidate Luxor and Cairo as suitable places to build up a solar cells system station rather than El-Beheira.
An efficient security framework for intrusion detection and prevention in int...IJECEIAES
Over the past few years, the internet of things (IoT) has advanced to connect billions of smart devices to improve quality of life. However, anomalies or malicious intrusions pose several security loopholes, leading to performance degradation and threat to data security in IoT operations. Thereby, IoT security systems must keep an eye on and restrict unwanted events from occurring in the IoT network. Recently, various technical solutions based on machine learning (ML) models have been derived towards identifying and restricting unwanted events in IoT. However, most ML-based approaches are prone to miss-classification due to inappropriate feature selection. Additionally, most ML approaches applied to intrusion detection and prevention consider supervised learning, which requires a large amount of labeled data to be trained. Consequently, such complex datasets are impossible to source in a large network like IoT. To address this problem, this proposed study introduces an efficient learning mechanism to strengthen the IoT security aspects. The proposed algorithm incorporates supervised and unsupervised approaches to improve the learning models for intrusion detection and mitigation. Compared with the related works, the experimental outcome shows that the model performs well in a benchmark dataset. It accomplishes an improved detection accuracy of approximately 99.21%.
Null Bangalore | Pentesters Approach to AWS IAMDivyanshu
#Abstract:
- Learn more about the real-world methods for auditing AWS IAM (Identity and Access Management) as a pentester. So let us proceed with a brief discussion of IAM as well as some typical misconfigurations and their potential exploits in order to reinforce the understanding of IAM security best practices.
- Gain actionable insights into AWS IAM policies and roles, using hands on approach.
#Prerequisites:
- Basic understanding of AWS services and architecture
- Familiarity with cloud security concepts
- Experience using the AWS Management Console or AWS CLI.
- For hands on lab create account on [killercoda.com](https://killercoda.com/cloudsecurity-scenario/)
# Scenario Covered:
- Basics of IAM in AWS
- Implementing IAM Policies with Least Privilege to Manage S3 Bucket
- Objective: Create an S3 bucket with least privilege IAM policy and validate access.
- Steps:
- Create S3 bucket.
- Attach least privilege policy to IAM user.
- Validate access.
- Exploiting IAM PassRole Misconfiguration
-Allows a user to pass a specific IAM role to an AWS service (ec2), typically used for service access delegation. Then exploit PassRole Misconfiguration granting unauthorized access to sensitive resources.
- Objective: Demonstrate how a PassRole misconfiguration can grant unauthorized access.
- Steps:
- Allow user to pass IAM role to EC2.
- Exploit misconfiguration for unauthorized access.
- Access sensitive resources.
- Exploiting IAM AssumeRole Misconfiguration with Overly Permissive Role
- An overly permissive IAM role configuration can lead to privilege escalation by creating a role with administrative privileges and allow a user to assume this role.
- Objective: Show how overly permissive IAM roles can lead to privilege escalation.
- Steps:
- Create role with administrative privileges.
- Allow user to assume the role.
- Perform administrative actions.
- Differentiation between PassRole vs AssumeRole
Try at [killercoda.com](https://killercoda.com/cloudsecurity-scenario/)
This presentation is about Food Delivery Systems and how they are developed using the Software Development Life Cycle (SDLC) and other methods. It explains the steps involved in creating a food delivery app, from planning and designing to testing and launching. The slide also covers different tools and technologies used to make these systems work efficiently.
Sri Guru Hargobind Ji - Bandi Chor Guru.pdfBalvir Singh
Sri Guru Hargobind Ji (19 June 1595 - 3 March 1644) is revered as the Sixth Nanak.
• On 25 May 1606 Guru Arjan nominated his son Sri Hargobind Ji as his successor. Shortly
afterwards, Guru Arjan was arrested, tortured and killed by order of the Mogul Emperor
Jahangir.
• Guru Hargobind's succession ceremony took place on 24 June 1606. He was barely
eleven years old when he became 6th Guru.
• As ordered by Guru Arjan Dev Ji, he put on two swords, one indicated his spiritual
authority (PIRI) and the other, his temporal authority (MIRI). He thus for the first time
initiated military tradition in the Sikh faith to resist religious persecution, protect
people’s freedom and independence to practice religion by choice. He transformed
Sikhs to be Saints and Soldier.
• He had a long tenure as Guru, lasting 37 years, 9 months and 3 days
AI in customer support Use cases solutions development and implementation.pdfmahaffeycheryld
AI in customer support will integrate with emerging technologies such as augmented reality (AR) and virtual reality (VR) to enhance service delivery. AR-enabled smart glasses or VR environments will provide immersive support experiences, allowing customers to visualize solutions, receive step-by-step guidance, and interact with virtual support agents in real-time. These technologies will bridge the gap between physical and digital experiences, offering innovative ways to resolve issues, demonstrate products, and deliver personalized training and support.
https://www.leewayhertz.com/ai-in-customer-support/#How-does-AI-work-in-customer-support
Blood finder application project report (1).pdfKamal Acharya
Blood Finder is an emergency time app where a user can search for the blood banks as
well as the registered blood donors around Mumbai. This application also provide an
opportunity for the user of this application to become a registered donor for this user have
to enroll for the donor request from the application itself. If the admin wish to make user
a registered donor, with some of the formalities with the organization it can be done.
Specialization of this application is that the user will not have to register on sign-in for
searching the blood banks and blood donors it can be just done by installing the
application to the mobile.
The purpose of making this application is to save the user’s time for searching blood of
needed blood group during the time of the emergency.
This is an android application developed in Java and XML with the connectivity of
SQLite database. This application will provide most of basic functionality required for an
emergency time application. All the details of Blood banks and Blood donors are stored
in the database i.e. SQLite.
This application allowed the user to get all the information regarding blood banks and
blood donors such as Name, Number, Address, Blood Group, rather than searching it on
the different websites and wasting the precious time. This application is effective and
user friendly.
Convolutional auto-encoded extreme learning machine for incremental learning of heterogeneous images
1. International Journal of Electrical and Computer Engineering (IJECE)
Vol. 13, No. 5, October 2023, pp. 5853~5864
ISSN: 2088-8708, DOI: 10.11591/ijece.v13i5.pp5853-5864 5853
Journal homepage: http://ijece.iaescore.com
Convolutional auto-encoded extreme learning machine for
incremental learning of heterogeneous images
Sathya Madhusudanan, Suresh Jaganathan, Dattuluri Venkatavara Prasad
Department of Computer Science and Engineering, Sri Sivasubramaniya Nadar College of Engineering, Chennai, India
Article Info ABSTRACT
Article history:
Received Dec 10, 2022
Revised Mar 29, 2023
Accepted Apr 7, 2023
In real-world scenarios, a system's continual updating of learning knowledge
becomes more critical as the data grows faster, producing vast volumes of
data. Moreover, the learning process becomes complex when the data features
become varied due to the addition or deletion of classes. In such cases, the
generated model should learn effectively. Incremental learning refers to the
learning of data which constantly arrives over time. This learning requires
continuous model adaptation but with limited memory resources without
sacrificing model accuracy. In this paper, we proposed a straightforward
knowledge transfer algorithm (convolutional auto-encoded extreme learning
machine (CAE-ELM)) implemented through the incremental learning
methodology for the task of supervised classification using an extreme
learning machine (ELM). Incremental learning is achieved by creating an
individual train model for each set of homogeneous data and incorporating the
knowledge transfer among the models without sacrificing accuracy with
minimal memory resources. In CAE-ELM, convolutional neural network
(CNN) extracts the features, stacked autoencoder (SAE) reduces the size, and
ELM learns and classifies the images. Our proposed algorithm is implemented
and experimented on various standard datasets: MNIST, ORL, JAFFE,
FERET and Caltech. The results show a positive sign of the correctness of the
proposed algorithm.
Keywords:
Autoencoder
Convolutional neural networks
Extreme learning machine
Heterogeneous data
Incremental learning
This is an open access article under the CC BY-SA license.
Corresponding Author:
Suresh Jaganathan
Department of Computer Science and Engineering, Sri Sivasubramaniya Nadar College of Engineering
Chennai, Tamilnadu, India
Email: sureshj@ssn.edu.in
1. INTRODUCTION
Information systems widely differ in their form and application, but converts all the data into
meaningful information. Real-world applications generate data in huge volumes, where the process of
acquiring knowledge becomes complex. Irrespective of the type of data, which may be homogeneous (same
feature set across the chunks) or heterogeneous (different feature set across the chunks) [1], the models
generated from the systems must continually learn to predict or classify. Incremental learning (or constructive
learning) [2], a machine learning technique introduced for continual learning, updates the existing model when
data streams in continually. Figure 1 shows an incremental learning model, which helps grow the network with
the data arrival belonging to new classes. It applies on classification and clustering applications, addressing the
data availability and resource scarcity challenges. An incremental learning algorithm must meet these criteria
[3], i) accommodating new classes, ii) minimal overhead for training new classes, iii) previously trained data must
not be retrained, and iv) preserve previously acquired knowledge. Challenges faced by incremental learning are:
a) Concept drift: refers to the changes observed in the data distribution over time [4], which falls under two
categories: i) virtual concept drift or covariate concept drift where changes are seen in the data distribution
2. ISSN: 2088-8708
Int J Elec & Comp Eng, Vol. 13, No. 5, October 2023: 5853-5864
5854
only and ii) real concept drift where changes are seen in the underlying functionality. Concept drift impacts
the system performance until the model re-adapts accordingly.
b) Stability-plasticity dilemma: refers to how quickly the model adapts with the changes. Two possibilities
are: i) the model adapts quickly to the new information, but forgets the old information and ii) in contrast,
the older information remains for a more extended period, but the model gets adapted slowly. The trade-
off between the two is referred as stability-plasticity dilemma [5].
c) Catastrophic forgetting: also known as catastrophic interference [6]. Many online learning models tend to
forget previously learned information completely or abruptly upon learning new information.
d) Adaptive model complexity and meta-parameters: incremental learning [7] also has to address the challenge
of model complexity, whose estimation in advance is impossible when huge data streams in. Intelligent
adaptation methods deal with the model complexities by reallocating the resources whenever the limit is
reached.
e) Efficient learning models: the incremental learning models [8] must be efficient enough to deal with the
constantly arriving data. Even in limited resources, these models must deal with the newly added data by
storing the information provided by the observed data in compact form.
Our proposed work focuses on designing an efficient incremental learning model for classifying image
data available in different static batches with varying resolutions, i.e., 64×64, 39×39, 32×32, 28×28. To achieve
efficient incremental learning, the proposed convolutional auto-encoded extreme learning machine (CAE-
ELM) neural network uses the correctly classified test samples of every batch as a source of new labeled
training samples for the successive batches: i) initially, convolutional neural network (CNN) [9] captures the
spatial image features; ii) later, stacked auto-encoder (SAE) [10] does the process of dimensionality reduction
on the extracted features; and iii) finally, extreme learning machine (ELM) [11] trains the image batches and
creates an incremental model using the unlabeled test samples of each batch.
CAE-ELM addresses all the above-mentioned challenges by retaining the learned information on old
classes and learning new classes arriving in varying batches with varying resolutions. CAE-ELM maintains
only the most recently updated model and reduces memory constraints by discarding all the old trained data
and model. It also saves training time by avoiding the retraining of old samples.
An extensive survey of the existing feed-forward neural network, ELM and incremental learning
methodologies are done to update our knowledge in the area of our proposed work. The survey confirms the
extensive use of ELM for cross-domain tasks and unsupervised learning. Furthermore, the detailed study of
various incremental architectures helped us understand each strategy's pros and cons, which made us choose a
replay-based technique for our incremental algorithm.
a) Extreme learning machine
Zhang et al. [12] optimized ELM using the Firefly algorithm. Song et al. [13], Huang et al. [14]
proposed the improved versions of ELM by incrementally adding hidden nodes to the ELM network, which
improved ELM's performance and validated the universal approximators, respectively. Wang et al. [15] briefs
and reviews the fast ELM algorithm and its applications in different domains. Yousefi-Azar and McDonnell
[16] proposed the unsupervised CNN-ELM (UCNN-ELM) for unsupervised learning. Adaptive ELM, an
extension to ELM, proposed by Zhang et al. [17] for cross task learning uses a new error term and Laplacian
graph based manifold regularization term in the objective function. Zhang et al. [18] makes a comparison
between ELM and SVM in cross-domain tasks, where ELM outperforms SVM with about 4% accuracy.
b) Incremental learning architectures
Discussed here are a few research works contributing to the area of incremental learning. We have
briefed two architectures for implementing incremental learning: i) machine learning techniques and ii) neural
networks (NN). In neural network architectures, we categorize three strategies to implement incremental
learning: i) replay-based methods, ii) regularization-based methods, and iii) parameter isolation methods.
c) Machine learning approaches
Losing et al. [19] discussed and compared the following incremental algorithms, LASVM (online
approximate SVM) [20] checks each current instance for becoming a support vector and removes all the
obsolete support vectors. Incremental learning vector quantization (ILVQ) [21] dynamically creates new
prototypes for classes depending on the number of classifiers. NBGauss (naive Bayes) [22] uses likelihood
estimation in the naïve-Bayes algorithm by fitting Gaussian distribution on one class at any instance. Stochastic
gradient descent (SGDLin+) [23] combines SGD with linear models to minimize loss function and optimize
the performance of discriminative models.
The studies reveal that SVM delivers the highest accuracy at the expense of the most complex model.
LASVM reduces the training time, though ORF gives a poor performance, it can be considered for high-speed
training and run-time. ILVQ offers an accurate and sparse alternative to the SVMs. NBGauss, SGDLin are
viable choices for large-scale learning in high-dimensional space.
3. Int J Elec & Comp Eng ISSN: 2088-8708
Convolutional auto-encoded extreme learning machine for incremental learning … (Sathya Madhusudanan)
5855
d) Neural network architectures
Replay-based methods: replay-based methods replay training samples from previous tasks into the
new batch of training data to retain the acquired knowledge. The replay can be done in two ways: a) creating
exemplars and b) generating synthetic data. A deep incremental CNN [24] model, where the exemplars of the
old classes update the loss function when trained with the new class samples. This architecture used for large-
scale incremental learning, where exemplars help in bias correction for the created model representing both the
old and new classes. ELM++ [25], an incremental learning algorithm, creates a unique model for each incoming
batch. The model is picked by using the intensity mean of the test sample and the classes trained. Here authors
included a data augmentation step in generating samples to accommodate old and new classes in the
incremental model, using adjusting brightness, contrast normalization, random cropping, and mirroring
techniques. Deep generative replay for continual learning of image classification tasks and constitutes a deep
generative model (“generator”) and a task solving model (“solver”). Brain-inspired replay (BI-R) [26] uses the
variational auto-encoder for generating samples learned previously. Variational auto-encoder generates the old
samples and serves the purpose of data augmentation. It also uses Gaussian mixture model (GMM) to generate
specific desired classes while regenerating trained samples.
Regularization-based methods: regularization-based methods use regularization terms to prevent the
current task parameters from deviating too much from the previous task parameters. A deep incremental CNN
architecture can use a strong distilling and classification loss in the last fully connected layer to effectively
overcome the catastrophic forgetting problem. He et al. [27] proposed a two-step incremental learning framework
with a modified cross-distillation loss function addressing the challenges in online learning scenario.
Parameter isolation methods: parameter isolation methods eradicate catastrophic forgetting by
dedicating a subset of a model’s parameters from previous tasks to each current incremental task.
Guo et al. [28] implemented incremental ELM (I-ELM), which trains online application data one-by-one or
chunk by chunk using three alternatives-minimal-norm incremental ELM (MN-IELM), least square
incremental ELM (LS-IELM), kernel-based incremental (KB-IELM). Among the three methods, KB-ELM
provides best accuracy results. Online sequential ELM (OS-ELM) [29] trains data sequentially one by one or
chunk by chunk based on recursive least-squares algorithm. In the sequential learning phase, for each new
observation, OS-ELM calculates the current hidden and output layer weight based on the previous hidden and
output weights. Error-minimized ELM (EM-ELM) [30] adds random hidden nodes to ELM network one by
one or chunk by chunk and reduces the computational complexity by updating the output weights
incrementally. Learn++ [31], an incremental training algorithm trains an ensemble of weak classifiers with
different distributions of samples. A majority voting scheme generates the final classification rule, which
eliminates over-fitting and fine-tuning. ADAIN [32], a general adaptive incremental learning framework, maps
the distribution function estimated on the initial dataset to the new chunk of data. The pseudo-error value
assigns the misclassified instances with a higher weight, making an efficient decision boundary. A non-
iterative, incremental, hyperparameter-free learning method iLANN-SVD, updates the network weights for the
new set of data by applying singular value decomposition on the previously obtained network weights.
Our proposed work comes more or less under the first category, as the correctly classified test samples
of the current batch serve as augmented samples. Thus, the next set of images generates the incremental model
along with the augmented samples. Table 1 (see in Appendix) summarizes the strategies adopted by various
neural network architectures to achieve incremental learning.
The rest of the article is organized as follows: the contributions listed in section 1 discuss the strategies
to achieve incremental learning. The proposed work, CAE-ELM, is detailed in section 2. The experimental
results and implementation of sample scenarios can be found in section 3 and also it discusses the pros and
cons of CAE-ELM. Finally, section 4 concludes the paper with possible future work.
2. METHOD
CNN-ELM [33] is a recently developed deep neural network that replaces the multi-layer perceptron
classifier part of CNN with the extremely fast ELM neural network. The flattened features of the input image
retrieved by applying convolutional filters are fed to the ELM to fasten the classification process. CNN is
combined with ELM to achieve the following benefits: i) the convolutional filters in CNN capture an image's
spatial dependencies, classifying it with significant precision, ii) ELM randomly assigns the hidden node
parameters initially, and they are never updated, and iii) ELM learns the output weight in a single step. Thus,
it gives the best generalization performance at a breakneck learning speed.
Exploiting the advantages of CNN-ELM, we propose the novel CAE-ELM, an incremental learning
algorithm for classifying supervised images. CAE-ELM is designed to solve two issues: i) classifies images
with varying dimensions and ii) performs incremental learning. CAE-ELM classifies the images arriving in
different batches with varying resolutions. CAE-ELM constitutes the following: i) CNN extracts feature from
input images, ii) SAE does the dimensionality reduction process, and iii) ELM incrementally trains every batch
4. ISSN: 2088-8708
Int J Elec & Comp Eng, Vol. 13, No. 5, October 2023: 5853-5864
5856
of images and generates an updated model with no retraining of samples. Figure 1 outlines the overview of the
proposed work CAE-ELM. (Note: CAE-ELM framework embeds only one CNN, SAE, and ELM architecture
for the entire incremental image classification process).
As CAE-ELM focuses on handling varied resolution images arriving in different batches, initially,
CNN outputs varied size features from the flattened layer for every batch. Before the training process, we zero-
pad the extracted varied size features from CNN to a fixed length (maximum length among the extracted
features). The SAE dimensionally reduces the zero-padded features to ease the training process. Finally, the
ELM neural network trains and classifies the dimensionally reduced features extracted from the varied
resolution input images.
Figure 1. Incremental learning using CAE-ELM
2.1. CAE-ELM for image classification with varied resolution
In CAE-ELM, the CNN framework had two convolutional layers and two pooling layers alternatively-
conv1 with ten 3×3 filters, pool1 with ten 2×2 filters, conv2 with twenty 3×3 filters and pool2 with twenty 2×2
filters. In our proposed work, the images arrive in batches with varying resolutions each day, i.e., batch 1 holds
images with 64×64, batch 2 carries images with 39×39, and batch 3 holds images with 32×32 resolution. The
CNN with the designed framework produces flattened features with varying sizes for each batch of images fed
into it. For example, batch 1, batch 2 and batch 3 has 3920, 800, 720 features, respectively. The output
dimension of each layer in the CNN is,
[
(𝑤+2𝑝−𝑐𝑓)
𝑠
+ 1] × [
(ℎ+2𝑝−𝑐𝑓)
+ 1] × 𝑛𝑐𝑓 (1)
where 𝑤 is image width, ℎ is height, 𝑝 is image padding, 𝑠 is stride, 𝑐𝑓 is convolutional filter size and 𝑛𝑐𝑓 is
the number of convolutional filters applied. We zero-pad the extracted features from varied batches to a fixed-
length features. For example, when the batch images end up with varied sized features 3920, 800, 720, we zero-
pad the features to a maximum length 3290.
SAE dimensionally reduces the zero-padded features to ease the training process. The autoencoder
generates a compact code representation, which successfully reconstructs the input image. The dimensionally
reduced features from the stacked autoencoder are fed into the ELM, a single hidden layer neural network.
ELM generates a model with parameters (𝑊, 𝑏, 𝛽). Algorithm 1 explains how SAE converts the varied
resolution images to the same features and the ELM training process for image classification.
5. Int J Elec & Comp Eng ISSN: 2088-8708
Convolutional auto-encoded extreme learning machine for incremental learning … (Sathya Madhusudanan)
5857
Algorithm 1. CAE-ELM
Input: Input training images
Output: Classified Output
for 𝑏𝑎𝑡𝑐ℎ𝑒𝑠 𝑖 = 1, 2, . . . , 𝑆
A. Do the following steps to extract features from every image 𝑗 = 1,2, . . . , 𝑁 in the dataset:
1. Convolve the input image (𝑤 𝑥 ℎ 𝑥 𝑑) with (𝑛𝑐𝑓) filters to produce the convolved image
(𝑤 × ℎ × 𝑛𝑐𝑓).
2. Pool the convolved image, reducing the dimensions to (
𝑤
2
×
ℎ
2
×
𝑛𝑐𝑓
2
), which helps reduce the
computation time, overfitting, and the need for substantial memory resources.
3. Flatten the pooled output into a single 1-D array feature 𝑓(𝑥)(𝑜𝑓 𝑠𝑖𝑧𝑒 𝑛1.
B. Reduce the single 1-D array feature 𝑓(𝑥) 𝑜𝑓 𝑠𝑖𝑧𝑒 𝑛1 to a constant feature 𝑠𝑖𝑧𝑒 𝑛𝑝 using Stacked
Autoencoder, such that ∀𝑖𝑛𝑝 < 𝑛𝑖.
C. ELM neural network trains the compact feature set 𝑓(𝑥) and generates a classifier model.
The (2) computes the single hidden layer (𝐻). The (3) computes the output matrix 𝛽.
𝐻 = (
𝑔(𝑓(𝑥1).𝑊1 + 𝑏1) ⋯ 𝑔(𝑓(𝑥1).𝑊𝑁′ + 𝑏𝑁′)
⋮ ⋱ ⋮
𝑔(𝑓(𝑥𝑁). 𝑊1 + 𝑏1) ⋯ 𝑔(𝑓(𝑥𝑁). 𝑊𝑁′ + 𝑏𝑁′)
) (2)
where 𝑁′
: hidden neurons and 𝑊: random weights are assigned. The output matrix 𝛽 is given
by (3):
𝛽 = 𝐻†
𝑇 (3)
where,
𝐻†
= {
𝐻𝑡(𝐻𝐻𝑡)−1
,𝑖𝑓 𝑚 < 𝑓(𝑥)
(𝐻𝐻𝑡)−1
𝐻𝑡
, 𝑖𝑓 𝑚 > 𝑓(𝑥)
(4)
and 𝑇 is a target matrix, 𝑡 represents the transpose of a matrix, 𝑚 represents output
neurons.
Endfor
2.2. CAE-ELM for incremental learning
Incremental learning methodology learns the arriving data batches continuously, updates the current
model information addressing stability-plasticity dilemma, and overcomes the problem of catastrophic
forgetting. CAE-ELM uses two types of testing to check the performance of the generated incremental
model. i) Individual testing: the model is tested with class samples as in the individual training batches and
ii) Incremental testing: samples belonging to all the classes are tested using the generated model.
CAE-ELM supports incremental learning of images with varied resolutions through the individual
test samples of each arriving batch. Individual test samples serve as support to synthesize new training samples
to create an incremental model, a form of data augmentation. Each batch of images is divided into training and
testing sets. CNN extracts the image features, and SAE reduces the features dimensionally to a countable
number of features, which remains common to all the incoming data batches. ELM creates a model by training
the samples in the training set. The model parameters generated (𝑊, 𝑏, 𝛽) classify the test samples of each
batch. The proposed work feeds one-half of the correctly classified test samples (50%) along with the following
training batch to support incremental learning. In this way, ELM learns the newly arrived classes, retaining the
previously acquired knowledge about the old classes with model parameters (𝑊
𝑛𝑒𝑤, 𝑏𝑛𝑒𝑤, 𝛽𝑛𝑒𝑤). The other
half of the correctly classified samples and misclassified samples test the updated model, along with the
subsequent batch of the test set. Thus, CAE-ELM serves the following advantages: i) learns newly arriving
classes, ii) remembers the previously learned information on odd classes, iii) does not retrain any of the input
samples and saves training time, and iv) discards old training samples and models when it generates the updated
model, thus saving memory space.
2.3. Illustration
Figure 2 illustrates CAE-ELM, which individually trains 3 batches. Batch 1 images constitute three
classes 1, 2 and 3 with 64×64 resolution (3920 features). Batch 2 images hold classes 2, 3 and 4 with
39×39 resolution (800 features), while Batch 3 images have resolution 32×32 (720 features) holding classes 1,
3 and 5. Starting from the second batch of data, CAE-ELM implements incremental learning by training one-
half of the correctly classified test samples from the previous batch. So, CAE-ELM trains batch 2 data along
with one-half of the correctly classified test samples from the batch 1 data, producing model M2 with classes
1, 2, 3 and 4, i.e., both old and new classes. Similarly, the updated model M3 generated on training batch
6. ISSN: 2088-8708
Int J Elec & Comp Eng, Vol. 13, No. 5, October 2023: 5853-5864
5858
3 data, holds information on all the classes 1, 2, 3, 4 and 5. CAE-ELM discards all the previous models, and
uses the most recent model M3 for classifying the test data.
Figure 2. Illustration of incremental learning in CAE-ELM
3. RESULTS AND DISCUSSION
Experiments test the incremental performance of CAE-ELM using standard datasets like MNIST,
JAFFE, ORL, FERET. Tables 2 to 5 prove the outstanding incremental performance of CAE-ELM against the
existing incremental methods. Subsection 3.3.2 discusses the pros and cons of using CNN, SAE, and ELM in
CAE-ELM for the process of feature extraction, feature size conversion (dimensionality reduction), and
training for image classification.
3.1. Incremental learning results
We compared CAE-ELM against different incremental methods for MNIST and CIFAR100 datasets
as follows:
a) MNIST dataset: MNIST dataset comprises of 10 classes, with a total of 20,000 (28×28) images.
Without adding new classes: for the MNIST dataset with a fewer number of classes, we compared our
algorithm CAE-ELM with existing incremental learning algorithms, Learn++, and ELM++, which use neural
networks for image classification. We split the MNIST dataset into six sets (S1 − S6), where each set holds
samples from classes 0 − 9. Table 2 (case 1) and Figures 3(a) and 3(b) show the accuracy comparison between
Learn++, ELM++, and CAE-ELM for the MNIST dataset when adding new images to the already existing
classes. The entire dataset was divided into six train sets (S1 − S6) and one test set. Each train set consists of
10 classes (0-9) with 200 images in each class. The set S2 will hold the same classes but different images from
S1. Set S3 will have images that are not present in S1 and S2, and so on. The test set contains images from all
these ten classes.
With adding new classes: We also compared CAE-ELM against Learn++ and ELM++ methods with
MNIST dataset under scenario where new classes gets added in every new arriving batch of data. We split the
MNIST dataset into three sets S1, S2, S3, where S1 trains classes (0-2), S2 trains classes (3-5) and S3 holds
samples from classes (6-9). Table 2 (case 2) and Figure 3 tabulates the results obtained.
b) CIFAR dataset: CIFAR dataset comprises of 100 classes, with a total of 60,000 (32×32) images.
For the CIFAR100 dataset with a larger number of classes, we compared CAE-ELM against brain-
inspired replay, end-to-end and large-scale incremental learning methods. We split the dataset into five train sets
S1-S5, where every set train 20 new classes in addition to the old classes. Table 3 and Figure 4 tabulates the
results obtained.
7. Int J Elec & Comp Eng ISSN: 2088-8708
Convolutional auto-encoded extreme learning machine for incremental learning … (Sathya Madhusudanan)
5859
Table 2. Comparison between learn++, ELM++, and CAE-ELM – MNIST dataset
Case Set Training Incremental testing
Accuracy (%)
Learn++ ELM++ CAE-ELM Learn++ ELM++ CAE-ELM
Without
adding
classes
S1(0-9) 94.2 92 95 82 72 87
S2(0-9) 93.5 90 95.5 84.7 78.2 90.5
S3(0-9) 95 90 96 89.7 85.3 91.78
S4(0-9) 93.5 92 96.7 91.7 88 94
S5(0-9) 95 96 97 92.2 90.8 95.6
S6(0-9) 95 96 96.8 92.7 94 97.8
With
adding
classes
S1(0, 1, 2) 98.7 97 96.45 40.2 35 42.3
S2(3, 4, 5) 96.1 93 98.9 60 52.7 63
S3(6, 7, 8, 9) 92.6 90.75 92.8 92.75 93.3 98.7
(a)
(b)
Figure 3. Comparison between Learn++, ELM++, and CAE-ELM – MNIST database: (a) comparison
between Learn++, ELM++, and CAE-ELM – MNIST database without adding classes and (b) comparison
between Learn++, ELM++, and CAE-ELM for MNIST database with adding classes
8. ISSN: 2088-8708
Int J Elec & Comp Eng, Vol. 13, No. 5, October 2023: 5853-5864
5860
Table 3. Comparison between BI-R, end-to-end, large-scale incremental learning and
CAE-ELM – CIFAR100 dataset (with adding classes)
Set Number of Classes Accuracy (%)
BI-R End-to-End Large-scale IL
S1 20 74 82 84 86.9
S2 40 43.7 77.6 74.69 77.2
S3 60 35.4 65 67.93 70.8
S4 80 33.1 60 61.25 64.4
S5 100 28.8 53.4 56.69 60.4
Figure 4. CIFAR100 database with adding classes
3.2. Implementation of application scenario
The proposed CAE-ELM incremental neural network is deployed in various lab to access the systems
by different group of students. The student’s images will be trained previously by the CAE-ELM, where the
training images will be captured using cameras with varying resolutions fixed in different labs. For example,
consider five batches (Batch B1, B2, B3, B4, and B5) and four labs (Python, R, MATLAB, and Java). Every
batch will hold 10 students, where each student represents a unique class. Every student will learn two different
subjects and have access to the corresponding labs. Python lab captures images with 64×64 resolution, R lab
captures images with 39×39 images, MATLAB with 32×32 resolution whereas, Java lab with 28×28 resolution
images. The student’s images will be captured using these lab cameras with varying resolutions, for example,
Batch B1 students will have MATLAB and Java classes, so their images will be captured in two different
resolutions 64×64 and 28×28. CAE-ELM handles and trains all the student’s images captured using varying
resolutions. CAE-ELM model tests the captured image and marks the student’s presence in the lab. Table 4
shows the different labs attended by different batch of students.
We implemented the application scenario with three datasets: ORL, JAFFE, and FERET. ORL dataset
comprises of 440 (92×112) images, which includes 40 classes. JAFFE dataset comprises of 10 classes, with a
total of 200 (256×256) images. FERET includes 11,338 (512×768) images with 994 classes. 10 classes from
each dataset were chosen. In batch B1, classes 0, 1, 2 were trained with resolution 64×64 from Python lab. In
batch B2, classes 3, 4, 5 were trained with resolution 32×32 from MATLAB lab. Similarly, in batch B3, classes
6, 7, 8, 9 were trained with resolution 28×28 from Java lab. In each dataset, the images were resized to specific
resolutions to implement the scenario. Table 5 illustrates the incremental results obtained for ORL, JAFFE,
and FERET datasets respectively. Individual testing involves samples only from trained classes in the specific
batches. Incremental Testing holds samples from all the classes (0, 1, 2, …, 8, 9).
9. Int J Elec & Comp Eng ISSN: 2088-8708
Convolutional auto-encoded extreme learning machine for incremental learning … (Sathya Madhusudanan)
5861
Table 4. Scenario for incremental learning using CAE-ELM
Batch Classes Labs
Python(64x64) R (39x39) MATLAB (32x32) Java (28x28)
B1 0-9 ✓ ✓
B2 10-19 ✓ ✓
B3 20-29 ✓ ✓
B4 30-39 ✓ ✓
B5 40-79 ✓ ✓
Table 5. CAE-ELM incremental results for ORL, JAFFE and FERET dataset
Dataset Batch Classes Individual testing Incremental testing
Python (64x64) R (39x39) MATLAB (32x32)
ORL B1 0, 1, 2 87.6 - - 35.8
B2 3, 4, 5 - 100 - 57.2
B3 6, 7, 8, 9 - - 95.8 96.5
JAFFE B1 0, 1, 2 100 - - 33.3
B2 3, 4, 5 - 100 - 64.2
B3 6, 7, 8, 9 - - 87.9 94.7
FERET B1 0, 1, 2 95 - - 32.1
B2 3, 4, 5 - 98 - 60.23
B3 6, 7, 8, 9 - - 96 97.5
3.3. Discussions
3.3.1. Pros and cons of CAE-ELM
The advantages of CAE-ELM lie mainly in addressing the varied resolution images arriving in
different batches and creating a single updated model for them without forgetting the learned knowledge. The
data augmentation method helps us in preserving the learned information of every batch, and propagates it to
the other upcoming batches. The use of CNN and ELM enhances the efficiency of the proposed work by
extracting the best features and providing a faster training time. CAE-ELM stays efficient in i) memory by
discarding all the old models and data and ii) time by avoiding the retraining of old samples.
ELM stands superior to other conventional methods like CNN with its fast-training time. The
advantage of using ELM in CAE-ELM lies with the replacement of the backpropagation part in CNN for
training purpose. With no doubt, ELM is a better choice than the conventional back-propagation process.
ELM lacks efficiency with the feature extraction process, specifically for the red, green, and blue (RGB) images. In
such cases, definitely CNN is a better choice over ELM. So, we have used CNN for the feature extraction process.
So, the combined use of CNN and ELM in CAE-ELM definitely proves to show a significant improvement in the
classification accuracy.
However, when does the performance of CAE-ELM becomes a concern? CAE-ELM becomes time-
consuming when the varied resolution images are very high dimensional i.e., 2048×1536, and 7680×6876.
Running the CNN for feature extraction followed by the stacked autoencoder to convert into the same feature
sizes may be time-consuming. Though the performance accuracy might be achieved, the time consumed in
running CNN and SAE together will wash away the advantage of avoiding retraining samples. In such cases,
some other efficient techniques must be used to handle the high-dimensional varied resolution images.
However, in very high-dimensional images, the use of ELM in place of back-propagation for image
classification compensates the time spent with SAE to some extent.
3.3.2. Pros of ELM training over the use of back-propagation
The CNN extracts the spatial features of images with the use of convolutional filters. After feature
extraction and reduction, the ELM classifier replaces the backpropagation for training the datasets. ELM does
not undergo any iterations for finding trained weights, thus requiring less time to train a model, even in the
case of a large number of hidden neurons. ELM is also resistant to overfitting, except in the case of tiny datasets.
As the training sample increases, ELM provides good performance accuracy.
Using the advantages of CNN and ELM, the proposed work CAE-ELM overcomes the memory
requirement of a large dataset by training each batch of arriving data independently to create individual models,
which avoids the retraining of previous batch data. CAE-ELM works better than the existing algorithm
Learn++ and ELM++ for the following reasons. i) CAE-ELM uses the best feature extractor CNN, whereas
ELM++ extracts the minute features of the image and ii) The training time decreases in CAE-ELM. It uses a
feedforward ELM network that does not involve any iterations, whereas Learn++ uses multiple neural network
classifiers, which involve iterations for backpropagating the errors.
10. ISSN: 2088-8708
Int J Elec & Comp Eng, Vol. 13, No. 5, October 2023: 5853-5864
5862
3.3.3. Challenges addressed
From the illustration shown in section 2.3, it is clear that CAE-ELM addresses the challenges of
incremental learning. Irrespective of the classes added or removed in different batches occurring at different
times, CAE-ELM adapts the model efficiently and learns all the classes, addressing the challenges of stability-
plasticity dilemma and catastrophic forgetting. Irrespective of the varying image features and classes,
CAE-ELM adapts the model dynamically and never forgets previously learned classes.
4. CONCLUSION
In this paper, our proposed incremental algorithm CAE-ELM learns varied resolution images arriving
in different batches efficiently, both in terms of memory and time, using the ELM neural network. Except for
the very high-dimensional images, the use of Stacked Autoencoder helps the incremental model to
accommodate information about varied resolution images. Addition/deletion of new/old classes to the
forthcoming batches never affects the learning performance of CAE-ELM. The most recently updated model
is only retained with all previously learned information, discarding all other trained models and data, saving
memory space. No input sample is retrained to save training time. Instead, CAE-ELM uses the correctly
classified test samples as an augmented input source to achieve an efficient incremental model. The current
incremental version of CAE-ELM works perfectly for varying image datasets. Our future work is to design an
incremental algorithm for time series data with concept drift.
APPENDIX
Table 1. Strategies adopted for incremental learning in NN
S. No Paper title (Author) NN architecture
used
Strategy adopted
1 End-to-end incremental learning
(Francisco et al. [24])
Deep CNN 1. exemplars + Generating synthetic data
2. regularization using the cross-entropy and distilling loss functions
2 Incremental learning for
classification of unstructured data
using extreme learning machine
(Madhusudhanan et al. [25])
ELM Using exemplars
3 Brain-inspired replay for
continual learning with artificial
neural networks (Van et al. [26])
Variational auto-
encoders
Regenerates old samples
4 Incremental learning in online
scenario (He et al. [27])
CNN Regularization using cross-distillation loss function
5 An incremental extreme learning
machine for online sequential
learning problems (Guo et al. [28])
ELM Parameter-isolation method using three alternatives-minimal-norm
incremental ELM (MN-IELM), least square incremental ELM (LS-
IELM), kernel-based incremental (KB-IELM)
6 Online sequential extreme learning
machine (Huang et al. [29])
ELM Parameter-isolation using recursive least-squares algorithm
7 Error-minimized extreme learning
machine (Feng et al. [30])
ELM Parameter-isolation by adding random hidden nodes
8 Learn++: an incremental learning
algorithm for supervised neural
networks (Polikar et al. [31])
MLP Updating of weights
9 Incremental learning from stream
data Haibo (He et al. [32])
MLP Updating of weights
10 Convolutional auto-encoded extreme
learning machine (CAE-ELM)
CNN-SAE-ELM Generating synthetic data
REFERENCES
[1] R. Kashef and M. Warraich, “Homogeneous Vs. heterogeneous distributed data clustering: A taxonomy,” in Data Management and
Analysis, 2020, pp. 51–66, doi: 10.1007/978-3-030-32587-9_4.
[2] A. Gepperth and B. Hammer, “Incremental learning algorithms and applications,” in European Symposium on Artificial Neural
Networks (ESANN), 2016, pp. 27–29.
[3] R. R. Ade and P. R. Deshmukh, “Methods for incremental learning: A survey,” International Journal of Data Mining & Knowledge
Management Process, vol. 3, no. 4, pp. 119–125, Jul. 2013, doi: 10.5121/ijdkp.2013.3408.
[4] Y. Kim and C. H. Park, “An efficient concept drift detection method for streaming data under limited labeling,” IEICE Transactions
on Information and Systems, vol. E100.D, no. 10, pp. 2537–2546, 2017, doi: 10.1587/transinf.2017EDP7091.
[5] M. Mermillod, A. Bugaiska, and P. Bonin, “The stability-plasticity dilemma: investigating the continuum from catastrophic
forgetting to age-limited learning effects,” Frontiers in Psychology, vol. 4, no. 504, 2013, doi: 10.3389/fpsyg.2013.00504.
[6] S. Sukhov, M. Leontev, A. Miheev, and K. Sviatov, “Prevention of catastrophic interference and imposing active forgetting with
generative methods,” Neurocomputing, vol. 400, pp. 73–85, Aug. 2020, doi: 10.1016/j.neucom.2020.03.024.
11. Int J Elec & Comp Eng ISSN: 2088-8708
Convolutional auto-encoded extreme learning machine for incremental learning … (Sathya Madhusudanan)
5863
[7] L. C. Jain, M. Seera, C. P. Lim, and P. Balasubramaniam, “A review of online learning in supervised neural networks,” Neural
Computing and Applications, vol. 25, no. 3–4, pp. 491–509, Sep. 2014, doi: 10.1007/s00521-013-1534-4.
[8] J. Zhong, Z. Liu, Y. Zeng, L. Cui, and Z. Ji, “A survey on incremental learning,” in 5th International Conference on Computer,
Automation and Power Electronics, 2017, vol. 1, doi: 10.25236/cape.2017.034.
[9] S. Indolia, A. K. Goswami, S. P. Mishra, and P. Asopa, “Conceptual understanding of convolutional neural network- a deep learning
approach,” Procedia Computer Science, vol. 132, pp. 679–688, 2018, doi: 10.1016/j.procs.2018.05.069.
[10] R. Katuwal and P. N. Suganthan, “Stacked autoencoder based deep random vector functional link neural network for classification,”
Applied Soft Computing, vol. 85, Dec. 2019, doi: 10.1016/j.asoc.2019.105854.
[11] C. Chen, K. Li, M. Duan, and K. Li, “Extreme learning machine and its applications in big data processing,” in Big Data Analytics
for Sensor-Network Collected Intelligence, Elsevier, 2017, pp. 117–150, doi: 10.1016/B978-0-12-809393-1.00006-4.
[12] Q. Zhang, H. Li, C. Liu, and W. Hu, “A new extreme learning machine optimized by firefly algorithm,” in 2013 Sixth International
Symposium on Computational Intelligence and Design, Oct. 2013, pp. 133–136, doi: 10.1109/ISCID.2013.147.
[13] S. Song, M. Wang, and Y. Lin, “An improved algorithm for incremental extreme learning machine,” Systems Science & Control
Engineering, vol. 8, no. 1, pp. 308–317, Jan. 2020, doi: 10.1080/21642583.2020.1759156.
[14] G.-B. Huang, L. Chen, and C.-K. Siew, “Universal approximation using incremental constructive feedforward networks with
random hidden nodes,” IEEE Transactions on Neural Networks, vol. 17, no. 4, pp. 879–892, Jul. 2006, doi:
10.1109/TNN.2006.875977.
[15] J. Wang, S. Lu, S.-H. Wang, and Y.-D. Zhang, “A review on extreme learning machine,” Multimedia Tools and Applications,
vol. 81, no. 29, pp. 41611–41660, Dec. 2022, doi: 10.1007/s11042-021-11007-7.
[16] M. Yousefi-Azar and M. D. McDonnell, “Semi-supervised convolutional extreme learning machine,” in 2017 International Joint
Conference on Neural Networks (IJCNN), May 2017, pp. 1968–1974, doi: 10.1109/IJCNN.2017.7966092.
[17] L. Zhang, Z. He, and Y. Liu, “Deep object recognition across domains based on adaptive extreme learning machine,”
Neurocomputing, vol. 239, pp. 194–203, May 2017, doi: 10.1016/j.neucom.2017.02.016.
[18] L. Zhang, D. Zhang, and F. Tian, “SVM and ELM: Who wins? object recognition with deep convolutional features from ImageNet,”
in Proceedings of ELM-2015, 2016, pp. 249–263, doi: 10.1007/978-3-319-28397-5_20.
[19] V. Losing, B. Hammer, and H. Wersing, “Incremental on-line learning: a review and comparison of state of the art algorithms,”
Neurocomputing, vol. 275, pp. 1261–1274, Jan. 2018, doi: 10.1016/j.neucom.2017.06.084.
[20] J. F. García Molina, L. Zheng, M. Sertdemir, D. J. Dinter, S. Schönberg, and M. Rädle, “Incremental learning with SVM for
multimodal classification of prostatic adenocarcinoma,” PLoS ONE, vol. 9, no. 4, Apr. 2014, doi: 10.1371/journal.pone.0093600.
[21] Y. Xu, F. Shen, and J. Zhao, “An incremental learning vector quantization algorithm for pattern classification,” Neural Computing
and Applications, vol. 21, no. 6, pp. 1205–1215, Sep. 2012, doi: 10.1007/s00521-010-0511-4.
[22] P. Kang, “One-class naïve Bayesian classifier for toll fraud detection,” IEICE Transactions on Information and Systems, vol. E97.D,
no. 5, pp. 1353–1357, 2014, doi: 10.1587/transinf.E97.D.1353.
[23] T. Zhang, “Solving large scale linear prediction problems using stochastic gradient descent algorithms,” in Twenty-first
international conference on Machine learning - ICML ’04, 2004, doi: 10.1145/1015330.1015332.
[24] F. M. Castro, M. J. Marín-Jiménez, N. Guil, C. Schmid, and K. Alahari, “End-to-end incremental learning,” Prepr.
arXiv.1807.09536, Jul. 2018.
[25] S. Madhusudhanan, S. Jaganathan, and J. L S, “Incremental learning for classification of unstructured data using extreme learning
machine,” Algorithms, vol. 11, no. 10, Oct. 2018, doi: 10.3390/a11100158.
[26] G. M. van de Ven, H. T. Siegelmann, and A. S. Tolias, “Brain-inspired replay for continual learning with artificial neural networks,”
Nature Communications, vol. 11, no. 1, Aug. 2020, doi: 10.1038/s41467-020-17866-2.
[27] J. He, R. Mao, Z. Shao, and F. Zhu, “Incremental learning in online scenario,” in 2020 IEEE/CVF Conference on Computer Vision
and Pattern Recognition (CVPR), Jun. 2020, pp. 13923–13932, doi: 10.1109/CVPR42600.2020.01394.
[28] L. Guo, J. Hao, and M. Liu, “An incremental extreme learning machine for online sequential learning problems,” Neurocomputing,
vol. 128, pp. 50–58, Mar. 2014, doi: 10.1016/j.neucom.2013.03.055.
[29] G. Bin Huang, N. Y. Liang, H. J. Rong, P. Saratchandran, and N. Sundararajan, “On-line sequential extreme learning machine,” in
Proceedings of the IASTED International Conference on Computational Intelligence, 2005, vol. 2005, pp. 232–237.
[30] G. Feng, G.-B. Huang, Q. Lin, and R. Gay, “Error minimized extreme learning machine with growth of hidden nodes and
incremental learning,” IEEE Transactions on Neural Networks, vol. 20, no. 8, pp. 1352–1357, Aug. 2009, doi:
10.1109/TNN.2009.2024147.
[31] R. Polikar, L. Upda, S. S. Upda, and V. Honavar, “Learn++: an incremental learning algorithm for supervised neural networks,”
IEEE Transactions on Systems, Man and Cybernetics, Part C (Applications and Reviews), vol. 31, no. 4, pp. 497–508, 2001, doi:
10.1109/5326.983933.
[32] H. He, S. Chen, K. Li, and X. Xu, “Incremental learning from stream data,” IEEE Transactions on Neural Networks, vol. 22,
no. 12, pp. 1901–1914, Dec. 2011, doi: 10.1109/TNN.2011.2171713.
[33] P. Wang, X. Zhang, and Y. Hao, “A method combining CNN and ELM for feature extraction and classification of SAR image,”
Journal of Sensors, vol. 2019, pp. 1–8, Nov. 2019, doi: 10.1155/2019/6134610.
BIOGRAPHIES OF AUTHORS
Sathya Madhusudanan research scholar at Anna University pursuing her research
in deep learning. She did her master’s in computer science and engineering and obtained a
bachelor’s degree from Anna University. To her credit, she has published papers in reputed
International Conferences and Journals and also filed one patent. Her areas of interest are data
analytics and deep learning. She can be contacted at email: sathyamadhusudhanan@gmail.com.
12. ISSN: 2088-8708
Int J Elec & Comp Eng, Vol. 13, No. 5, October 2023: 5853-5864
5864
Suresh Jaganathan Associate Professor in the Department of Computer Science
and Engineering, Sri Sivasubramaniya Nadar College of Engineering, has more than 26 years of
teaching experience. He received his PhD in Computer Science from Jawaharlal Nehru
Technological University, Hyderabad, M.E Software Engineering from Anna University and
B.E Computer Science & Engineering, from Madurai Kamarajar University, Madurai. He has
more than 30 publications in referred International Journals and Conferences. Apart from this to
his credit, he has two patents in the area of image processing and has written a book on "Cloud
Computing: A Practical Approach for Learning and Implementation", published by Pearson
Publications. He is an active reviewer in reputed journals (Elsevier - Journal of Networks and
Computer Applications, Computer in Biology and Medicine) and also co-investigator for the
SSN-NIVIDA GPU Education center. His areas of interest are distributed computing, deep
learning, data analytics, machine learning and blockchain technology. He can be contacted at
email: sureshj@ssn.edu.in
Dattuluri Venkatavara Prasad Professor in the Department of Computer Science
and Engineering and has more than 20 years of teaching and research experience. He received
his B.E degree from the University of Mysore, M.E. from the Andhra University,
Visakhapatnam, and PhD from the Jawaharlal Nehru Technological University Anantapur. His
PhD work is on “Chip area minimization using interconnect length optimization”. His area of
research is computer architecture, GPU computing and machine learning. He is a member of
IEEE, ACM and also a Life member of CSI and ISTE. He has published a number of research
papers in International and National Journals and conferences. He has published two textbooks:
Computer Architecture, and MicroprocessorsMicrocontrollers. He is a principle Investigator for
SSN-nVIDIA GPU Education/Research Center. He can be contacted at dvvprasad@ssn.edu.in.