In this paper, a fruit image data set is used to compare the efficiency and accuracy of two widely used
Convolutional Neural Network, namely the ResNet and the DenseNet, for the recognition of 50 different
kinds of fruits. In the experiment, the structure of ResNet-34 and DenseNet_BC-121 (with bottleneck layer)
are used. The mathematic principle, experiment detail and the experiment result will be explained through
comparison.
Reconfiguration layers of convolutional neural network for fundus patches cla...journalBEEI
Convolutional neural network (CNN) is a method of supervised deep learning. The architectures including AlexNet, VGG16, VGG19, ResNet 50, ResNet101, GoogleNet, Inception-V3, Inception ResNet-V2, and Squeezenet that have 25 to 825 layers. This study aims to simplify layers of CNN architectures and increased accuracy for fundus patches classification. Fundus patches classify two categories: normal and neovascularization. Data used for classification is MESSIDOR and Retina Image Bank that have 2,080 patches. Results show the best accuracy of 93.17% for original data and 99,33% for augmentation data using CNN 31 layers. It consists input layer, 7 convolutional layers, 7 batch normalization, 7 rectified linear unit, 6 max-pooling, fully connected layer, softmax, and output layer.
ON THE PERFORMANCE OF INTRUSION DETECTION SYSTEMS WITH HIDDEN MULTILAYER NEUR...IJCNCJournal
Deep learning applications, especially multilayer neural network models, result in network intrusion detection with high accuracy. This study proposes a model that combines a multilayer neural network with Dense Sparse Dense (DSD) multi-stage training to simultaneously improve the criteria related to the performance of intrusion detection systems on a comprehensive dataset UNSW-NB15. We conduct experiments on many neural network models such as Recurrent Neural Network (RNN), Long-Short Term Memory (LSTM), Gated Recurrent Unit (GRU), etc. to evaluate the combined efficiency with each model through many criteria such as accuracy, detection rate, false alarm rate, precision, and F1-Score.
On The Performance of Intrusion Detection Systems with Hidden Multilayer Neur...IJCNCJournal
Deep learning applications, especially multilayer neural network models, result in network intrusion detection with high accuracy. This study proposes a model that combines a multilayer neural network with Dense Sparse Dense (DSD) multi-stage training to simultaneously improve the criteria related to the performance of intrusion detection systems on a comprehensive dataset UNSW-NB15. We conduct experiments on many neural network models such as Recurrent Neural Network (RNN), Long-Short Term Memory (LSTM), Gated Recurrent Unit (GRU), etc. to evaluate the combined efficiency with each model through many criteria such as accuracy, detection rate, false alarm rate, precision, and F1-Score.
Image classification is perhaps the most important part of digital image analysis. In this paper, we compare the most widely used model CNN Convolutional Neural Network , and MLP Multilayer Perceptron . We aim to show how both models differ and how both models approach towards the final goal, which is image classification. Souvik Banerjee | Dr. A Rengarajan "Hand-Written Digit Classification" Published in International Journal of Trend in Scientific Research and Development (ijtsrd), ISSN: 2456-6470, Volume-5 | Issue-4 , June 2021, URL: https://www.ijtsrd.compapers/ijtsrd42444.pdf Paper URL: https://www.ijtsrd.comcomputer-science/artificial-intelligence/42444/handwritten-digit-classification/souvik-banerjee
RunPool: A Dynamic Pooling Layer for Convolution Neural NetworkPutra Wanda
Deep learning (DL) has achieved a significant performance in computer vision problems, mainly in automatic feature extraction and representation. However, it is not easy to determine the best pooling method in a different case study. For instance, experts can implement the best types of pooling in image processing cases, which might not be optimal for various tasks. Thus, it is
required to keep in line with the philosophy of DL. In dynamic neural network architecture, it is not practically possible to find
a proper pooling technique for the layers. It is the primary reason why various pooling cannot be applied in the dynamic and multidimensional dataset. To deal with the limitations, it needs to construct an optimal pooling method as a better option than max pooling and average pooling. Therefore, we introduce a dynamic pooling layer called RunPool to train the convolutional
neuralnetwork(CNN)architecture.RunPoolpoolingisproposedtoregularizetheneuralnetworkthatreplacesthedeterministic
pooling functions. In the final section, we test the proposed pooling layer to address classification problems with online social network (OSN) dataset
Hyper-parameter optimization of convolutional neural network based on particl...journalBEEI
Deep neural networks have accomplished enormous progress in tackling many problems. More specifically, convolutional neural network (CNN) is a category of deep networks that have been a dominant technique in computer vision tasks. Despite that these deep neural networks are highly effective; the ideal structure is still an issue that needs a lot of investigation. Deep Convolutional Neural Network model is usually designed manually by trials and repeated tests which enormously constrain its application. Many hyper-parameters of the CNN can affect the model performance. These parameters are depth of the network, numbers of convolutional layers, and numbers of kernels with their sizes. Therefore, it may be a huge challenge to design an appropriate CNN model that uses optimized hyper-parameters and reduces the reliance on manual involvement and domain expertise. In this paper, a design architecture method for CNNs is proposed by utilization of particle swarm optimization (PSO) algorithm to learn the optimal CNN hyper-parameters values. In the experiment, we used Modified National Institute of Standards and Technology (MNIST) database of handwritten digit recognition. The experiments showed that our proposed approach can find an architecture that is competitive to the state-of-the-art models with a testing error of 0.87%.
Stochastic Computing Correlation Utilization in Convolutional Neural Network ...TELKOMNIKA JOURNAL
In recent years, many applications have been implemented in embedded systems and mobile Internet of Things (IoT) devices that typically have constrained resources, smaller power budget, and exhibit "smartness" or intelligence. To implement computation-intensive and resource-hungry Convolutional Neural Network (CNN) in this class of devices, many research groups have developed specialized parallel accelerators using Graphical Processing Units (GPU), Field-Programmable Gate Arrays (FPGA), or Application-Specific Integrated Circuits (ASIC). An alternative computing paradigm called Stochastic Computing (SC) can implement CNN with low hardware footprint and power consumption. To enable building more efficient SC CNN, this work incorporates the CNN basic functions in SC that exploit correlation, share Random Number Generators (RNG), and is more robust to rounding error. Experimental results show our proposed solution provides significant savings in hardware footprint and increased accuracy for the SC CNN basic functions circuits compared to previous work.
Reconfiguration layers of convolutional neural network for fundus patches cla...journalBEEI
Convolutional neural network (CNN) is a method of supervised deep learning. The architectures including AlexNet, VGG16, VGG19, ResNet 50, ResNet101, GoogleNet, Inception-V3, Inception ResNet-V2, and Squeezenet that have 25 to 825 layers. This study aims to simplify layers of CNN architectures and increased accuracy for fundus patches classification. Fundus patches classify two categories: normal and neovascularization. Data used for classification is MESSIDOR and Retina Image Bank that have 2,080 patches. Results show the best accuracy of 93.17% for original data and 99,33% for augmentation data using CNN 31 layers. It consists input layer, 7 convolutional layers, 7 batch normalization, 7 rectified linear unit, 6 max-pooling, fully connected layer, softmax, and output layer.
ON THE PERFORMANCE OF INTRUSION DETECTION SYSTEMS WITH HIDDEN MULTILAYER NEUR...IJCNCJournal
Deep learning applications, especially multilayer neural network models, result in network intrusion detection with high accuracy. This study proposes a model that combines a multilayer neural network with Dense Sparse Dense (DSD) multi-stage training to simultaneously improve the criteria related to the performance of intrusion detection systems on a comprehensive dataset UNSW-NB15. We conduct experiments on many neural network models such as Recurrent Neural Network (RNN), Long-Short Term Memory (LSTM), Gated Recurrent Unit (GRU), etc. to evaluate the combined efficiency with each model through many criteria such as accuracy, detection rate, false alarm rate, precision, and F1-Score.
On The Performance of Intrusion Detection Systems with Hidden Multilayer Neur...IJCNCJournal
Deep learning applications, especially multilayer neural network models, result in network intrusion detection with high accuracy. This study proposes a model that combines a multilayer neural network with Dense Sparse Dense (DSD) multi-stage training to simultaneously improve the criteria related to the performance of intrusion detection systems on a comprehensive dataset UNSW-NB15. We conduct experiments on many neural network models such as Recurrent Neural Network (RNN), Long-Short Term Memory (LSTM), Gated Recurrent Unit (GRU), etc. to evaluate the combined efficiency with each model through many criteria such as accuracy, detection rate, false alarm rate, precision, and F1-Score.
Image classification is perhaps the most important part of digital image analysis. In this paper, we compare the most widely used model CNN Convolutional Neural Network , and MLP Multilayer Perceptron . We aim to show how both models differ and how both models approach towards the final goal, which is image classification. Souvik Banerjee | Dr. A Rengarajan "Hand-Written Digit Classification" Published in International Journal of Trend in Scientific Research and Development (ijtsrd), ISSN: 2456-6470, Volume-5 | Issue-4 , June 2021, URL: https://www.ijtsrd.compapers/ijtsrd42444.pdf Paper URL: https://www.ijtsrd.comcomputer-science/artificial-intelligence/42444/handwritten-digit-classification/souvik-banerjee
RunPool: A Dynamic Pooling Layer for Convolution Neural NetworkPutra Wanda
Deep learning (DL) has achieved a significant performance in computer vision problems, mainly in automatic feature extraction and representation. However, it is not easy to determine the best pooling method in a different case study. For instance, experts can implement the best types of pooling in image processing cases, which might not be optimal for various tasks. Thus, it is
required to keep in line with the philosophy of DL. In dynamic neural network architecture, it is not practically possible to find
a proper pooling technique for the layers. It is the primary reason why various pooling cannot be applied in the dynamic and multidimensional dataset. To deal with the limitations, it needs to construct an optimal pooling method as a better option than max pooling and average pooling. Therefore, we introduce a dynamic pooling layer called RunPool to train the convolutional
neuralnetwork(CNN)architecture.RunPoolpoolingisproposedtoregularizetheneuralnetworkthatreplacesthedeterministic
pooling functions. In the final section, we test the proposed pooling layer to address classification problems with online social network (OSN) dataset
Hyper-parameter optimization of convolutional neural network based on particl...journalBEEI
Deep neural networks have accomplished enormous progress in tackling many problems. More specifically, convolutional neural network (CNN) is a category of deep networks that have been a dominant technique in computer vision tasks. Despite that these deep neural networks are highly effective; the ideal structure is still an issue that needs a lot of investigation. Deep Convolutional Neural Network model is usually designed manually by trials and repeated tests which enormously constrain its application. Many hyper-parameters of the CNN can affect the model performance. These parameters are depth of the network, numbers of convolutional layers, and numbers of kernels with their sizes. Therefore, it may be a huge challenge to design an appropriate CNN model that uses optimized hyper-parameters and reduces the reliance on manual involvement and domain expertise. In this paper, a design architecture method for CNNs is proposed by utilization of particle swarm optimization (PSO) algorithm to learn the optimal CNN hyper-parameters values. In the experiment, we used Modified National Institute of Standards and Technology (MNIST) database of handwritten digit recognition. The experiments showed that our proposed approach can find an architecture that is competitive to the state-of-the-art models with a testing error of 0.87%.
Stochastic Computing Correlation Utilization in Convolutional Neural Network ...TELKOMNIKA JOURNAL
In recent years, many applications have been implemented in embedded systems and mobile Internet of Things (IoT) devices that typically have constrained resources, smaller power budget, and exhibit "smartness" or intelligence. To implement computation-intensive and resource-hungry Convolutional Neural Network (CNN) in this class of devices, many research groups have developed specialized parallel accelerators using Graphical Processing Units (GPU), Field-Programmable Gate Arrays (FPGA), or Application-Specific Integrated Circuits (ASIC). An alternative computing paradigm called Stochastic Computing (SC) can implement CNN with low hardware footprint and power consumption. To enable building more efficient SC CNN, this work incorporates the CNN basic functions in SC that exploit correlation, share Random Number Generators (RNG), and is more robust to rounding error. Experimental results show our proposed solution provides significant savings in hardware footprint and increased accuracy for the SC CNN basic functions circuits compared to previous work.
CONVOLUTIONAL NEURAL NETWORK BASED RETINAL VESSEL SEGMENTATIONCSEIJJournal
In human eye, the state of the blood vessel is a crucial diagnostic factor. The segmentation of blood vessel
from the fundus image is difficult due to the spatial complexity, adjacency, overlapping and variability of
blood vessel. The detection of ophthalmic pathologies like hypertensive disorders, diabetic retinopathy and
cardiovascular diseases are remain challenging task due to the wide-ranging distribution of blood vessels.
In this paper, Stacked Autoencoder and CNN (Convolutional Neural Network) technique is proposed to
extract the blood vessel from the fundus image. Based on the experiments conducted using the Stacked
Autoencoder and Convolutional Neural Network gives 90% & 95% accuracy for segmentation.
Convolutional Neural Network based Retinal Vessel SegmentationCSEIJJournal
In human eye, the state of the blood vessel is a crucial diagnostic factor. The segmentation of blood vessel
from the fundus image is difficult due to the spatial complexity, adjacency, overlapping and variability of
blood vessel. The detection of ophthalmic pathologies like hypertensive disorders, diabetic retinopathy and
cardiovascular diseases are remain challenging task due to the wide-ranging distribution of blood vessels.
In this paper, Stacked Autoencoder and CNN (Convolutional Neural Network) technique is proposed to
extract the blood vessel from the fundus image. Based on the experiments conducted using the Stacked
Autoencoder and Convolutional Neural Network gives 90% & 95% accuracy for segmentation.
Efficient And Improved Video Steganography using DCT and Neural NetworkIJSRD
As per the demand of modern communication it is important to establish secret communication which is obtain by seganography .Video Steganography is the technique of hiding some covert message inside a video. The addition of this information to the video is not recognizable through the human eye as modify of a pixel color is negligible. In the proposed method Discrete Cosine Transform (DCT) and neural network is used. Input image is divided into blocks and is processed to generate quantization matrix of cover and stego images by using Discrete Cosine Transform (DCT).And using neural network performance of this method can be further improved. The neural network is trained and on the basis of training and segmentation done, neural network provide efficient positions where data can be merge. The performance and efficiency is measured by PSNR and MSE value.
Machine learning based augmented reality for improved learning application th...IJECEIAES
Detection of objects and their location in an image are important elements of current research in computer vision. In May 2020, Meta released its state-ofthe-art object-detection model based on a transformer architecture called detection transformer (DETR). There are several object-detection models such as region-based convolutional neural network (R-CNN), you only look once (YOLO) and single shot detectors (SSD), but none have used a transformer to accomplish this task. These models mentioned earlier, use all sorts of hyperparameters and layers. However, the advantages of using a transformer pattern make the architecture simple and easy to implement. In this paper, we determine the name of a chemical experiment through two steps: firstly, by building a DETR model, trained on a customized dataset, and then integrate it into an augmented reality mobile application. By detecting the objects used during the realization of an experiment, we can predict the name of the experiment using a multi-class classification approach. The combination of various computer vision techniques with augmented reality is indeed promising and offers a better user experience.
Comparative Study of Neural Networks Algorithms for Cloud Computing CPU Sched...IJECEIAES
Cloud Computing is the most powerful computing model of our time. While the major IT providers and consumers are competing to exploit the benefits of this computing model in order to thrive their profits, most of the cloud computing platforms are still built on operating systems that uses basic CPU (Core Processing Unit) scheduling algorithms that lacks the intelligence needed for such innovative computing model. Correspdondingly, this paper presents the benefits of applying Artificial Neural Networks algorithms in regards to enhancing CPU scheduling for Cloud Computing model. Furthermore, a set of characteristics and theoretical metrics are proposed for the sake of comparing the different Artificial Neural Networks algorithms and finding the most accurate algorithm for Cloud Computing CPU Scheduling.
We trained a large, deep convolutional neural network to classify the 1.2 million
high-resolution images in the ImageNet LSVRC-2010 contest into the 1000 different
classes. On the test data, we achieved top-1 and top-5 error rates of 37.5%
and 17.0% which is considerably better than the previous state-of-the-art. The
neural network, which has 60 million parameters and 650,000 neurons, consists
of five convolutional layers, some of which are followed by max-pooling layers,
and three fully-connected layers with a final 1000-way softmax. To make training
faster, we used non-saturating neurons and a very efficient GPU implementation
of the convolution operation. To reduce overfitting in the fully-connected
layers we employed a recently-developed regularization method called “dropout”
that proved to be very effective. We also entered a variant of this model in the
ILSVRC-2012 competition and achieved a winning top-5 test error rate of 15.3%,
compared to 26.2% achieved by the second-best entry
AN IMPROVED METHOD FOR IDENTIFYING WELL-TEST INTERPRETATION MODEL BASED ON AG...IAEME Publication
This paper presents an approach based on applying an aggregated predictor formed by multiple versions of a multilayer neural network with a back-propagation optimization algorithm for helping the engineer to get a list of the most appropriate well-test interpretation models for a given set of pressure/ production data. The proposed method consists of three stages: (1) data decorrelation through principal component analysis to reduce the covariance between the variables and the dimension of the input layer in the artificial neural network, (2) bootstrap replicates of the learning set where the data is repeatedly sampled with a random split of the data into train sets and using these as new learning sets, and (3) automatic reservoir model identification through aggregated predictor formed by a plurality vote when predicting a new class. This method is described in detail to ensure successful replication of results. The required training and test dataset were generated by using analytical solution models. In our case, there were used 600 samples: 300 for training, 100 for cross-validation, and 200 for testing. Different network structures were tested during this study to arrive at optimum network design. We notice that the single net methodology always brings about confusion in selecting the correct model even though the training results for the constructed networks are close to 1. We notice also that the principal component analysis is an effective strategy in reducing the number of input features, simplifying the network structure, and lowering the training time of the ANN. The results obtained show that the proposed model provides better performance when predicting new data with a coefficient of correlation approximately equal to 95% Compared to a previous approach 80%, the combination of the PCA and ANN is more stable and determine the more accurate results with lesser computational complexity than was feasible previously. Clearly, the aggregated predictor is more stable and shows less bad classes compared to the previous approach.
A New Method for Figuring the Number of Hidden Layer Nodes in BP Algorithmrahulmonikasharma
In the field of artificial neural network, BP neural network is a multi-layer feed-forward neural network. Because it is difficult to figure the number of hidden layer nodes in a BP neural network, the theoretical basis and the existing methods for BP network hidden layer nodes are studied. Then based on traditional empirical formulas, we propose a new approach to rapidly figure the quantity of hidden layer nodes in two-layer network. That is, with the assistance of experience formulas, the horizon of unit number in hidden layer can be confirmed and its optimal value will be found in this horizon. Finally, a new formula for figuring the quantity of hidden layer codes is obtained through fitting input dimension, output dimension and the optimal value of hidden layer codes. Under some given input dimension and output dimension, efficiency and precision of BP algorithm may be improved by applying the proposed formula.
TEST-COST-SENSITIVE CONVOLUTIONAL NEURAL NETWORKS WITH EXPERT BRANCHESsipij
It has been proven that deeper convolutional neural networks (CNN) can result in better accuracy in many
problems, but this accuracy comes with a high computational cost. Also, input instances have not the same
difficulty. As a solution for accuracy vs. computational cost dilemma, we introduce a new test-cost-sensitive
method for convolutional neural networks. This method trains a CNN with a set of auxiliary outputs and
expert branches in some middle layers of the network. The expert branches decide to use a shallower part
of the network or going deeper to the end, based on the difficulty of input instance. The expert branches
learn to determine: is the current network prediction is wrong and if the given instance passed to deeper
layers of the network it will generate right output; If not, then the expert branches stop the computation
process. The experimental results on standard dataset CIFAR-10 show that the proposed method can train
models with lower test-cost and competitive accuracy in comparison with basic models.
Architecture neural network deep optimizing based on self organizing feature ...journalBEEI
Forward neural network (FNN) execution relying on the algorithm of training and architecture selection. Different parameters using for nip out the architecture of FNN such as the connections number among strata, neurons hidden number in each strata hidden and hidden strata number. Feature architectural combinations exponential could be uncontrollable manually so specific architecture can be design automatically by using special algorithm which build system with ability generalization better. Determination of architecture FNN can be done by using the algorithm of optimization numerous. In this paper methodology new proposes achievement where FNN neurons respective with hidden layers estimation work where in this work collect algorithm training self organizing feature map (SOFM) with advantages to explain how the best architectural selected automatically by SOFM from criteria error testing based on architecture populated. Different size of dataset benchmark of 4 classifications tested for approach proposed.
Effective Multi-Stage Training Model for Edge Computing Devices in Intrusion ...IJCNCJournal
Intrusion detection poses a significant challenge within expansive and persistently interconnected environments. As malicious code continues to advance and sophisticated attack methodologies proliferate, various advanced deep learning-based detection approaches have been proposed. Nevertheless, the complexity and accuracy of intrusion detection models still need further enhancement to render them more adaptable to diverse system categories, particularly within resource-constrained devices, such as those embedded in edge computing systems. This research introduces a three-stage training paradigm, augmented by an enhanced pruning methodology and model compression techniques. The objective is to elevate the system's effectiveness, concurrently maintaining a high level of accuracy for intrusion detection. Empirical assessments conducted on the UNSW-NB15 dataset evince that this solution notably reduces the model's dimensions, while upholding accuracy levels equivalent to similar proposals.
Effective Multi-Stage Training Model for Edge Computing Devices in Intrusion ...IJCNCJournal
Intrusion detection poses a significant challenge within expansive and persistently interconnected environments. As malicious code continues to advance and sophisticated attack methodologies proliferate, various advanced deep learning-based detection approaches have been proposed. Nevertheless, the complexity and accuracy of intrusion detection models still need further enhancement to render them more adaptable to diverse system categories, particularly within resource-constrained devices, such as those embedded in edge computing systems. This research introduces a three-stage training paradigm, augmented by an enhanced pruning methodology and model compression techniques. The objective is to elevate the system's effectiveness, concurrently maintaining a high level of accuracy for intrusion detection. Empirical assessments conducted on the UNSW-NB15 dataset evince that this solution notably reduces the model's dimensions, while upholding accuracy levels equivalent to similar proposals.
The International Journal of Computational Science, Information Technology an...rinzindorjej
The International Journal of Computational Science, Information Technology and Control Engineering (IJCSITCE) is an open access peer-reviewed journal that publishes quality articles which make innovative contributions in all areas of Computational Science, Mathematical Modeling, Information Technology, Networks, Computer Science, Control and Automation Engineering. IJCSITCE is an abstracted and indexed journal that focuses on all technical and practical aspects of Scientific Computing, Modeling and Simulation, Information Technology, Computer Science, Networks and Communication Engineering, Control Theory and Automation. The goal of this journal is to bring together researchers and practitioners from academia and industry to focus on advanced techniques in computational science, information technology, computer science, chaos, control theory and automation, and establishing new collaborations in these areas.
2 nd International Conference on Soft Computing, Data mining and Data Scienc...rinzindorjej
2
nd International Conference on Soft Computing, Data mining and Data Science (SCDD
2024) will provide an excellent international forum for sharing knowledge and results in
theory, methodology and applications of Soft Computing, Data mining, and Data Science.
The Conference looks for significant contributions to all major fields of the Soft Computing,
Data mining, and Data Science in theoretical and practical aspects. The aim of the
Conference is to provide a platform to the researchers and practitioners from both academia
as well as industry to meet and share cutting-edge development in the field.
Authors are solicited to contribute to the Conference by submitting articles that illustrate
research results, projects, surveying works and industrial experiences that describe significant
advances in the following areas, but are not limited to
More Related Content
Similar to CONTRAST OF RESNET AND DENSENET BASED ON THE RECOGNITION OF SIMPLE FRUIT DATA SET
CONVOLUTIONAL NEURAL NETWORK BASED RETINAL VESSEL SEGMENTATIONCSEIJJournal
In human eye, the state of the blood vessel is a crucial diagnostic factor. The segmentation of blood vessel
from the fundus image is difficult due to the spatial complexity, adjacency, overlapping and variability of
blood vessel. The detection of ophthalmic pathologies like hypertensive disorders, diabetic retinopathy and
cardiovascular diseases are remain challenging task due to the wide-ranging distribution of blood vessels.
In this paper, Stacked Autoencoder and CNN (Convolutional Neural Network) technique is proposed to
extract the blood vessel from the fundus image. Based on the experiments conducted using the Stacked
Autoencoder and Convolutional Neural Network gives 90% & 95% accuracy for segmentation.
Convolutional Neural Network based Retinal Vessel SegmentationCSEIJJournal
In human eye, the state of the blood vessel is a crucial diagnostic factor. The segmentation of blood vessel
from the fundus image is difficult due to the spatial complexity, adjacency, overlapping and variability of
blood vessel. The detection of ophthalmic pathologies like hypertensive disorders, diabetic retinopathy and
cardiovascular diseases are remain challenging task due to the wide-ranging distribution of blood vessels.
In this paper, Stacked Autoencoder and CNN (Convolutional Neural Network) technique is proposed to
extract the blood vessel from the fundus image. Based on the experiments conducted using the Stacked
Autoencoder and Convolutional Neural Network gives 90% & 95% accuracy for segmentation.
Efficient And Improved Video Steganography using DCT and Neural NetworkIJSRD
As per the demand of modern communication it is important to establish secret communication which is obtain by seganography .Video Steganography is the technique of hiding some covert message inside a video. The addition of this information to the video is not recognizable through the human eye as modify of a pixel color is negligible. In the proposed method Discrete Cosine Transform (DCT) and neural network is used. Input image is divided into blocks and is processed to generate quantization matrix of cover and stego images by using Discrete Cosine Transform (DCT).And using neural network performance of this method can be further improved. The neural network is trained and on the basis of training and segmentation done, neural network provide efficient positions where data can be merge. The performance and efficiency is measured by PSNR and MSE value.
Machine learning based augmented reality for improved learning application th...IJECEIAES
Detection of objects and their location in an image are important elements of current research in computer vision. In May 2020, Meta released its state-ofthe-art object-detection model based on a transformer architecture called detection transformer (DETR). There are several object-detection models such as region-based convolutional neural network (R-CNN), you only look once (YOLO) and single shot detectors (SSD), but none have used a transformer to accomplish this task. These models mentioned earlier, use all sorts of hyperparameters and layers. However, the advantages of using a transformer pattern make the architecture simple and easy to implement. In this paper, we determine the name of a chemical experiment through two steps: firstly, by building a DETR model, trained on a customized dataset, and then integrate it into an augmented reality mobile application. By detecting the objects used during the realization of an experiment, we can predict the name of the experiment using a multi-class classification approach. The combination of various computer vision techniques with augmented reality is indeed promising and offers a better user experience.
Comparative Study of Neural Networks Algorithms for Cloud Computing CPU Sched...IJECEIAES
Cloud Computing is the most powerful computing model of our time. While the major IT providers and consumers are competing to exploit the benefits of this computing model in order to thrive their profits, most of the cloud computing platforms are still built on operating systems that uses basic CPU (Core Processing Unit) scheduling algorithms that lacks the intelligence needed for such innovative computing model. Correspdondingly, this paper presents the benefits of applying Artificial Neural Networks algorithms in regards to enhancing CPU scheduling for Cloud Computing model. Furthermore, a set of characteristics and theoretical metrics are proposed for the sake of comparing the different Artificial Neural Networks algorithms and finding the most accurate algorithm for Cloud Computing CPU Scheduling.
We trained a large, deep convolutional neural network to classify the 1.2 million
high-resolution images in the ImageNet LSVRC-2010 contest into the 1000 different
classes. On the test data, we achieved top-1 and top-5 error rates of 37.5%
and 17.0% which is considerably better than the previous state-of-the-art. The
neural network, which has 60 million parameters and 650,000 neurons, consists
of five convolutional layers, some of which are followed by max-pooling layers,
and three fully-connected layers with a final 1000-way softmax. To make training
faster, we used non-saturating neurons and a very efficient GPU implementation
of the convolution operation. To reduce overfitting in the fully-connected
layers we employed a recently-developed regularization method called “dropout”
that proved to be very effective. We also entered a variant of this model in the
ILSVRC-2012 competition and achieved a winning top-5 test error rate of 15.3%,
compared to 26.2% achieved by the second-best entry
AN IMPROVED METHOD FOR IDENTIFYING WELL-TEST INTERPRETATION MODEL BASED ON AG...IAEME Publication
This paper presents an approach based on applying an aggregated predictor formed by multiple versions of a multilayer neural network with a back-propagation optimization algorithm for helping the engineer to get a list of the most appropriate well-test interpretation models for a given set of pressure/ production data. The proposed method consists of three stages: (1) data decorrelation through principal component analysis to reduce the covariance between the variables and the dimension of the input layer in the artificial neural network, (2) bootstrap replicates of the learning set where the data is repeatedly sampled with a random split of the data into train sets and using these as new learning sets, and (3) automatic reservoir model identification through aggregated predictor formed by a plurality vote when predicting a new class. This method is described in detail to ensure successful replication of results. The required training and test dataset were generated by using analytical solution models. In our case, there were used 600 samples: 300 for training, 100 for cross-validation, and 200 for testing. Different network structures were tested during this study to arrive at optimum network design. We notice that the single net methodology always brings about confusion in selecting the correct model even though the training results for the constructed networks are close to 1. We notice also that the principal component analysis is an effective strategy in reducing the number of input features, simplifying the network structure, and lowering the training time of the ANN. The results obtained show that the proposed model provides better performance when predicting new data with a coefficient of correlation approximately equal to 95% Compared to a previous approach 80%, the combination of the PCA and ANN is more stable and determine the more accurate results with lesser computational complexity than was feasible previously. Clearly, the aggregated predictor is more stable and shows less bad classes compared to the previous approach.
A New Method for Figuring the Number of Hidden Layer Nodes in BP Algorithmrahulmonikasharma
In the field of artificial neural network, BP neural network is a multi-layer feed-forward neural network. Because it is difficult to figure the number of hidden layer nodes in a BP neural network, the theoretical basis and the existing methods for BP network hidden layer nodes are studied. Then based on traditional empirical formulas, we propose a new approach to rapidly figure the quantity of hidden layer nodes in two-layer network. That is, with the assistance of experience formulas, the horizon of unit number in hidden layer can be confirmed and its optimal value will be found in this horizon. Finally, a new formula for figuring the quantity of hidden layer codes is obtained through fitting input dimension, output dimension and the optimal value of hidden layer codes. Under some given input dimension and output dimension, efficiency and precision of BP algorithm may be improved by applying the proposed formula.
TEST-COST-SENSITIVE CONVOLUTIONAL NEURAL NETWORKS WITH EXPERT BRANCHESsipij
It has been proven that deeper convolutional neural networks (CNN) can result in better accuracy in many
problems, but this accuracy comes with a high computational cost. Also, input instances have not the same
difficulty. As a solution for accuracy vs. computational cost dilemma, we introduce a new test-cost-sensitive
method for convolutional neural networks. This method trains a CNN with a set of auxiliary outputs and
expert branches in some middle layers of the network. The expert branches decide to use a shallower part
of the network or going deeper to the end, based on the difficulty of input instance. The expert branches
learn to determine: is the current network prediction is wrong and if the given instance passed to deeper
layers of the network it will generate right output; If not, then the expert branches stop the computation
process. The experimental results on standard dataset CIFAR-10 show that the proposed method can train
models with lower test-cost and competitive accuracy in comparison with basic models.
Architecture neural network deep optimizing based on self organizing feature ...journalBEEI
Forward neural network (FNN) execution relying on the algorithm of training and architecture selection. Different parameters using for nip out the architecture of FNN such as the connections number among strata, neurons hidden number in each strata hidden and hidden strata number. Feature architectural combinations exponential could be uncontrollable manually so specific architecture can be design automatically by using special algorithm which build system with ability generalization better. Determination of architecture FNN can be done by using the algorithm of optimization numerous. In this paper methodology new proposes achievement where FNN neurons respective with hidden layers estimation work where in this work collect algorithm training self organizing feature map (SOFM) with advantages to explain how the best architectural selected automatically by SOFM from criteria error testing based on architecture populated. Different size of dataset benchmark of 4 classifications tested for approach proposed.
Effective Multi-Stage Training Model for Edge Computing Devices in Intrusion ...IJCNCJournal
Intrusion detection poses a significant challenge within expansive and persistently interconnected environments. As malicious code continues to advance and sophisticated attack methodologies proliferate, various advanced deep learning-based detection approaches have been proposed. Nevertheless, the complexity and accuracy of intrusion detection models still need further enhancement to render them more adaptable to diverse system categories, particularly within resource-constrained devices, such as those embedded in edge computing systems. This research introduces a three-stage training paradigm, augmented by an enhanced pruning methodology and model compression techniques. The objective is to elevate the system's effectiveness, concurrently maintaining a high level of accuracy for intrusion detection. Empirical assessments conducted on the UNSW-NB15 dataset evince that this solution notably reduces the model's dimensions, while upholding accuracy levels equivalent to similar proposals.
Effective Multi-Stage Training Model for Edge Computing Devices in Intrusion ...IJCNCJournal
Intrusion detection poses a significant challenge within expansive and persistently interconnected environments. As malicious code continues to advance and sophisticated attack methodologies proliferate, various advanced deep learning-based detection approaches have been proposed. Nevertheless, the complexity and accuracy of intrusion detection models still need further enhancement to render them more adaptable to diverse system categories, particularly within resource-constrained devices, such as those embedded in edge computing systems. This research introduces a three-stage training paradigm, augmented by an enhanced pruning methodology and model compression techniques. The objective is to elevate the system's effectiveness, concurrently maintaining a high level of accuracy for intrusion detection. Empirical assessments conducted on the UNSW-NB15 dataset evince that this solution notably reduces the model's dimensions, while upholding accuracy levels equivalent to similar proposals.
Similar to CONTRAST OF RESNET AND DENSENET BASED ON THE RECOGNITION OF SIMPLE FRUIT DATA SET (20)
The International Journal of Computational Science, Information Technology an...rinzindorjej
The International Journal of Computational Science, Information Technology and Control Engineering (IJCSITCE) is an open access peer-reviewed journal that publishes quality articles which make innovative contributions in all areas of Computational Science, Mathematical Modeling, Information Technology, Networks, Computer Science, Control and Automation Engineering. IJCSITCE is an abstracted and indexed journal that focuses on all technical and practical aspects of Scientific Computing, Modeling and Simulation, Information Technology, Computer Science, Networks and Communication Engineering, Control Theory and Automation. The goal of this journal is to bring together researchers and practitioners from academia and industry to focus on advanced techniques in computational science, information technology, computer science, chaos, control theory and automation, and establishing new collaborations in these areas.
2 nd International Conference on Soft Computing, Data mining and Data Scienc...rinzindorjej
2
nd International Conference on Soft Computing, Data mining and Data Science (SCDD
2024) will provide an excellent international forum for sharing knowledge and results in
theory, methodology and applications of Soft Computing, Data mining, and Data Science.
The Conference looks for significant contributions to all major fields of the Soft Computing,
Data mining, and Data Science in theoretical and practical aspects. The aim of the
Conference is to provide a platform to the researchers and practitioners from both academia
as well as industry to meet and share cutting-edge development in the field.
Authors are solicited to contribute to the Conference by submitting articles that illustrate
research results, projects, surveying works and industrial experiences that describe significant
advances in the following areas, but are not limited to
The International Journal of Computational Science, Information Technology an...rinzindorjej
The International Journal of Computational Science, Information Technology and Control Engineering (IJCSITCE) is an open access peer-reviewed journal that publishes quality articles which make innovative contributions in all areas of Computational Science, Mathematical Modeling, Information Technology, Networks, Computer Science, Control and Automation Engineering. IJCSITCE is an abstracted and indexed journal that focuses on all technical and practical aspects of Scientific Computing, Modeling and Simulation, Information Technology, Computer Science, Networks and Communication Engineering, Control Theory and Automation. The goal of this journal is to bring together researchers and practitioners from academia and industry to focus on advanced techniques in computational science, information technology, computer science, chaos, control theory and automation, and establishing new collaborations in these areas.
The International Journal of Computational Science, Information Technology an...rinzindorjej
The International Journal of Computational Science, Information Technology and Control Engineering (IJCSITCE) is an open access peer-reviewed journal that publishes quality articles which make innovative contributions in all areas of Computational Science, Mathematical Modeling, Information Technology, Networks, Computer Science, Control and Automation Engineering. IJCSITCE is an abstracted and indexed journal that focuses on all technical and practical aspects of Scientific Computing, Modeling and Simulation, Information Technology, Computer Science, Networks and Communication Engineering, Control Theory and Automation. The goal of this journal is to bring together researchers and practitioners from academia and industry to focus on advanced techniques in computational science, information technology, computer science, chaos, control theory and automation, and establishing new collaborations in these areas.
The International Journal of Computational Science, Information Technology an...rinzindorjej
The International Journal of Computational Science, Information Technology and Control Engineering (IJCSITCE) is an open access peer-reviewed journal that publishes quality articles which make innovative contributions in all areas of Computational Science, Mathematical Modeling, Information Technology, Networks, Computer Science, Control and Automation Engineering. IJCSITCE is an abstracted and indexed journal that focuses on all technical and practical aspects of Scientific Computing, Modeling and Simulation, Information Technology, Computer Science, Networks and Communication Engineering, Control Theory and Automation. The goal of this journal is to bring together researchers and practitioners from academia and industry to focus on advanced techniques in computational science, information technology, computer science, chaos, control theory and automation, and establishing new collaborations in these areas.
The International Journal of Computational Science, Information Technology an...rinzindorjej
The International Journal of Computational Science, Information Technology and Control Engineering (IJCSITCE) is an open access peer-reviewed journal that publishes quality articles which make innovative contributions in all areas of Computational Science, Mathematical Modeling, Information Technology, Networks, Computer Science, Control and Automation Engineering. IJCSITCE is an abstracted and indexed journal that focuses on all technical and practical aspects of Scientific Computing, Modeling and Simulation, Information Technology, Computer Science, Networks and Communication Engineering, Control Theory and Automation. The goal of this journal is to bring together researchers and practitioners from academia and industry to focus on advanced techniques in computational science, information technology, computer science, chaos, control theory and automation, and establishing new collaborations in these areas.
The International Journal of Computational Science, Information Technology an...rinzindorjej
The International Journal of Computational Science, Information Technology and Control Engineering (IJCSITCE) is an open access peer-reviewed journal that publishes quality articles which make innovative contributions in all areas of Computational Science, Mathematical Modeling, Information Technology, Networks, Computer Science, Control and Automation Engineering. IJCSITCE is an abstracted and indexed journal that focuses on all technical and practical aspects of Scientific Computing, Modeling and Simulation, Information Technology, Computer Science, Networks and Communication Engineering, Control Theory and Automation. The goal of this journal is to bring together researchers and practitioners from academia and industry to focus on advanced techniques in computational science, information technology, computer science, chaos, control theory and automation, and establishing new collaborations in these areas.
International Journal of Computational Science, Information Technology and Co...rinzindorjej
The International Journal of Computational Science, Information Technology and Control Engineering (IJCSITCE) is an open access peer-reviewed journal that publishes quality articles which make innovative contributions in all areas of Computational Science, Mathematical Modeling, Information Technology, Networks, Computer Science, Control and Automation Engineering. IJCSITCE is an abstracted and indexed journal that focuses on all technical and practical aspects of Scientific Computing, Modeling and Simulation, Information Technology, Computer Science, Networks and Communication Engineering, Control Theory and Automation. The goal of this journal is to bring together researchers and practitioners from academia and industry to focus on advanced techniques in computational science, information technology, computer science, chaos, control theory and automation, and establishing new collaborations in these areas.
International Journal of Computational Science, Information Technology and Co...rinzindorjej
The International Journal of Computational Science, Information Technology and Control Engineering (IJCSITCE) is an open access peer-reviewed journal that publishes quality articles which make innovative contributions in all areas of Computational Science, Mathematical Modeling, Information Technology, Networks, Computer Science, Control and Automation Engineering. IJCSITCE is an abstracted and indexed journal that focuses on all technical and practical aspects of Scientific Computing, Modeling and Simulation, Information Technology, Computer Science, Networks and Communication Engineering, Control Theory and Automation. The goal of this journal is to bring together researchers and practitioners from academia and industry to focus on advanced techniques in computational science, information technology, computer science, chaos, control theory and automation, and establishing new collaborations in these areas.
The International Journal of Computational Science, Information Technology an...rinzindorjej
The International Journal of Computational Science, Information Technology and Control Engineering (IJCSITCE) is an open access peer-reviewed journal that publishes quality articles which make innovative contributions in all areas of Computational Science, Mathematical Modeling, Information Technology, Networks, Computer Science, Control and Automation Engineering. IJCSITCE is an abstracted and indexed journal that focuses on all technical and practical aspects of Scientific Computing, Modeling and Simulation, Information Technology, Computer Science, Networks and Communication Engineering, Control Theory and Automation. The goal of this journal is to bring together researchers and practitioners from academia and industry to focus on advanced techniques in computational science, information technology, computer science, chaos, control theory and automation, and establishing new collaborations in these areas.
The International Journal of Computational Science, Information Technology an...rinzindorjej
The International Journal of Computational Science, Information Technology and Control Engineering (IJCSITCE) is an open access peer-reviewed journal that publishes quality articles which make innovative contributions in all areas of Computational Science, Mathematical Modeling, Information Technology, Networks, Computer Science, Control and Automation Engineering. IJCSITCE is an abstracted and indexed journal that focuses on all technical and practical aspects of Scientific Computing, Modeling and Simulation, Information Technology, Computer Science, Networks and Communication Engineering, Control Theory and Automation. The goal of this journal is to bring together researchers and practitioners from academia and industry to focus on advanced techniques in computational science, information technology, computer science, chaos, control theory and automation, and establishing new collaborations in these areas.
The International Journal of Computational Science, Information Technology an...rinzindorjej
The International Journal of Computational Science, Information Technology and Control Engineering (IJCSITCE) is an open access peer-reviewed journal that publishes quality articles which make innovative contributions in all areas of Computational Science, Mathematical Modeling, Information Technology, Networks, Computer Science, Control and Automation Engineering. IJCSITCE is an abstracted and indexed journal that focuses on all technical and practical aspects of Scientific Computing, Modeling and Simulation, Information Technology, Computer Science, Networks and Communication Engineering, Control Theory and Automation. The goal of this journal is to bring together researchers and practitioners from academia and industry to focus on advanced techniques in computational science, information technology, computer science, chaos, control theory and automation, and establishing new collaborations in these areas.
The International Journal of Computational Science, Information Technology an...rinzindorjej
The International Journal of Computational Science, Information Technology and Control Engineering (IJCSITCE) is an open access peer-reviewed journal that publishes quality articles which make innovative contributions in all areas of Computational Science, Mathematical Modeling, Information Technology, Networks, Computer Science, Control and Automation Engineering. IJCSITCE is an abstracted and indexed journal that focuses on all technical and practical aspects of Scientific Computing, Modeling and Simulation, Information Technology, Computer Science, Networks and Communication Engineering, Control Theory and Automation. The goal of this journal is to bring together researchers and practitioners from academia and industry to focus on advanced techniques in computational science, information technology, computer science, chaos, control theory and automation, and establishing new collaborations in these areas.
The International Journal of Computational Science, Information Technology an...rinzindorjej
The International Journal of Computational Science, Information Technology and Control Engineering (IJCSITCE) is an open access peer-reviewed journal that publishes quality articles which make innovative contributions in all areas of Computational Science, Mathematical Modeling, Information Technology, Networks, Computer Science, Control and Automation Engineering. IJCSITCE is an abstracted and indexed journal that focuses on all technical and practical aspects of Scientific Computing, Modeling and Simulation, Information Technology, Computer Science, Networks and Communication Engineering, Control Theory and Automation. The goal of this journal is to bring together researchers and practitioners from academia and industry to focus on advanced techniques in computational science, information technology, computer science, chaos, control theory and automation, and establishing new collaborations in these areas.
The International Journal of Computational Science, Information Technology an...rinzindorjej
The International Journal of Computational Science, Information Technology and Control Engineering (IJCSITCE) is an open access peer-reviewed journal that publishes quality articles which make innovative contributions in all areas of Computational Science, Mathematical Modeling, Information Technology, Networks, Computer Science, Control and Automation Engineering. IJCSITCE is an abstracted and indexed journal that focuses on all technical and practical aspects of Scientific Computing, Modeling and Simulation, Information Technology, Computer Science, Networks and Communication Engineering, Control Theory and Automation. The goal of this journal is to bring together researchers and practitioners from academia and industry to focus on advanced techniques in computational science, information technology, computer science, chaos, control theory and automation, and establishing new collaborations in these areas.
The International Journal of Computational Science, Information Technology an...rinzindorjej
The International Journal of Computational Science, Information Technology and Control Engineering (IJCSITCE) is an open access peer-reviewed journal that publishes quality articles which make innovative contributions in all areas of Computational Science, Mathematical Modeling, Information Technology, Networks, Computer Science, Control and Automation Engineering. IJCSITCE is an abstracted and indexed journal that focuses on all technical and practical aspects of Scientific Computing, Modeling and Simulation, Information Technology, Computer Science, Networks and Communication Engineering, Control Theory and Automation. The goal of this journal is to bring together researchers and practitioners from academia and industry to focus on advanced techniques in computational science, information technology, computer science, chaos, control theory and automation, and establishing new collaborations in these areas.
The International Journal of Computational Science, Information Technology an...rinzindorjej
The International Journal of Computational Science, Information Technology and Control Engineering (IJCSITCE) is an open access peer-reviewed journal that publishes quality articles which make innovative contributions in all areas of Computational Science, Mathematical Modeling, Information Technology, Networks, Computer Science, Control and Automation Engineering. IJCSITCE is an abstracted and indexed journal that focuses on all technical and practical aspects of Scientific Computing, Modeling and Simulation, Information Technology, Computer Science, Networks and Communication Engineering, Control Theory and Automation. The goal of this journal is to bring together researchers and practitioners from academia and industry to focus on advanced techniques in computational science, information technology, computer science, chaos, control theory and automation, and establishing new collaborations in these areas.
The International Journal of Computational Science, Information Technology an...rinzindorjej
The International Journal of Computational Science, Information Technology and Control Engineering (IJCSITCE) is an open access peer-reviewed journal that publishes quality articles which make innovative contributions in all areas of Computational Science, Mathematical Modeling, Information Technology, Networks, Computer Science, Control and Automation Engineering. IJCSITCE is an abstracted and indexed journal that focuses on all technical and practical aspects of Scientific Computing, Modeling and Simulation, Information Technology, Computer Science, Networks and Communication Engineering, Control Theory and Automation. The goal of this journal is to bring together researchers and practitioners from academia and industry to focus on advanced techniques in computational science, information technology, computer science, chaos, control theory and automation, and establishing new collaborations in these areas.
Immunizing Image Classifiers Against Localized Adversary Attacksgerogepatton
This paper addresses the vulnerability of deep learning models, particularly convolutional neural networks
(CNN)s, to adversarial attacks and presents a proactive training technique designed to counter them. We
introduce a novel volumization algorithm, which transforms 2D images into 3D volumetric representations.
When combined with 3D convolution and deep curriculum learning optimization (CLO), itsignificantly improves
the immunity of models against localized universal attacks by up to 40%. We evaluate our proposed approach
using contemporary CNN architectures and the modified Canadian Institute for Advanced Research (CIFAR-10
and CIFAR-100) and ImageNet Large Scale Visual Recognition Challenge (ILSVRC12) datasets, showcasing
accuracy improvements over previous techniques. The results indicate that the combination of the volumetric
input and curriculum learning holds significant promise for mitigating adversarial attacks without necessitating
adversary training.
Industrial Training at Shahjalal Fertilizer Company Limited (SFCL)MdTanvirMahtab2
This presentation is about the working procedure of Shahjalal Fertilizer Company Limited (SFCL). A Govt. owned Company of Bangladesh Chemical Industries Corporation under Ministry of Industries.
Student information management system project report ii.pdfKamal Acharya
Our project explains about the student management. This project mainly explains the various actions related to student details. This project shows some ease in adding, editing and deleting the student details. It also provides a less time consuming process for viewing, adding, editing and deleting the marks of the students.
Sachpazis:Terzaghi Bearing Capacity Estimation in simple terms with Calculati...Dr.Costas Sachpazis
Terzaghi's soil bearing capacity theory, developed by Karl Terzaghi, is a fundamental principle in geotechnical engineering used to determine the bearing capacity of shallow foundations. This theory provides a method to calculate the ultimate bearing capacity of soil, which is the maximum load per unit area that the soil can support without undergoing shear failure. The Calculation HTML Code included.
CW RADAR, FMCW RADAR, FMCW ALTIMETER, AND THEIR PARAMETERSveerababupersonal22
It consists of cw radar and fmcw radar ,range measurement,if amplifier and fmcw altimeterThe CW radar operates using continuous wave transmission, while the FMCW radar employs frequency-modulated continuous wave technology. Range measurement is a crucial aspect of radar systems, providing information about the distance to a target. The IF amplifier plays a key role in signal processing, amplifying intermediate frequency signals for further analysis. The FMCW altimeter utilizes frequency-modulated continuous wave technology to accurately measure altitude above a reference point.
Using recycled concrete aggregates (RCA) for pavements is crucial to achieving sustainability. Implementing RCA for new pavement can minimize carbon footprint, conserve natural resources, reduce harmful emissions, and lower life cycle costs. Compared to natural aggregate (NA), RCA pavement has fewer comprehensive studies and sustainability assessments.
NO1 Uk best vashikaran specialist in delhi vashikaran baba near me online vas...Amil Baba Dawood bangali
Contact with Dawood Bhai Just call on +92322-6382012 and we'll help you. We'll solve all your problems within 12 to 24 hours and with 101% guarantee and with astrology systematic. If you want to take any personal or professional advice then also you can call us on +92322-6382012 , ONLINE LOVE PROBLEM & Other all types of Daily Life Problem's.Then CALL or WHATSAPP us on +92322-6382012 and Get all these problems solutions here by Amil Baba DAWOOD BANGALI
#vashikaranspecialist #astrologer #palmistry #amliyaat #taweez #manpasandshadi #horoscope #spiritual #lovelife #lovespell #marriagespell#aamilbabainpakistan #amilbabainkarachi #powerfullblackmagicspell #kalajadumantarspecialist #realamilbaba #AmilbabainPakistan #astrologerincanada #astrologerindubai #lovespellsmaster #kalajaduspecialist #lovespellsthatwork #aamilbabainlahore#blackmagicformarriage #aamilbaba #kalajadu #kalailam #taweez #wazifaexpert #jadumantar #vashikaranspecialist #astrologer #palmistry #amliyaat #taweez #manpasandshadi #horoscope #spiritual #lovelife #lovespell #marriagespell#aamilbabainpakistan #amilbabainkarachi #powerfullblackmagicspell #kalajadumantarspecialist #realamilbaba #AmilbabainPakistan #astrologerincanada #astrologerindubai #lovespellsmaster #kalajaduspecialist #lovespellsthatwork #aamilbabainlahore #blackmagicforlove #blackmagicformarriage #aamilbaba #kalajadu #kalailam #taweez #wazifaexpert #jadumantar #vashikaranspecialist #astrologer #palmistry #amliyaat #taweez #manpasandshadi #horoscope #spiritual #lovelife #lovespell #marriagespell#aamilbabainpakistan #amilbabainkarachi #powerfullblackmagicspell #kalajadumantarspecialist #realamilbaba #AmilbabainPakistan #astrologerincanada #astrologerindubai #lovespellsmaster #kalajaduspecialist #lovespellsthatwork #aamilbabainlahore #Amilbabainuk #amilbabainspain #amilbabaindubai #Amilbabainnorway #amilbabainkrachi #amilbabainlahore #amilbabaingujranwalan #amilbabainislamabad
NO1 Uk best vashikaran specialist in delhi vashikaran baba near me online vas...
CONTRAST OF RESNET AND DENSENET BASED ON THE RECOGNITION OF SIMPLE FRUIT DATA SET
1. The International Journal of Computational Science, Information Technology and Control Engineering
(IJCSITCE) Vol.6, No.1, January 2019
DOI: 10.5121/ijcsitce.2019.6101 1
CONTRAST OF RESNET AND DENSENET BASED ON
THE RECOGNITION OF SIMPLE FRUIT DATA SET
Ding Tianye
Hangzhou Foreign Language School, Hangzhou, Zhejiang, China
ABSTRACT
In this paper, a fruit image data set is used to compare the efficiency and accuracy of two widely used
Convolutional Neural Network, namely the ResNet and the DenseNet, for the recognition of 50 different
kinds of fruits. In the experiment, the structure of ResNet-34 and DenseNet_BC-121 (with bottleneck layer)
are used. The mathematic principle, experiment detail and the experiment result will be explained through
comparison.
KEYWORDS
Deep learning, Object recognition, Computer vision, Image processing, Convolutional Neural Networks.
1. INTRODUCTION
The aim of this paper is to discover the learning efficiency and convergence rate in machine
learning of ResNet and DenseNet_BC through comparable experiments. The fruit image data set
[1] used in the experiment consists of images of 100*100 with black/white background and
Figure 1. one of the images in the image data set [1]
without noise interference. Including 50 kinds of different fruits, 25,100 images in total as
training data and 12,700 images as testing data. In the experiment, the only consideration is how
many times is needed to train the neural network so that it can have an accuracy of more than
98%. The first step in the training process is to pre-process the input images, using the Python
module OpenCV to turn the images into RGB channel images, and divide each channel value by
225 so that the resultant value is in the range of 0 to 1. Combining 50 images as an input batch,
determining the loss between each prediction and actual value through computing the cross
2. The International Journal of Computational Science, Information Technology and Control Engineering
(IJCSITCE) Vol.6, No.1, January 2019
2
entropy, , every time after 10 times of iterate input, compute the recognition
accuracy and output as a way of visualization, and train the two networks for 2,000 times before
input the testing data. Each convolutional layer uses ReLU(Rectified Linear Unit) as activation
function, using a layer of batch normalization between each code block and a layer of dropout
with keep probability of 0.5 to avoid the appearance of over fitting, the training of both neuron
networks uses Adam Optimizer and epsilon of 0.1, and the setting of learning rate uses the
method of Learning Rate Exponential
Decay ( ), the decay step is
the same as the total training time, setting stair case as True, the initial learning rate is 1e-3, using
the decay rate of 0.96. The hardware environment used in the experiment is running on GPU
GTX 1070 with allocated memory of 5.0Gb, the framework used is TensorFlow [2] developed by
Google in 2015.
2. DEEP LEARNING
The deep learning neuron networks are usually consisted by multiple layers, the input data of
each layer is the output of the previous layer, compare with shallow learning, deep learning is
usually been regarded a great from Weak AI towards Strong AI.
The Convolutional Neural Networks (CNN) can be categorized as supervised learning in deep
learning, which means that the training process of the neuron networks needs to provide not only
the input data, but also the actual data used to calculate the loss between prediction value and
actual value, and the optimizers will use different back propagation algorithms to contribute the
total loss onto each neuron and the activation functions inside each neuron will change the
parameters inside each neuron, so that after enough data input to the network and trained for
enough times, the neuron network will tend to find the local or global optimize and achieve a
good enough performance on the particular question that it is trained for.
CNN is most widely used in the field of image recognition, a paper [3] has proven the idea that
the convolutional neural networks have a better and those advanced image recognition networks
like ResNet and DenseNet are developed on the base of CNN. That is one reason why fruit data
set [1] is chosen as training data. The pooling layers in the CNN are used to reinforce and
compress the feature in each feature map to reduce the possibility of feature disappear.
3. TWO CONVOLUTIONAL NETWORKS
In the neuron networks that were used in the computer vision and image recognition before, with
the increase of the depth of the network, gradient disappear of explosion and OOM (Out of
Memory) lead to the reduction in the accuracy of the neuron network has turned into a difficulty
that many teams or individuals are trying to overcome.
3. The International Journal of Computational Science, Information Technology and Control Engineering
(IJCSITCE) Vol.6, No.1, January 2019
3
2.1. ResNet
Figure 2. The parameters of different structures of ResNet, the structure used was 34-layer
In 2015, the introduction of ResNet (Residual Convolutional Network), which has won the
champion of classification in the ImageNet competition, greatly narrow down the problem
through the method called residual transmission- using simple code block to transfer the input and
the output in the previous layer as residual to be the input data of the next layer, as shown in
Figure 3, which provides a channel for the input gradient in each layer only has change in
dimension but no need to process and get into the next layer of neuron network. The principle of
ResNet is simple code blocks, stacking and connecting by channels, not only simplify the
complexity of neuron networks, but also reduce the memory occupied by the session in the
running process, which greatly reduce the probability of OOM, improve the efficiency of machine
learning and the rate of gradient convergence, so that many programmers are keen on using
ResNet in supervised learning.
Figure 3. Visible structure of ResNet
The formula of ResNet in the Essay “Densely Connected Convolutional Networks” [4] is briefly
introduced as “ ” where represents the input data of current layer,
represents non-linear transformation, the input of current layer is residual formed by combining
the input and output of the previous layer and transmit to the next layer. Also, each code block in
the ResNet contains residual block used to convolution and the identity block used to directly
transmit gradient, the resultant output of each block is , and represents
the part of residual. Theoretically, with the increase in the depth of the neuron network, the output
of part of the residual blocks will gradually tend to be 0, but the channel through identity block is
still transmitting the gradient, so that the problem of gradient disappear won’t happened in the
network, and the network can stay in the optimal state.
4. The International Journal of Computational Science, Information Technology and Control Engineering
(IJCSITCE) Vol.6, No.1, January 2019
4
When the two adjacent blocks have a change in the convolutional kernel depth, there will be an
identity block with transformation operation to change the dimension of the input tensor, so the
dimension of input tensor and the output tensor will stay the same on the y-axis.
The convolution layers of the ResNet-34 in the experiment except from the very early stage and
the full-connected layers are all using a kernel size of 3*3, the output number of feature maps in
each stage is separately 64, 128, 256 and 512, ensuring that the computing speed and learning
efficiency won’t be influenced because of the output number of feature maps is too large.
Normalizing the value of RGB into the range of 0 to 1 can reduce the computation complexity
while training. And the labels of actual values are processed into one hot code.
During the training process, in the first 190 times of input data batch iteration, the output
accuracy of the neuron network doesn’t have a great increase, is about 0% to 6%. After 190 times
of iteration, it appears to be a rapid increase of the recognition accuracy and when the training
step reaches about 630 to 640, the accuracy first reached about 98%; in the further training, the
accuracy is about 94% to 100%. After 2000 times of training, the output testing accuracy is about
96% to 98%
.2.2. DenseNet
Figure 4. The parameters of different DenseNet structures, the structure used was DenseNet_BC-121
from Densely Connected Neuron Network [4]
In CVPR 2017, the oral paper of “Densely Connected Neuron Networks” [4] by Gao Huang,
Zhuang Liu, Kilian Q. Weinberger and Laurens van der Maaten, which firstly introduce a
completely new neuron network structure- DenseNet, which focuses on solving the problem of
ResNet (the gradient passing through residual might be impeded when the network gets deeper),
5. The International Journal of Computational Science, Information Technology and Control Engineering
(IJCSITCE) Vol.6, No.1, January 2019
5
Figure 5. the visible structure of DenseNet form Densely Connected Neuron Networks [4]
the paper describes the brief formula of DenseNet as “ ”, this
function was achieved in the experiment through stacking the output of each previous layer by the
3rd
axis as input of the current layer, adopt a growth rate of 32, which means the output of each
layer is 32, and drop rate of 0.5, means in the transition layer, the number of output feature maps
is reduced into half of the initial number. Compare to ResNet, theoretically, because the
DenseNet is narrower than ResNet, the DenseNet with same number of neuron layers will have
less parameter and the because of the bottleneck layer and small number of output feature maps
(every layer usually have a small convolutional kernel depth), which can save the memory used
while running the session, and because input of every neuron layers is directly access to the input
and output gradient of each layer, so the problem of gradient disappear won’t exist with the
increase in the depth of neuron network.
In the training process, the assumption is that the training of DenseNet is more speed-efficient
and takes fewer steps for converging to a higher accuracy compares with the performance of
ResNet. However, the output result is not as ideal as the expectation, the time taken for the
network to reach an output accuracy is much long than the training process of ResNet-34, which
might be cause by the depth of the neuron network, furthermore, when the training step reached
about 200, there isn’t an obvious improvement in the training accuracy, even after 2000 times of
training, there is only an increase in accuracy about 10% to 35%, then several tries for eliminating
the bottleneck layer to keep the most number of feature maps, whereas still the training result
isn’t as ideal as expectation.
2.3. Code Segments
In the experiment, the TensorFlow [2] framework was used, import tensorflow as tf, also import
the module slim, from tensorflow.contrib.slim as slim, to simplify the complex procedure of
defining a complete convolutional layer.
In both experiments, the tensor of weights and biases are randomly produced as below:
define function weights with parameter(shape)
init = tf.truncated_normal with parameter(shape, standard deviation=0.01) (tensor
with shape filled with random truncated normal with standard deviation 0.01)
return variable init as the output of the function
6. The International Journal of Computational Science, Information Technology and Control Engineering
(IJCSITCE) Vol.6, No.1, January 2019
6
define function biases with parameter(shape)
init = tf.constant with parameter(0.02, shape) (tensor with shape filled with 0.02)
return variable init as the output of the function
In the experiment of ResNet-34, the convolutional layers with the transformation in
each stage is defined as below:
define function conv with parameters (inputs, out_size, k_size)
x_short_cut = inputs
conv1 = 2-dimensional convolution (inputs, number of output feature
maps=out_size, kernel size=k_size, stride=2, padding='SAME', without activation
function)
conv1_output = ReLU activation function (batch normalize (conv1 on axis=3))
conv2 = 2-dimensinal convolution (conv1_output, number of output feature
maps=out_size, kernel size=k_size, stride=1, padding='SAME', without activation
function)
conv2_output = batch normalize (conv2 on axis=3)
input_conv = 2-dimensinal convolution (x_short_cut, number of output feature
maps =out_size, kernel size=k_size, stride=2, padding='SAME', without activation
function)
input_reshape = batch normalize (input_conv on axis=3)
output = ReLU activation function (input_reshape+conv2_output)
return output as the output of the function
The identity blocks which include the channel for passing the residual of input data
and the output result is defined as below:
define identity with parameters (inputs, out_size, k_size)
x_short_cut = inputs
conv1 = 2-dimensinal convolution (inputs, number of output feature
maps=out_size, kernel size=k_size, stride=1, padding='SAME', without activation
function)
conv1_output = ReLU activation function (batch normalize (conv1 on axis=3))
conv2 = 2-dimensinal convolution (conv1_output, number of output feature
maps=out_size, kernel size=k_size, stride=1, padding='SAME', without activation
function)
conv2_BN = batch normalize (conv2 on axis=3)
conv2_output = ReLU activation function (conv2_BN+x_short_cut)
return conv2_output as the output of the function
In the experiment of DenseNet_BC-121, the dense blocks with growth rate of 32
and a bottle neck layer is defined as below:
Define function dense with parameters (inputs, growth_rate=32,
internal_layer=True, keep_prob)
x_input = batch normalize (inputs, axis=3)
x_relu = ReLU activation function (x_input)
# bottleneck layer
conv1 = 2-dimensinal convolution (x_relu, number of output feature
maps=growth_rate*4, kernel size=1*1, stride=1, padding='SAME', without
activation function)
conv1_dropout = Dropout (conv1, keep percentage=keep_prob)
conv1_relu = ReLU activation function (batch normalize (conv1_dropout on
axis=3))
conv2 = 2-dimensinal convolution (conv1_relu, number of output feature
maps=growth_rate, kernel size=3*3, stride=1, padding='SAME', without activation
function)
conv2_dropout = Dropout (conv2, keep percentage=keep_prob)
7. The International Journal of Computational Science, Information Technology and Control Engineering
(IJCSITCE) Vol.6, No.1, January 2019
7
if internal_layer is True then
output = stack ([inputs, conv2_dropout] on axis=3)
else
output = conv2_dropout
return output as the output of the function
The transition block used to compress the dimension of input tensor is defined as below:
Define function transition with parameters (inputs, drop=0.5)
x_input = batch normalize (inputs on axis=3)
conv = 2-dimensinal convolution (x_input, number of output feature maps=the depth of tensor
x_input*drop, kernel size=1*1, stride=1, padding='SAME', without activation function)
h1 = average pooling (conv, pooling kernel size=2*2, strides=2, padding='SAME')
return h1 as the output of the function
2.4. Experiment details
The procedure of tuning parameters in the experiment was mainly focus on the learning rate of
the neural networks and the default decay rate in exponential decay was 0.96. The method of
tuning refers to A Practical Guide [5], range while tuning the learning rate is set between 1E-4
and 0.7, first start with 0.7 and observe the rate of convergence through the output accuracy.
Bigger learning rate was easier for the accuracy to rise to about 95%, whereas the prediction will
start to move back and forth the local optimum and hardly make further improvements. Then the
learning rate will be change to 0.3, but the problem still occurs. However, if the initial learning
rate was too small, it will take a long time to converge since the beginning of training process.
Ultimately, it turned out to be that 0.003 is the most efficient learning rate for ResNet after
limited times of experiments.
2.5 Limitations
On the one hand, the neural network structure used in the experiment procedure was programmed
by myself based on the understanding of the papers published by the developers of these two
neural networks, rather than using the existed code segments provided by the developers. On the
other hand, one factor that may cause the slow convergence rate of DenseNet compares to ResNet
may caused by the depth of the neural network which was 121, much deeper than 34 layers
ResNet and had more connections.
4. CONCLUSION
Through the comparison of the two experiments, it is obvious that ResNet-34 performs better than
the DenseNet_BC-121 on the training and adaption of the fruit data set [1], since the efficiency of
learning, the rate of fitting the data set is much faster and takes less time to return a resultant
accuracy. It can be concluded that ResNet performs much better than DenseNet on simple data
sets, since the ResNet avoids the disappearance of gradient through residual passing, so that the
current layer is able to get both the input gradient and tensor of the previous layer and the output
tensor and gradient. However, in the DenseNet, each layer needs to compute all of the input and
output tensor and gradient of all the previous layers, when it is applied on those simple data sets,
this might cause an increment in the computation complexity also it might blur the output
processed tensor and gradient of the previous layer, so it resulted in only a limited improvement
in its performance after 2000 times of training and might be the reasons why the using of
DenseNet structure now is not as much as the using of ResNet structure. According to one of the
developers of DenseNet [6], the structures of DenseNet are usually narrower but much deeper, so
that the training process of DenseNet is more parameter-efficiency but less memory and speed-
efficiency, which is mainly why the training process is slow and lack of increment in the
accuracy.
8. The International Journal of Computational Science, Information Technology and Control Engineering
(IJCSITCE) Vol.6, No.1, January 2019
8
ACKNOWLEDGEMENTS
It gives me great pleasure in acknowledging the support and help of Professor Juntao Ye. I would
like to thank my parents for giving me support when I met difficulties. And everyone who gives
me assistance when I met troubles.
REFERENCES
[1] Fruits-360, Version: 2018.07.01.0. https://www.kaggle.com/moltean/fruits. last visited on 12.07.2018.
[2] TensorFlow. https://www.tensorflow.org. last visited on 15.07.2018.
[3] M. Liang, X. Hu, Recurrent Convolutional Neural Network for Object Recognition, IEEE Conference
on Computer Vision and Pattern Recognition (CVPR) Boston, pp. 3367-3375, 2015.
[4] G. Huang, Z. Liu, K. Weinberger, L. Maaten, Densely Connected Convolutional Networks, 2017.
[5] keitakurita, Learning Rate Tuning in Deep Learning: A Practical Guide, 2018.
http://mlexplained.com/2018/01/29/learning-rate-tuning-in-deep-learning-a-practical-guide/. last
visited on 15.01.2019.
[6] Z. Liu, answering why DenseNet requires more memory in training, 2017.
https://www.reddit.com/r/MachineLearning/comments/67fds7/d_how_does_densenet_compare_to_re
snet_and/. last visited on 12.01.2019
AUTHORS
Ding Tianye, year 11 senior high student in Hangzhou Foreign Language School,
has a month of summer program experience in National Laboratory of Pattern
Recognition (NLPR), Institute of Automation, Chinese Academy of Sciences
(CASIA). Currently the president of Developer Association, studying machine
learning.