This document provides an overview of deep learning techniques and their applications. It discusses various deep learning algorithms like convolutional neural networks (CNNs), recurrent neural networks (RNNs), autoencoders, and sparse coding networks. It also reviews literature applying deep learning methods to areas such as computer vision, natural language processing, recommender systems, and more. Tables are provided comparing different deep learning algorithms and summarizing related works applying these techniques. The document discusses techniques for addressing overfitting in deep neural networks. In summary, this document surveys the state-of-the-art in deep learning methods and their applications across multiple domains.
This is a deep learning presentation based on Deep Neural Network. It reviews the deep learning concept, related works and specific application areas.It describes a use case scenario of deep learning and highlights the current trends and research issues of deep learning
A simplified and novel technique to retrieve color images from hand-drawn sk...IJECEIAES
With the increasing adoption of human-computer interaction, there is a growing trend of extracting the image through hand-drawn sketches by humans to find out correlated objects from the storage unit. A review of the existing system shows the dominant use of sophisticated and complex mechanisms where the focus is more on accuracy and less on system efficiency. Hence, this proposed system introduces a simplified extraction of the related image using an attribution clustering process and a cost-effective training scheme. The proposed method uses K-means clustering and bag-ofattributes to extract essential information from the sketch. The proposed system also introduces a unique indexing scheme that makes the retrieval process faster and results in retrieving the highest-ranked images. Implemented in MATLAB, the study outcome shows the proposed system offers better accuracy and processing time than the existing feature extraction technique.
Big data cloud-based recommendation system using NLP techniques with machine ...TELKOMNIKA JOURNAL
Recommendation systems (RS) are crucial for social networking sites. Without it, finding precise products is harder. However, existing systems lack adequate efficiency, especially with big data. This paper presents a prototype cloud-based recommendation system for processing big data. The proposed work is implemented by utilizing the matrix factorization method with three approaches. In the first approach, singular value decomposition (SVD) is used, which is an old and traditional recommendation technique. The second recommendation approach is fine-tuned using the alternating least squares (ALS) algorithm with Apache Spark. Finally, the deep neural network (DNN) algorithm is utilized with TensorFlow. This study solves the challenge of handling large-scale datasets in the collaborative filtering (CF) technique after tuning the algorithms by adjusting the parameters in the second approach, which uses machine learning, as well as in the third approach, which uses deep learning. Furthermore, the results of these two approaches outperformed conventional techniques and achieved an acceptable computational time. The dataset size is about 1.5 GB and it is collected from the Goodreads website API. Moreover, the Hadoop distributed file system (HDFS) is used as cloud storage instead of the computer’s local disk for handling larger dataset sizes in the future.
Classifier Model using Artificial Neural NetworkAI Publications
When it comes to AI and ML, precision in categorization is of the utmost importance. In this research, the use of supervised instance selection (SIS) to improve the performance of artificial neural networks (ANNs) in classification is investigated. The goal of SIS is to enhance the accuracy of future classification tasks by identifying and selecting a subset of examples from the original dataset. The purpose of this research is to provide light on how useful SIS is as a preprocessing tool for artificial neural network-based classification. The work aims to improve the input dataset to ANNs by using SIS, which may help with problems caused by noisy or redundant data. The ultimate goal is to improve ANNs' ability to identify data points properly across a wide range of application areas.
Sensing complicated meanings from unstructured data: a novel hybrid approachIJECEIAES
The majority of data on computers nowadays is in the form of unstructured data and unstructured text. The inherent ambiguity of natural language makes it incredibly difficult but also highly profitable to find hidden information or comprehend complex semantics in unstructured text. In this paper, we present the combination of natural language processing (NLP) and convolution neural network (CNN) hybrid architecture called automated analysis of unstructured text using machine learning (AAUT-ML) for the detection of complex semantics from unstructured data that enables different users to make understand formal semantic knowledge to be extracted from an unstructured text corpus. The AAUT-ML has been evaluated using three datasets data mining (DM), operating system (OS), and data base (DB), and compared with the existing models, i.e., YAKE, term frequency-inverse document frequency (TF-IDF) and text-R. The results show better outcomes in terms of precision, recall, and macro-averaged F1-score. This work presents a novel method for identifying complex semantics using unstructured data.
This is a deep learning presentation based on Deep Neural Network. It reviews the deep learning concept, related works and specific application areas.It describes a use case scenario of deep learning and highlights the current trends and research issues of deep learning
A simplified and novel technique to retrieve color images from hand-drawn sk...IJECEIAES
With the increasing adoption of human-computer interaction, there is a growing trend of extracting the image through hand-drawn sketches by humans to find out correlated objects from the storage unit. A review of the existing system shows the dominant use of sophisticated and complex mechanisms where the focus is more on accuracy and less on system efficiency. Hence, this proposed system introduces a simplified extraction of the related image using an attribution clustering process and a cost-effective training scheme. The proposed method uses K-means clustering and bag-ofattributes to extract essential information from the sketch. The proposed system also introduces a unique indexing scheme that makes the retrieval process faster and results in retrieving the highest-ranked images. Implemented in MATLAB, the study outcome shows the proposed system offers better accuracy and processing time than the existing feature extraction technique.
Big data cloud-based recommendation system using NLP techniques with machine ...TELKOMNIKA JOURNAL
Recommendation systems (RS) are crucial for social networking sites. Without it, finding precise products is harder. However, existing systems lack adequate efficiency, especially with big data. This paper presents a prototype cloud-based recommendation system for processing big data. The proposed work is implemented by utilizing the matrix factorization method with three approaches. In the first approach, singular value decomposition (SVD) is used, which is an old and traditional recommendation technique. The second recommendation approach is fine-tuned using the alternating least squares (ALS) algorithm with Apache Spark. Finally, the deep neural network (DNN) algorithm is utilized with TensorFlow. This study solves the challenge of handling large-scale datasets in the collaborative filtering (CF) technique after tuning the algorithms by adjusting the parameters in the second approach, which uses machine learning, as well as in the third approach, which uses deep learning. Furthermore, the results of these two approaches outperformed conventional techniques and achieved an acceptable computational time. The dataset size is about 1.5 GB and it is collected from the Goodreads website API. Moreover, the Hadoop distributed file system (HDFS) is used as cloud storage instead of the computer’s local disk for handling larger dataset sizes in the future.
Classifier Model using Artificial Neural NetworkAI Publications
When it comes to AI and ML, precision in categorization is of the utmost importance. In this research, the use of supervised instance selection (SIS) to improve the performance of artificial neural networks (ANNs) in classification is investigated. The goal of SIS is to enhance the accuracy of future classification tasks by identifying and selecting a subset of examples from the original dataset. The purpose of this research is to provide light on how useful SIS is as a preprocessing tool for artificial neural network-based classification. The work aims to improve the input dataset to ANNs by using SIS, which may help with problems caused by noisy or redundant data. The ultimate goal is to improve ANNs' ability to identify data points properly across a wide range of application areas.
Sensing complicated meanings from unstructured data: a novel hybrid approachIJECEIAES
The majority of data on computers nowadays is in the form of unstructured data and unstructured text. The inherent ambiguity of natural language makes it incredibly difficult but also highly profitable to find hidden information or comprehend complex semantics in unstructured text. In this paper, we present the combination of natural language processing (NLP) and convolution neural network (CNN) hybrid architecture called automated analysis of unstructured text using machine learning (AAUT-ML) for the detection of complex semantics from unstructured data that enables different users to make understand formal semantic knowledge to be extracted from an unstructured text corpus. The AAUT-ML has been evaluated using three datasets data mining (DM), operating system (OS), and data base (DB), and compared with the existing models, i.e., YAKE, term frequency-inverse document frequency (TF-IDF) and text-R. The results show better outcomes in terms of precision, recall, and macro-averaged F1-score. This work presents a novel method for identifying complex semantics using unstructured data.
Image segmentation and classification tasks in computer vision have proven to be highly effective using neural networks, specifically Convolutional Neural Networks (CNNs). These tasks have numerous
practical applications, such as in medical imaging, autonomous driving, and surveillance. CNNs are capable
of learning complex features directly from images and achieving outstanding performance across several
datasets. In this work, we have utilized three different datasets to investigate the efficacy of various preprocessing and classification techniques in accurssedately segmenting and classifying different structures
within the MRI and natural images. We have utilized both sample gradient and Canny Edge Detection
methods for pre-processing, and K-means clustering have been applied to segment the images. Image
augmentation improves the size and diversity of datasets for training the models for image classification
Image segmentation and classification tasks in computer vision have proven to be highly effective using neural networks, specifically Convolutional Neural Networks (CNNs). These tasks have numerous
practical applications, such as in medical imaging, autonomous driving, and surveillance. CNNs are capable
of learning complex features directly from images and achieving outstanding performance across several
datasets. In this work, we have utilized three different datasets to investigate the efficacy of various preprocessing and classification techniques in accurssedately segmenting and classifying different structures
within the MRI and natural images. We have utilized both sample gradient and Canny Edge Detection
methods for pre-processing, and K-means clustering have been applied to segment the images. Image
augmentation improves the size and diversity of datasets for training the models for image classification.
This work highlights transfer learning’s effectiveness in image classification using CNNs and VGG 16 that
provides insights into the selection of pre-trained models and hyper parameters for optimal performance.
We have proposed a comprehensive approach for image segmentation and classification, incorporating preprocessing techniques, the K-means algorithm for segmentation, and employing deep learning models such
as CNN and VGG 16 for classification.
Noise-robust classification with hypergraph neural networknooriasukmaningtyas
This paper presents a novel version of hypergraph neural network method. This method is utilized to solve the noisy label learning problem. First, we apply the PCA dimensional reduction technique to the feature matrices of the image datasets in order to reduce the “noise” and the redundant features in the feature matrices of the image datasets and to reduce the runtime constructing the hypergraph of the hypergraph neural network method. Then, the classic graph based semisupervised learning method, the classic hypergraph based semi-supervised learning method, the graph neural network, the hypergraph neural network, and our proposed hypergraph neural network are employed to solve the noisy label learning problem. The accuracies of these five methods are evaluated and compared. Experimental results show that the hypergraph neural network methods achieve the best performance when the noise level increases. Moreover, the hypergraph neural network methods are at least as good as the graph neural network.
Multi-scale 3D-convolutional neural network for hyperspectral image classific...nooriasukmaningtyas
Deep Learning methods are state-of-the-art approaches for pixel-based hyperspectral images (HSI) classification. High classification accuracy has been achieved by extracting deep features from both spatial-spectral channels. However, the efficiency of such spatial-spectral approaches depends on the spatial dimension of each patch and there is no theoretically valid approach to find the optimum spatial dimension to be considered. It is more valid to extract spatial features by considering varying neighborhood scales in spatial dimensions. In this regard, this article proposes a deep convolutional neural network (CNN) model wherein three different multi-scale spatial-spectral patches are used to extract the features in both the spatial and spectral channels. In order to extract these potential features, the proposed deep learning architecture takes three patches various scales in spatial dimension. 3D convolution is performed on each selected patch and the process runs through entire image. The proposed is named as multi-scale three-dimensional convolutional neural network (MS-3DCNN). The efficiency of the proposed model is being verified through the experimental studies on three publicly available benchmark datasets including Pavia University, Indian Pines, and Salinas. It is empirically proved that the classification accuracy of the proposed model is improved when compared with the remaining state-of-the-art methods.
MULTI-LEVEL FEATURE FUSION BASED TRANSFER LEARNING FOR PERSON RE-IDENTIFICATIONijaia
Most of the currently known methods treat person re-identification task as classification problem and used commonly neural networks. However, these methods used only high-level convolutional feature or to express the feature representation of pedestrians. Moreover, the current data sets for person reidentification is relatively small. Under the limitation of the number of training set, deep convolutional networks are difficult to train adequately. Therefore, it is very worthwhile to introduce auxiliary data sets to help training. In order to solve this problem, this paper propose a novel method of deep transfer learning, and combines the comparison model with the classification model and multi-level fusion of the convolution features on the basis of transfer learning. In a multi-layers convolutional network, the characteristics of each layer of network are the dimensionality reduction of the previous layer of results, but the information of multi-level features is not only inclusive, but also has certain complementarity. We can using the information gap of different layers of convolutional neural networks to extract a better feature expression. Finally, the algorithm proposed in this paper is fully tested on four data sets (VIPeR, CUHK01, GRID and PRID450S). The obtained re-identification results prove the effectiveness of the algorithm.
Eat it, Review it: A New Approach for Review Predictionvivatechijri
Deep Learning has achieved significant improvement in various machine learning tasks. Nowadays,
Recurrent Neural Network (RNN) and Long Short Term Memory (LSTM) have been increasing its popularity on
Text Sequence i.e. word prediction. The ability to abstract information from image or text is being widely
adopted by organizations around the world. A basic task in deep learning is classification be it image or text.
Current trending techniques such as RNN, CNN has proven that such techniques open the door for data analysis.
Emerging technologies such has Region CNN, Recurrent CNN have been under consideration for the analysis.
Recurrent CNN is being under development with the current world. The proposed system uses Recurrent Neural
Network for review prediction. Also LSTM is used along with RNN so as to predict long sentences. This system
focuses on context based review prediction and will provide full length sentence. This will help to write a proper
reviews by understanding the context of user.
Text classification based on gated recurrent unit combines with support vecto...IJECEIAES
As the amount of unstructured text data that humanity produce largely and a lot of texts are grows on the Internet, so the one of the intelligent technique is require processing it and extracting different types of knowledge from it. Gated recurrent unit (GRU) and support vector machine (SVM) have been successfully used to Natural Language Processing (NLP) systems with comparative, remarkable results. GRU networks perform well in sequential learning tasks and overcome the issues of “vanishing and explosion of gradients in standard recurrent neural networks (RNNs) when captureing long-term dependencies. In this paper, we proposed a text classification model based on improved approaches to this norm by presenting a linear support vector machine (SVM) as the replacement of Softmax in the final output layer of a GRU model. Furthermore, the cross-entropy function shall be replaced with a margin-based function. Empirical results present that the proposed GRU-SVM model achieved comparatively better results than the baseline approaches BLSTM-C, DABN.
Comparative Analysis of RMSE and MAP Metrices for Evaluating CNN and LSTM Mod...GagandeepKaur872517
Convolutional Neural Networks (CNNs) and Long Short-Term Memory Networks (LSTMs) have made substantial advances in the domains of computer vision and speech recognition in recent years. These deep learning architectures have shown exceptional ability in a variety of tasks. Evaluating the performance of such models is critical for comprehending their efficacy and directing future developments. In this study, we undertake a thorough comparison utilizing two essential evaluation metrics: Root Mean Square Error (RMSE) and Mean Average Precision (MAP). Our research intends to give light on the applicability of these metrics for analyzing the performance of CNNs and LSTMs, as well as their strengths and limitations.
Handwriting identification using deep convolutional neural network methodTELKOMNIKA JOURNAL
Handwriting is a unique thing that produced differently for each person. Handwriting has a characteristic that remain the same with single writer, so a handwriting can be used as a variable in biometric systems. Each person have a different form of handwriting style but with a small possibility that same characters have something commons. We propose a handwriting identification method using sentence segmented handwriting forms. Sentence form is used to get more complete handwriting characteristics than using a single characters or words. Dataset used is divided into three categories of images, binary, grayscale, and inverted binary. All datasets have same image with different in color and consist of 100 class. Transfer learning used in this paper are pre-trained model VGG19. Training was conducted in 100 epochs. Highest result is grayscale images with genuince acceptance rate of 92.3% and equal error rate of 7.7%.
The advents in this technological era have resulted into enormous pool of information. This information is
stored at multiple places globally, in multiple formats. This article highlights a methodology for extracting
the video lectures delivered by experts in the domain of Computer Science by using Generalized Gamma
Mixture Model. The feature extraction is based on the DCT transformations. In order to propose the model,
the data set is pooled from the YouTube video lectures in the domain of Computer Science. The outputs
generated are evaluated using Precision and Recall.
Investigating the Effect of BD-CRAFT to Text Detection Algorithmsgerogepatton
With the rise and development of deep learning, computer vision and document analysis has influenced the
area of text detection. Despite significant efforts in improving text detection performance, it remains to be
challenging, as evident by the series of the Robust Reading Competitions. This study investigates the impact
of employing BD-CRAFT – a variant of CRAFT that involves automatic image classification utilizing a
Laplacian operator and further preprocess the classified blurry images using blind deconvolution to the
top-ranked algorithms, SenseTime and TextFuseNet. Results revealed that the proposed method
significantly enhanced the detection performances of the said algorithms. TextFuseNet + BD-CRAFT
achieved an outstanding h-mean result of 93.55% and shows an impressive improvement of over 4%
increase to its precision yielding 95.71% while SenseTime + BD-CRAFT placed first with a very
remarkable 95.22% h-mean and exhibited a huge precision improvement of over 4%.
INVESTIGATING THE EFFECT OF BD-CRAFT TO TEXT DETECTION ALGORITHMSijaia
With the rise and development of deep learning, computer vision and document analysis has influenced the
area of text detection. Despite significant efforts in improving text detection performance, it remains to be
challenging, as evident by the series of the Robust Reading Competitions. This study investigates the impact
of employing BD-CRAFT – a variant of CRAFT that involves automatic image classification utilizing a
Laplacian operator and further preprocess the classified blurry images using blind deconvolution to the
top-ranked algorithms, SenseTime and TextFuseNet. Results revealed that the proposed method
significantly enhanced the detection performances of the said algorithms. TextFuseNet + BD-CRAFT
achieved an outstanding h-mean result of 93.55% and shows an impressive improvement of over 4%
increase to its precision yielding 95.71% while SenseTime + BD-CRAFT placed first with a very
remarkable 95.22% h-mean and exhibited a huge precision improvement of over 4%.
End-to-end deep auto-encoder for segmenting a moving object with limited tra...IJECEIAES
Deep learning-based approaches have been widely used in various applications, including segmentation and classification. However, a large amount of data is required to train such techniques. Indeed, in the surveillance video domain, there are few accessible data due to acquisition and experiment complexity. In this paper, we propose an end-to-end deep auto-encoder system for object segmenting from surveillance videos. Our main purpose is to enhance the process of distinguishing the foreground object when only limited data are available. To this end, we propose two approaches based on transfer learning and multi-depth auto-encoders to avoid over-fitting by combining classical data augmentation and principal component analysis (PCA) techniques to improve the quality of training data. Our approach achieves good results outperforming other popular models, which used the same principle of training with limited data. In addition, a detailed explanation of these techniques and some recommendations are provided. Our methodology constitutes a useful strategy for increasing samples in the deep learning domain and can be applied to improve segmentation accuracy. We believe that our strategy has a considerable interest in various applications such as medical and biological fields, especially in the early stages of experiments where there are few samples.
Off-line English Character Recognition: A Comparative Surveyidescitation
It has been decades since the evolution of idea that
human brain can be mimicked by artificial neuron like
mathematical structures. Till date, the development of this
endeavor has not reached the threshold of excellence. Neural
networks are commonly used to solve sample-recognition
problems. One of these is character recognition. The solution
of this problem is one of the easier implementations of neural
networks. This paper presents a detailed comparative
literature survey on the research accomplished for the last
few decades. The comparative literature review will help us
understand the platform on which we stand today to achieve
the highest efficiency in terms of Character Recognition
accuracy as well as computational resource and cost.
Stacked Generalization of Random Forest and Decision Tree Techniques for Libr...IJEACS
The huge amount of library data stored in our modern research and statistic centers of organizations is springing up on daily bases. These databases grow exponentially in size with respect to time, it becomes exceptionally difficult to easily understand the behavior and interpret data with the relationships that exist between attributes. This exponential growth of data poses new organizational challenges like the conventional record management system infrastructure could no longer cope to give precise and detailed information about the behavior data over time. There is confusion and novel concern in selecting tools that can support and handle big data visualization that deals with multi-dimension. Viewing all related data at once in a database is a problem that has attracted the interest of data professionals with machine learning skills. This is a lingering issue in the data industry because the existing techniques cannot be used to remove or filter noise from relevant data and pad up missing values in order to get the required information. The aim is to develop a stacked generalization model that combines the functionality of random forest and decision tree to visualization library database visualization. In this paper, the random forest and decision tree techniques were employed to effectively visualize large amounts of school library data. The proposed system was implemented with a few lines of Python code to create visualizations that can help users at a glance understand and interpret the behavior of data and its relationships. The model was trained and tested to learn and extract hidden patterns of data with a cross-validation test. It combined the functionalities of both models to form a stacked generalization model that performed better than the individual techniques. The stacked model produced 95% followed by the RF which produced a 95% accuracy rate and 0.223600 RMSE error value in comparison with the DT which recorded an 80.00% success rate and 0.15990 RMSE value.
Image segmentation and classification tasks in computer vision have proven to be highly effective using neural networks, specifically Convolutional Neural Networks (CNNs). These tasks have numerous
practical applications, such as in medical imaging, autonomous driving, and surveillance. CNNs are capable
of learning complex features directly from images and achieving outstanding performance across several
datasets. In this work, we have utilized three different datasets to investigate the efficacy of various preprocessing and classification techniques in accurssedately segmenting and classifying different structures
within the MRI and natural images. We have utilized both sample gradient and Canny Edge Detection
methods for pre-processing, and K-means clustering have been applied to segment the images. Image
augmentation improves the size and diversity of datasets for training the models for image classification
Image segmentation and classification tasks in computer vision have proven to be highly effective using neural networks, specifically Convolutional Neural Networks (CNNs). These tasks have numerous
practical applications, such as in medical imaging, autonomous driving, and surveillance. CNNs are capable
of learning complex features directly from images and achieving outstanding performance across several
datasets. In this work, we have utilized three different datasets to investigate the efficacy of various preprocessing and classification techniques in accurssedately segmenting and classifying different structures
within the MRI and natural images. We have utilized both sample gradient and Canny Edge Detection
methods for pre-processing, and K-means clustering have been applied to segment the images. Image
augmentation improves the size and diversity of datasets for training the models for image classification.
This work highlights transfer learning’s effectiveness in image classification using CNNs and VGG 16 that
provides insights into the selection of pre-trained models and hyper parameters for optimal performance.
We have proposed a comprehensive approach for image segmentation and classification, incorporating preprocessing techniques, the K-means algorithm for segmentation, and employing deep learning models such
as CNN and VGG 16 for classification.
Noise-robust classification with hypergraph neural networknooriasukmaningtyas
This paper presents a novel version of hypergraph neural network method. This method is utilized to solve the noisy label learning problem. First, we apply the PCA dimensional reduction technique to the feature matrices of the image datasets in order to reduce the “noise” and the redundant features in the feature matrices of the image datasets and to reduce the runtime constructing the hypergraph of the hypergraph neural network method. Then, the classic graph based semisupervised learning method, the classic hypergraph based semi-supervised learning method, the graph neural network, the hypergraph neural network, and our proposed hypergraph neural network are employed to solve the noisy label learning problem. The accuracies of these five methods are evaluated and compared. Experimental results show that the hypergraph neural network methods achieve the best performance when the noise level increases. Moreover, the hypergraph neural network methods are at least as good as the graph neural network.
Multi-scale 3D-convolutional neural network for hyperspectral image classific...nooriasukmaningtyas
Deep Learning methods are state-of-the-art approaches for pixel-based hyperspectral images (HSI) classification. High classification accuracy has been achieved by extracting deep features from both spatial-spectral channels. However, the efficiency of such spatial-spectral approaches depends on the spatial dimension of each patch and there is no theoretically valid approach to find the optimum spatial dimension to be considered. It is more valid to extract spatial features by considering varying neighborhood scales in spatial dimensions. In this regard, this article proposes a deep convolutional neural network (CNN) model wherein three different multi-scale spatial-spectral patches are used to extract the features in both the spatial and spectral channels. In order to extract these potential features, the proposed deep learning architecture takes three patches various scales in spatial dimension. 3D convolution is performed on each selected patch and the process runs through entire image. The proposed is named as multi-scale three-dimensional convolutional neural network (MS-3DCNN). The efficiency of the proposed model is being verified through the experimental studies on three publicly available benchmark datasets including Pavia University, Indian Pines, and Salinas. It is empirically proved that the classification accuracy of the proposed model is improved when compared with the remaining state-of-the-art methods.
MULTI-LEVEL FEATURE FUSION BASED TRANSFER LEARNING FOR PERSON RE-IDENTIFICATIONijaia
Most of the currently known methods treat person re-identification task as classification problem and used commonly neural networks. However, these methods used only high-level convolutional feature or to express the feature representation of pedestrians. Moreover, the current data sets for person reidentification is relatively small. Under the limitation of the number of training set, deep convolutional networks are difficult to train adequately. Therefore, it is very worthwhile to introduce auxiliary data sets to help training. In order to solve this problem, this paper propose a novel method of deep transfer learning, and combines the comparison model with the classification model and multi-level fusion of the convolution features on the basis of transfer learning. In a multi-layers convolutional network, the characteristics of each layer of network are the dimensionality reduction of the previous layer of results, but the information of multi-level features is not only inclusive, but also has certain complementarity. We can using the information gap of different layers of convolutional neural networks to extract a better feature expression. Finally, the algorithm proposed in this paper is fully tested on four data sets (VIPeR, CUHK01, GRID and PRID450S). The obtained re-identification results prove the effectiveness of the algorithm.
Eat it, Review it: A New Approach for Review Predictionvivatechijri
Deep Learning has achieved significant improvement in various machine learning tasks. Nowadays,
Recurrent Neural Network (RNN) and Long Short Term Memory (LSTM) have been increasing its popularity on
Text Sequence i.e. word prediction. The ability to abstract information from image or text is being widely
adopted by organizations around the world. A basic task in deep learning is classification be it image or text.
Current trending techniques such as RNN, CNN has proven that such techniques open the door for data analysis.
Emerging technologies such has Region CNN, Recurrent CNN have been under consideration for the analysis.
Recurrent CNN is being under development with the current world. The proposed system uses Recurrent Neural
Network for review prediction. Also LSTM is used along with RNN so as to predict long sentences. This system
focuses on context based review prediction and will provide full length sentence. This will help to write a proper
reviews by understanding the context of user.
Text classification based on gated recurrent unit combines with support vecto...IJECEIAES
As the amount of unstructured text data that humanity produce largely and a lot of texts are grows on the Internet, so the one of the intelligent technique is require processing it and extracting different types of knowledge from it. Gated recurrent unit (GRU) and support vector machine (SVM) have been successfully used to Natural Language Processing (NLP) systems with comparative, remarkable results. GRU networks perform well in sequential learning tasks and overcome the issues of “vanishing and explosion of gradients in standard recurrent neural networks (RNNs) when captureing long-term dependencies. In this paper, we proposed a text classification model based on improved approaches to this norm by presenting a linear support vector machine (SVM) as the replacement of Softmax in the final output layer of a GRU model. Furthermore, the cross-entropy function shall be replaced with a margin-based function. Empirical results present that the proposed GRU-SVM model achieved comparatively better results than the baseline approaches BLSTM-C, DABN.
Comparative Analysis of RMSE and MAP Metrices for Evaluating CNN and LSTM Mod...GagandeepKaur872517
Convolutional Neural Networks (CNNs) and Long Short-Term Memory Networks (LSTMs) have made substantial advances in the domains of computer vision and speech recognition in recent years. These deep learning architectures have shown exceptional ability in a variety of tasks. Evaluating the performance of such models is critical for comprehending their efficacy and directing future developments. In this study, we undertake a thorough comparison utilizing two essential evaluation metrics: Root Mean Square Error (RMSE) and Mean Average Precision (MAP). Our research intends to give light on the applicability of these metrics for analyzing the performance of CNNs and LSTMs, as well as their strengths and limitations.
Handwriting identification using deep convolutional neural network methodTELKOMNIKA JOURNAL
Handwriting is a unique thing that produced differently for each person. Handwriting has a characteristic that remain the same with single writer, so a handwriting can be used as a variable in biometric systems. Each person have a different form of handwriting style but with a small possibility that same characters have something commons. We propose a handwriting identification method using sentence segmented handwriting forms. Sentence form is used to get more complete handwriting characteristics than using a single characters or words. Dataset used is divided into three categories of images, binary, grayscale, and inverted binary. All datasets have same image with different in color and consist of 100 class. Transfer learning used in this paper are pre-trained model VGG19. Training was conducted in 100 epochs. Highest result is grayscale images with genuince acceptance rate of 92.3% and equal error rate of 7.7%.
The advents in this technological era have resulted into enormous pool of information. This information is
stored at multiple places globally, in multiple formats. This article highlights a methodology for extracting
the video lectures delivered by experts in the domain of Computer Science by using Generalized Gamma
Mixture Model. The feature extraction is based on the DCT transformations. In order to propose the model,
the data set is pooled from the YouTube video lectures in the domain of Computer Science. The outputs
generated are evaluated using Precision and Recall.
Investigating the Effect of BD-CRAFT to Text Detection Algorithmsgerogepatton
With the rise and development of deep learning, computer vision and document analysis has influenced the
area of text detection. Despite significant efforts in improving text detection performance, it remains to be
challenging, as evident by the series of the Robust Reading Competitions. This study investigates the impact
of employing BD-CRAFT – a variant of CRAFT that involves automatic image classification utilizing a
Laplacian operator and further preprocess the classified blurry images using blind deconvolution to the
top-ranked algorithms, SenseTime and TextFuseNet. Results revealed that the proposed method
significantly enhanced the detection performances of the said algorithms. TextFuseNet + BD-CRAFT
achieved an outstanding h-mean result of 93.55% and shows an impressive improvement of over 4%
increase to its precision yielding 95.71% while SenseTime + BD-CRAFT placed first with a very
remarkable 95.22% h-mean and exhibited a huge precision improvement of over 4%.
INVESTIGATING THE EFFECT OF BD-CRAFT TO TEXT DETECTION ALGORITHMSijaia
With the rise and development of deep learning, computer vision and document analysis has influenced the
area of text detection. Despite significant efforts in improving text detection performance, it remains to be
challenging, as evident by the series of the Robust Reading Competitions. This study investigates the impact
of employing BD-CRAFT – a variant of CRAFT that involves automatic image classification utilizing a
Laplacian operator and further preprocess the classified blurry images using blind deconvolution to the
top-ranked algorithms, SenseTime and TextFuseNet. Results revealed that the proposed method
significantly enhanced the detection performances of the said algorithms. TextFuseNet + BD-CRAFT
achieved an outstanding h-mean result of 93.55% and shows an impressive improvement of over 4%
increase to its precision yielding 95.71% while SenseTime + BD-CRAFT placed first with a very
remarkable 95.22% h-mean and exhibited a huge precision improvement of over 4%.
End-to-end deep auto-encoder for segmenting a moving object with limited tra...IJECEIAES
Deep learning-based approaches have been widely used in various applications, including segmentation and classification. However, a large amount of data is required to train such techniques. Indeed, in the surveillance video domain, there are few accessible data due to acquisition and experiment complexity. In this paper, we propose an end-to-end deep auto-encoder system for object segmenting from surveillance videos. Our main purpose is to enhance the process of distinguishing the foreground object when only limited data are available. To this end, we propose two approaches based on transfer learning and multi-depth auto-encoders to avoid over-fitting by combining classical data augmentation and principal component analysis (PCA) techniques to improve the quality of training data. Our approach achieves good results outperforming other popular models, which used the same principle of training with limited data. In addition, a detailed explanation of these techniques and some recommendations are provided. Our methodology constitutes a useful strategy for increasing samples in the deep learning domain and can be applied to improve segmentation accuracy. We believe that our strategy has a considerable interest in various applications such as medical and biological fields, especially in the early stages of experiments where there are few samples.
Off-line English Character Recognition: A Comparative Surveyidescitation
It has been decades since the evolution of idea that
human brain can be mimicked by artificial neuron like
mathematical structures. Till date, the development of this
endeavor has not reached the threshold of excellence. Neural
networks are commonly used to solve sample-recognition
problems. One of these is character recognition. The solution
of this problem is one of the easier implementations of neural
networks. This paper presents a detailed comparative
literature survey on the research accomplished for the last
few decades. The comparative literature review will help us
understand the platform on which we stand today to achieve
the highest efficiency in terms of Character Recognition
accuracy as well as computational resource and cost.
Stacked Generalization of Random Forest and Decision Tree Techniques for Libr...IJEACS
The huge amount of library data stored in our modern research and statistic centers of organizations is springing up on daily bases. These databases grow exponentially in size with respect to time, it becomes exceptionally difficult to easily understand the behavior and interpret data with the relationships that exist between attributes. This exponential growth of data poses new organizational challenges like the conventional record management system infrastructure could no longer cope to give precise and detailed information about the behavior data over time. There is confusion and novel concern in selecting tools that can support and handle big data visualization that deals with multi-dimension. Viewing all related data at once in a database is a problem that has attracted the interest of data professionals with machine learning skills. This is a lingering issue in the data industry because the existing techniques cannot be used to remove or filter noise from relevant data and pad up missing values in order to get the required information. The aim is to develop a stacked generalization model that combines the functionality of random forest and decision tree to visualization library database visualization. In this paper, the random forest and decision tree techniques were employed to effectively visualize large amounts of school library data. The proposed system was implemented with a few lines of Python code to create visualizations that can help users at a glance understand and interpret the behavior of data and its relationships. The model was trained and tested to learn and extract hidden patterns of data with a cross-validation test. It combined the functionalities of both models to form a stacked generalization model that performed better than the individual techniques. The stacked model produced 95% followed by the RF which produced a 95% accuracy rate and 0.223600 RMSE error value in comparison with the DT which recorded an 80.00% success rate and 0.15990 RMSE value.
The Internet of Things (IoT) is a revolutionary concept that connects everyday objects and devices to the internet, enabling them to communicate, collect, and exchange data. Imagine a world where your refrigerator notifies you when you’re running low on groceries, or streetlights adjust their brightness based on traffic patterns – that’s the power of IoT. In essence, IoT transforms ordinary objects into smart, interconnected devices, creating a network of endless possibilities.
Here is a blog on the role of electrical and electronics engineers in IOT. Let's dig in!!!!
For more such content visit: https://nttftrg.com/
NO1 Uk best vashikaran specialist in delhi vashikaran baba near me online vas...Amil Baba Dawood bangali
Contact with Dawood Bhai Just call on +92322-6382012 and we'll help you. We'll solve all your problems within 12 to 24 hours and with 101% guarantee and with astrology systematic. If you want to take any personal or professional advice then also you can call us on +92322-6382012 , ONLINE LOVE PROBLEM & Other all types of Daily Life Problem's.Then CALL or WHATSAPP us on +92322-6382012 and Get all these problems solutions here by Amil Baba DAWOOD BANGALI
#vashikaranspecialist #astrologer #palmistry #amliyaat #taweez #manpasandshadi #horoscope #spiritual #lovelife #lovespell #marriagespell#aamilbabainpakistan #amilbabainkarachi #powerfullblackmagicspell #kalajadumantarspecialist #realamilbaba #AmilbabainPakistan #astrologerincanada #astrologerindubai #lovespellsmaster #kalajaduspecialist #lovespellsthatwork #aamilbabainlahore#blackmagicformarriage #aamilbaba #kalajadu #kalailam #taweez #wazifaexpert #jadumantar #vashikaranspecialist #astrologer #palmistry #amliyaat #taweez #manpasandshadi #horoscope #spiritual #lovelife #lovespell #marriagespell#aamilbabainpakistan #amilbabainkarachi #powerfullblackmagicspell #kalajadumantarspecialist #realamilbaba #AmilbabainPakistan #astrologerincanada #astrologerindubai #lovespellsmaster #kalajaduspecialist #lovespellsthatwork #aamilbabainlahore #blackmagicforlove #blackmagicformarriage #aamilbaba #kalajadu #kalailam #taweez #wazifaexpert #jadumantar #vashikaranspecialist #astrologer #palmistry #amliyaat #taweez #manpasandshadi #horoscope #spiritual #lovelife #lovespell #marriagespell#aamilbabainpakistan #amilbabainkarachi #powerfullblackmagicspell #kalajadumantarspecialist #realamilbaba #AmilbabainPakistan #astrologerincanada #astrologerindubai #lovespellsmaster #kalajaduspecialist #lovespellsthatwork #aamilbabainlahore #Amilbabainuk #amilbabainspain #amilbabaindubai #Amilbabainnorway #amilbabainkrachi #amilbabainlahore #amilbabaingujranwalan #amilbabainislamabad
Cosmetic shop management system project report.pdfKamal Acharya
Buying new cosmetic products is difficult. It can even be scary for those who have sensitive skin and are prone to skin trouble. The information needed to alleviate this problem is on the back of each product, but it's thought to interpret those ingredient lists unless you have a background in chemistry.
Instead of buying and hoping for the best, we can use data science to help us predict which products may be good fits for us. It includes various function programs to do the above mentioned tasks.
Data file handling has been effectively used in the program.
The automated cosmetic shop management system should deal with the automation of general workflow and administration process of the shop. The main processes of the system focus on customer's request where the system is able to search the most appropriate products and deliver it to the customers. It should help the employees to quickly identify the list of cosmetic product that have reached the minimum quantity and also keep a track of expired date for each cosmetic product. It should help the employees to find the rack number in which the product is placed.It is also Faster and more efficient way.
Hybrid optimization of pumped hydro system and solar- Engr. Abdul-Azeez.pdffxintegritypublishin
Advancements in technology unveil a myriad of electrical and electronic breakthroughs geared towards efficiently harnessing limited resources to meet human energy demands. The optimization of hybrid solar PV panels and pumped hydro energy supply systems plays a pivotal role in utilizing natural resources effectively. This initiative not only benefits humanity but also fosters environmental sustainability. The study investigated the design optimization of these hybrid systems, focusing on understanding solar radiation patterns, identifying geographical influences on solar radiation, formulating a mathematical model for system optimization, and determining the optimal configuration of PV panels and pumped hydro storage. Through a comparative analysis approach and eight weeks of data collection, the study addressed key research questions related to solar radiation patterns and optimal system design. The findings highlighted regions with heightened solar radiation levels, showcasing substantial potential for power generation and emphasizing the system's efficiency. Optimizing system design significantly boosted power generation, promoted renewable energy utilization, and enhanced energy storage capacity. The study underscored the benefits of optimizing hybrid solar PV panels and pumped hydro energy supply systems for sustainable energy usage. Optimizing the design of solar PV panels and pumped hydro energy supply systems as examined across diverse climatic conditions in a developing country, not only enhances power generation but also improves the integration of renewable energy sources and boosts energy storage capacities, particularly beneficial for less economically prosperous regions. Additionally, the study provides valuable insights for advancing energy research in economically viable areas. Recommendations included conducting site-specific assessments, utilizing advanced modeling tools, implementing regular maintenance protocols, and enhancing communication among system components.
HEAP SORT ILLUSTRATED WITH HEAPIFY, BUILD HEAP FOR DYNAMIC ARRAYS.
Heap sort is a comparison-based sorting technique based on Binary Heap data structure. It is similar to the selection sort where we first find the minimum element and place the minimum element at the beginning. Repeat the same process for the remaining elements.
NUMERICAL SIMULATIONS OF HEAT AND MASS TRANSFER IN CONDENSING HEAT EXCHANGERS...ssuser7dcef0
Power plants release a large amount of water vapor into the
atmosphere through the stack. The flue gas can be a potential
source for obtaining much needed cooling water for a power
plant. If a power plant could recover and reuse a portion of this
moisture, it could reduce its total cooling water intake
requirement. One of the most practical way to recover water
from flue gas is to use a condensing heat exchanger. The power
plant could also recover latent heat due to condensation as well
as sensible heat due to lowering the flue gas exit temperature.
Additionally, harmful acids released from the stack can be
reduced in a condensing heat exchanger by acid condensation. reduced in a condensing heat exchanger by acid condensation.
Condensation of vapors in flue gas is a complicated
phenomenon since heat and mass transfer of water vapor and
various acids simultaneously occur in the presence of noncondensable
gases such as nitrogen and oxygen. Design of a
condenser depends on the knowledge and understanding of the
heat and mass transfer processes. A computer program for
numerical simulations of water (H2O) and sulfuric acid (H2SO4)
condensation in a flue gas condensing heat exchanger was
developed using MATLAB. Governing equations based on
mass and energy balances for the system were derived to
predict variables such as flue gas exit temperature, cooling
water outlet temperature, mole fraction and condensation rates
of water and sulfuric acid vapors. The equations were solved
using an iterative solution technique with calculations of heat
and mass transfer coefficients and physical properties.
We have compiled the most important slides from each speaker's presentation. This year’s compilation, available for free, captures the key insights and contributions shared during the DfMAy 2024 conference.
26. (1) Learning Algorithms
Table 1: A categorization of the basic deep NN learning algorithms and related approaches.(Source: Guo et al.,
2016).
CNN RBM AUTOENCODER SPARSE CODING
AlexNet
(Krizhevsky et al, 2012)
Deep Belief Net
(Hinton, et al, 2006)
Sparse Autoencoder
(Poultney et al 2006)
Sparse Coding
(Yang et al, 2009)
Clarifai
(Zeiler, et al 2014)
Deep Boltzmann
Machine (Salakhutdinov
et al., 2009)
Denoising Autoencoder
(Vincent, et al. 2008)
Laplacian Sparse coding
(Gao et al, 2010)
SPP
(He et al, 2014)
Deep Energy Models
(Ngiam et al., 2011)
Contractive Autoencoder
(Rifai, et al.2011)
Local Co-ordinate coding
(Yu et al, 2009)
VGG
(Simonyan et al., 2014)
Super-Vector coding
(Zhou et al, 2010)
GoogLeNet (Szegedy et
al., 2015)
31
26
28. Related Works
S/N Research Focus Contribution
1 To discover a fast and more efficient way of Implementation of a novel learning
initializing weights for effective learning of low- algorithm for initializing weights
dimensional codes from high-dimensional data in that allows deep AE networks and
multi-layer neural networks (Hinton et al., 2006; deep boltmzann machines to learn
Salakhutdinov et al., 2009; Vincent et al., 2010; useful higher representations
2
al., 2015; Zhong et al., 2016).
Cho et al., 2011).
To explore the possibility of allowing hashing Introduction of a state–of-the-art
function learning(learning of efficient binary codes deep hashing, supervised deep
that preserve neighborhood structure in the original hashing and semantic hashing
data space) and feature learning occur methods for large scale visual search,
simultaneously (Salakhutdinov et al., 2009; Erin et image retrieval and text mining
3
for supervised learning and unsupervised learning
(Springenberg et al., 2014; Radford et al., 2015).
To bridge the gap between the success of CNNSs Introduction of a class of CNNs
called deep convolutional generative
adversarial networks (DCGANs) for
unsupervised learning.
31
30. Related Works
S/N Research Focus Contribution
1
2016;).
To mitigate the problem of overfitting in large Implementation of several
neural networks with sparse datasets (Zeiler et regularization techniques such as
al., 2013; Srivastava et al., 2014; Pasupa et al., “dropout”, stochastic pooling,
weight decay, flipped image
augmentation amongst others for
ensuring stability in DNN
2 To investigate how to automatically rank Design of a reliable theoretical
source CNNs for transfer learning and use framework that perform zeroshot
transfer learning to improve a Sum-Product ranking of CNNs for transfer
Network for probabilistic inference when using learning for a given target task in
sparse datasets (Afridi et al., 2017; Zhao et al., Sum-Product networks
2017).
31
30
32. Related Works
S/N Research Focus Contribution
1 To develop a computationally efficient algorithm for
gradient based optimization in deep neural networks
(Hinton et al., 2006; Duchi et al., 2011; Ngiam et al.,
2011; Tieleman et al., 2012; Sutskever et al., 2013;
Kingma et al., 2014; Patel, 2016).
2 To develop an accelerator that can deliver state-of-the-art
accuracy with minimum energy consumption when
using large CNNs(Chen et al., 2017).
Introduction of several first-order and
second-order stochastic gradient based
optimization methods for minimizing
large objective functions in deep
networks such as Complementary
priors, Adam, Adagrad, RMSprop,
Momentum, L-BFGS and Kalman- based
SGD
Implementation of an Energy-Efficient
Reconfigurable Accelerator for Deep
CNN using efficient dataflow to
minimize energy through zeros
skipping/gating.
31
32
34. Related Works
S/N Research Focus Contribution
1 To demonstrate the advantage of combining deep neural
networks with support vector machines (Zhong et al.,
2000; Nagi et al., 2012; Tang et al., 2013; Li et al., 2017).
2 To exploit the power of deep neural networks in
optimizing the performance of nearest neighbor
classifiers in kNN Classification tasks (Min et al., 2009;
Ren et al., 2014).
Introduction of a novel classifier
architecture that combines two
heterogeneous supervised
classification techniques, CNN and
SVM for feature extraction and for
classification
They presented a framework for
learning convolutional nonlinear
features for K nearest neighbor (kNN)
classification.
31
34
36. (1) Methodology Application of DL
Author and Title Objective Methodology Contribution
Araque et
al.(2017).
Enhancing deep
learning sentiment
analysis with
ensemble
techniques in
social
applications.
To improve the
performance of
sentiment analysis in
social applications by
integrating deep
learning techniques
with traditional feature
based approaches based
on hand-crafted or
manually extracted
features
The utilization of a
word embedding's
model and a linear
machine learning
algorithm to develop a
deep learning based
sentiment
classifier(baseline), the
use of two ensemble
techniques namely
ensemble of classifiers
(CEM) and ensemble of
features (MSG and MGA)
Development of
ensemble models for
sentiment analysis
which surpass that of
the original baseline
classifier
31
36
37. Author and Title Objective Methodology Contribution
Betru et al. (2017).
Deep Learning
Methods on
Recommender
System. A Survey
of State-of-the-art:
To distinguish between
the various traditional
recommendation
techniques and
introducing deep
learning collaborative
and content based
approaches
As pointed out by the
authors, the
methodology adopted in
(Wang, Wang, & Yeung,
2015) integrated a
Bayesian Stack De-
noising Auto Encoder
(SDAE) and
Collaborative Topic
Regression to perform
collaborative deep
learning.
The implementation of a
novel collaborative deep
learning approach, the
first of its kind to learn
from review texts and
ratings.
Methodology Application of DL (Contd)
31
37
38. Author and Title Objective Methodology Contribution
Luo et al.(2016).
A deep learning
approach for credit
scoring using credit
default swaps.
To implement a
novel method
which leverages a
DBN model for
carrying out credit
scoring in credit
default swaps
(CDS) markets
The methodology adopted
by the researchers in their
experiments was to
compare the results of
MLR, MLP, and SVM with
the Deep Belief Networks
(DBN) with the Restricted
Boltzmann Machine by
applying 10-fold cross-
validation on a dataset
The contribution made
by the researchers to
this literature is
investigating the
performance of DBN in
corporate credit
scoring. The results
demonstrate that the
deep learning
algorithm significantly
outperforms the
baselines.
38
Methodology Application of DL (Contd)
39. Author and Title Objective Methodology Contribution
Grinblat et al.(2016).
Deep learning for plant
identification using vein
morphological patterns.
The authors aimed
to eliminate the use
of handcrafted
features extractors
by proposing the
use of deep
convolutional
network for the
problem of plant
identification from
leaf vein patterns.
The methodology
adopted to classify three
plant species: white
bean, red bean and
soybean was the use of
dataset containing leaf
images, a CNN of 6
layers trained with the
SGD method, a training
set using 20 samples as
mini batches with a 50%
dropout for
regularization.
The relevance of deep
learning to agriculture
using CNN as a model
for plant identification
based on vein
morphological pattern.
39
Methodology Application of DL (Contd)
40. Author and Title Objective Methodology Contribution
Evermann et
al.(2017).
Predicting process
behaviour using
deep learning.
To come up with a
novel method of
carrying out process
prediction without
the use of explicit
models using deep
learning.
The approach used to
implement this novel
idea included the use of
a framework called
Tensorflow as it
provides (RNN)
functionality embedded
with LSTM cells which
can be run on high
performance parallel,
cluster and GPU
platforms.
Improvement in state-
of-the-art in process
prediction, the needless
use of explicit model
and the inherent
advantages of using an
artificial intelligence
approach.
40
Methodology Application of DL (Contd)
41. Author and Title Objective Methodology Contribution
Kang et al. (2016).
A deep-learning-
based emergency
alert system.
proposed a deep
learning emergency
alert system to
overcome the
limitations of the
traditional emergency
alert systems
A heuristic based
machine learning
technology was used to
generate descriptors
starts for labels in the
problem domain, an API
analyzer that utilized
convolutional neural
network for object
detection and parsing to
generate compositional
models was also used.
Contribution of this
research shows that the
EAS can be adapted to
other monitoring
devices asides from
CCTV
41
Methodology Application of DL (Contd)
42. (2) Domain Applications Of Deep Learning
Domain Deep learning is Applied to Topic &Reference
perform
Recommender
System
Sentiment Analysis/Opinion
mining)
Collaborative Deep Learning for
Recommender Systems(Wang, Wang, &
Yeung, 2015)
Social
Applications
(Sentiment Analysis/Opinion
mining/Facial Recognition)
Enhancing deep learning sentiment
analysis with ensemble techniques in
social applications (Araque et al., 2017).
Medicine (Medical Diagnosis) A survey on deep learning in medical
image analysis(Litjens et al., 2017)
Finance (Credit Scoring, stock market
prediction)
A deep learning approach for credit
scoring using credit default swaps (Luo
et al., 2016).
42
43. Domain Deep learning is Applied Topic &Reference
to perform
Transportation Traffic flow prediction Deep learning for short-term traffic flow
prediction(Polson et al.,2017).
Business Process prediction Predicting Process Behaviour Using Deep
Learning (Evermann et al., 2017).
Emergency Emergency Alert A Deep-Learning-Based Emergency Alert
System (Kang et al., 2016)
Agriculture (Plant Identification) Deep Learning for Plant Identification Using
Vein Morphological Patterns (Grinblat et
al.,2016).
43
Domain Applications 0f Deep Learning (Contd)
50. Trends in Deep Learning Research
50
1. Design of more powerful deep models to learn from fewer
training data. (Guo et al, 2016; pasupa et al., 2016 ;Li, et al 2017)
2. Use of better optimization algorithms to adjust network
parameters i.e. regularization techniques (zeng et al, 2016; Li, et
al 2017)
3. Implementation of deep learning algorithms on mobile devices
(Li, et al 2017)
4. Stability analysis of deep neural network (Li, et al 2017)
51. Trends in Deep Learning Research (Contd)
51
5. Combining probabilistic , auto-encoder and manifold learning
models.(bengio et al., 2013)
6. Applications of deep neural networks in nonlinear networked
control systems (NCSs) (Li, et al 2017)
7. Applications of unsupervised, semi-supervised and
reinforcement-learning approaches to DNNs for complex
systems (Li, et al 2017)
8. Learning deep networks for other machine learning techniques
e.g. deep kNN (Zoran et al, 2017), deep SVM (Li, et al 2017).
52. Research Issues/challenges in Deep Learning
52
1. High Computational cost/burden in training phase (pasupa et al.,
2016)
2. Over-fitting problem when the data-set is small. (pasupa et al., 2016,
Guo et al, 2016)
3. Optimization issues due to local minima or use of first order methods.
(pasupa et al., 2016)
4. Little or no clear understanding of the underlying theoretical
foundation of which deep learning architecture should perform well or
outperform other approaches. (Guo et al, 2016)
5. Time complexity (Guo et al, 2016)
53. Deep Learning – Use Case
53
Let’s look at a use case where we can use DL for image recognition
54. Practical Application of deep learning in Facial
Recognition
54
Problem Scenario
Suppose we want to create a system that can recognize
faces of different people in an image. How do we solve
this as a typical machine learning problem and/or
using a deep learning approach?
55. Classical Machine Learning
Approach
55
We will define facial features such as eyes,
nose, ears etc. and then, the system will
identify which features are more
important for which person on its own or
by itself.
56. Deep Learning Approach
56
Now, deep learning takes this one step ahead.
Deep learning automatically finds out the
features which are important for classification
because of deep neural networks, whereas in
case of Machine Learning we had to manually
define these features.
57. Practical Application of deep learning - Facial Recognition
(Contd)
Figure 19: Face Recognition Using deep networks (Source: www.edureka.co)
57
58. Deep Face Recognition
Face recognition applications have two parts or phases viz:
(1)Phase-I: Enrollment
phase – Model / system is
trained using millions of
prototype face images and
a trained model is
generated. Generated face
features are stored in
database and
(2)Phase-II: Recognition
phase – Query face image
is given as input to the
model generated in phase-
I to recognise it correctly.
Figure 20: Face Recognition Architecture (Source: aiehive.com)
58
59. Deep Face Recognition (Contd)
59
Steps within Enrollment
Phase Includes
1. Face Detection
2. Feature extraction
3.Store Model and extracted
feature in Database
Steps within Recognition Phase /
Query Phase Includes
1. Face Detection
2. Preprocessing
3. Feature Extraction
4. Recognition
60. Step 1: Face Detection - Enrollment Phase
Face Detection: Face needs to be
located and region of interest is
computed.
• Histogram of Oriented
Gradients (HOG) is a faster
and easier algorithm for face
detection.
• Detected faces are given to
next step of feature extraction.
Figure 21: Multiple Face Detection (Source: aiehive.com)
60
61. Step 2: Feature Extraction- Enrollment Phase
What is the best feature measure that represents human face in a best way?
Deep learning can
determine which parts of a
face are important to
measure. Deep Convolution
Neural Network (DCNN)
can be trained to learn
important features.
(Simonyan et al., 2014)
61
63. Step 1: Face Detection- Recognition Phase
Face Detection: Face needs to be
located and region of interest is
computed.
• Histogram of Oriented
Gradients (HOG) is faster and
easier algorithm for face
detection.
• Detected faces are given to next
step of preprocessing.
63
64. Step-2. Pre-processing- Recognition Phase
• Pre-process to overcome issues like noise,
illumination using any suitable filters
[Kalman Filter, Adaptive Retinex (AR),
Multi-Scale Self Quotient (SQI), Gabor
Filter, etc.]
• Pose/rotation can be accounted by using
3D transformation or affine transformation
or face landmark estimation
• Determine 68 landmark points on every
face— the top of the chin, the outside edge
of each eye, the inner edge of each
eyebrow, etc.
Figure 22: Landmark point estimation (Source: aiehive.com)
64
65. Step 3: Feature Extraction-Recognition Phase
65
• In this third step of Deep Face Recognition, we have to use trained DCNN
model, which was generated during feature extraction step of enrollment
phase
• A query image is given as input.
• The DCNN generates 128 feature values.
• This feature vector is then compared with feature vector stored in database
Step 4: Recognition-Recognition Phase
• This can be done by using any basic machine learning classification
algorithm SVM classifier, Bayesian classifier, Euclidean Distance classifier,
for matching database feature vector with query feature vector.
• Gives ID of best matching face image from database as a recognition output.
66. Conclusion
66
• Deep learning is a representation learning method and the new state-of-the-art
technique for performing automatic feature extraction in large unlabeled data
• Various categories of deep learning architectures and basic algorithms together with
their related approaches have been discussed
• Several theoretical concepts and practical application areas have been presented
• It is a promising research area for tackling feature extraction for complex real-world
problems without having to undergo the process of manual feature engineering.
• With the rapid development of hardware resources and computation technologies,
it is certain that deep neural networks will receive wider attention and find broader
applications in the future.
67. References
67
• Afridi, M. J., Ross, A., & Shapiro, E. M. (2017). On automated source selection for transfer learning
in convolutional neural networks. Pattern Recognition.
• Araque, O., Corcuera-Platas, I., Sánchez-Rada, J. F., & Iglesias, C. A. (2017). Enhancing deep
learning sentiment analysis with ensemble techniques in social applications. Expert Systems with
Applications, 77, 236-246.
• Betru, B. T., Onana, C. A., & Batchakui, B. (2017). A Survey of State-of-the-art: Deep Learning
Methods on Recommender System. International Journal of Computer Applications, 162(10).
• Chen, Y. H., Krishna, T., Emer, J. S., & Sze, V. (2017). Eyeriss: An energy-efficient reconfigurable
accelerator for deep convolutional neural networks. IEEE Journal of Solid-State Circuits, 52(1), 127-
138.
• Cho, K., Raiko, T., & Ihler, A. T. (2011). Enhanced gradient and adaptive learning rate for training
restricted Boltzmann machines. In Proceedings of the 28th International Conference on Machine
Learning (ICML-11) (pp. 105-112).
68. References (Contd).
68
• Deng, L., & Yu, D. (2014). Deep learning: methods and applications. Foundations and Trends® in
Signal Processing, 7(3–4), 197-387.
• Duchi, J., Hazan, E., & Singer, Y. (2011). Adaptive subgradient methods for online learning and
stochastic optimization. Journal of Machine Learning Research, 12(Jul), 2121-2159.
• Erin Liong, V., Lu, J., Wang, G., Moulin, P., & Zhou, J. (2015). Deep hashing for compact binary
codes learning. In Proceedings of the IEEE Conference on Computer Vision and Pattern
Recognition (pp. 2475-2483).
• Evermann, J., Rehse, J. R., & Fettke, P. (2017). Predicting process behaviour using deep learning.
Decision Support Systems.
• Gao, S., Tsang, I. W. H., Chia, L. T., & Zhao, P. (2010, June). Local features are not lonely–
Laplacian sparse coding for image classification. In Computer Vision and Pattern Recognition
(CVPR), 2010 IEEE Conference on (pp. 3555-3561). IEEE.
69. References (Contd).
69
• Grinblat, G. L., Uzal, L. C., Larese, M. G., & Granitto, P. M. (2016). Deep learning for plant
identification using vein morphological patterns. Computers and Electronics in Agriculture, 127,
418-424.
• Guo, Y., Liu, Y., Oerlemans, A., Lao, S., Wu, S., & Lew, M. S. (2016). Deep learning for visual
understanding: A review. Neurocomputing, 187, 27-48.
• He, K., Zhang, X., Ren, S., & Sun, J. (2014, September). Spatial pyramid pooling in deep
convolutional networks for visual recognition. In European Conference on Computer Vision (pp.
346-361). Springer, Cham.
• Hinton, G. E., Osindero, S., & Teh, Y. W. (2006). A fast learning algorithm for deep belief nets.
Neural computation, 18(7), 1527-1554.
• Kang, B., & Choo, H. (2016). A deep-learning-based emergency alert system. ICT Express, 2(2), 67-
70.
70. References (Contd).
70
• Kingma, D., & Ba, J. (2014). Adam: A method for stochastic optimization. arXiv preprint
arXiv:1412.6980.
• Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2012). Imagenet classification with deep
convolutional neural networks. In Advances in neural information processing systems (pp. 1097-
1105).
• LeCun, Y., Bengio, Y., & Hinton, G. (2015). Deep learning. Nature, 521(7553), 436-444.
• LeCun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998). Gradient-based learning applied to
document recognition. Proceedings of the IEEE, 86(11), 2278-2324.
• Lee, H., Grosse, R., Ranganath, R., & Ng, A. Y. (2011). Unsupervised learning of hierarchical
representations with convolutional deep belief networks. Communications of the ACM, 54(10), 95-
103.
71. References (Contd).
• Li, Y., & Zhang, T. (2017). Deep neural mapping support vector machines. Neural Networks, 93,
185-194.
• Litjens, G., Kooi, T., Bejnordi, B. E., Setio, A. A. A., Ciompi, F., Ghafoorian, M., & Sánchez, C. I.
(2017). A survey on deep learning in medical image analysis. arXiv preprint arXiv:1702.05747.
• Luo, C., Wu, D., & Wu, D. (2016). A deep learning approach for credit scoring using credit default
swaps. Engineering Applications of Artificial Intelligence.
• Makwana, M. A.(2016, Dec)Deep Face Recognition Using Deep Convolutional Neural
Network.Retrieved from http://aiehive.com
• Min, R., Stanley, D. A., Yuan, Z., Bonner, A., & Zhang, Z. (2009, December). A deep non-linear
feature mapping for large-margin knn classification. In Data Mining, 2009. ICDM'09. Ninth IEEE
International Conference on (pp. 357-366). IEEE.
• Nagi, J., Di Caro, G. A., Giusti, A., Nagi, F., & Gambardella, L. M. (2012, December). Convolutional
neural support vector machines: hybrid visual pattern classifiers for multi-robot systems. In
Machine Learning and Applications (ICMLA), 2012 11th International Conference on (Vol. 1, pp. 71
72. References (Contd).
72
• Ngiam, J., Chen, Z., Koh, P. W., & Ng, A. Y. (2011). Learning deep energy models. In Proceedings
of the 28th International Conference on Machine Learning (ICML-11) (pp. 1105-1112).
• Pasupa, K., & Sunhem, W. (2016, October). A comparison between shallow and deep architecture
classifiers on small dataset. In Information Technology and Electrical Engineering (ICITEE), 2016
8th International Conference on (pp. 1-6). IEEE.
• Patel, V. (2016). Kalman-based stochastic gradient method with stop condition and insensitivity to
conditioning. SIAM Journal on Optimization, 26(4), 2620-2648.
• Polson, N. G., & Sokolov, V. O. (2017). Deep learning for short-term traffic flow prediction.
Transportation Research Part C: Emerging Technologies, 79, 1-17.
• Poultney, C., Chopra, S., & Cun, Y. L. (2007). Efficient learning of sparse representations with an
energy-based model. In Advances in neural information processing systems (pp. 1137-1144).
73. References (Contd).
73
• Radford, A., Metz, L., & Chintala, S. (2015). Unsupervised representation learning with deep
convolutional generative adversarial networks. arXiv preprint arXiv:1511.06434.
• Ren, W., Yu, Y., Zhang, J., & Huang, K. (2014, August). Learning convolutional nonlinear features
for k nearest neighbor image classification. In Pattern Recognition (ICPR), 2014 22nd International
Conference on (pp. 4358-4363). IEEE.
• Rifai, S., Vincent, P., Muller, X., Glorot, X., & Bengio, Y. (2011). Contractive auto-encoders: Explicit
invariance during feature extraction. In Proceedings of the 28th international conference on
machine learning (ICML-11) (pp. 833-840).
• Salakhutdinov, R., & Hinton, G. (2009). Semantic hashing. International Journal of Approximate
Reasoning, 50(7), 969-978.
• Salakhutdinov, R., & Hinton, G. (2009, April). Deep boltzmann machines. In Artificial Intelligence
and Statistics (pp. 448-455).
74. References (Contd).
recognition (pp. 1-9).
74
• Simonyan, K., & Zisserman, A. (2014). Very deep convolutional networks for large-scale image
recognition. arXiv preprint arXiv:1409.1556.
• Springenberg, J. T., Dosovitskiy, A., Brox, T., & Riedmiller, M. (2014). Striving for simplicity: The all
convolutional net. arXiv preprint arXiv:1412.6806.
• Srivastava, N., Hinton, G. E., Krizhevsky, A., Sutskever, I., & Salakhutdinov, R. (2014). Dropout: a
simple way to prevent neural networks from overfitting. Journal of machine learning research,
15(1), 1929-1958.
• Sutskever, I., Martens, J., Dahl, G., & Hinton, G. (2013, February). On the importance of
initialization and momentum in deep learning. In International conference on machine learning
(pp. 1139-1147).
• Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., ... & Rabinovich, A. (2015). Going
deeper with convolutions. In Proceedings of the IEEE conference on computer vision and pattern
75. References (Contd).
• Tang, Y. (2013). Deep learning using support vector machines. CoRR, abs/1306.0239, 2.
• Tieleman, T., & Hinton, G. (2012). Lecture 6.5-rmsprop: Divide the gradient by a running average of
its recent magnitude. COURSERA: Neural networks for machine learning, 4(2), 26-31.
• Vincent, P., Larochelle, H., Bengio, Y., & Manzagol, P. A. (2008, July). Extracting and composing
robust features with denoising autoencoders. In Proceedings of the 25th international conference on
Machine learning (pp. 1096-1103). ACM.
• Vincent, P., Larochelle, H., Lajoie, I., Bengio, Y., & Manzagol, P. A. (2010). Stacked denoising
autoencoders: Learning useful representations in a deep network with a local denoising criterion.
Journal of Machine Learning Research, 11(Dec), 3371-3408.
• Wang, H., Wang, N., & Yeung, D. Y. (2015, August). Collaborative deep learning for recommender
systems. In Proceedings of the 21th ACM SIGKDD International Conference on Knowledge
Discovery and Data Mining (pp. 1235-1244). ACM.
• what is deep learning.(Web log post).Retrieved September 6, 2017 from
75
76. References (Contd).
76
• Yang, J., Yu, K., Gong, Y., & Huang, T. (2009, June). Linear spatial pyramid matching using sparse
coding for image classification. In Computer Vision and Pattern Recognition, 2009. CVPR 2009.
IEEE Conference on (pp. 1794-1801). IEEE.
• Yu, K., Zhang, T., & Gong, Y. (2009). Nonlinear learning using local coordinate coding. In
Advances in neural information processing systems (pp. 2223-2231).
• Zeiler, M. D., & Fergus, R. (2014, September). Visualizing and understanding convolutional
networks. In European conference on computer vision (pp. 818-833). Springer, Cham.
• Zhao, J., & Ho, S. S. (2017). Structural knowledge transfer for learning Sum-Product Networks.
Knowledge-Based Systems, 122, 159-166.
• Zhong, G., Xu, H., Yang, P., Wang, S., & Dong, J. (2016, July). Deep hashing learning networks. In
Neural Networks (IJCNN), 2016 International Joint Conference on (pp. 2236-2243). IEEE.
• Zhong, S., & Ghosh, J. (2000). Decision boundary focused neural network classifier.
77. References (Contd).
77
• Zhou, X., Yu, K., Zhang, T., & Huang, T. S. (2010, September). Image classification using super-
vector coding of local image descriptors. In European conference on computer vision (pp. 141-154).
Springer, Berlin, Heidelberg.
• Zoran, D., Lakshminarayanan, B., & Blundell, C. (2017). Learning Deep Nearest Neighbor
Representations Using Differentiable Boundary Trees. arXiv preprint arXiv:1702.08833.
78. • Almighty God for His sufficient grace.
• I would like to appreciate the HOD, Dr Osamor V.C. and the
PG Coordinator, Dr. Azeta for their contribution toward the
reality of the presentation today.
• Special recognition to my Mentor, Dr. Olufunke Oladipupo
who gave this work the depth of knowledge it possesses
• I also appreciate the entire faculty members in the department
for their support.
78
Acknowledgement