We develop a precise writing survey on sequence-to-sequence learning with neural network and its models. The primary aim of this report is to enhance the knowledge of the sequence-to-sequence neural network and to locate the best way to deal with executing it. Three models are mostly used in sequence-to-sequence neural network applications, namely: recurrent neural networks (RNN), connectionist temporal classification (CTC), and attention model. The evidence we adopted in conducting this survey included utilizing the examination inquiries or research questions to determine keywords, which were used to search for bits of peer-reviewed papers, articles, or books at scholastic directories. Through introductory hunts, 790 papers, and scholarly works were found, and with the assistance of choice criteria and PRISMA methodology, the number of papers reviewed decreased to 16. Every one of the 16 articles was categorized by their contribution to each examination question, and they were broken down. At last, the examination papers experienced a quality appraisal where the subsequent range was from 83.3% to 100%. The proposed systematic review enabled us to collect, evaluate, analyze, and explore different approaches of implementing sequence-to-sequence neural network models and pointed out the most common use in machine learning. We followed a methodology that shows the potential of applying these models to real-world applications.
Artificial neural networks and its applicationHưng Đặng
Artificial neural networks (ANNs) are non-linear data driven approaches that can identify patterns in complex data. ANNs imitate the human brain in learning from examples rather than being explicitly programmed. There are various types of ANN architectures, but feedforward and recurrent networks are most common. ANNs have been successfully applied to problems in diverse domains, including classification, prediction, and modeling where relationships are unknown. Developing an effective ANN model requires selecting variables, dividing data into training/testing/validation sets, determining network architecture, evaluating performance, and training the network through iterative adjustment of weights.
Artificial neural networks (ANNs) are computational models inspired by the human brain that are used for predictive analytics and nonlinear statistical modeling. ANNs can learn complex patterns and relationships from large datasets through a process of training, and then make predictions on new data. The three most common types of ANN architectures are multilayer perceptrons, radial basis function networks, and self-organizing maps. ANNs have been successfully applied across many domains, including finance, medicine, engineering, and biology, to solve problems involving classification, prediction, and nonlinear pattern recognition.
Analysis of Neocognitron of Neural Network Method in the String RecognitionIDES Editor
This paper aims that analysing neural network method
in pattern recognition. A neural network is a processing device,
whose design was inspired by the design and functioning of
human brain and their components. The proposed solutions
focus on applying Neocognitron Algorithm model for pattern
recognition. The primary function of which is to retrieve in a
pattern stored in memory, when an incomplete or noisy version
of that pattern is presented. An associative memory is a
storehouse of associated patterns that are encoded in some
form. In auto-association, an input pattern is associated with
itself and the states of input and output units coincide. When
the storehouse is incited with a given distorted or partial
pattern, the associated pattern pair stored in its perfect form
is recalled. Pattern recognition techniques are associated a
symbolic identity with the image of the pattern. This problem
of replication of patterns by machines (computers) involves
the machine printed patterns. There is no idle memory
containing data and programmed, but each neuron is
programmed and continuously active.
New Generation Routing Protocol over Mobile Ad Hoc Wireless Networks based on...ijasuc
There is a vast amount of researched literature available on Route Finding and Link Establishment in
MANET protocols based on various concepts such as “pro-active”, “reactive”, “power awareness”,
“cross-layering” etc. Most of these techniques are rather restrictive, taking into account a few of the
several aspects that go into effective route establishment. When we look at practical implementations of
MANETs, we have to take into account various factors in totality, not in isolation. The several factors that
decide and influence the routing have to be considered as a whole in the difficult task of finding the best
solution in route finding and optimization. The inputs to the system are manifold and apparently unrelated.
Most of the parameters are imprecise or non-crisp in nature. The uncertainty and imprecision lead to think
that intelligent routing techniques are essential and important in evolving robust and dependable solutions
to route finding. The obvious method by which this can be achieved is the deployment of soft computing
techniques such as Neural Nets, Fuzzy Logic and Genetic algorithms. Neural Networks help us to solve the
complex problem of transforming the inputs to outputs without apriori knowledge of what the relationship
is between inputs and outputs. Fuzzy Logic helps us to deal with imprecise and ill-conditioned data.
Genetic Algorithms help us to select the best possible solution from the solution space in an optimal sense.
Our paper presented here below seeks to explore new horizons in this direction. The results of our
experimentation have been very satisfactory and we have achieved the goal of optimal route finding to a
large extent. There is of course considerable room for further refinements.
A Parallel Framework For Multilayer Perceptron For Human Face RecognitionCSCJournals
Artificial neural networks have already shown their success in face recognition and similar complex pattern recognition tasks. However, a major disadvantage of the technique is that it is extremely slow during training for larger classes and hence not suitable for real-time complex problems such as pattern recognition. This is an attempt to develop a parallel framework for the training algorithm of a perceptron. In this paper, two general architectures for a Multilayer Perceptron (MLP) have been demonstrated. The first architecture is All-Class-in-One-Network (ACON) where all the classes are placed in a single network and the second one is One-Class-in-One-Network (OCON) where an individual single network is responsible for each and every class. Capabilities of these two architectures were compared and verified in solving human face recognition, which is a complex pattern recognition task where several factors affect the recognition performance like pose variations, facial expression changes, occlusions, and most importantly illumination changes. Experimental results show that the proposed OCON structure performs better than the conventional ACON in terms of network training convergence speed and which can be easily exercised in a parallel environment.
Artificial Neural Networks: Applications In ManagementIOSR Journals
With the advancement of computer and communication technology, the tools used for management decisions have undergone a gigantic change. Finding the more effective solution and tools for managerial problems is one of the most important topics in the management studies today. Artificial Neural Networks (ANNs) are one of these tools that have become a critical component for business intelligence. The purpose of this article is to describe the basic behavior of neural networks as well as the works done in application of the same in management sciences and stimulate further research interests and efforts in the identified topics.
NETWORK LEARNING AND TRAINING OF A CASCADED LINK-BASED FEED FORWARD NEURAL NE...ijaia
Presently, considering the technological advancement of our modern world, we are in dire need for a system that can learn new concepts and give decisions on its own. Hence the Artificial Neural Network is all that is required in the contemporary situation. In this paper, CLBFFNN is presented as a special and intelligent form of artificial neural networks that has the capability to adapt to training and learning of new ideas and be able to give decisions in a trimodal biometric system involving fingerprints, face and iris biometric data. It gives an overview of neural networks.
Artificial neural networks and its applicationHưng Đặng
Artificial neural networks (ANNs) are non-linear data driven approaches that can identify patterns in complex data. ANNs imitate the human brain in learning from examples rather than being explicitly programmed. There are various types of ANN architectures, but feedforward and recurrent networks are most common. ANNs have been successfully applied to problems in diverse domains, including classification, prediction, and modeling where relationships are unknown. Developing an effective ANN model requires selecting variables, dividing data into training/testing/validation sets, determining network architecture, evaluating performance, and training the network through iterative adjustment of weights.
Artificial neural networks (ANNs) are computational models inspired by the human brain that are used for predictive analytics and nonlinear statistical modeling. ANNs can learn complex patterns and relationships from large datasets through a process of training, and then make predictions on new data. The three most common types of ANN architectures are multilayer perceptrons, radial basis function networks, and self-organizing maps. ANNs have been successfully applied across many domains, including finance, medicine, engineering, and biology, to solve problems involving classification, prediction, and nonlinear pattern recognition.
Analysis of Neocognitron of Neural Network Method in the String RecognitionIDES Editor
This paper aims that analysing neural network method
in pattern recognition. A neural network is a processing device,
whose design was inspired by the design and functioning of
human brain and their components. The proposed solutions
focus on applying Neocognitron Algorithm model for pattern
recognition. The primary function of which is to retrieve in a
pattern stored in memory, when an incomplete or noisy version
of that pattern is presented. An associative memory is a
storehouse of associated patterns that are encoded in some
form. In auto-association, an input pattern is associated with
itself and the states of input and output units coincide. When
the storehouse is incited with a given distorted or partial
pattern, the associated pattern pair stored in its perfect form
is recalled. Pattern recognition techniques are associated a
symbolic identity with the image of the pattern. This problem
of replication of patterns by machines (computers) involves
the machine printed patterns. There is no idle memory
containing data and programmed, but each neuron is
programmed and continuously active.
New Generation Routing Protocol over Mobile Ad Hoc Wireless Networks based on...ijasuc
There is a vast amount of researched literature available on Route Finding and Link Establishment in
MANET protocols based on various concepts such as “pro-active”, “reactive”, “power awareness”,
“cross-layering” etc. Most of these techniques are rather restrictive, taking into account a few of the
several aspects that go into effective route establishment. When we look at practical implementations of
MANETs, we have to take into account various factors in totality, not in isolation. The several factors that
decide and influence the routing have to be considered as a whole in the difficult task of finding the best
solution in route finding and optimization. The inputs to the system are manifold and apparently unrelated.
Most of the parameters are imprecise or non-crisp in nature. The uncertainty and imprecision lead to think
that intelligent routing techniques are essential and important in evolving robust and dependable solutions
to route finding. The obvious method by which this can be achieved is the deployment of soft computing
techniques such as Neural Nets, Fuzzy Logic and Genetic algorithms. Neural Networks help us to solve the
complex problem of transforming the inputs to outputs without apriori knowledge of what the relationship
is between inputs and outputs. Fuzzy Logic helps us to deal with imprecise and ill-conditioned data.
Genetic Algorithms help us to select the best possible solution from the solution space in an optimal sense.
Our paper presented here below seeks to explore new horizons in this direction. The results of our
experimentation have been very satisfactory and we have achieved the goal of optimal route finding to a
large extent. There is of course considerable room for further refinements.
A Parallel Framework For Multilayer Perceptron For Human Face RecognitionCSCJournals
Artificial neural networks have already shown their success in face recognition and similar complex pattern recognition tasks. However, a major disadvantage of the technique is that it is extremely slow during training for larger classes and hence not suitable for real-time complex problems such as pattern recognition. This is an attempt to develop a parallel framework for the training algorithm of a perceptron. In this paper, two general architectures for a Multilayer Perceptron (MLP) have been demonstrated. The first architecture is All-Class-in-One-Network (ACON) where all the classes are placed in a single network and the second one is One-Class-in-One-Network (OCON) where an individual single network is responsible for each and every class. Capabilities of these two architectures were compared and verified in solving human face recognition, which is a complex pattern recognition task where several factors affect the recognition performance like pose variations, facial expression changes, occlusions, and most importantly illumination changes. Experimental results show that the proposed OCON structure performs better than the conventional ACON in terms of network training convergence speed and which can be easily exercised in a parallel environment.
Artificial Neural Networks: Applications In ManagementIOSR Journals
With the advancement of computer and communication technology, the tools used for management decisions have undergone a gigantic change. Finding the more effective solution and tools for managerial problems is one of the most important topics in the management studies today. Artificial Neural Networks (ANNs) are one of these tools that have become a critical component for business intelligence. The purpose of this article is to describe the basic behavior of neural networks as well as the works done in application of the same in management sciences and stimulate further research interests and efforts in the identified topics.
NETWORK LEARNING AND TRAINING OF A CASCADED LINK-BASED FEED FORWARD NEURAL NE...ijaia
Presently, considering the technological advancement of our modern world, we are in dire need for a system that can learn new concepts and give decisions on its own. Hence the Artificial Neural Network is all that is required in the contemporary situation. In this paper, CLBFFNN is presented as a special and intelligent form of artificial neural networks that has the capability to adapt to training and learning of new ideas and be able to give decisions in a trimodal biometric system involving fingerprints, face and iris biometric data. It gives an overview of neural networks.
DiscoversNet is an adaptive simulation-based learning environment for designing neural networks. It contains a knowledge-based neural network consultant module to provide educational guidance during the neural network design process. The consultant module represents domain knowledge using a knowledge-based neural network and provides advice to users based on their current understanding level. The system allows users to build neural network simulators through interactive manipulation of neural network components and receives feedback to correct misconceptions.
IRJET-Breast Cancer Detection using Convolution Neural NetworkIRJET Journal
This document discusses using a convolutional neural network (CNN) to detect breast cancer from medical images. CNNs are a type of deep learning model that can learn image features without manual feature engineering. The proposed system would take a sample medical image as input, preprocess it, and compare it to images in a database labeled as cancerous or non-cancerous. If cancer is detected, the system would determine the cancer stage and recommend appropriate treatment. The CNN model would be built and trained using libraries like Keras, TensorFlow, and Numpy to classify images and detect breast cancer at early stages for better treatment outcomes.
The document is a dissertation submitted to Gujarat University in partial fulfillment of a Master's degree in Computer Application, which discusses character recognition using neural networks. It provides an index of the contents including the introduction to neural networks, their architecture and applications, an introduction to character recognition, the use of Matlab and its neural network toolbox, a literature survey, the proposed work on digit recognition, potential enhancements, and conclusions. The dissertation was submitted by Sachinkumar M. Bharadva and Dhara Solanki under the guidance of their internal guide Mr. Sandeep R. Vasant at the AES Institute of Computer Studies.
Employing Neocognitron Neural Network Base Ensemble Classifiers To Enhance Ef...cscpconf
This paper presents an ensemble of neo-cognitron neural network base classifiers to enhance
the accuracy of the system, along the experimental results. The method offers lesser
computational preprocessing in comparison to other ensemble techniques as it ex-preempts
feature extraction process before feeding the data into base classifiers. This is achieved by the
basic nature of neo-cognitron, it is a multilayer feed-forward neural network. Ensemble of such
base classifiers gives class labels for each pattern that in turn is combined to give the final class
label for that pattern. The purpose of this paper is not only to exemplify learning behaviour of
neo-cognitron as base classifiers, but also to purport better fashion to combine neural network
based ensemble classifiers.
This document contains summaries of several papers related to artificial intelligence and neural networks:
1. The first paper discusses using recurrent neural networks to plan robot motions in variable environments.
2. The second paper describes using neural network models to classify brain signals and mind states from EEG data.
3. The third paper proposes using coarticulation composite models in acoustic-phonetic decoding to improve recognition rates beyond phonemes, diphones, and triphones.
Artificial Neural Network and its Applicationsshritosh kumar
Abstract
This report is an introduction to Artificial Neural
Networks. The various types of neural networks are
explained and demonstrated, applications of neural
networks like ANNs in medicine are described, and a
detailed historical background is provided. The
connection between the artificial and the real thing is
also investigated and explained. Finally, the
mathematical models involved are presented and
demonstrated.
Key Management Schemes for Secure Communication in Heterogeneous Sensor NetworksIDES Editor
Hierarchical Sensor Network organization is
widely used to achieve energy efficiency in Wireless Sensor
Networks(WSN). To achieve security in hierarchical WSN,
it is important to be able to encrypt the messages sent
between sensor nodes and its cluster head. The key
management task is challenging due to resource constrained
nature of WSN. In this paper we are proposing two key
management schemes for hierarchical networks which
handles various events like node addition, node compromise
and key refresh at regular intervals. The Tree-Based
Scheme ensures in-network processing by maintaining some
additional intermediate keys. Whereas the CRT-Based
Scheme performs the key management with minimum
communication and storage at each node.
The document provides an overview of neural networks including:
- Their history from early models in the 1940s to the breakthrough of backpropagation in the 1980s.
- What a neural network is and how it works at the level of individual neurons and when connected together.
- Common applications of neural networks like prediction, classification, and clustering.
- Key considerations in choosing an appropriate neural network architecture and training data for a given problem.
The document discusses artificial neural networks (ANNs). It defines ANNs as computational models inspired by biological neural networks. The basic structure and types of ANNs are explained, including feed forward and feedback networks. The document also covers ANN learning methods like supervised, unsupervised, and reinforcement learning. Applications of ANNs span various domains like aerospace, automotive, military, electronics, and more. While ANNs can perform complex tasks, they require extensive training and processing power for large networks.
Artificial neural networks are fundamental means for providing an attempt at modelling the information
processing capabilities of artificial nervous system which plays an important role in the field of cognitive
science. This paper focuses the features of artificial neural networks studied by reviewing the existing research
works, these features were then assessed and evaluated and comparative analysis. The study and literature
survey metrics such as functional capabilities of neurons, learning capabilities, style of computation, processing
elements, processing speed, connections, strength, information storage, information transmission,
communication media selection, signal transduction and fault tolerance were used as basis for comparison. A
major finding in this paper showed that artificial neural networks served as the platform for neuron computing
technology in the field of cognitive science.
Neural Networks for Pattern RecognitionVipra Singh
- Neural networks are computing systems inspired by biological neural networks in the brain that can be used for pattern recognition. An artificial neuron receives multiple inputs and produces a single output. Neural networks are trained to recognize complex patterns and identify categories.
- An important application of neural networks is pattern recognition, where a network is trained to associate input patterns with output categories. Recent advances include using neural networks for tasks like predicting student performance, medical diagnosis, and analyzing customer interactions. Neural networks are also being used increasingly in business for applications like predictive analytics and artificial intelligence.
This document is a preface to a book about neural networks. It provides an overview of the book's contents and objectives. The book aims to present a variety of standard neural network architectures along with their training algorithms and examples of applications. It is intended as both a textbook and reference for students and researchers interested in using neural networks. The preface outlines the scope and organization of the material covered in the book.
Army Study: Ontology-based Adaptive Systems of Cyber DefenseRDECOM
The U.S. Army Research Laboratory is part of the U.S. Army Research, Development and Engineering Command, which has the mission to ensure decisive overmatch for unified land operations to empower the Army, the joint warfighter and our nation. RDECOM is a major subordinate command of the U.S. Army Materiel Command.
Neural Network Classification and its Applications in Insurance IndustryInderjeet Singh
This document summarizes a neural networks project report on using neural networks for classification in the insurance industry. The report discusses extracting rules from trained neural networks, using neural networks to predict customer retention and pricing policies. It also discusses using neural networks to detect auto insurance fraud by identifying important fraud indicators.
This document provides an overview of artificial neural networks (ANNs). It discusses how ANNs are inspired by biological neural networks and consist of interconnected artificial neurons that process information. The document describes common ANN architectures like multilayer perceptrons and radial basis function networks. It also summarizes different ANN learning paradigms such as supervised, unsupervised, and reinforcement learning. Specific learning rules and algorithms are mentioned, including the perceptron rule, Hebbian learning, competitive learning, and backpropagation. Applications of ANNs discussed include pattern recognition, clustering, prediction, and data compression.
Nature Inspired Reasoning Applied in Semantic Webguestecf0af
1) Neural networks are computational structures inspired by biological neural networks and have been successfully used to solve complex tasks like image recognition and natural language processing.
2) Neural networks consist of interconnected nodes that perform simple mathematical functions to produce outputs. The connections between nodes and their weights can be modified through training to solve problems.
3) Nature inspired algorithms like neural networks are well-suited for semantic web problems because they can process large amounts of information quickly to find good enough solutions.
Comparison of relational and attribute-IEEE-1999-published ...butest
1. The document compares relational data mining methods to attribute-based methods for use in intelligent systems and data mining. Relational methods use first-order logic to represent background knowledge and relationships between objects, while attribute-based methods like neural networks are limited to attribute-value representations.
2. Relational methods have advantages over attribute-based methods for applications that require expressing complex logical relationships and background knowledge. They can also better handle sparse data. However, existing inductive logic programming systems for relational data mining are relatively inefficient for numerical data.
3. The paper proposes a hybrid relational data mining technique called MMDR that combines inductive logic programming with probabilistic inference. This allows it to efficiently handle
Artificial neural networks (ANNs) are computing systems inspired by biological neural networks. ANNs can learn complex patterns and make predictions based on large amounts of data. The document discusses the basic structure and functioning of ANNs, including their ability to learn through adjustment of synaptic weights between neurons. It also describes several common types of ANNs, focusing on perceptrons and multi-layer perceptrons.
This document provides a critical review of recurrent neural networks for sequence learning. It begins with an abstract summarizing the paper. It then discusses why recurrent neural networks are well-suited for modeling sequential data compared to other models like feedforward neural networks and Markov models. Specifically, it notes that RNNs can capture long-range temporal dependencies, unlike models with a finite context window. It also explains that RNNs can represent a vast number of states using real-valued activations, unlike discrete state Markov models.
This document summarizes a research paper that proposes a new neural network algorithm called C-Mantec. C-Mantec adds competition between neurons using a thermal perceptron learning rule. This allows existing neurons to continue learning even as new neurons are added. The algorithm was tested on a diabetes dataset and shown to generate compact neural network architectures with good generalization capabilities. It was implemented in an FPGA using Xilinx ISE for synthesis and ModelSim for simulation. The output was obtained over three clock cycles, first loading inputs/weights, then multiplying/accumulating weights, and finally checking the output against a threshold. The study showed C-Mantec can effectively model glucose level fluctuations to determine if a patient has diabetes.
This document summarizes an introductory seminar presentation on evaluating different machine learning algorithms in wireless sensor networks. The presentation discusses the problem statement, objectives, introduction, literature review, block diagram, expected outputs, future applications, and plan of work. The literature review summarizes past research evaluating neural networks, genetic algorithms, and other machine learning methods for optimizing data transmission and increasing sensor node lifetime in wireless sensor networks. The block diagram shows the overall framework. The objectives are to optimize data transmission using machine learning and increase sensor node lifetime.
DiscoversNet is an adaptive simulation-based learning environment for designing neural networks. It contains a knowledge-based neural network consultant module to provide educational guidance during the neural network design process. The consultant module represents domain knowledge using a knowledge-based neural network and provides advice to users based on their current understanding level. The system allows users to build neural network simulators through interactive manipulation of neural network components and receives feedback to correct misconceptions.
IRJET-Breast Cancer Detection using Convolution Neural NetworkIRJET Journal
This document discusses using a convolutional neural network (CNN) to detect breast cancer from medical images. CNNs are a type of deep learning model that can learn image features without manual feature engineering. The proposed system would take a sample medical image as input, preprocess it, and compare it to images in a database labeled as cancerous or non-cancerous. If cancer is detected, the system would determine the cancer stage and recommend appropriate treatment. The CNN model would be built and trained using libraries like Keras, TensorFlow, and Numpy to classify images and detect breast cancer at early stages for better treatment outcomes.
The document is a dissertation submitted to Gujarat University in partial fulfillment of a Master's degree in Computer Application, which discusses character recognition using neural networks. It provides an index of the contents including the introduction to neural networks, their architecture and applications, an introduction to character recognition, the use of Matlab and its neural network toolbox, a literature survey, the proposed work on digit recognition, potential enhancements, and conclusions. The dissertation was submitted by Sachinkumar M. Bharadva and Dhara Solanki under the guidance of their internal guide Mr. Sandeep R. Vasant at the AES Institute of Computer Studies.
Employing Neocognitron Neural Network Base Ensemble Classifiers To Enhance Ef...cscpconf
This paper presents an ensemble of neo-cognitron neural network base classifiers to enhance
the accuracy of the system, along the experimental results. The method offers lesser
computational preprocessing in comparison to other ensemble techniques as it ex-preempts
feature extraction process before feeding the data into base classifiers. This is achieved by the
basic nature of neo-cognitron, it is a multilayer feed-forward neural network. Ensemble of such
base classifiers gives class labels for each pattern that in turn is combined to give the final class
label for that pattern. The purpose of this paper is not only to exemplify learning behaviour of
neo-cognitron as base classifiers, but also to purport better fashion to combine neural network
based ensemble classifiers.
This document contains summaries of several papers related to artificial intelligence and neural networks:
1. The first paper discusses using recurrent neural networks to plan robot motions in variable environments.
2. The second paper describes using neural network models to classify brain signals and mind states from EEG data.
3. The third paper proposes using coarticulation composite models in acoustic-phonetic decoding to improve recognition rates beyond phonemes, diphones, and triphones.
Artificial Neural Network and its Applicationsshritosh kumar
Abstract
This report is an introduction to Artificial Neural
Networks. The various types of neural networks are
explained and demonstrated, applications of neural
networks like ANNs in medicine are described, and a
detailed historical background is provided. The
connection between the artificial and the real thing is
also investigated and explained. Finally, the
mathematical models involved are presented and
demonstrated.
Key Management Schemes for Secure Communication in Heterogeneous Sensor NetworksIDES Editor
Hierarchical Sensor Network organization is
widely used to achieve energy efficiency in Wireless Sensor
Networks(WSN). To achieve security in hierarchical WSN,
it is important to be able to encrypt the messages sent
between sensor nodes and its cluster head. The key
management task is challenging due to resource constrained
nature of WSN. In this paper we are proposing two key
management schemes for hierarchical networks which
handles various events like node addition, node compromise
and key refresh at regular intervals. The Tree-Based
Scheme ensures in-network processing by maintaining some
additional intermediate keys. Whereas the CRT-Based
Scheme performs the key management with minimum
communication and storage at each node.
The document provides an overview of neural networks including:
- Their history from early models in the 1940s to the breakthrough of backpropagation in the 1980s.
- What a neural network is and how it works at the level of individual neurons and when connected together.
- Common applications of neural networks like prediction, classification, and clustering.
- Key considerations in choosing an appropriate neural network architecture and training data for a given problem.
The document discusses artificial neural networks (ANNs). It defines ANNs as computational models inspired by biological neural networks. The basic structure and types of ANNs are explained, including feed forward and feedback networks. The document also covers ANN learning methods like supervised, unsupervised, and reinforcement learning. Applications of ANNs span various domains like aerospace, automotive, military, electronics, and more. While ANNs can perform complex tasks, they require extensive training and processing power for large networks.
Artificial neural networks are fundamental means for providing an attempt at modelling the information
processing capabilities of artificial nervous system which plays an important role in the field of cognitive
science. This paper focuses the features of artificial neural networks studied by reviewing the existing research
works, these features were then assessed and evaluated and comparative analysis. The study and literature
survey metrics such as functional capabilities of neurons, learning capabilities, style of computation, processing
elements, processing speed, connections, strength, information storage, information transmission,
communication media selection, signal transduction and fault tolerance were used as basis for comparison. A
major finding in this paper showed that artificial neural networks served as the platform for neuron computing
technology in the field of cognitive science.
Neural Networks for Pattern RecognitionVipra Singh
- Neural networks are computing systems inspired by biological neural networks in the brain that can be used for pattern recognition. An artificial neuron receives multiple inputs and produces a single output. Neural networks are trained to recognize complex patterns and identify categories.
- An important application of neural networks is pattern recognition, where a network is trained to associate input patterns with output categories. Recent advances include using neural networks for tasks like predicting student performance, medical diagnosis, and analyzing customer interactions. Neural networks are also being used increasingly in business for applications like predictive analytics and artificial intelligence.
This document is a preface to a book about neural networks. It provides an overview of the book's contents and objectives. The book aims to present a variety of standard neural network architectures along with their training algorithms and examples of applications. It is intended as both a textbook and reference for students and researchers interested in using neural networks. The preface outlines the scope and organization of the material covered in the book.
Army Study: Ontology-based Adaptive Systems of Cyber DefenseRDECOM
The U.S. Army Research Laboratory is part of the U.S. Army Research, Development and Engineering Command, which has the mission to ensure decisive overmatch for unified land operations to empower the Army, the joint warfighter and our nation. RDECOM is a major subordinate command of the U.S. Army Materiel Command.
Neural Network Classification and its Applications in Insurance IndustryInderjeet Singh
This document summarizes a neural networks project report on using neural networks for classification in the insurance industry. The report discusses extracting rules from trained neural networks, using neural networks to predict customer retention and pricing policies. It also discusses using neural networks to detect auto insurance fraud by identifying important fraud indicators.
This document provides an overview of artificial neural networks (ANNs). It discusses how ANNs are inspired by biological neural networks and consist of interconnected artificial neurons that process information. The document describes common ANN architectures like multilayer perceptrons and radial basis function networks. It also summarizes different ANN learning paradigms such as supervised, unsupervised, and reinforcement learning. Specific learning rules and algorithms are mentioned, including the perceptron rule, Hebbian learning, competitive learning, and backpropagation. Applications of ANNs discussed include pattern recognition, clustering, prediction, and data compression.
Nature Inspired Reasoning Applied in Semantic Webguestecf0af
1) Neural networks are computational structures inspired by biological neural networks and have been successfully used to solve complex tasks like image recognition and natural language processing.
2) Neural networks consist of interconnected nodes that perform simple mathematical functions to produce outputs. The connections between nodes and their weights can be modified through training to solve problems.
3) Nature inspired algorithms like neural networks are well-suited for semantic web problems because they can process large amounts of information quickly to find good enough solutions.
Comparison of relational and attribute-IEEE-1999-published ...butest
1. The document compares relational data mining methods to attribute-based methods for use in intelligent systems and data mining. Relational methods use first-order logic to represent background knowledge and relationships between objects, while attribute-based methods like neural networks are limited to attribute-value representations.
2. Relational methods have advantages over attribute-based methods for applications that require expressing complex logical relationships and background knowledge. They can also better handle sparse data. However, existing inductive logic programming systems for relational data mining are relatively inefficient for numerical data.
3. The paper proposes a hybrid relational data mining technique called MMDR that combines inductive logic programming with probabilistic inference. This allows it to efficiently handle
Artificial neural networks (ANNs) are computing systems inspired by biological neural networks. ANNs can learn complex patterns and make predictions based on large amounts of data. The document discusses the basic structure and functioning of ANNs, including their ability to learn through adjustment of synaptic weights between neurons. It also describes several common types of ANNs, focusing on perceptrons and multi-layer perceptrons.
This document provides a critical review of recurrent neural networks for sequence learning. It begins with an abstract summarizing the paper. It then discusses why recurrent neural networks are well-suited for modeling sequential data compared to other models like feedforward neural networks and Markov models. Specifically, it notes that RNNs can capture long-range temporal dependencies, unlike models with a finite context window. It also explains that RNNs can represent a vast number of states using real-valued activations, unlike discrete state Markov models.
This document summarizes a research paper that proposes a new neural network algorithm called C-Mantec. C-Mantec adds competition between neurons using a thermal perceptron learning rule. This allows existing neurons to continue learning even as new neurons are added. The algorithm was tested on a diabetes dataset and shown to generate compact neural network architectures with good generalization capabilities. It was implemented in an FPGA using Xilinx ISE for synthesis and ModelSim for simulation. The output was obtained over three clock cycles, first loading inputs/weights, then multiplying/accumulating weights, and finally checking the output against a threshold. The study showed C-Mantec can effectively model glucose level fluctuations to determine if a patient has diabetes.
This document summarizes an introductory seminar presentation on evaluating different machine learning algorithms in wireless sensor networks. The presentation discusses the problem statement, objectives, introduction, literature review, block diagram, expected outputs, future applications, and plan of work. The literature review summarizes past research evaluating neural networks, genetic algorithms, and other machine learning methods for optimizing data transmission and increasing sensor node lifetime in wireless sensor networks. The block diagram shows the overall framework. The objectives are to optimize data transmission using machine learning and increase sensor node lifetime.
This document summarizes an introductory seminar presentation on evaluating different machine learning algorithms in wireless sensor networks. The presentation discusses the problem statement, objectives, introduction, literature review, block diagram, expected outputs, future applications, and plan of work. The literature review summarizes past research evaluating neural networks, genetic algorithms, and machine learning methods for tasks like energy-efficient routing, localization, and reducing data transmission in wireless sensor networks. The objectives are to optimize information transmission using machine learning, detect redundancies, increase sensor lifetime, and extend application lifetime.
A Time Series ANN Approach for Weather Forecastingijctcm
Weather forecasting is most challenging problem around the world. There are various reason because of its experimented values in meteorology, but it is also a typical unbiased time series forecasting problem in scientific research. A lots of methods proposed by various scientists. The motive behind research is to predict more accurate. This paper contribute the same using artificial neural network (ANN) and simulated in MATLAB to predict two important weather parameters i.e. maximum and minimum temperature. The model has been trained using past 60 years of real data collected from(1901-1960) and tested over 40 years to forecast maximum and minimum temperature. The results based on mean square error function (MSE) confirm, this model which is based on multilayer perceptron has the potential to successful application to weather forecasting
This article provides an introduction to artificial neural networks (ANNs) and presents guidelines for designing effective ANN solutions. It discusses the key components of ANNs, including their biological inspiration, history, and different types of learning algorithms. The article emphasizes that successful ANN development requires extensive domain knowledge engineering and following best practices for selecting input variables, learning methods, architecture, and training samples. Specifically, it recommends knowledge-based input selection, choosing appropriate learning algorithms based on the data type, designing network topology based on the algorithm, and selecting optimal training set sizes, especially for time series problems. Overall, the article stresses that incorporating domain expertise at each design step is essential for building ANNs that generalize well to new problems.
The document describes DiscoverNet, an adaptive simulation-based learning environment for designing neural networks. DiscoverNet allows users to build neural network simulators by manipulating neural objects. It includes a consultant module to provide educational guidance during the design process. The consultant reproduces the user's initial neural network design and identifies any issues to help the user correct misconceptions. Experimental results show that DiscoverNet supports learning and helps users develop an understanding of neural network mechanisms.
Model of Differential Equation for Genetic Algorithm with Neural Network (GAN...Sarvesh Kumar
The work is carried on the application of differential equation (DE) and its computational technique of genetic algorithm and neural (GANN) in C#, which is frequently used in globalised world by human wings. Diagrammatical and flow chart presentation is the major concerned for easy undertaking of these two concepts with indication of its present and future application is the new initiative taken in this paper along with computational approaches in C#. Little observation has been also pointed during working, functioning and development process of above algorithm in C# under given boundary value condition of DE for genetic and neural. Operations of fitness function and Genetic operations were completed for behavioural transmission of chromosome.
Time Series Forecasting Using Novel Feature Extraction Algorithm and Multilay...Editor IJCATR
Time series forecasting is important because it can often provide the foundation for decision making in a large variety of fields. A tree-ensemble method, referred to as time series forest (TSF), is proposed for time series classification. The approach is based on the concept of data series envelopes and essential attributes generated by a multilayer neural network... These claims are further investigated by applying statistical tests. With the results presented in this article and results from related investigations that are considered as well, we want to support practitioners or scholars in answering the following question: Which measure should be looked at first if accuracy is the most important criterion, if an application is time-critical, or if a compromise is needed? In this paper demonstrated feature extraction by novel method can improvement in time series data forecasting process
Pattern Recognition using Artificial Neural NetworkEditor IJCATR
An artificial neural network (ANN) usually called neural network. It can be considered as a resemblance to a paradigm
which is inspired by biological nervous system. In network the signals are transmitted by the means of connections links. The links
possess an associated way which is multiplied along with the incoming signal. The output signal is obtained by applying activation to
the net input NN are one of the most exciting and challenging research areas. As ANN mature into commercial systems, they are likely
to be implemented in hardware. Their fault tolerance and reliability are therefore vital to the functioning of the system in which they
are embedded. The pattern recognition system is implemented with Back propagation network and Hopfield network to remove the
distortion from the input. The Hopfield network has high fault tolerance which supports this system to get the accurate output.
This document provides an overview and summary of a student project report on simulating a feed forward artificial neural network in C++. The report includes an abstract, table of contents, list of figures, and 5 chapters that discuss the objectives of the project, provide background on artificial neural networks, describe the design and implementation of a 3-layer feed forward neural network using backpropagation, present the results, and provide references. The design section explains the backpropagation algorithm and provides pseudocode for calculating outputs at each layer. The implementation section provides pseudocode for training patterns and minimizing error.
NEURAL NETWORK FOR THE RELIABILITY ANALYSIS OF A SERIES - PARALLEL SYSTEM SUB...IAEME Publication
Artificial neural networks can achieve high computation rates by employing a massive number of simple processing elements with a high degree of connectivity between the elements. Neural networks with feedback connections provide a computing model capable of exploiting fine- grained parallelism to solve a rich class of complex problems. In this paper we discuss a complex series-parallel system subjected to finite common cause and finite human error failures and its reliability using neural network method.
INVERTIBLE NEURAL NETWORK FOR INFERENCE PIPELINE ANOMALY DETECTIONIJNSA Journal
This study combines research in machine learning and system engineering practices to conceptualize a paradigm-enhancing trustworthiness of a machine learning inference pipeline. We explore the topic of reversibility in deep neural networks and introduce its anomaly detection capabilities to build a framework of integrity verification checkpoints across the inference pipeline of a deployed model. We leverage previous findings and principles regarding several types of autoencoders, deep generative maximumlikelihood training and invertibility of neural networks to propose an improved network architecture for anomaly detection. We hypothesize and experimentally confirm that an Invertible Neural Network (INN) trained as a convolutional autoencoder is a superior alternative naturally suited to solve that task. This remarkable INN’s ability to reconstruct data from its compressed representation and to solve inverse problems is then generalized and applied in the field of Trustworthy AI to achieve integrity verification of an inference pipeline through the concept of an INN-based Trusted Neural Network (TNN) nodes placed around the mission critical parts of the system, as well as the end-to-end outcome verification. This work aspires to enhance robustness and reliability of applications employing artificial intelligence, which are playing increasingly noticeable role in highly consequential decision-making processes across many industries and problem domains. INNs are invertible by construction and tractably trained simultaneously in both directions. This feature has untapped potential to improve the explainability of machine learning pipelines in support of their trustworthiness and is a topic of our current studies.
DESIGN ISSUES ON SOFTWARE ASPECTS AND SIMULATION TOOLS FOR WIRELESS SENSOR NE...IJNSA Journal
In this paper, various existing simulation environments for general purpose and specific purpose WSNs are discussed. The features of number of different sensor network simulators and operating systems are compared. We have presented an overview of the most commonly used operating systems that can be used in different approaches to address the common problems of WSNs. For different simulation environments there are different layer, components and protocols implemented so that it is difficult to compare them. When same protocol is simulated using two different simulators still each protocol implementation differs, since their functionality is exactly not the same. Selection of simulator is purely based on the application, since each simulator has a varied range of performance depending on application.
Parallel multivariate deep learning models for time-series prediction: A comp...IAESIJAI
This study investigates deep learning models for financial data prediction and examines whether the architecture of a deep learning model and time-series data properties affect prediction accuracy. Comparing the performance of convolutional neural network (CNN), long short-term memory (LSTM), Stacked-LSTM, CNN-LSTM, and convolutional LSTM (ConvLSTM) when used as a prediction approach to a collection of financial time-series data is the main methodology of this study. In this instance, only those deep learning architectures that can predict multivariate time-series data sets in parallel are considered. This research uses the daily movements of 4 (four) Asian stock market indices from 1 January 2020 to 31 December 2020. Using data from the early phase of the spread of the Covid-19 pandemic that has created worldwide economic turmoil is intended to validate the performance of the analyzed deep learning models. Experiment results and analytical findings indicate that there is no superior deep learning model that consistently makes the most accurate predictions for all states' financial data. In addition, a single deep learning model tends to provide more accurate predictions for more stable time-series data, but the hybrid model is preferred for more chaotic time-series data.
Writing long sentences is bit boring, but with text prediction in the keyboard technology has made
this simple. Learning technology behind the keyboard is developing fast and has become more accurate.
Learning technologies such as machine learning, deep learning here play an important role in predicting the
text. Current trending techniques in deep learning has opened door for data analysis. Emerging technologies
such has Region CNN, Recurrent CNN have been under consideration for the analysis. Many techniques have
been used for text sequence prediction such as Convolutional Neural Networks (CNN), Recurrent Neural
Networks (RNN), and Recurrent Convolution Neural Networks (RCNN). This paper aims to provide a
comparative study of different techniques used for text prediction.
This document summarizes and evaluates various rule extraction algorithms from trained artificial neural networks. It begins with an introduction explaining the importance of explanation capabilities for neural networks. It then provides a taxonomy for classifying rule extraction approaches based on the expressiveness of the extracted rules, whether the approach takes an open-box or black-box view of the neural network, any specialized training regimes used, the quality of explanations generated, and computational complexity. The document discusses sensitivity analysis as a basic method for understanding neural network relationships before focusing on decompositional and pedagogical rule extraction approaches.
This document proposes a new method for extracting rules from trained multilayer artificial neural networks that can represent rules in both "if-then" and "M of N" formats. The method extracts an intermediate structure called a "generator list" from which both types of rules can be derived. This provides a more generic representation than existing methods that can only output one rule format. The generator list approach avoids preprocessing steps used in other methods that can modify the original network. It uses heuristics to prune the search space when extracting the generator list to address the computational complexity involved.
EXPERIMENTS ON DIFFERENT RECURRENT NEURAL NETWORKS FOR ENGLISH-HINDI MACHINE ...csandit
Recurrent Neural Networks are a type of Artificial Neural Networks which are adept at dealing
with problems which have a temporal aspect to them. These networks exhibit dynamic
properties due to their recurrent connections. Most of the advances in deep learning employ
some form of Recurrent Neural Networks for their model architecture. RNN's have proven to be
an effective technique in applications like computer vision and natural language processing. In
this paper, we demonstrate the effectiveness of RNNs for the task of English to Hindi Machine
Translation. We perform experiments using different neural network architectures - employing
Gated Recurrent Units, Long Short Term Memory Units and Attention Mechanism and report
the results for each architecture. Our results show a substantial increase in translation quality
over Rule-Based and Statistical Machine Translation approaches.
This document summarizes and compares different operating systems and simulation tools for wireless sensor networks. It discusses TinyOS, LiteOS, and MANTIS OS as examples of operating systems designed for wireless sensor networks. It also outlines some of the key features and considerations of simulating wireless sensor networks, such as the need for simulation given the resource constraints of sensor nodes. The document provides an overview of different operating systems and simulation environments used for wireless sensor network research.
Similar to A systematic review on sequence-to-sequence learning with neural network and its models (20)
Redefining brain tumor segmentation: a cutting-edge convolutional neural netw...IJECEIAES
Medical image analysis has witnessed significant advancements with deep learning techniques. In the domain of brain tumor segmentation, the ability to
precisely delineate tumor boundaries from magnetic resonance imaging (MRI)
scans holds profound implications for diagnosis. This study presents an ensemble convolutional neural network (CNN) with transfer learning, integrating
the state-of-the-art Deeplabv3+ architecture with the ResNet18 backbone. The
model is rigorously trained and evaluated, exhibiting remarkable performance
metrics, including an impressive global accuracy of 99.286%, a high-class accuracy of 82.191%, a mean intersection over union (IoU) of 79.900%, a weighted
IoU of 98.620%, and a Boundary F1 (BF) score of 83.303%. Notably, a detailed comparative analysis with existing methods showcases the superiority of
our proposed model. These findings underscore the model’s competence in precise brain tumor localization, underscoring its potential to revolutionize medical
image analysis and enhance healthcare outcomes. This research paves the way
for future exploration and optimization of advanced CNN models in medical
imaging, emphasizing addressing false positives and resource efficiency.
Embedded machine learning-based road conditions and driving behavior monitoringIJECEIAES
Car accident rates have increased in recent years, resulting in losses in human lives, properties, and other financial costs. An embedded machine learning-based system is developed to address this critical issue. The system can monitor road conditions, detect driving patterns, and identify aggressive driving behaviors. The system is based on neural networks trained on a comprehensive dataset of driving events, driving styles, and road conditions. The system effectively detects potential risks and helps mitigate the frequency and impact of accidents. The primary goal is to ensure the safety of drivers and vehicles. Collecting data involved gathering information on three key road events: normal street and normal drive, speed bumps, circular yellow speed bumps, and three aggressive driving actions: sudden start, sudden stop, and sudden entry. The gathered data is processed and analyzed using a machine learning system designed for limited power and memory devices. The developed system resulted in 91.9% accuracy, 93.6% precision, and 92% recall. The achieved inference time on an Arduino Nano 33 BLE Sense with a 32-bit CPU running at 64 MHz is 34 ms and requires 2.6 kB peak RAM and 139.9 kB program flash memory, making it suitable for resource-constrained embedded systems.
Advanced control scheme of doubly fed induction generator for wind turbine us...IJECEIAES
This paper describes a speed control device for generating electrical energy on an electricity network based on the doubly fed induction generator (DFIG) used for wind power conversion systems. At first, a double-fed induction generator model was constructed. A control law is formulated to govern the flow of energy between the stator of a DFIG and the energy network using three types of controllers: proportional integral (PI), sliding mode controller (SMC) and second order sliding mode controller (SOSMC). Their different results in terms of power reference tracking, reaction to unexpected speed fluctuations, sensitivity to perturbations, and resilience against machine parameter alterations are compared. MATLAB/Simulink was used to conduct the simulations for the preceding study. Multiple simulations have shown very satisfying results, and the investigations demonstrate the efficacy and power-enhancing capabilities of the suggested control system.
Neural network optimizer of proportional-integral-differential controller par...IJECEIAES
Wide application of proportional-integral-differential (PID)-regulator in industry requires constant improvement of methods of its parameters adjustment. The paper deals with the issues of optimization of PID-regulator parameters with the use of neural network technology methods. A methodology for choosing the architecture (structure) of neural network optimizer is proposed, which consists in determining the number of layers, the number of neurons in each layer, as well as the form and type of activation function. Algorithms of neural network training based on the application of the method of minimizing the mismatch between the regulated value and the target value are developed. The method of back propagation of gradients is proposed to select the optimal training rate of neurons of the neural network. The neural network optimizer, which is a superstructure of the linear PID controller, allows increasing the regulation accuracy from 0.23 to 0.09, thus reducing the power consumption from 65% to 53%. The results of the conducted experiments allow us to conclude that the created neural superstructure may well become a prototype of an automatic voltage regulator (AVR)-type industrial controller for tuning the parameters of the PID controller.
An improved modulation technique suitable for a three level flying capacitor ...IJECEIAES
This research paper introduces an innovative modulation technique for controlling a 3-level flying capacitor multilevel inverter (FCMLI), aiming to streamline the modulation process in contrast to conventional methods. The proposed
simplified modulation technique paves the way for more straightforward and
efficient control of multilevel inverters, enabling their widespread adoption and
integration into modern power electronic systems. Through the amalgamation of
sinusoidal pulse width modulation (SPWM) with a high-frequency square wave
pulse, this controlling technique attains energy equilibrium across the coupling
capacitor. The modulation scheme incorporates a simplified switching pattern
and a decreased count of voltage references, thereby simplifying the control
algorithm.
A review on features and methods of potential fishing zoneIJECEIAES
This review focuses on the importance of identifying potential fishing zones in seawater for sustainable fishing practices. It explores features like sea surface temperature (SST) and sea surface height (SSH), along with classification methods such as classifiers. The features like SST, SSH, and different classifiers used to classify the data, have been figured out in this review study. This study underscores the importance of examining potential fishing zones using advanced analytical techniques. It thoroughly explores the methodologies employed by researchers, covering both past and current approaches. The examination centers on data characteristics and the application of classification algorithms for classification of potential fishing zones. Furthermore, the prediction of potential fishing zones relies significantly on the effectiveness of classification algorithms. Previous research has assessed the performance of models like support vector machines, naïve Bayes, and artificial neural networks (ANN). In the previous result, the results of support vector machine (SVM) were 97.6% more accurate than naive Bayes's 94.2% to classify test data for fisheries classification. By considering the recent works in this area, several recommendations for future works are presented to further improve the performance of the potential fishing zone models, which is important to the fisheries community.
Electrical signal interference minimization using appropriate core material f...IJECEIAES
As demand for smaller, quicker, and more powerful devices rises, Moore's law is strictly followed. The industry has worked hard to make little devices that boost productivity. The goal is to optimize device density. Scientists are reducing connection delays to improve circuit performance. This helped them understand three-dimensional integrated circuit (3D IC) concepts, which stack active devices and create vertical connections to diminish latency and lower interconnects. Electrical involvement is a big worry with 3D integrates circuits. Researchers have developed and tested through silicon via (TSV) and substrates to decrease electrical wave involvement. This study illustrates a novel noise coupling reduction method using several electrical involvement models. A 22% drop in electrical involvement from wave-carrying to victim TSVs introduces this new paradigm and improves system performance even at higher THz frequencies.
Electric vehicle and photovoltaic advanced roles in enhancing the financial p...IJECEIAES
Climate change's impact on the planet forced the United Nations and governments to promote green energies and electric transportation. The deployments of photovoltaic (PV) and electric vehicle (EV) systems gained stronger momentum due to their numerous advantages over fossil fuel types. The advantages go beyond sustainability to reach financial support and stability. The work in this paper introduces the hybrid system between PV and EV to support industrial and commercial plants. This paper covers the theoretical framework of the proposed hybrid system including the required equation to complete the cost analysis when PV and EV are present. In addition, the proposed design diagram which sets the priorities and requirements of the system is presented. The proposed approach allows setup to advance their power stability, especially during power outages. The presented information supports researchers and plant owners to complete the necessary analysis while promoting the deployment of clean energy. The result of a case study that represents a dairy milk farmer supports the theoretical works and highlights its advanced benefits to existing plants. The short return on investment of the proposed approach supports the paper's novelty approach for the sustainable electrical system. In addition, the proposed system allows for an isolated power setup without the need for a transmission line which enhances the safety of the electrical network
Bibliometric analysis highlighting the role of women in addressing climate ch...IJECEIAES
Fossil fuel consumption increased quickly, contributing to climate change
that is evident in unusual flooding and draughts, and global warming. Over
the past ten years, women's involvement in society has grown dramatically,
and they succeeded in playing a noticeable role in reducing climate change.
A bibliometric analysis of data from the last ten years has been carried out to
examine the role of women in addressing the climate change. The analysis's
findings discussed the relevant to the sustainable development goals (SDGs),
particularly SDG 7 and SDG 13. The results considered contributions made
by women in the various sectors while taking geographic dispersion into
account. The bibliometric analysis delves into topics including women's
leadership in environmental groups, their involvement in policymaking, their
contributions to sustainable development projects, and the influence of
gender diversity on attempts to mitigate climate change. This study's results
highlight how women have influenced policies and actions related to climate
change, point out areas of research deficiency and recommendations on how
to increase role of the women in addressing the climate change and
achieving sustainability. To achieve more successful results, this initiative
aims to highlight the significance of gender equality and encourage
inclusivity in climate change decision-making processes.
Voltage and frequency control of microgrid in presence of micro-turbine inter...IJECEIAES
The active and reactive load changes have a significant impact on voltage
and frequency. In this paper, in order to stabilize the microgrid (MG) against
load variations in islanding mode, the active and reactive power of all
distributed generators (DGs), including energy storage (battery), diesel
generator, and micro-turbine, are controlled. The micro-turbine generator is
connected to MG through a three-phase to three-phase matrix converter, and
the droop control method is applied for controlling the voltage and
frequency of MG. In addition, a method is introduced for voltage and
frequency control of micro-turbines in the transition state from gridconnected mode to islanding mode. A novel switching strategy of the matrix
converter is used for converting the high-frequency output voltage of the
micro-turbine to the grid-side frequency of the utility system. Moreover,
using the switching strategy, the low-order harmonics in the output current
and voltage are not produced, and consequently, the size of the output filter
would be reduced. In fact, the suggested control strategy is load-independent
and has no frequency conversion restrictions. The proposed approach for
voltage and frequency regulation demonstrates exceptional performance and
favorable response across various load alteration scenarios. The suggested
strategy is examined in several scenarios in the MG test systems, and the
simulation results are addressed.
Enhancing battery system identification: nonlinear autoregressive modeling fo...IJECEIAES
Precisely characterizing Li-ion batteries is essential for optimizing their
performance, enhancing safety, and prolonging their lifespan across various
applications, such as electric vehicles and renewable energy systems. This
article introduces an innovative nonlinear methodology for system
identification of a Li-ion battery, employing a nonlinear autoregressive with
exogenous inputs (NARX) model. The proposed approach integrates the
benefits of nonlinear modeling with the adaptability of the NARX structure,
facilitating a more comprehensive representation of the intricate
electrochemical processes within the battery. Experimental data collected
from a Li-ion battery operating under diverse scenarios are employed to
validate the effectiveness of the proposed methodology. The identified
NARX model exhibits superior accuracy in predicting the battery's behavior
compared to traditional linear models. This study underscores the
importance of accounting for nonlinearities in battery modeling, providing
insights into the intricate relationships between state-of-charge, voltage, and
current under dynamic conditions.
Smart grid deployment: from a bibliometric analysis to a surveyIJECEIAES
Smart grids are one of the last decades' innovations in electrical energy.
They bring relevant advantages compared to the traditional grid and
significant interest from the research community. Assessing the field's
evolution is essential to propose guidelines for facing new and future smart
grid challenges. In addition, knowing the main technologies involved in the
deployment of smart grids (SGs) is important to highlight possible
shortcomings that can be mitigated by developing new tools. This paper
contributes to the research trends mentioned above by focusing on two
objectives. First, a bibliometric analysis is presented to give an overview of
the current research level about smart grid deployment. Second, a survey of
the main technological approaches used for smart grid implementation and
their contributions are highlighted. To that effect, we searched the Web of
Science (WoS), and the Scopus databases. We obtained 5,663 documents
from WoS and 7,215 from Scopus on smart grid implementation or
deployment. With the extraction limitation in the Scopus database, 5,872 of
the 7,215 documents were extracted using a multi-step process. These two
datasets have been analyzed using a bibliometric tool called bibliometrix.
The main outputs are presented with some recommendations for future
research.
Use of analytical hierarchy process for selecting and prioritizing islanding ...IJECEIAES
One of the problems that are associated to power systems is islanding
condition, which must be rapidly and properly detected to prevent any
negative consequences on the system's protection, stability, and security.
This paper offers a thorough overview of several islanding detection
strategies, which are divided into two categories: classic approaches,
including local and remote approaches, and modern techniques, including
techniques based on signal processing and computational intelligence.
Additionally, each approach is compared and assessed based on several
factors, including implementation costs, non-detected zones, declining
power quality, and response times using the analytical hierarchy process
(AHP). The multi-criteria decision-making analysis shows that the overall
weight of passive methods (24.7%), active methods (7.8%), hybrid methods
(5.6%), remote methods (14.5%), signal processing-based methods (26.6%),
and computational intelligent-based methods (20.8%) based on the
comparison of all criteria together. Thus, it can be seen from the total weight
that hybrid approaches are the least suitable to be chosen, while signal
processing-based methods are the most appropriate islanding detection
method to be selected and implemented in power system with respect to the
aforementioned factors. Using Expert Choice software, the proposed
hierarchy model is studied and examined.
Enhancing of single-stage grid-connected photovoltaic system using fuzzy logi...IJECEIAES
The power generated by photovoltaic (PV) systems is influenced by
environmental factors. This variability hampers the control and utilization of
solar cells' peak output. In this study, a single-stage grid-connected PV
system is designed to enhance power quality. Our approach employs fuzzy
logic in the direct power control (DPC) of a three-phase voltage source
inverter (VSI), enabling seamless integration of the PV connected to the
grid. Additionally, a fuzzy logic-based maximum power point tracking
(MPPT) controller is adopted, which outperforms traditional methods like
incremental conductance (INC) in enhancing solar cell efficiency and
minimizing the response time. Moreover, the inverter's real-time active and
reactive power is directly managed to achieve a unity power factor (UPF).
The system's performance is assessed through MATLAB/Simulink
implementation, showing marked improvement over conventional methods,
particularly in steady-state and varying weather conditions. For solar
irradiances of 500 and 1,000 W/m2
, the results show that the proposed
method reduces the total harmonic distortion (THD) of the injected current
to the grid by approximately 46% and 38% compared to conventional
methods, respectively. Furthermore, we compare the simulation results with
IEEE standards to evaluate the system's grid compatibility.
Enhancing photovoltaic system maximum power point tracking with fuzzy logic-b...IJECEIAES
Photovoltaic systems have emerged as a promising energy resource that
caters to the future needs of society, owing to their renewable, inexhaustible,
and cost-free nature. The power output of these systems relies on solar cell
radiation and temperature. In order to mitigate the dependence on
atmospheric conditions and enhance power tracking, a conventional
approach has been improved by integrating various methods. To optimize
the generation of electricity from solar systems, the maximum power point
tracking (MPPT) technique is employed. To overcome limitations such as
steady-state voltage oscillations and improve transient response, two
traditional MPPT methods, namely fuzzy logic controller (FLC) and perturb
and observe (P&O), have been modified. This research paper aims to
simulate and validate the step size of the proposed modified P&O and FLC
techniques within the MPPT algorithm using MATLAB/Simulink for
efficient power tracking in photovoltaic systems.
Adaptive synchronous sliding control for a robot manipulator based on neural ...IJECEIAES
Robot manipulators have become important equipment in production lines, medical fields, and transportation. Improving the quality of trajectory tracking for
robot hands is always an attractive topic in the research community. This is a
challenging problem because robot manipulators are complex nonlinear systems
and are often subject to fluctuations in loads and external disturbances. This
article proposes an adaptive synchronous sliding control scheme to improve trajectory tracking performance for a robot manipulator. The proposed controller
ensures that the positions of the joints track the desired trajectory, synchronize
the errors, and significantly reduces chattering. First, the synchronous tracking
errors and synchronous sliding surfaces are presented. Second, the synchronous
tracking error dynamics are determined. Third, a robust adaptive control law is
designed,the unknown components of the model are estimated online by the neural network, and the parameters of the switching elements are selected by fuzzy
logic. The built algorithm ensures that the tracking and approximation errors
are ultimately uniformly bounded (UUB). Finally, the effectiveness of the constructed algorithm is demonstrated through simulation and experimental results.
Simulation and experimental results show that the proposed controller is effective with small synchronous tracking errors, and the chattering phenomenon is
significantly reduced.
Remote field-programmable gate array laboratory for signal acquisition and de...IJECEIAES
A remote laboratory utilizing field-programmable gate array (FPGA) technologies enhances students’ learning experience anywhere and anytime in embedded system design. Existing remote laboratories prioritize hardware access and visual feedback for observing board behavior after programming, neglecting comprehensive debugging tools to resolve errors that require internal signal acquisition. This paper proposes a novel remote embeddedsystem design approach targeting FPGA technologies that are fully interactive via a web-based platform. Our solution provides FPGA board access and debugging capabilities beyond the visual feedback provided by existing remote laboratories. We implemented a lab module that allows users to seamlessly incorporate into their FPGA design. The module minimizes hardware resource utilization while enabling the acquisition of a large number of data samples from the signal during the experiments by adaptively compressing the signal prior to data transmission. The results demonstrate an average compression ratio of 2.90 across three benchmark signals, indicating efficient signal acquisition and effective debugging and analysis. This method allows users to acquire more data samples than conventional methods. The proposed lab allows students to remotely test and debug their designs, bridging the gap between theory and practice in embedded system design.
Detecting and resolving feature envy through automated machine learning and m...IJECEIAES
Efficiently identifying and resolving code smells enhances software project quality. This paper presents a novel solution, utilizing automated machine learning (AutoML) techniques, to detect code smells and apply move method refactoring. By evaluating code metrics before and after refactoring, we assessed its impact on coupling, complexity, and cohesion. Key contributions of this research include a unique dataset for code smell classification and the development of models using AutoGluon for optimal performance. Furthermore, the study identifies the top 20 influential features in classifying feature envy, a well-known code smell, stemming from excessive reliance on external classes. We also explored how move method refactoring addresses feature envy, revealing reduced coupling and complexity, and improved cohesion, ultimately enhancing code quality. In summary, this research offers an empirical, data-driven approach, integrating AutoML and move method refactoring to optimize software project quality. Insights gained shed light on the benefits of refactoring on code quality and the significance of specific features in detecting feature envy. Future research can expand to explore additional refactoring techniques and a broader range of code metrics, advancing software engineering practices and standards.
Smart monitoring technique for solar cell systems using internet of things ba...IJECEIAES
Rapidly and remotely monitoring and receiving the solar cell systems status parameters, solar irradiance, temperature, and humidity, are critical issues in enhancement their efficiency. Hence, in the present article an improved smart prototype of internet of things (IoT) technique based on embedded system through NodeMCU ESP8266 (ESP-12E) was carried out experimentally. Three different regions at Egypt; Luxor, Cairo, and El-Beheira cities were chosen to study their solar irradiance profile, temperature, and humidity by the proposed IoT system. The monitoring data of solar irradiance, temperature, and humidity were live visualized directly by Ubidots through hypertext transfer protocol (HTTP) protocol. The measured solar power radiation in Luxor, Cairo, and El-Beheira ranged between 216-1000, 245-958, and 187-692 W/m 2 respectively during the solar day. The accuracy and rapidity of obtaining monitoring results using the proposed IoT system made it a strong candidate for application in monitoring solar cell systems. On the other hand, the obtained solar power radiation results of the three considered regions strongly candidate Luxor and Cairo as suitable places to build up a solar cells system station rather than El-Beheira.
An efficient security framework for intrusion detection and prevention in int...IJECEIAES
Over the past few years, the internet of things (IoT) has advanced to connect billions of smart devices to improve quality of life. However, anomalies or malicious intrusions pose several security loopholes, leading to performance degradation and threat to data security in IoT operations. Thereby, IoT security systems must keep an eye on and restrict unwanted events from occurring in the IoT network. Recently, various technical solutions based on machine learning (ML) models have been derived towards identifying and restricting unwanted events in IoT. However, most ML-based approaches are prone to miss-classification due to inappropriate feature selection. Additionally, most ML approaches applied to intrusion detection and prevention consider supervised learning, which requires a large amount of labeled data to be trained. Consequently, such complex datasets are impossible to source in a large network like IoT. To address this problem, this proposed study introduces an efficient learning mechanism to strengthen the IoT security aspects. The proposed algorithm incorporates supervised and unsupervised approaches to improve the learning models for intrusion detection and mitigation. Compared with the related works, the experimental outcome shows that the model performs well in a benchmark dataset. It accomplishes an improved detection accuracy of approximately 99.21%.
Software Engineering and Project Management - Software Testing + Agile Method...Prakhyath Rai
Software Testing: A Strategic Approach to Software Testing, Strategic Issues, Test Strategies for Conventional Software, Test Strategies for Object -Oriented Software, Validation Testing, System Testing, The Art of Debugging.
Agile Methodology: Before Agile – Waterfall, Agile Development.
Introduction- e - waste – definition - sources of e-waste– hazardous substances in e-waste - effects of e-waste on environment and human health- need for e-waste management– e-waste handling rules - waste minimization techniques for managing e-waste – recycling of e-waste - disposal treatment methods of e- waste – mechanism of extraction of precious metal from leaching solution-global Scenario of E-waste – E-waste in India- case studies.
Generative AI Use cases applications solutions and implementation.pdfmahaffeycheryld
Generative AI solutions encompass a range of capabilities from content creation to complex problem-solving across industries. Implementing generative AI involves identifying specific business needs, developing tailored AI models using techniques like GANs and VAEs, and integrating these models into existing workflows. Data quality and continuous model refinement are crucial for effective implementation. Businesses must also consider ethical implications and ensure transparency in AI decision-making. Generative AI's implementation aims to enhance efficiency, creativity, and innovation by leveraging autonomous generation and sophisticated learning algorithms to meet diverse business challenges.
https://www.leewayhertz.com/generative-ai-use-cases-and-applications/
Null Bangalore | Pentesters Approach to AWS IAMDivyanshu
#Abstract:
- Learn more about the real-world methods for auditing AWS IAM (Identity and Access Management) as a pentester. So let us proceed with a brief discussion of IAM as well as some typical misconfigurations and their potential exploits in order to reinforce the understanding of IAM security best practices.
- Gain actionable insights into AWS IAM policies and roles, using hands on approach.
#Prerequisites:
- Basic understanding of AWS services and architecture
- Familiarity with cloud security concepts
- Experience using the AWS Management Console or AWS CLI.
- For hands on lab create account on [killercoda.com](https://killercoda.com/cloudsecurity-scenario/)
# Scenario Covered:
- Basics of IAM in AWS
- Implementing IAM Policies with Least Privilege to Manage S3 Bucket
- Objective: Create an S3 bucket with least privilege IAM policy and validate access.
- Steps:
- Create S3 bucket.
- Attach least privilege policy to IAM user.
- Validate access.
- Exploiting IAM PassRole Misconfiguration
-Allows a user to pass a specific IAM role to an AWS service (ec2), typically used for service access delegation. Then exploit PassRole Misconfiguration granting unauthorized access to sensitive resources.
- Objective: Demonstrate how a PassRole misconfiguration can grant unauthorized access.
- Steps:
- Allow user to pass IAM role to EC2.
- Exploit misconfiguration for unauthorized access.
- Access sensitive resources.
- Exploiting IAM AssumeRole Misconfiguration with Overly Permissive Role
- An overly permissive IAM role configuration can lead to privilege escalation by creating a role with administrative privileges and allow a user to assume this role.
- Objective: Show how overly permissive IAM roles can lead to privilege escalation.
- Steps:
- Create role with administrative privileges.
- Allow user to assume the role.
- Perform administrative actions.
- Differentiation between PassRole vs AssumeRole
Try at [killercoda.com](https://killercoda.com/cloudsecurity-scenario/)
Optimizing Gradle Builds - Gradle DPE Tour Berlin 2024Sinan KOZAK
Sinan from the Delivery Hero mobile infrastructure engineering team shares a deep dive into performance acceleration with Gradle build cache optimizations. Sinan shares their journey into solving complex build-cache problems that affect Gradle builds. By understanding the challenges and solutions found in our journey, we aim to demonstrate the possibilities for faster builds. The case study reveals how overlapping outputs and cache misconfigurations led to significant increases in build times, especially as the project scaled up with numerous modules using Paparazzi tests. The journey from diagnosing to defeating cache issues offers invaluable lessons on maintaining cache integrity without sacrificing functionality.
Build the Next Generation of Apps with the Einstein 1 Platform.
Rejoignez Philippe Ozil pour une session de workshops qui vous guidera à travers les détails de la plateforme Einstein 1, l'importance des données pour la création d'applications d'intelligence artificielle et les différents outils et technologies que Salesforce propose pour vous apporter tous les bénéfices de l'IA.
Design and optimization of ion propulsion dronebjmsejournal
Electric propulsion technology is widely used in many kinds of vehicles in recent years, and aircrafts are no exception. Technically, UAVs are electrically propelled but tend to produce a significant amount of noise and vibrations. Ion propulsion technology for drones is a potential solution to this problem. Ion propulsion technology is proven to be feasible in the earth’s atmosphere. The study presented in this article shows the design of EHD thrusters and power supply for ion propulsion drones along with performance optimization of high-voltage power supply for endurance in earth’s atmosphere.
Use PyCharm for remote debugging of WSL on a Windo cf5c162d672e4e58b4dde5d797...shadow0702a
This document serves as a comprehensive step-by-step guide on how to effectively use PyCharm for remote debugging of the Windows Subsystem for Linux (WSL) on a local Windows machine. It meticulously outlines several critical steps in the process, starting with the crucial task of enabling permissions, followed by the installation and configuration of WSL.
The guide then proceeds to explain how to set up the SSH service within the WSL environment, an integral part of the process. Alongside this, it also provides detailed instructions on how to modify the inbound rules of the Windows firewall to facilitate the process, ensuring that there are no connectivity issues that could potentially hinder the debugging process.
The document further emphasizes on the importance of checking the connection between the Windows and WSL environments, providing instructions on how to ensure that the connection is optimal and ready for remote debugging.
It also offers an in-depth guide on how to configure the WSL interpreter and files within the PyCharm environment. This is essential for ensuring that the debugging process is set up correctly and that the program can be run effectively within the WSL terminal.
Additionally, the document provides guidance on how to set up breakpoints for debugging, a fundamental aspect of the debugging process which allows the developer to stop the execution of their code at certain points and inspect their program at those stages.
Finally, the document concludes by providing a link to a reference blog. This blog offers additional information and guidance on configuring the remote Python interpreter in PyCharm, providing the reader with a well-rounded understanding of the process.
Software Engineering and Project Management - Introduction, Modeling Concepts...Prakhyath Rai
Introduction, Modeling Concepts and Class Modeling: What is Object orientation? What is OO development? OO Themes; Evidence for usefulness of OO development; OO modeling history. Modeling
as Design technique: Modeling, abstraction, The Three models. Class Modeling: Object and Class Concept, Link and associations concepts, Generalization and Inheritance, A sample class model, Navigation of class models, and UML diagrams
Building the Analysis Models: Requirement Analysis, Analysis Model Approaches, Data modeling Concepts, Object Oriented Analysis, Scenario-Based Modeling, Flow-Oriented Modeling, class Based Modeling, Creating a Behavioral Model.
AI for Legal Research with applications, toolsmahaffeycheryld
AI applications in legal research include rapid document analysis, case law review, and statute interpretation. AI-powered tools can sift through vast legal databases to find relevant precedents and citations, enhancing research accuracy and speed. They assist in legal writing by drafting and proofreading documents. Predictive analytics help foresee case outcomes based on historical data, aiding in strategic decision-making. AI also automates routine tasks like contract review and due diligence, freeing up lawyers to focus on complex legal issues. These applications make legal research more efficient, cost-effective, and accessible.
A systematic review on sequence-to-sequence learning with neural network and its models
1. International Journal of Electrical and Computer Engineering (IJECE)
Vol. 11, No. 3, June 2021, pp. 2315~2326
ISSN: 2088-8708, DOI: 10.11591/ijece.v11i3.pp2315-2326 2315
Journal homepage: http://ijece.iaescore.com
A systematic review on sequence-to-sequence learning with
neural network and its models
Hana Yousuf1
, Michael Lahzi2
, Said A. Salloum3
, Khaled Shaalan4
1,2,3,4
Faculty of Engineering and IT, the British University in Dubai, United Arab Emirates
3
Research Institute of Sciences and Engineering, University of Sharjah, United Arab Emirates
Article Info ABSTRACT
Article history:
Received Mar 24, 2020
Revised Sep 18, 2020
Accepted Oct 5, 2020
We develop a precise writing survey on sequence-to-sequence learning with
neural network and its models. The primary aim of this report is to enhance
the knowledge of the sequence-to-sequence neural network and to locate the
best way to deal with executing it. Three models are mostly used in
sequence-to-sequence neural network applications, namely: recurrent neural
networks (RNN), connectionist temporal classification (CTC), and attention
model. The evidence we adopted in conducting this survey included utilizing
the examination inquiries or research questions to determine keywords,
which were used to search for bits of peer-reviewed papers, articles, or books
at scholastic directories. Through introductory hunts, 790 papers, and
scholarly works were found, and with the assistance of choice criteria and
PRISMA methodology, the number of papers reviewed decreased to 16.
Every one of the 16 articles was categorized by their contribution to each
examination question, and they were broken down. At last, the examination
papers experienced a quality appraisal where the subsequent range was from
83.3% to 100%. The proposed systematic review enabled us to collect,
evaluate, analyze, and explore different approaches of implementing
sequence-to-sequence neural network models and pointed out the most
common use in machine learning. We followed a methodology that shows
the potential of applying these models to real-world applications.
Keywords:
Connectionist temporal
classifications
Recurrent neural networks
attention models
Sequence-to-sequence models
Systematic review
This is an open access article under the CC BY-SA license.
Corresponding Author:
Said A. Salloum
Research Institute of Sciences and Engineering
University of Sharjah
Academic city Road, P. O.Box 27272 Sharjah, United Arab Emirates
Tel: +971 6 5585000, Fax: +971 6 5585099
Email: ssalloum@sharjah.ac.ae
1. INTRODUCTION
Machine learning (ML) is a logical investigation of calculations and accurate models within
computational frameworks that act without utilizing clear guidelines, depending on examples and surmising.
It is viewed as a subset of computerized reasoning [1-4]. Performing ML includes making a model, which is
prepared on some preparation information and afterward can process extra information to make forecasts [5-7].
Different kinds of models have been utilized and investigated for ML frameworks. These models include
neural networks, decision trees, regression analysis and have a massive application that includes speech and
object recognition [8-15]. The scope of this paper is focused on neural networks and their subsets,
particularly neural networks and sequence-to-sequence learning. Neural systems or connectionist frameworks
are registering frameworks dubiously motivated by the organic neural systems that establish creature
cerebrums. Such frameworks "learn" to perform assignments by thinking about models, for the most part,
2. ISSN: 2088-8708
Int J Elec & Comp Eng, Vol. 11, No. 3, June 2021 : 2315 - 2326
2316
without being modified with task-explicit guidelines. Their applications are massive, and they have been
developed to perform many difficult tasks, mainly including prediction and classification [16].
Sequence-to-Sequence learning is a part of ML and a method of neural networks that is mostly
utilized in language processing models [17-23]. It can be implemented with recurrent neural networks
(RNNs) using encoder-decoder based machine interpretation that maps an input sequence to a yield of a
succession of output sequence with a tag and consideration esteem. The idea is to exploit two RNN that will
cooperate with a unique token and attempt to anticipate the following state arrangement from the past
succession. Sequence-to-Sequence models can also be implemented through what is known as connectionist
temporal classification (CTC) and attention-based models. The sequence-to-sequence model was initially
created by Google for machine translation and was introduced to train the model with a single command. The
model was also made to be easily reproducible and extendable such that the code files were organized in a
measured manner and that simple to expand upon and recreate.
The extent of this paper incorporates neural systems and their subsets, especially recurrent neural
networks (RNN), connectionist temporal classification (CTC), and attention-based models to determine
which is the best-suited approach to implement with sequence-to-sequence learning. This paper aims to
explore the working of sequence-to-sequence learning and their different applications, which is reflected in
the research questions of the paper.
This paper will conduct a systematic literature review on sequence-to-sequence neural network
through exploring different academic research directories to look for peer-reviewed content. More on the
method of conducting this systematic literature review is discussed in Section 3. The structure of this paper is
as: Section 2 will talk about and dissect the accessible writing about RNNs, CTC, and attention-based models
learning; it will explain the working of each network and discuss their strengths and weaknesses. Section 3
will study the examination methodology used to lead the systematic literature review, which incorporates the
detailing and formulation of the research questions. Section 4 will utilize the academic writings to conduct a
quality assessment, respond to the questions addressed, and lastly, section 5 will have a short conclusion to
the paper.
2. LITERATURE REVIEW
Neural networks is an established model of machine learning and has had extensive research done
on it over the years; Sequence-to-sequence neural network is a new learning technique [9, 24, 25]. Despite
this, there is still quite a substantial some of research done on both models and techniques, which will be
expanded on in this section.
2.1. Background
Neural networks are inspired by biological neural networks systems that comprise animals' brains;
they are designed to develop, progress, and solve complex problems that require a high level of
comprehension to perform. There are many types of neural networks that are found to perform extremely
well with such difficult tasks such as speech recognition and machine translation; one of such networks are
the recurrent neural networks (RNN). The working principle behind RNNs is essentially constructed around
neural network models that incorporate an encoder-decoder framework that can be used and trained end-to-
end to map input sequences into output target sequences [26]. A wide-ranging definition of sequence-to-
sequence models can be said to "refers to the broader class of models that include all models that map one
sequence to another" [27]. Thus, by comparing it to the definition of [26], it is clear to see the relation
between the two definitions.
Connectionist temporal classifications (CTC) is a kind of neural system yield related to scoring
capacity, for preparing intermittent neural systems to handle grouping issues where the timing is variable. It
may be utilized for assignments like online penmanship recognition or perceiving phonemes in discourse
sound. CTC was presented in 2006 and alluded to the yields and scoring and is autonomous of the hidden
neural system structure [28]. Historically, CTC has been used for the classification of unsegmented
sequences with RNNs, such as cases of handwriting or speech recognition. RNNs on their own were not
sufficient for the task as their standard neural system target capacities are characterized independently for
each point in the preparation arrangement; basically, RNNs must be prepared to make a progression of
autonomous mark orders.
In order to remove this dependency and enable RNNs to perform this task, the network had to
decode the system outputs as a likelihood appropriation over all possible mark successions, adapted on a
given input grouping. Given this dispersion, a target capacity can be determined that straightforwardly
expands the probabilities of the right marking. Since the target work is differentiable, the system would then
3. Int J Elec & Comp Eng ISSN: 2088-8708
A systematic review on sequence-to-sequence learning with neural network and its models (Hana Yousuf)
2317
be able to be prepared with standard backpropagation through time [29]. Thus, using this concept, therefore,
use of RNNs in this way was known as CTC.
The attention mechanism is a type of neural network that allows the decoder aspect of the network
to focus on certain parts of the sequence while the output is generated. Attention-based models help remove
any dependencies on variable-length inputs without compressing them into fixed vectors by using variable-
length memory where then, the model is free to use this memory in a truly adaptable way to create the output
succession. In addition, various pieces of memory can be obtained at multiple yield time steps. These models
are well-persuaded in light of the fact that data is lost by compacting long factor length groupings into a
fixed-size vector, and choosing the pressure is an extra assignment to comprehend [30].
2.2. Sequence-to-sequence models
There are several approaches used to implement sequence-to-sequence algorithm models. The most
common models are the connectionist temporal classification (CTC), RNNs, and attention-based model.
2.2.1. Connectionist temporal classification
The CTC algorithm proposed by [28]. This algorithm is a method of preparing start to finish models
without a requirement of casing level arrangement of the objective names for a preparation articulation. CTC
defines the probability of the output condition, estimated to use recurrent neural networks, simply known as
encoders [31]. In addition, CTC uses for sequence-to-sequence method to help to address any issues related
to the length of the output labels when it is shorter than the length of the input sequence since "CTC
introduces a special blank label and allows for repetition of labels to force the output and input sequences to
have the same length. CTC outputs are usually dominated by blank symbols" [32]. This gives CTC a major
advantage when using sequence-to-sequence models in many applications such as translation.
2.2.2. Recurrent neural networks
The idea behind sequence-to-sequence models using the RNN approach utilizes two RNN that will
cooperate with a unique token and attempt to anticipate the following state arrangement from the past
succession. The RNN transducer differs on the encoder usage from the CTC alignment model by different
repeat lease expectation arrangement over the output sequences. Instinctively, the encoder can be thought of
as an acoustic model, while the expectation arranges practically equivalent to a language model. The
expectation arrange gets as info and processes an output vector, subject to the whole sequence of labels [31].
2.2.3. Attention model
It is a consideration-based model contains an encoder organize, as in the RNN transducer model. In
any case, in contrast to the RNN transducer, in which the encoder and the expectation arrange are displayed
autonomously and consolidated in the joint system, a consideration based model uses a solitary decoder to
deliver an appropriation over the marks molded on the full grouping of past forecasts and the acoustics [31].
The decoder network consists of several recurrent layers. The attention aspect of the model puts a higher
weight on certain layers to produce an output using the end-to-end sequence method.
3. METHODOLOGY
The principle of research methodology is developed based on a systematic literature review. The
paper follows the systematic review methodology illustrated by [33] to direct the deliberate literature audit.
The reason for selecting the systematic review for sequence-to-sequence neural network is that no systematic
review focuses on sequence-to-sequence neural network usage, limitations, and applications. Moreover, this
methodology enabled us to collect, evaluate, analyze, and explore different approaches to implementing
sequence-to-sequence neural network models and find the most common use in machine learning. The initial
step to this method is to figure out the research hypothesis of our paper.
3.1. Research hypothesis
The research hypothesis developed for the paper were as:
1. What are the different applications of sequence-to-sequence neural network models?
2. How has this model been implemented and developed?
3. What are the advantages and limitations of implementing sequence-to-sequence models?
4. What is the best model to approach sequence-to-sequence implementation?
5. What are the countries that contributed to the development and implementation of sequence-to-sequence?
Table 1 shows the research hypothesis, including their motivation.
4. ISSN: 2088-8708
Int J Elec & Comp Eng, Vol. 11, No. 3, June 2021 : 2315 - 2326
2318
Table 1. Research motivation and hypothesis
RQ# Research Hypothesis Motivation
RQ1 What are the different
applications of sequence-to-
sequence models?
This question helps gain a broader understanding of all the applications of sequence-
to-sequence models to relieve a lot of ambiguity surrounding the definition. This will
later help identify the best possible approach to implement the model based on the
best advantages it has on each application.
RQ2 How has this model been
implemented and developed?
This question allows a deeper understanding of the development of the model.
Moreover, It will highlight its shortcomings to identify the additional work needs to be
done on the field and what is the best and least problematic approach to be used with
common machine learning applications like speech recognition and translation.
RQ3 What are the advantages and
limitations of implementing
sequence-to-sequence models?
This question addresses the significance of the model and how the installation has
helped in relieving some of the problems that were involved in the field of machine
learning prior to its development. Moreover, this helps to highlight some challenges
that are still not solved and new challenges that developed with the new model.
RQ4 What is the best model to
approach sequence-to-sequence
implementation?
This question will highlight the advantages and disadvantages of every approach. It
will help choose the best approach for the purposes of suggesting a standard to use
when implementing this model in common machine learning applications.
RQ5 What are the countries that
contributed to the development
and implementation of sequence-
to-sequence?
This question will highlight the most countries contributed to the implementation and
development Sequence-to-Sequence
3.2. Research strategy
The principle of research strategy in this paper is to conduct a careful checking on the subsequent
database, peer-reviewed journals, and periodicals. Most of the papers utilized from arXiv, Google Scholar,
Science Direct, IEEE Xplore, and Springer complete journals. When the most significant research registries
were chosen, important keywords retrieved from the research hypothesis were utilized in search through
pertinent titles and modified works of papers and articles. A Boolean technique adapted to string together the
important search terms to locate the most significant peer-evaluated items for this paper. Specific
perspectives were viewed when utilizing the search terms, such as equivalent words or plurals of terms.
Table 2 shows the search terms used in various databases and the outcomes it accomplished. Overall, there
were 871 papers appeared through the initial searching stage. From those, 86 papers appeared to be
duplicated, and five papers were found through citations of found academic papers, which brought the total
number of papers to 790. These papers were refined. The final pieces of literature used in this paper were
screening according to the selection criteria and the preferred reporting items for systematic reviews and
meta-analyses (PRISMA) Statement [34]. Figure 1 shows the PRISMA flowchart.
Table 2. Boolean technique search results
Database Search String Number of results
Google Scholar ("Connectionist temporal classifications") and ("sequence-to-sequence") and
("RNN") and ("CTC")
56
Science Direct (("Attention model" or "attention mechanism") and ("sequence-to-Sequence")) 68
IEEE Xplore "Sequence-to-sequence" and "neural network" and "challenges" or "limitation" 284
Springer Complete Journals ("Sequence-to-sequence") and ("recurrent neural networks" or "connectionist
temporal classifications" or "attention model")
463
3.3. Inclusion and exclusion criteria
Inclusion and exclusion criteria were chosen criteria to measure the most formal writing for the
extent of this paper. In view of these criteria, the papers which followed the research perspective criteria were
included for the extent of research. Table 3 shows the inclusion and exclusion criteria.
Table 3. Inclusion and exclusion criteria
Inclusion criteria Excision criteria
Must be peer-reviewed journal papers, articles, or books Articles are written in another language
Should include the development and implementation of the
sequence-to-sequence model and neural networks
Papers focus on specialized parts or working the sequence-to-
sequence model and neural network
Must be written in English Discuses as a subtopic in Sequence-to-Sequence, RNN, CTC,
and attention models.
Academic papers that failed to meet the mentioned criteria were excluded from research. The initial
screening was done on all the extracted 790 records in order to narrow the result. These records were
5. Int J Elec & Comp Eng ISSN: 2088-8708
A systematic review on sequence-to-sequence learning with neural network and its models (Hana Yousuf)
2319
compared against pre-selection criteria that included refining the search fields to engineering and computer
science, limiting the results to the no older than ten years. This resulted in the evaluated numbers to decrease
to 413 papers, which were assessed by the selection criteria mentioned. Figure 1 represents the process for
both the pre-selection and the selection criteria. This process resulted in 16 papers to be included in the
systematic review in sequence-to-sequence neural network.
Figure 1. PRISMA flowchart
3.4. Quality assessment
Inclusion quality assessment is one of the most essential and critical parts of any systematic
review [35-39]. The assessment quality assurance checklist for our systematic review consists of 6 questions
for the 16 chosen papers, as shown in Table 4. The scoring of this process is done based on the work of [40]
as: a 'Yes' to the question of the quality assessment is indicated by a 1, a 'No' was indicated by a 0, and a
'Partially' was indicated by a 0.5. As seen by the results in Table 5, all the chosen papers have passed the
quality assessment.
Table 4. Quality assurance questions
Ques. Quality Assurance Question
1 Are the research aims specified clearly?
2 Is the information presented clear and concise?
3 Does the study provide enough explanation of its methodology?
4 Do the findings of the study add to the understanding of sequence-to-sequence models?
5 Are the conclusions clearly identified?
6 Are the conclusions logical and concise with the flow of the paper?
6. ISSN: 2088-8708
Int J Elec & Comp Eng, Vol. 11, No. 3, June 2021 : 2315 - 2326
2320
Table 5. Quality assessment results
Study Q1 Q2 Q3 Q4 Q5 Q6 Total Percentage
S1 1 1 0.5 1 1 1 5.5 91.67%
S2 1 1 1 1 0.5 1 5.5 91.67%
S3 1 1 1 0 1 1 5 83.33%
S4 1 1 1 0.5 1 0.5 5 83.33%
S5 1 1 1 0.5 1 1 5.5 91.67%
S6 1 1 0.5 0.5 1 1 5 83.33%
S7 1 1 1 1 1 1 6 100%
S8 1 1 0.5 1 1 0.5 5 83.33%
S9 1 1 1 0 1 1 5 83.33%
S10 1 1 1 0 1 1 5 83.33%
S11 1 1 1 1 1 1 6 100%
S12 1 1 1 1 0.5 1 5.5 91.67%
S13 1 1 0.5 1 0.5 1 5.5 91.67%
S14 1 0.5 1 0.5 1 1 5.5 91.67%
S15 1 1 0 1 1 1 5 83.33%
S16 1 1 1 1 1 0.5 5.5 91.67%
4. RESULTS AND DISCUSSION
Utilizing the research procedure outlined in the previous section, the research questions that were
tended to each paper are classified and analyzed based on their contribution to each question.
4.1. Classifications and analysis of studies
By studying all the 16 research papers included in the systematic review, a classification system was
comprised based on their contribution to answering the research questions. Markings were made if the main
focus of the paper was related to a particular category. Most papers, for example, talked in brief about the
different applications of sequence-to-sequence (seq2seq) models. However, only studies that were mainly
focused on a certain application or discussed several applications in depth were categorized as 'Applications
of seq2seq'. The classification results can be seen in Table 6. Additionally, each study was analyzed in detail,
and the results of this detailed study are outlined in Table 7 (see in Appendix). Figure 2 shows the publication
distributions country wise. As the results show, there has been an increase in interest in this topic over the
last two decades, as indicated by the increase in a number of publications since 1990. Most of this research is
focused on the US, as mentioned.
Figure 2. Publication distribution country wise
4.2. Quality assessment results
Using the questions outlined in Table 4, a score was given to each of the 16 studies used in this
paper. The maximum score that a paper can have is 6. The results of each paper are presented in Table 6.
7. Int J Elec & Comp Eng ISSN: 2088-8708
A systematic review on sequence-to-sequence learning with neural network and its models (Hana Yousuf)
2321
Table 6. Classification of literature review
Study Source Applications
of seq2seq
Seq2seq via
RNN
Seq2seq
via CTC
Seq2seq via
attention models
Challenges of
seq2seq
S1 [41] x x
S2 [32] x x
S3 [28] x x x
S4 [42] x x
S5 [43] x x
S6 [27] x x
S7 [16] x x x
S8 [31] x x x
S9 [26] x x
S10 [44] x
S11 [45] x x
S12 [8] x x
S13 [46] x x
S14 [29] x x
S15 [47] x x
S16 [48] x x
4.3. Answers to research questions
a. RQ1. What are the different applications of the sequence-to-sequence neural network model?
As seen from Error! Reference source not found., 11, or 68.75%, of the studies were relevant to
the applications of sequence-to-sequence neural network models, indicating not only the relevance of this
question but also its widespread interest in the field. The general consensus was that sequence-to-sequence
models were best utilized for speech recognition and general linguistics, as suggested by [16, 31, 48]. In
addition, sequence-to-sequence models can be used for video to text conversion [46] and handling large
vocabularies, optimizing translation performance, and multi-lingual learning [27].
b. RQ2. How has this model been implemented and developed?
All 16 of the papers used in the systematic review mentioned different approaches to implementing
and developing the sequence-to-sequence neural network models. For example: R. Prabhavalkar et al. [31]
summarises three methods of implementation, which include: RNNs, Connectionist Temporal Classifications
(CTC), and Attention models.
c. RQ3. What are the advantages and limitations of implementing sequence-to-sequence models?
I. Sutskever et al. [8] and Y. H. Chan et al. [41] talked mostly about implementations using RNNs
and discussed many advantages and disadvantages of applying this model, while in [28, 42] discussed the
implementation through CTC in details with the limitations in implementation. Similarly, in [32, 45]
discussed the limitations of the attention model.
d. RQ4. What is the best model to approach sequence-to-sequence implementation?
The majority of the papers (62.5%) talk about RNNs and their implementation, limitations, and
endorsement. Notably, the work of [31] compared the three different approaches and found the most
promising approach to be "the RNN transducer, attention-based models, and a novel RNN transducer
augmented with attention."
5. CONCLUSION
In conclusion, the paper aimed to conduct a systematic review on the topic of the sequence-to-
sequence neural network and its models. The main aim of the review to gain insight into the sequence-to-
sequence neural network models and to find the best approach to implement it. Three such approaches were
found: through recurrent neural networks, connectionist temporal classifications (CTC), and attention
models. The research question derived for the literature review were encompassing the applications of
sequence-to-sequence models, their advantages and disadvantages, as well as the best implementation
approach for them. The procedure done to conduct this systematic literature review included using the
research questions to derive keywords that were then used to look at the subsequent database, peer-reviewed
journals, and periodicals. Most of papers utilized from arXiv, Google Scholar, Science Direct, IEEE Xplore
and Springer complete journals. Through initial searches, 790 papers and academic works were found, and
with the help of selection criteria and PRISMA procedure, the number of papers reviewed in this paper was
reduced to 16. Each of the 16 papers was categorized by their contribution to each research question, and
they were analyzed. Finally, the research papers underwent a quality assessment where the resulting range
was from 83.3% to 100%.
8. ISSN: 2088-8708
Int J Elec & Comp Eng, Vol. 11, No. 3, June 2021 : 2315 - 2326
2322
APPENDIX
Table 7. Analysis of the literature review
Study Source Purpose Country Database Findings
S1 [41] The authors suggested and
experimented with an alternative
approach to embedded emotional
information at the encoder stage
of sequence to sequence based
emotional generation.
Hong Kong IEEE
Xplore
Different methods were tested on
emotional encoding information for
sequence to sequence generation
response and evaluated the result, which
was found to have a positive effect on a
sentence level.
S2 [32] The paper proposed a new
method used in neural speech
recognition by aligning the
attention modeling within the
CTC framework
Canada IEEE
Xplore
The proposed method boosted the end-to-
end acoustic-to-word CTC model and
achieved better WER than the traditional
context-dependent phoneme CTC model
decoded with a very large-sized language
model
S3 [28] The paper presented a novel
method for training RNNs to
label unsegmented sequences.
"An experiment on the TIMIT
speech corpus demonstrates its
advantages over both a baseline
HMM and a hybrid HMM-RNN".
Switzerland ACM
Digital
Libraries
The method is derived from probabilistic
principles and fits the framework of the
neural network classifier. It allows the
network to be trained directly for
sequence labeling by removing the
requirement of pre-segmented data.
S4 [42] The authors developed a
technique to base the RNN-T
system towards a specific
keyword.
USA IEEE
Xplore
The paper developed a streaming
keyword by using (RNN-T) model to
predict either phonemes or graphemes as
sub-word, which allows detecting random
expressions.
S5 [43] The authors proposed an artificial
Bangla text generator with
LSTM and model it to validate
the accuracy of the text
generators.
Bangladesh Science
Direct
The paper worked with RNN structures and
LTSM networks to be able to fulfill their
purpose. They were able to "model for
achieving multi-task sequence to sequence
text generation and multi-way translation like
Bengali articles, caption generation"
S6 [27] The author introduced 'neural
machine translation' or 'neural
sequence-to-sequence models'
USA arXiv The paper covered the basics of neural
machine translation and sequence to
sequence models. It covered several
application such as Handling large
vocabularies, Optimizing translation
performance, and Multi-lingual learning.
S7 [16] This paper carries out a
comprehensive review of articles
that involve a comparative study
of feed forward neural networks
and statistical techniques used for
prediction and classification
problems in various areas of
applications. Tabular presentations
highlighting the important features
of these articles are also provided
India Science
Direct
"Neural networks are being used in areas
of prediction and classification, the areas
where statistical methods have
traditionally been used".
"One of the important advantages of
neural networks cited in the literature is
that it can automatically approximate any
nonlinear mathematical function".
S8 [31] The authors conducted a detailed
evaluation of sequence to
sequence models which are used in
the task of speech recognition such
as connectionist temporal
classification (CTC), the recurrent
neural network (RNN)
transducer, an attention-based
model, and a model which
augments the RNN transducer
with an attention mechanism.
USA Google
Scholar
"[they] compared a number of sequence-
to-sequence modeling approaches on an
LVCSR task. In experimental
evaluations, we find that the RNN
transducer, attention-based models and a
novel RNN transducer augmented with
attention are comparable in performance
to a strong state-of-the-art baseline on a
dictation test set, even when evaluated
without the use of an external pronunciation
or language model"
S9 [26] "We introduce an encoder-
decoder recurrent neural network
model called recurrent neural
aligner (RNA) that can be used for
the sequence to sequence mapping
tasks. Like connectionist temporal
classification (CTC) models, RNA
defines a probability distribution
over target label sequences,
including blank labels
corresponding to each time step
in the input".
USA Research
Gate
"We presented the RNA model, which is
a recurrent neural network model in the
encoder-decoder framework. We applied
it to end-to-end speech recognition and
showed initial experimental results on
YouTube transcription and mobile dictation
tasks. We found that the RNA grapheme
model can not perform as well as CTC
CD phone models for mobile dictation
task where the vocabulary size is larger,
and the training data is relatively smaller."
9. Int J Elec & Comp Eng ISSN: 2088-8708
A systematic review on sequence-to-sequence learning with neural network and its models (Hana Yousuf)
2323
Table 7. Analysis of the literature review (Continue)
Study Source Purpose Country Database Findings
S10 [44] “We propose SAN-CTC, a deep,
fully self-attentional network for
CTC, and show it is tractable and
competitive for end-to-end speech
recognition. We motivate the
architecture for speech, evaluate
the position and down-sampling
approaches, and explore how to
label alphabets (character, phoneme,
subword) affect attention heads and
performance”.
UK IEEE
Xplore
"SAN-CTC trains quickly and
outperforms existing CTC models and
most encoder-decoder models, with
character error rates (CERs) of 4.7% in
1 day on WSJ eval92 and 2.8% in 1
week on LibriSpeech test-clean, with a
fixed architecture and one GPU. Similar
improvements hold for WERs after LM
decoding.
S11 [45] The work is focused on using
sequence to sequence the attention
model by the Google brain team to
generate the abstract of research
papers. Moreover, temporal attention
mechanism has been used in
replacement to the global attention to
cater to the problem of repetitive
words.
Pakistan IEEE
Xplore
The result indicates that the temporal
attention model is a useful method for
generating summaries. Moreover, results
indicate that with the increase in dataset
size, the accuracy of the results also
increases.
S12 [8] The paper uses a multilayered
long short-term memory (LSTM)
to map the input sequence to a
vector of fixed dimensionality.
Then, another deep LSTM to
decode the target sequence from
the vector."
Canada Google
Scholar
The main result is that on an English to
French translation task from the WMT-
14 dataset, the translations produced by the
LSTM achieve a BLEU score of 34.8 on the
entire test set, where the LSTM's BLEU
score was penalized on out-of-vocabulary
words. Additionally, the LSTM did not have
difficulty in long sentences.
S13 [46] "We propose a novel end-to-end
sequence-to-sequence model to
generate captions for videos. For
this, we exploit recurrent neural
net- works, specifically LSTMs,
which have demonstrated state-of-
the-art performance in image
caption generation."
USA IEEE
Xplore
[This paper] constructed descriptions
using a sequence to sequence model,
where frames are first to read
sequentially, and then words are
generated sequentially. This allows us to
handle variable-length input and output
while simultaneously modeling temporal
structure. Our model achieves state-of-
the-art performance on the MSVD dataset,
and outperforms related work on two large
and challenging movie-description
datasets."
S14 [29] "Basic backpropagation, which is
a simple method now being
widely used in areas like pattern
recognition and fault diagnosis, is
reviewed. The basic equations for
backpropagation through time,
and applications to areas like pattern
recognition involving dynamic
systems, systems identification, and
control are discussed."
USA IEEE
Xplore
This paper presents the key equations of
backpropagation, as applied to neural
networks of varying degrees of
complexity. It has also discussed other
papers which elaborate on the
extensions of this method to more
general applications and some of the
tradeoffs involved
S15 [47] "This paper examines the most
popular DNNs approaches:
LSTM, Encoder-Decoder network
and Memory network in sequence
prediction field to handle the
software sequence learning and
prediction task."
China IEEE
Xplore
"Our results demonstrate that attention
mechanism does not fit all seq2seq
problems, especially in a weak mapping
relationship. And additional information
can benefit sequence prediction in
neural networks."
S16 [48] The paper presented a character
level sequence to sequence the
learning method. The author set in
an RNN into an encoder-decoder
framework and generate the
character-level sequence in
representation as an input
China IEEE
Xplore
Experimental results by the authors
confirmed that the proposed approach
achieved performance close to
conventional word and phrase-based
translation systems. The proposed
methods allows to "reads quantized
characters into the translation system,
instead of using a predefined vocabulary
with a limited number of words"
10. ISSN: 2088-8708
Int J Elec & Comp Eng, Vol. 11, No. 3, June 2021 : 2315 - 2326
2324
ACKNOWLEDGEMENTS
This work is a part of a project undertaken at the British University in Dubai.
REFERENCES
[1] S. A. Salloum, M. Alshurideh, A. Elnagar, and K. Shaalan, "Machine Learning and Deep Learning Techniques for
Cybersecurity: A Review," Proceedings of the International Conference on Artificial Intelligence and Computer
Vision (AICV2020), vol. 1153, 2020, pp. 50-57.
[2] S. A. Salloum, M. Alshurideh, A. Elnagar, and K. Shaalan, "Mining in Educational Data: Review and Future
Directions," Proceedings of the International Conference on Artificial Intelligence and Computer Vision
(AICV2020), vol. 1153, 2020, pp. 92-102.
[3] S. A. Salloum, R. Khan, and K. Shaalan, "A Survey of Semantic Analysis Approaches," Proceedings of the
International Conference on Artificial Intelligence and Computer Vision (AICV2020), vol. 1153, 2020, pp. 61-70.
[4] A. Y. Zainal, H. Yousuf, and S. A. Salloum, "Dimensions of Agility Capabilities Organizational Competitiveness
in Sustaining," Proceedings of the International Conference on Artificial Intelligence and Computer Vision
(AICV2020), vol. 1153, 2020, pp. 762-772.
[5] H. AlGhanem, M. Shanaa, S. Salloum, and K. Shaalan, "The Role of KM in Enhancing AI Algorithms and
Systems," Advances is Science, Technology and Engineering System Journal, vol. 5, no. 4, pp. 388-396, 2020.
[6] H. Yousuf, A. Y. Zainal, M. Alshurideh, and S. A. Salloum, "Artificial Intelligence Models in Power System
Analysis," in Artificial Intelligence for Sustainable Development: Theory, Practice and Future Applications,
vol. 912, pp. 231-242, 2020.
[7] M. AlShamsi, S. A. Salloum, M. Alshurideh, and S. Abdallah, "Artificial Intelligence and Blockchain for
Transparency in Governance," in Artificial Intelligence for Sustainable Development: Theory, Practice and Future
Applications, vol. 912, pp. 219-230, 2020.
[8] I. Sutskever, O. Vinyals, and Q.V. Le, "Sequence to sequence learning with neural networks," in Advances in
neural information processing systems, pp. 3104-3112, 2014.
[9] T. -S. Nguyen, S. Stüker, J. Niehues, and A. Waibel, "Improving sequence-to-sequence speech recognition training
with on-the-fly data augmentation," ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and
Signal Processing (ICASSP), Barcelona, Spain, 2020, pp. 7689-7693.
[10] I. Arel, J. B. Looks, A. Ziaei, and M. Lefkowitz, "Reconciliation between simulator and speech recognition output
using sequence-to-sequence mapping," Google Patents, 2020.
[11] M. Blaauw and J. Bonada, "Sequence-to-Sequence Singing Synthesis Using the Feed-Forward Transformer," in
ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP),
Barcelona, Spain, 2020, pp. 7229-7233.
[12] S. A. Salloum, M. Al-Emran, A. A. Monem, and K. Shaalan, "A survey of text mining in social media: facebook
and twitter perspectives," Advances is Science, Technology and Engineering System Journal, vol. 2, no. 1,
pp. 127-133, 2017.
[13] S. A. Salloum, A. Q. AlHamad, M. Al-Emran, and K. Shaalan, “A survey of Arabic text mining,” Intelligent
Natural Language Processing: Trends and Applications, vol. 740, pp. 417-431, 2017.
[14] S. A. Salloum, M. Al-Emran, A. A. Monem, and K. Shaalan, “Using text mining techniques for extracting
information from research articles,” Intelligent Natural Language Processing: Trends and Applications, vol. 740,
pp. 373-397, 2017.
[15] S. A. Salloum, M. Al-Emran, S. Abdallah, and K. Shaalan, "Analyzing the Arab Gulf Newspapers Using Text
Mining Techniques," Proceedings of the International Conference on Advanced Intelligent Systems and
Informatics, vol. 639, 2017, pp. 396-405.
[16] M. Paliwal and U. A. Kumar, "Neural networks and statistical techniques: A review of applications," Expert
Systems with Applications, vol. 36, no. 1, pp. 2-17, 2009.
[17] P. Serai, A. Stiff, and E. Fosler-Lussier, "End to End Speech Recognition Error Prediction with Sequence to
Sequence Learning," ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal
Processing (ICASSP), Barcelona, Spain, 2020, pp. 6339-6343.
[18] K. Palasundram, N. M. Sharef, K. A. Kasmiran, and A. Azman, "Enhancements to the Sequence-to-Sequence-
Based Natural Answer Generation Models," IEEE Access, vol. 8, pp. 45738-45752, 2020.
[19] I. Saito, K. Nishida, K. Nishida, and J. Tomita, "Abstractive Summarization with Combination of Pre-trained
Sequence-to-Sequence and Saliency Models," arXiv Prepr. arXiv2003.13028, 2020.
[20] M. Surdeanu, "A Gentle Introduction to Deep Learning for Natural Language Processing," google Scholar,
pp. 1-63, 2020.
[21] D. King, A. D. Sims, and M. Elsner, "Interpreting sequence-to-sequence models for Russian inflectional
morphology," Proceedings of the Society for Computation in Linguistics, vol. 3, pp. 402-411, 2020.
[22] Y. Yan et al., "ProphetNet: Predicting Future N-gram for Sequence-to-Sequence Pre-training," arXiv Prepr.
arXiv2001.04063, 2020.
[23] A. Sun, J. Wang, N. Cheng, H. Peng, Z. Zeng, and J. Xiao, "GraphTTS: graph-to-sequence modelling in neural
text-to-speech," ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing
(ICASSP), Barcelona, Spain, 2020, pp. 6719-6723.
11. Int J Elec & Comp Eng ISSN: 2088-8708
A systematic review on sequence-to-sequence learning with neural network and its models (Hana Yousuf)
2325
[24] R. Lokeshkumar, K. Jayakumar, V. Prem, and M. S. Nonghuloo, "Analyses and Modeling of Deep Learning Neural
Networks for Sequence-to-Sequence Translation," International Journal of Advanced Science and Technology,
vol. 29, no. 5, pp. 3152-3159, 2020.
[25] C. Chiu, N. Jaitly, I. Sutskever, and Y. Luo, "Recurrent neural networks for online sequence generation," US
Patent, 2020.
[26] H. Sak, M. Shannon, K. Rao, and F. Beaufays, "Recurrent Neural Aligner: An Encoder-Decoder Neural Network
Model for Sequence to Sequence Mapping," Interspeech, vol. 8, pp. 1298-1302, 2017.
[27] G. Neubig, "Neural machine translation and sequence-to-sequence models: A tutorial," arXiv Prepr.
arXiv1703.01619, 2017.
[28] A. Graves, S. Fernández, F. Gomez, and J. Schmidhuber, "Connectionist temporal classification: labelling
unsegmented sequence data with recurrent neural networks," in Proceedings of the 23rd international conference
on Machine learning, 2006, pp. 369-376.
[29] P. J. Werbos, "Backpropagation through time: what it does and how to do it," Proceedings of the IEEE, vol. 78,
no. 10, pp. 1550-1560, 1990.
[30] A. K. Tiwari, G. Ramakrishna, L. K. Sharma, and S. K. Kashyap, "Academic performance prediction algorithm
based on fuzzy data mining," IAES International Journal of Artificial Intelligence (IJ-AI), vol. 8, no. 1, pp. 26-32, 2019.
[31] R. Prabhavalkar, K. Rao, T. N. Sainath, B. Li, L. Johnson, and N. Jaitly, "A Comparison of Sequence-to-Sequence
Models for Speech Recognition.," Interspeech, pp. 939-943, 2017.
[32] A. Das, J. Li, R. Zhao, and Y. Gong, "Advancing connectionist temporal classification with attention modeling,"
2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Calgary, AB, 2018,
pp. 4769-4773.
[33] B. Kitchenham, O. P. Brereton, D. Budgen, M. Turner, J. Bailey, and S. Linkman, "Systematic literature reviews in
software engineering-a systematic literature review," Information and Software Technology, vol. 51, no. 1,
pp. 7-15, 2009.
[34] D. Moher, A. Liberati, J. Tetzlaff, and D.G. Altman, "Preferred reporting items for systematic reviews and meta-
analyses: the PRISMA statement," PLoS Medicine, vol. 8, no. 7716, pp. 336-341, 2009.
[35] S. A. S. Salloum and K. Shaalan, "Investigating students' acceptance of E-learning system in Higher Educational
Environments in the UAE: Applying the Extended Technology Acceptance Model (TAM)," The British University
in Dubai, pp. 1-102, 2018.
[36] S. A. Salloum, A. Qasim Mohammad Alhamad, M. Al-Emran, A. Abdel Monem, and K. Shaalan, "Exploring
students' acceptance of e-learning through the development of a comprehensive technology acceptance model,"
IEEE Access, vol. 7, pp. 128445-128462, 2019.
[37] S. F. S. Alhashmi, S. A. Salloum, and S. Abdallah, “Critical Success Factors for Implementing Artificial
Intelligence (AI) Projects in Dubai Government United Arab Emirates (UAE) Health Sector: Applying the
Extended Technology Acceptance Model (TAM),” Proceedings of the International Conference on Advanced
Intelligent Systems and Informatics 2019, vol. 1058, 2020, pp. 393-405.
[38] S. F. S. Alhashmi, S. A. Salloum, and C. Mhamdi, "Implementing Artificial Intelligence in the United Arab
Emirates Healthcare Sector: An Extended Technology Acceptance Model," International Journal of Information
Technology and Language Studies, vol. 3, no. 3, pp. 27-42, 2019.
[39] S. F. S. Alhashmi, M. Alshurideh, B. Al Kurdi, and S. A. Salloum, "A Systematic Review of the Factors Affecting
the Artificial Intelligence Implementation in the Health Care Sector," Proceedings of the International Conference
on Artificial Intelligence and Computer Vision (AICV2020), vol. 1153, 2020, pp. 37-49.
[40] M. Al-Emran, V. Mezhuyev, A. Kamaludin, and K. Shaalan, "The impact of knowledge management processes on
information systems: A systematic review," International Journal of Information Management, vol. 43,
pp. 173-187, 2018.
[41] Y. H. Chan and A. K. F. Lui, "Encoding emotional information for sequence-to-sequence response generation,"
2018 International Conference on Artificial Intelligence and Big Data (ICAIBD), Chengdu, 2018, pp. 113-116.
[42] Y. He, R. Prabhavalkar, K. Rao, W. Li, A. Bakhtin, and I. McGraw, "Streaming small-footprint keyword spotting
using sequence-to-sequence models," 2017 IEEE Automatic Speech Recognition and Understanding Workshop
(ASRU), Okinawa, 2017, pp. 474-481.
[43] M. S. Islam, S. S. S. Mousumi, S. Abujar, and S. A. Hossain, "Sequence-to-sequence Bangla sentence generation
with LSTM Recurrent Neural Networks," Procedia Computer Science, vol. 152, pp. 51-58, 2019.
[44] J. Salazar, K. Kirchhoff, and Z. Huang, "Self-attention networks for connectionist temporal classification in speech
recognition," ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing
(ICASSP), Brighton, United Kingdom, 2019, pp. 7115-7119.
[45] T. Siddiqui and J. A. Shamsi, "Generating Abstractive Summaries Using Sequence to Sequence Attention Model,"
2018 International Conference on Frontiers of Information Technology (FIT), Islamabad, Pakistan, 2018, pp. 212-217.
[46] S. Venugopalan, M. Rohrbach, J. Donahue, R. Mooney, T. Darrell, and K. Saenko, "Sequence to sequence-video to
text," 2015 IEEE International Conference on Computer Vision (ICCV), Santiago, 2015, pp. 4534-4542.
[47] Q. Yang, Z. He, F. Ge, and Y. Zhang, "Sequence-to-sequence prediction of personal computer software by
recurrent neural network," 2017 International Joint Conference on Neural Networks (IJCNN), Anchorage, AK,
2017, pp. 934-940.
[48] H. Zhang, J. Li, Y. Ji, and H. Yue, "A character-level sequence-to-sequence method for subtitle learning," 2016
IEEE 14th International Conference on Industrial Informatics (INDIN), Poitiers, 2016, pp. 780-783.
12. ISSN: 2088-8708
Int J Elec & Comp Eng, Vol. 11, No. 3, June 2021 : 2315 - 2326
2326
BIOGRAPHIES OF AUTHORS
Hana Y. Zainal had a master's degree in Electrical Engineering from Rochester Institute of
Technology .and a Bachelor Degree in Electrical and Computer Engineering from United Arab
Emirates University. She is currently a Computer Science PhD candidate in British University in
Dubai. The area of interest are Natural Language Processing, Sequence-to-Sequence, CryptDB,
Word2Vec, Victorization, Deep Learning and Secure Encryption. Hana is working in Dubai and
Water Authority (DEWA) in Planning Division as Deputy Senior Manager.
Michael Lahzi Gaid is a PhD student in CS in the British University in Dubai (BUiD). He
received his Masters of CS from The Arab Academy for Science, Technology & Maritime
Transport (AAST), Cairo Egypt in 2019. His particular interests in computer security and data
confidentiality. He works as head developer in Setup Sys Company, developing ERP-System.
Also he works as IT Manager in Ceramica Verdi, Aba Al Dahab Trading Company, Ceramica
Glamour, Mica-Vit Factory, Egyptian Canadian Co. For Manufacturing Carton - Golden Packs,
and Mountee Verde Co. For Manufacturing Oils. Michael has a successful business carrier, he
has developed custom software in many properties all over the world e.g. Vatican, Vodafone
Egypt, Top-Business, Moevenpick Hotels, Rotana Hotels, Ceramica Alfa).
Said A. Salloum had graduated from The British University in Dubai with a distinction with
MSc in Informatics (Knowledge and Data Management). He got his Bachelor's degree in
Computer Science from Yarmouk University. Currently, He is working at the University of
Sharjah "Research Institute of Sciences and Engineering (RISE)" as a researcher on different
research areas in Computer Science such as data analysis, machine learning, knowledge
management, and Arabic Language Processing. Salloum is an Oracle expert since 2013, along
with various recognized international certificates that are issued by Oracle. You can contact.
Said Salloum via emails at: ssalloum@sharjah.ac.ae or salloum78@live.com
Khaled Shaalan is a full professor of Computer Science/Artificial Intelligence at the British
University in Dubai (BUiD), UAE. He is an Honorary Fellow at the School of Informatics,
University of Edinburgh (UoE), UK. Over the last two decades, Prof Khaled has been
contributing to a wide range of research topics in AI, Arabic NLP, Knowledge management,
health informatics, and educational technology. Prof Khaled has published 200+ referred
publications. Prof Khaled's research work is cited extensively worldwide and the impact of his
research using GoogleScholar's H-index metric is 35+. Prof Khaled has been actively and
extensively supporting the local and international academic community. He acts as the chair of
international Conferences, journals & books editor, keynote speaker, external member of
promotions committees, among others.