In this work, the TREPAN algorithm is enhanced and extended for extracting decision trees from neural networks. We empirically evaluated the performance of the algorithm on a set of databases from real world events. This benchmark enhancement was achieved by adapting Single-test TREPAN and C4.5 decision tree induction algorithms to analyze the datasets. The models are then compared with X-TREPAN for comprehensibility and classification accuracy. Furthermore, we validate the experimentations by applying statistical methods. Finally, the modified algorithm is extended to work with multi-class regression problems and the ability to comprehend generalized feed forward networks is achieved.
In this work, the TREPAN algorithm is enhanced and extended for extracting decision trees from neural networks. We empirically evaluated the performance of the algorithm on a set of databases from real world events. This benchmark enhancement was achieved by adapting Single-test TREPAN and C4.5 decision tree induction algorithms to analyze the datasets. The models are then compared with X-TREPAN for
comprehensibility and classification accuracy. Furthermore, we validate the experimentations by applying statistical methods. Finally, the modified algorithm is extended to work with multi-class regression problems and the ability to comprehend generalized feed forward networks is achieved.
On the High Dimentional Information Processing in Quaternionic Domain and its...IJAAS Team
There are various high dimensional engineering and scientific applications in communication, control, robotics, computer vision, biometrics, etc.; where researchers are facing problem to design an intelligent and robust neural system which can process higher dimensional information efficiently. The conventional real-valued neural networks are tried to solve the problem associated with high dimensional parameters, but the required network structure possesses high complexity and are very time consuming and weak to noise. These networks are also not able to learn magnitude and phase values simultaneously in space. The quaternion is the number, which possesses the magnitude in all four directions and phase information is embedded within it. This paper presents a well generalized learning machine with a quaternionic domain neural network that can finely process magnitude and phase information of high dimension data without any hassle. The learning and generalization capability of the proposed learning machine is presented through a wide spectrum of simulations which demonstrate the significance of the work.
International Journal of Engineering Research and Applications (IJERA) is an open access online peer reviewed international journal that publishes research and review articles in the fields of Computer Science, Neural Networks, Electrical Engineering, Software Engineering, Information Technology, Mechanical Engineering, Chemical Engineering, Plastic Engineering, Food Technology, Textile Engineering, Nano Technology & science, Power Electronics, Electronics & Communication Engineering, Computational mathematics, Image processing, Civil Engineering, Structural Engineering, Environmental Engineering, VLSI Testing & Low Power VLSI Design etc.
ON THE PERFORMANCE OF INTRUSION DETECTION SYSTEMS WITH HIDDEN MULTILAYER NEUR...IJCNCJournal
Deep learning applications, especially multilayer neural network models, result in network intrusion detection with high accuracy. This study proposes a model that combines a multilayer neural network with Dense Sparse Dense (DSD) multi-stage training to simultaneously improve the criteria related to the performance of intrusion detection systems on a comprehensive dataset UNSW-NB15. We conduct experiments on many neural network models such as Recurrent Neural Network (RNN), Long-Short Term Memory (LSTM), Gated Recurrent Unit (GRU), etc. to evaluate the combined efficiency with each model through many criteria such as accuracy, detection rate, false alarm rate, precision, and F1-Score.
In this work, the TREPAN algorithm is enhanced and extended for extracting decision trees from neural networks. We empirically evaluated the performance of the algorithm on a set of databases from real world events. This benchmark enhancement was achieved by adapting Single-test TREPAN and C4.5 decision tree induction algorithms to analyze the datasets. The models are then compared with X-TREPAN for
comprehensibility and classification accuracy. Furthermore, we validate the experimentations by applying statistical methods. Finally, the modified algorithm is extended to work with multi-class regression problems and the ability to comprehend generalized feed forward networks is achieved.
On the High Dimentional Information Processing in Quaternionic Domain and its...IJAAS Team
There are various high dimensional engineering and scientific applications in communication, control, robotics, computer vision, biometrics, etc.; where researchers are facing problem to design an intelligent and robust neural system which can process higher dimensional information efficiently. The conventional real-valued neural networks are tried to solve the problem associated with high dimensional parameters, but the required network structure possesses high complexity and are very time consuming and weak to noise. These networks are also not able to learn magnitude and phase values simultaneously in space. The quaternion is the number, which possesses the magnitude in all four directions and phase information is embedded within it. This paper presents a well generalized learning machine with a quaternionic domain neural network that can finely process magnitude and phase information of high dimension data without any hassle. The learning and generalization capability of the proposed learning machine is presented through a wide spectrum of simulations which demonstrate the significance of the work.
International Journal of Engineering Research and Applications (IJERA) is an open access online peer reviewed international journal that publishes research and review articles in the fields of Computer Science, Neural Networks, Electrical Engineering, Software Engineering, Information Technology, Mechanical Engineering, Chemical Engineering, Plastic Engineering, Food Technology, Textile Engineering, Nano Technology & science, Power Electronics, Electronics & Communication Engineering, Computational mathematics, Image processing, Civil Engineering, Structural Engineering, Environmental Engineering, VLSI Testing & Low Power VLSI Design etc.
ON THE PERFORMANCE OF INTRUSION DETECTION SYSTEMS WITH HIDDEN MULTILAYER NEUR...IJCNCJournal
Deep learning applications, especially multilayer neural network models, result in network intrusion detection with high accuracy. This study proposes a model that combines a multilayer neural network with Dense Sparse Dense (DSD) multi-stage training to simultaneously improve the criteria related to the performance of intrusion detection systems on a comprehensive dataset UNSW-NB15. We conduct experiments on many neural network models such as Recurrent Neural Network (RNN), Long-Short Term Memory (LSTM), Gated Recurrent Unit (GRU), etc. to evaluate the combined efficiency with each model through many criteria such as accuracy, detection rate, false alarm rate, precision, and F1-Score.
The aim of this research is to find accurate solution for the Troesch’s problem by using high performance technique based on parallel processing implementation.
Design/methodology/approach – Feed forward neural network is designed to solve important type of differential equations that arises in many applied sciences and engineering applications. The suitable designed based on choosing suitable learning rate, transfer function, and training algorithm. The authors used back propagation with new implement of Levenberg - Marquardt training algorithm. Also, the authors depend new idea for choosing the weights. The effectiveness of the suggested design for the network is shown by using it for solving Troesch problem in many cases.
Findings – New idea for choosing the weights of the neural network, new implement of Levenberg - Marquardt training algorithm which assist to speeding the convergence and the implementation of the suggested design demonstrates the usefulness in finding exact solutions.
ANALYSIS AND COMPARISON STUDY OF DATA MINING ALGORITHMS USING RAPIDMINERIJCSEA Journal
Comparison study of algorithms is very much required before implementing them for the needs of any
organization. The comparisons of algorithms are depending on the various parameters such as data
frequency, types of data and relationship among the attributes in a given data set. There are number of
learning and classifications algorithms are used to analyse, learn patterns and categorize data are
available. But the problem is the one to find the best algorithm according to the problem and desired
output. The desired result has always been higher accuracy in predicting future values or events from the
given dataset. Algorithms taken for the comparisons study are Neural net, SVM, Naïve Bayes, BFT and
Decision stump. These top algorithms are most influential data mining algorithms in the research
community. These algorithms have been considered and mostly used in the field of knowledge discovery
and data mining.
Incorporating Kalman Filter in the Optimization of Quantum Neural Network Par...Waqas Tariq
Kalman filter have been used for the estimation of instantaneous states of linear dynamic systems. It is a good tool for inferring of missing information from noisy measurement. The quantum neural network is another approach to the merging of fuzzy logic with the neural network and that by the investment of quantum mechanics theory in building the structure of neural network. The gradient descent algorithm has been used, widely, in training the neural network, but the problem of local minima is one of the disadvantages of this algorithm. This paper presents an algorithm to train the quantum neural network by using the extended kalman filter.
NETWORK LEARNING AND TRAINING OF A CASCADED LINK-BASED FEED FORWARD NEURAL NE...ijaia
Presently, considering the technological advancement of our modern world, we are in dire need for a system that can learn new concepts and give decisions on its own. Hence the Artificial Neural Network is all that is required in the contemporary situation. In this paper, CLBFFNN is presented as a special and intelligent form of artificial neural networks that has the capability to adapt to training and learning of new ideas and be able to give decisions in a trimodal biometric system involving fingerprints, face and iris biometric data. It gives an overview of neural networks.
Comparison Between Levenberg-Marquardt And Scaled Conjugate Gradient Training...CSCJournals
The Internet paved way for information sharing all over the world decades ago and its popularity for distribution of data has spread like a wildfire ever since. Data in the form of images, sounds, animations and videos is gaining users’ preference in comparison to plain text all across the globe. Despite unprecedented progress in the fields of data storage, computing speed and data transmission speed, the demands of available data and its size (due to the increase in both, quality and quantity) continue to overpower the supply of resources. One of the reasons for this may be how the uncompressed data is compressed in order to send it across the network. This paper compares the two most widely used training algorithms for multilayer perceptron (MLP) image compression – the Levenberg-Marquardt algorithm and the Scaled Conjugate Gradient algorithm. We test the performance of the two training algorithms by compressing the standard test image (Lena or Lenna) in terms of accuracy and speed. Based on our results, we conclude that both algorithms were comparable in terms of speed and accuracy. However, the Levenberg- Marquardt algorithm has shown slightly better performance in terms of accuracy (as found in the average training accuracy and mean squared error), whereas the Scaled Conjugate Gradient algorithm faired better in terms of speed (as found in the average training iteration) on a simple MLP structure (2 hidden layers).
This work is proposed the feed forward neural network with symmetric table addition method to design the
neuron synapses algorithm of the sine function approximations, and according to the Taylor series
expansion. Matlab code and LabVIEW are used to build and create the neural network, which has been
designed and trained database set to improve its performance, and gets the best a global convergence with
small value of MSE errors and 97.22% accuracy.
An artificial neural network (ANN) is the piece of a computing system designed to simulate the way the human brain analyzes and processes information. It is the foundation of artificial intelligence (AI) and solves problems that would prove impossible or difficult by human or statistical standards. ANNs have self-learning capabilities that enable them to produce better results as more data becomes available.
Deep learning algorithms have drawn the attention of researchers working in the field of computer vision, speech
recognition, malware detection, pattern recognition and natural language processing. In this paper, we present an overview of
deep learning techniques like Convolutional neural network, deep belief network, Autoencoder, Restricted Boltzmann machine
and recurrent neural network. With this, current work of deep learning algorithms on malware detection is shown with the
help of literature survey. Suggestions for future research are given with full justification. We also showed the experimental
analysis in order to show the importance of deep learning techniques.
International Journal of Engineering Research and Applications (IJERA) is an open access online peer reviewed international journal that publishes research and review articles in the fields of Computer Science, Neural Networks, Electrical Engineering, Software Engineering, Information Technology, Mechanical Engineering, Chemical Engineering, Plastic Engineering, Food Technology, Textile Engineering, Nano Technology & science, Power Electronics, Electronics & Communication Engineering, Computational mathematics, Image processing, Civil Engineering, Structural Engineering, Environmental Engineering, VLSI Testing & Low Power VLSI Design etc.
Improving of artifical neural networks performance by using gpu's a surveycsandit
In this paper we study the improvement in the performance of Artificial Neural Networks (ANN)
by using parallel programming in GPU or FPGA architectures. It is well known that ANN can
be parallelized according to particular characteristics of the training algorithm. We discuss
both approaches: the software (GPU) and the Hardware (FPGA). Different training strategies
are discussed: the Perceptron training unit, the Support Vector Machines (SVM) and Spiking
Neural Networks (SNN). The different approaches are evaluated by the training speed and
performance. On the other hand, algorithms were coded by authors in the hardware, like Nvidia
card, FPGA or sequential circuits that depends on methodology used, to compare learning time
with between GPU and CPU. Also, the main applications were made for recognition pattern,
like acoustic speech, odor and clustering According to literature, GPU has a great advantage
compared to CPU, this in the learning time except when it implies rendering of images, despite
several architectures of Nvidia cards and CPU’s. Also, in the survey we introduce a brief
description of the types of ANN and its techniques of execution to be related with results of
researching.
IMPROVING OF ARTIFICIAL NEURAL NETWORKS PERFORMANCE BY USING GPU’S: A SURVEYcsandit
In this paper we study the improvement in the performance of Artificial Neural Networks (ANN) by using parallel programming in GPU or FPGA architectures. It is well known that ANN can be parallelized according to particular characteristics of the training algorithm. We discuss both approaches: the software (GPU) and the Hardware (FPGA). Different training strategies
are discussed: the Perceptron training unit, the Support Vector Machines (SVM) and Spiking Neural Networks (SNN). The different approaches are evaluated by the training speed and performance. On the other hand, algorithms were coded by authors in the hardware, like Nvidia card, FPGA or sequential circuits that depends on methodology used, to compare learning time with between GPU and CPU. Also, the main applications were made for recognition pattern,like acoustic speech, odor and clustering According to literature, GPU has a great advantage compared to CPU, this in the learning time except when it implies rendering of images, despite
several architectures of Nvidia cards and CPU’s. Also, in the survey we introduce a brief description of the types of ANN and its techniques of execution to be related with results of researching.
Provides a brief overview of what machine learning is, how it works (theory), how to prepare data for a machine learning problem, an example case study, and additional resources.
The aim of this research is to find accurate solution for the Troesch’s problem by using high performance technique based on parallel processing implementation.
Design/methodology/approach – Feed forward neural network is designed to solve important type of differential equations that arises in many applied sciences and engineering applications. The suitable designed based on choosing suitable learning rate, transfer function, and training algorithm. The authors used back propagation with new implement of Levenberg - Marquardt training algorithm. Also, the authors depend new idea for choosing the weights. The effectiveness of the suggested design for the network is shown by using it for solving Troesch problem in many cases.
Findings – New idea for choosing the weights of the neural network, new implement of Levenberg - Marquardt training algorithm which assist to speeding the convergence and the implementation of the suggested design demonstrates the usefulness in finding exact solutions.
ANALYSIS AND COMPARISON STUDY OF DATA MINING ALGORITHMS USING RAPIDMINERIJCSEA Journal
Comparison study of algorithms is very much required before implementing them for the needs of any
organization. The comparisons of algorithms are depending on the various parameters such as data
frequency, types of data and relationship among the attributes in a given data set. There are number of
learning and classifications algorithms are used to analyse, learn patterns and categorize data are
available. But the problem is the one to find the best algorithm according to the problem and desired
output. The desired result has always been higher accuracy in predicting future values or events from the
given dataset. Algorithms taken for the comparisons study are Neural net, SVM, Naïve Bayes, BFT and
Decision stump. These top algorithms are most influential data mining algorithms in the research
community. These algorithms have been considered and mostly used in the field of knowledge discovery
and data mining.
Incorporating Kalman Filter in the Optimization of Quantum Neural Network Par...Waqas Tariq
Kalman filter have been used for the estimation of instantaneous states of linear dynamic systems. It is a good tool for inferring of missing information from noisy measurement. The quantum neural network is another approach to the merging of fuzzy logic with the neural network and that by the investment of quantum mechanics theory in building the structure of neural network. The gradient descent algorithm has been used, widely, in training the neural network, but the problem of local minima is one of the disadvantages of this algorithm. This paper presents an algorithm to train the quantum neural network by using the extended kalman filter.
NETWORK LEARNING AND TRAINING OF A CASCADED LINK-BASED FEED FORWARD NEURAL NE...ijaia
Presently, considering the technological advancement of our modern world, we are in dire need for a system that can learn new concepts and give decisions on its own. Hence the Artificial Neural Network is all that is required in the contemporary situation. In this paper, CLBFFNN is presented as a special and intelligent form of artificial neural networks that has the capability to adapt to training and learning of new ideas and be able to give decisions in a trimodal biometric system involving fingerprints, face and iris biometric data. It gives an overview of neural networks.
Comparison Between Levenberg-Marquardt And Scaled Conjugate Gradient Training...CSCJournals
The Internet paved way for information sharing all over the world decades ago and its popularity for distribution of data has spread like a wildfire ever since. Data in the form of images, sounds, animations and videos is gaining users’ preference in comparison to plain text all across the globe. Despite unprecedented progress in the fields of data storage, computing speed and data transmission speed, the demands of available data and its size (due to the increase in both, quality and quantity) continue to overpower the supply of resources. One of the reasons for this may be how the uncompressed data is compressed in order to send it across the network. This paper compares the two most widely used training algorithms for multilayer perceptron (MLP) image compression – the Levenberg-Marquardt algorithm and the Scaled Conjugate Gradient algorithm. We test the performance of the two training algorithms by compressing the standard test image (Lena or Lenna) in terms of accuracy and speed. Based on our results, we conclude that both algorithms were comparable in terms of speed and accuracy. However, the Levenberg- Marquardt algorithm has shown slightly better performance in terms of accuracy (as found in the average training accuracy and mean squared error), whereas the Scaled Conjugate Gradient algorithm faired better in terms of speed (as found in the average training iteration) on a simple MLP structure (2 hidden layers).
This work is proposed the feed forward neural network with symmetric table addition method to design the
neuron synapses algorithm of the sine function approximations, and according to the Taylor series
expansion. Matlab code and LabVIEW are used to build and create the neural network, which has been
designed and trained database set to improve its performance, and gets the best a global convergence with
small value of MSE errors and 97.22% accuracy.
An artificial neural network (ANN) is the piece of a computing system designed to simulate the way the human brain analyzes and processes information. It is the foundation of artificial intelligence (AI) and solves problems that would prove impossible or difficult by human or statistical standards. ANNs have self-learning capabilities that enable them to produce better results as more data becomes available.
Deep learning algorithms have drawn the attention of researchers working in the field of computer vision, speech
recognition, malware detection, pattern recognition and natural language processing. In this paper, we present an overview of
deep learning techniques like Convolutional neural network, deep belief network, Autoencoder, Restricted Boltzmann machine
and recurrent neural network. With this, current work of deep learning algorithms on malware detection is shown with the
help of literature survey. Suggestions for future research are given with full justification. We also showed the experimental
analysis in order to show the importance of deep learning techniques.
International Journal of Engineering Research and Applications (IJERA) is an open access online peer reviewed international journal that publishes research and review articles in the fields of Computer Science, Neural Networks, Electrical Engineering, Software Engineering, Information Technology, Mechanical Engineering, Chemical Engineering, Plastic Engineering, Food Technology, Textile Engineering, Nano Technology & science, Power Electronics, Electronics & Communication Engineering, Computational mathematics, Image processing, Civil Engineering, Structural Engineering, Environmental Engineering, VLSI Testing & Low Power VLSI Design etc.
Improving of artifical neural networks performance by using gpu's a surveycsandit
In this paper we study the improvement in the performance of Artificial Neural Networks (ANN)
by using parallel programming in GPU or FPGA architectures. It is well known that ANN can
be parallelized according to particular characteristics of the training algorithm. We discuss
both approaches: the software (GPU) and the Hardware (FPGA). Different training strategies
are discussed: the Perceptron training unit, the Support Vector Machines (SVM) and Spiking
Neural Networks (SNN). The different approaches are evaluated by the training speed and
performance. On the other hand, algorithms were coded by authors in the hardware, like Nvidia
card, FPGA or sequential circuits that depends on methodology used, to compare learning time
with between GPU and CPU. Also, the main applications were made for recognition pattern,
like acoustic speech, odor and clustering According to literature, GPU has a great advantage
compared to CPU, this in the learning time except when it implies rendering of images, despite
several architectures of Nvidia cards and CPU’s. Also, in the survey we introduce a brief
description of the types of ANN and its techniques of execution to be related with results of
researching.
IMPROVING OF ARTIFICIAL NEURAL NETWORKS PERFORMANCE BY USING GPU’S: A SURVEYcsandit
In this paper we study the improvement in the performance of Artificial Neural Networks (ANN) by using parallel programming in GPU or FPGA architectures. It is well known that ANN can be parallelized according to particular characteristics of the training algorithm. We discuss both approaches: the software (GPU) and the Hardware (FPGA). Different training strategies
are discussed: the Perceptron training unit, the Support Vector Machines (SVM) and Spiking Neural Networks (SNN). The different approaches are evaluated by the training speed and performance. On the other hand, algorithms were coded by authors in the hardware, like Nvidia card, FPGA or sequential circuits that depends on methodology used, to compare learning time with between GPU and CPU. Also, the main applications were made for recognition pattern,like acoustic speech, odor and clustering According to literature, GPU has a great advantage compared to CPU, this in the learning time except when it implies rendering of images, despite
several architectures of Nvidia cards and CPU’s. Also, in the survey we introduce a brief description of the types of ANN and its techniques of execution to be related with results of researching.
Provides a brief overview of what machine learning is, how it works (theory), how to prepare data for a machine learning problem, an example case study, and additional resources.
Artificial Neural Networks: Applications In ManagementIOSR Journals
With the advancement of computer and communication technology, the tools used for management decisions have undergone a gigantic change. Finding the more effective solution and tools for managerial problems is one of the most important topics in the management studies today. Artificial Neural Networks (ANNs) are one of these tools that have become a critical component for business intelligence. The purpose of this article is to describe the basic behavior of neural networks as well as the works done in application of the same in management sciences and stimulate further research interests and efforts in the identified topics.
NEURAL NETWORK FOR THE RELIABILITY ANALYSIS OF A SERIES - PARALLEL SYSTEM SUB...IAEME Publication
Artificial neural networks can achieve high computation rates by employing a massive number of simple processing elements with a high degree of connectivity between the elements. Neural networks with feedback connections provide a computing model capable of exploiting fine- grained parallelism to solve a rich class of complex problems. In this paper we discuss a complex series-parallel system subjected to finite common cause and finite human error failures and its reliability using neural network method.
On The Performance of Intrusion Detection Systems with Hidden Multilayer Neur...IJCNCJournal
Deep learning applications, especially multilayer neural network models, result in network intrusion detection with high accuracy. This study proposes a model that combines a multilayer neural network with Dense Sparse Dense (DSD) multi-stage training to simultaneously improve the criteria related to the performance of intrusion detection systems on a comprehensive dataset UNSW-NB15. We conduct experiments on many neural network models such as Recurrent Neural Network (RNN), Long-Short Term Memory (LSTM), Gated Recurrent Unit (GRU), etc. to evaluate the combined efficiency with each model through many criteria such as accuracy, detection rate, false alarm rate, precision, and F1-Score.
A Mixed Binary-Real NSGA II Algorithm Ensuring Both Accuracy and Interpretabi...IJECEIAES
In this work, a Neuro-Fuzzy Controller network, called NFC that implements a Mamdani fuzzy inference system is proposed. This network includes neurons able to perform fundamental fuzzy operations. Connections between neurons are weighted through binary and real weights. Then a mixed binaryreal Non dominated Sorting Genetic Algorithm II (NSGA II) is used to perform both accuracy and interpretability of the NFC by minimizing two objective functions; one objective relates to the number of rules, for compactness, while the second is the mean square error, for accuracy. In order to preserve interpretability of fuzzy rules during the optimization process, some constraints are imposed. The approach is tested on two control examples: a single input single output (SISO) system and a multivariable (MIMO) system.
AN IMPROVED METHOD FOR IDENTIFYING WELL-TEST INTERPRETATION MODEL BASED ON AG...IAEME Publication
This paper presents an approach based on applying an aggregated predictor formed by multiple versions of a multilayer neural network with a back-propagation optimization algorithm for helping the engineer to get a list of the most appropriate well-test interpretation models for a given set of pressure/ production data. The proposed method consists of three stages: (1) data decorrelation through principal component analysis to reduce the covariance between the variables and the dimension of the input layer in the artificial neural network, (2) bootstrap replicates of the learning set where the data is repeatedly sampled with a random split of the data into train sets and using these as new learning sets, and (3) automatic reservoir model identification through aggregated predictor formed by a plurality vote when predicting a new class. This method is described in detail to ensure successful replication of results. The required training and test dataset were generated by using analytical solution models. In our case, there were used 600 samples: 300 for training, 100 for cross-validation, and 200 for testing. Different network structures were tested during this study to arrive at optimum network design. We notice that the single net methodology always brings about confusion in selecting the correct model even though the training results for the constructed networks are close to 1. We notice also that the principal component analysis is an effective strategy in reducing the number of input features, simplifying the network structure, and lowering the training time of the ANN. The results obtained show that the proposed model provides better performance when predicting new data with a coefficient of correlation approximately equal to 95% Compared to a previous approach 80%, the combination of the PCA and ANN is more stable and determine the more accurate results with lesser computational complexity than was feasible previously. Clearly, the aggregated predictor is more stable and shows less bad classes compared to the previous approach.
ANALYSIS OF LAND SURFACE DEFORMATION GRADIENT BY DINSAR cscpconf
The progressive development of Synthetic Aperture Radar (SAR) systems diversify the exploitation of the generated images by these systems in different applications of geoscience. Detection and monitoring surface deformations, procreated by various phenomena had benefited from this evolution and had been realized by interferometry (InSAR) and differential interferometry (DInSAR) techniques. Nevertheless, spatial and temporal decorrelations of the interferometric couples used, limit strongly the precision of analysis results by these techniques. In this context, we propose, in this work, a methodological approach of surface deformation detection and analysis by differential interferograms to show the limits of this technique according to noise quality and level. The detectability model is generated from the deformation signatures, by simulating a linear fault merged to the images couples of ERS1 / ERS2 sensors acquired in a region of the Algerian south.
4D AUTOMATIC LIP-READING FOR SPEAKER'S FACE IDENTIFCATIONcscpconf
A novel based a trajectory-guided, concatenating approach for synthesizing high-quality image real sample renders video is proposed . The lips reading automated is seeking for modeled the closest real image sample sequence preserve in the library under the data video to the HMM predicted trajectory. The object trajectory is modeled obtained by projecting the face patterns into an KDA feature space is estimated. The approach for speaker's face identification by using synthesise the identity surface of a subject face from a small sample of patterns which sparsely each the view sphere. An KDA algorithm use to the Lip-reading image is discrimination, after that work consisted of in the low dimensional for the fundamental lip features vector is reduced by using the 2D-DCT.The mouth of the set area dimensionality is ordered by a normally reduction base on the PCA to obtain the Eigen lips approach, their proposed approach by[33]. The subjective performance results of the cost function under the automatic lips reading modeled , which wasn’t illustrate the superior performance of the
method.
MOVING FROM WATERFALL TO AGILE PROCESS IN SOFTWARE ENGINEERING CAPSTONE PROJE...cscpconf
Universities offer software engineering capstone course to simulate a real world-working environment in which students can work in a team for a fixed period to deliver a quality product. The objective of the paper is to report on our experience in moving from Waterfall process to Agile process in conducting the software engineering capstone project. We present the capstone course designs for both Waterfall driven and Agile driven methodologies that highlight the structure, deliverables and assessment plans.To evaluate the improvement, we conducted a survey for two different sections taught by two different instructors to evaluate students’ experience in moving from traditional Waterfall model to Agile like process. Twentyeight students filled the survey. The survey consisted of eight multiple-choice questions and an open-ended question to collect feedback from students. The survey results show that students were able to attain hands one experience, which simulate a real world-working environment. The results also show that the Agile approach helped students to have overall better design and avoid mistakes they have made in the initial design completed in of the first phase of the capstone project. In addition, they were able to decide on their team capabilities, training needs and thus learn the required technologies earlier which is reflected on the final product quality
PROMOTING STUDENT ENGAGEMENT USING SOCIAL MEDIA TECHNOLOGIEScscpconf
Using social media in education provides learners with an informal way for communication. Informal communication tends to remove barriers and hence promotes student engagement. This paper presents our experience in using three different social media technologies in teaching software project management course. We conducted different surveys at the end of every semester to evaluate students’ satisfaction and engagement. Results show that using social media enhances students’ engagement and satisfaction. However, familiarity with the tool is an important factor for student satisfaction.
A SURVEY ON QUESTION ANSWERING SYSTEMS: THE ADVANCES OF FUZZY LOGICcscpconf
In real world computing environment with using a computer to answer questions has been a human dream since the beginning of the digital era, Question-answering systems are referred to as intelligent systems, that can be used to provide responses for the questions being asked by the user based on certain facts or rules stored in the knowledge base it can generate answers of questions asked in natural , and the first main idea of fuzzy logic was to working on the problem of computer understanding of natural language, so this survey paper provides an overview on what Question-Answering is and its system architecture and the possible relationship and
different with fuzzy logic, as well as the previous related research with respect to approaches that were followed. At the end, the survey provides an analytical discussion of the proposed QA models, along or combined with fuzzy logic and their main contributions and limitations.
DYNAMIC PHONE WARPING – A METHOD TO MEASURE THE DISTANCE BETWEEN PRONUNCIATIONS cscpconf
Human beings generate different speech waveforms while speaking the same word at different times. Also, different human beings have different accents and generate significantly varying speech waveforms for the same word. There is a need to measure the distances between various words which facilitate preparation of pronunciation dictionaries. A new algorithm called Dynamic Phone Warping (DPW) is presented in this paper. It uses dynamic programming technique for global alignment and shortest distance measurements. The DPW algorithm can be used to enhance the pronunciation dictionaries of the well-known languages like English or to build pronunciation dictionaries to the less known sparse languages. The precision measurement experiments show 88.9% accuracy.
INTELLIGENT ELECTRONIC ASSESSMENT FOR SUBJECTIVE EXAMS cscpconf
In education, the use of electronic (E) examination systems is not a novel idea, as Eexamination systems have been used to conduct objective assessments for the last few years. This research deals with randomly designed E-examinations and proposes an E-assessment system that can be used for subjective questions. This system assesses answers to subjective questions by finding a matching ratio for the keywords in instructor and student answers. The matching ratio is achieved based on semantic and document similarity. The assessment system is composed of four modules: preprocessing, keyword expansion, matching, and grading. A survey and case study were used in the research design to validate the proposed system. The examination assessment system will help instructors to save time, costs, and resources, while increasing efficiency and improving the productivity of exam setting and assessments.
TWO DISCRETE BINARY VERSIONS OF AFRICAN BUFFALO OPTIMIZATION METAHEURISTICcscpconf
African Buffalo Optimization (ABO) is one of the most recent swarms intelligence based metaheuristics. ABO algorithm is inspired by the buffalo’s behavior and lifestyle. Unfortunately, the standard ABO algorithm is proposed only for continuous optimization problems. In this paper, the authors propose two discrete binary ABO algorithms to deal with binary optimization problems. In the first version (called SBABO) they use the sigmoid function and probability model to generate binary solutions. In the second version (called LBABO) they use some logical operator to operate the binary solutions. Computational results on two knapsack problems (KP and MKP) instances show the effectiveness of the proposed algorithm and their ability to achieve good and promising solutions.
DETECTION OF ALGORITHMICALLY GENERATED MALICIOUS DOMAINcscpconf
In recent years, many malware writers have relied on Dynamic Domain Name Services (DDNS) to maintain their Command and Control (C&C) network infrastructure to ensure a persistence presence on a compromised host. Amongst the various DDNS techniques, Domain Generation Algorithm (DGA) is often perceived as the most difficult to detect using traditional methods. This paper presents an approach for detecting DGA using frequency analysis of the character distribution and the weighted scores of the domain names. The approach’s feasibility is demonstrated using a range of legitimate domains and a number of malicious algorithmicallygenerated domain names. Findings from this study show that domain names made up of English characters “a-z” achieving a weighted score of < 45 are often associated with DGA. When a weighted score of < 45 is applied to the Alexa one million list of domain names, only 15% of the domain names were treated as non-human generated.
GLOBAL MUSIC ASSET ASSURANCE DIGITAL CURRENCY: A DRM SOLUTION FOR STREAMING C...cscpconf
The amount of piracy in the streaming digital content in general and the music industry in specific is posing a real challenge to digital content owners. This paper presents a DRM solution to monetizing, tracking and controlling online streaming content cross platforms for IP enabled devices. The paper benefits from the current advances in Blockchain and cryptocurrencies. Specifically, the paper presents a Global Music Asset Assurance (GoMAA) digital currency and presents the iMediaStreams Blockchain to enable the secure dissemination and tracking of the streamed content. The proposed solution provides the data owner the ability to control the flow of information even after it has been released by creating a secure, selfinstalled, cross platform reader located on the digital content file header. The proposed system provides the content owners’ options to manage their digital information (audio, video, speech, etc.), including the tracking of the most consumed segments, once it is release. The system benefits from token distribution between the content owner (Music Bands), the content distributer (Online Radio Stations) and the content consumer(Fans) on the system blockchain.
IMPORTANCE OF VERB SUFFIX MAPPING IN DISCOURSE TRANSLATION SYSTEMcscpconf
This paper discusses the importance of verb suffix mapping in Discourse translation system. In
discourse translation, the crucial step is Anaphora resolution and generation. In Anaphora
resolution, cohesion links like pronouns are identified between portions of text. These binders
make the text cohesive by referring to nouns appearing in the previous sentences or nouns
appearing in sentences after them. In Machine Translation systems, to convert the source
language sentences into meaningful target language sentences the verb suffixes should be
changed as per the cohesion links identified. This step of translation process is emphasized in
the present paper. Specifically, the discussion is on how the verbs change according to the
subjects and anaphors. To explain the concept, English is used as the source language (SL) and
an Indian language Telugu is used as Target language (TL)
EXACT SOLUTIONS OF A FAMILY OF HIGHER-DIMENSIONAL SPACE-TIME FRACTIONAL KDV-T...cscpconf
In this paper, based on the definition of conformable fractional derivative, the functional
variable method (FVM) is proposed to seek the exact traveling wave solutions of two higherdimensional
space-time fractional KdV-type equations in mathematical physics, namely the
(3+1)-dimensional space–time fractional Zakharov-Kuznetsov (ZK) equation and the (2+1)-
dimensional space–time fractional Generalized Zakharov-Kuznetsov-Benjamin-Bona-Mahony
(GZK-BBM) equation. Some new solutions are procured and depicted. These solutions, which
contain kink-shaped, singular kink, bell-shaped soliton, singular soliton and periodic wave
solutions, have many potential applications in mathematical physics and engineering. The
simplicity and reliability of the proposed method is verified.
AUTOMATED PENETRATION TESTING: AN OVERVIEWcscpconf
The using of information technology resources is rapidly increasing in organizations,
businesses, and even governments, that led to arise various attacks, and vulnerabilities in the
field. All resources make it a must to do frequently a penetration test (PT) for the environment
and see what can the attacker gain and what is the current environment's vulnerabilities. This
paper reviews some of the automated penetration testing techniques and presents its
enhancement over the traditional manual approaches. To the best of our knowledge, it is the
first research that takes into consideration the concept of penetration testing and the standards
in the area.This research tackles the comparison between the manual and automated
penetration testing, the main tools used in penetration testing. Additionally, compares between
some methodologies used to build an automated penetration testing platform.
CLASSIFICATION OF ALZHEIMER USING fMRI DATA AND BRAIN NETWORKcscpconf
Since the mid of 1990s, functional connectivity study using fMRI (fcMRI) has drawn increasing
attention of neuroscientists and computer scientists, since it opens a new window to explore
functional network of human brain with relatively high resolution. BOLD technique provides
almost accurate state of brain. Past researches prove that neuro diseases damage the brain
network interaction, protein- protein interaction and gene-gene interaction. A number of
neurological research paper also analyse the relationship among damaged part. By
computational method especially machine learning technique we can show such classifications.
In this paper we used OASIS fMRI dataset affected with Alzheimer’s disease and normal
patient’s dataset. After proper processing the fMRI data we use the processed data to form
classifier models using SVM (Support Vector Machine), KNN (K- nearest neighbour) & Naïve
Bayes. We also compare the accuracy of our proposed method with existing methods. In future,
we will other combinations of methods for better accuracy.
VALIDATION METHOD OF FUZZY ASSOCIATION RULES BASED ON FUZZY FORMAL CONCEPT AN...cscpconf
In order to treat and analyze real datasets, fuzzy association rules have been proposed. Several
algorithms have been introduced to extract these rules. However, these algorithms suffer from
the problems of utility, redundancy and large number of extracted fuzzy association rules. The
expert will then be confronted with this huge amount of fuzzy association rules. The task of
validation becomes fastidious. In order to solve these problems, we propose a new validation
method. Our method is based on three steps. (i) We extract a generic base of non redundant
fuzzy association rules by applying EFAR-PN algorithm based on fuzzy formal concept analysis.
(ii) we categorize extracted rules into groups and (iii) we evaluate the relevance of these rules
using structural equation model.
PROBABILITY BASED CLUSTER EXPANSION OVERSAMPLING TECHNIQUE FOR IMBALANCED DATAcscpconf
In many applications of data mining, class imbalance is noticed when examples in one class are
overrepresented. Traditional classifiers result in poor accuracy of the minority class due to the
class imbalance. Further, the presence of within class imbalance where classes are composed of
multiple sub-concepts with different number of examples also affect the performance of
classifier. In this paper, we propose an oversampling technique that handles between class and
within class imbalance simultaneously and also takes into consideration the generalization
ability in data space. The proposed method is based on two steps- performing Model Based
Clustering with respect to classes to identify the sub-concepts; and then computing the
separating hyperplane based on equal posterior probability between the classes. The proposed
method is tested on 10 publicly available data sets and the result shows that the proposed
method is statistically superior to other existing oversampling methods.
CHARACTER AND IMAGE RECOGNITION FOR DATA CATALOGING IN ECOLOGICAL RESEARCHcscpconf
Data collection is an essential, but manpower intensive procedure in ecological research. An
algorithm was developed by the author which incorporated two important computer vision
techniques to automate data cataloging for butterfly measurements. Optical Character
Recognition is used for character recognition and Contour Detection is used for imageprocessing.
Proper pre-processing is first done on the images to improve accuracy. Although
there are limitations to Tesseract’s detection of certain fonts, overall, it can successfully identify
words of basic fonts. Contour detection is an advanced technique that can be utilized to
measure an image. Shapes and mathematical calculations are crucial in determining the precise
location of the points on which to draw the body and forewing lines of the butterfly. Overall,
92% accuracy were achieved by the program for the set of butterflies measured.
SOCIAL MEDIA ANALYTICS FOR SENTIMENT ANALYSIS AND EVENT DETECTION IN SMART CI...cscpconf
Smart cities utilize Internet of Things (IoT) devices and sensors to enhance the quality of the city
services including energy, transportation, health, and much more. They generate massive
volumes of structured and unstructured data on a daily basis. Also, social networks, such as
Twitter, Facebook, and Google+, are becoming a new source of real-time information in smart
cities. Social network users are acting as social sensors. These datasets so large and complex
are difficult to manage with conventional data management tools and methods. To become
valuable, this massive amount of data, known as 'big data,' needs to be processed and
comprehended to hold the promise of supporting a broad range of urban and smart cities
functions, including among others transportation, water, and energy consumption, pollution
surveillance, and smart city governance. In this work, we investigate how social media analytics
help to analyze smart city data collected from various social media sources, such as Twitter and
Facebook, to detect various events taking place in a smart city and identify the importance of
events and concerns of citizens regarding some events. A case scenario analyses the opinions of
users concerning the traffic in three largest cities in the UAE
SOCIAL NETWORK HATE SPEECH DETECTION FOR AMHARIC LANGUAGEcscpconf
The anonymity of social networks makes it attractive for hate speech to mask their criminal
activities online posing a challenge to the world and in particular Ethiopia. With this everincreasing
volume of social media data, hate speech identification becomes a challenge in
aggravating conflict between citizens of nations. The high rate of production, has become
difficult to collect, store and analyze such big data using traditional detection methods. This
paper proposed the application of apache spark in hate speech detection to reduce the
challenges. Authors developed an apache spark based model to classify Amharic Facebook
posts and comments into hate and not hate. Authors employed Random forest and Naïve Bayes
for learning and Word2Vec and TF-IDF for feature selection. Tested by 10-fold crossvalidation,
the model based on word2vec embedding performed best with 79.83%accuracy. The
proposed method achieve a promising result with unique feature of spark for big data.
GENERAL REGRESSION NEURAL NETWORK BASED POS TAGGING FOR NEPALI TEXTcscpconf
This article presents Part of Speech tagging for Nepali text using General Regression Neural
Network (GRNN). The corpus is divided into two parts viz. training and testing. The network is
trained and validated on both training and testing data. It is observed that 96.13% words are
correctly being tagged on training set whereas 74.38% words are tagged correctly on testing
data set using GRNN. The result is compared with the traditional Viterbi algorithm based on
Hidden Markov Model. Viterbi algorithm yields 97.2% and 40% classification accuracies on
training and testing data sets respectively. GRNN based POS Tagger is more consistent than the
traditional Viterbi decoding technique.
Accelerate your Kubernetes clusters with Varnish CachingThijs Feryn
A presentation about the usage and availability of Varnish on Kubernetes. This talk explores the capabilities of Varnish caching and shows how to use the Varnish Helm chart to deploy it to Kubernetes.
This presentation was delivered at K8SUG Singapore. See https://feryn.eu/presentations/accelerate-your-kubernetes-clusters-with-varnish-caching-k8sug-singapore-28-2024 for more details.
Alt. GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using ...James Anderson
Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM
The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management.
The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM).
Speakers:
Bob Boule
Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle.
Gopinath Rebala
Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.
Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...SOFTTECHHUB
The choice of an operating system plays a pivotal role in shaping our computing experience. For decades, Microsoft's Windows has dominated the market, offering a familiar and widely adopted platform for personal and professional use. However, as technological advancements continue to push the boundaries of innovation, alternative operating systems have emerged, challenging the status quo and offering users a fresh perspective on computing.
One such alternative that has garnered significant attention and acclaim is Nitrux Linux 3.5.0, a sleek, powerful, and user-friendly Linux distribution that promises to redefine the way we interact with our devices. With its focus on performance, security, and customization, Nitrux Linux presents a compelling case for those seeking to break free from the constraints of proprietary software and embrace the freedom and flexibility of open-source computing.
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...DanBrown980551
Do you want to learn how to model and simulate an electrical network from scratch in under an hour?
Then welcome to this PowSyBl workshop, hosted by Rte, the French Transmission System Operator (TSO)!
During the webinar, you will discover the PowSyBl ecosystem as well as handle and study an electrical network through an interactive Python notebook.
PowSyBl is an open source project hosted by LF Energy, which offers a comprehensive set of features for electrical grid modelling and simulation. Among other advanced features, PowSyBl provides:
- A fully editable and extendable library for grid component modelling;
- Visualization tools to display your network;
- Grid simulation tools, such as power flows, security analyses (with or without remedial actions) and sensitivity analyses;
The framework is mostly written in Java, with a Python binding so that Python developers can access PowSyBl functionalities as well.
What you will learn during the webinar:
- For beginners: discover PowSyBl's functionalities through a quick general presentation and the notebook, without needing any expert coding skills;
- For advanced developers: master the skills to efficiently apply PowSyBl functionalities to your real-world scenarios.
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...UiPathCommunity
💥 Speed, accuracy, and scaling – discover the superpowers of GenAI in action with UiPath Document Understanding and Communications Mining™:
See how to accelerate model training and optimize model performance with active learning
Learn about the latest enhancements to out-of-the-box document processing – with little to no training required
Get an exclusive demo of the new family of UiPath LLMs – GenAI models specialized for processing different types of documents and messages
This is a hands-on session specifically designed for automation developers and AI enthusiasts seeking to enhance their knowledge in leveraging the latest intelligent document processing capabilities offered by UiPath.
Speakers:
👨🏫 Andras Palfi, Senior Product Manager, UiPath
👩🏫 Lenka Dulovicova, Product Program Manager, UiPath
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdfPaige Cruz
Monitoring and observability aren’t traditionally found in software curriculums and many of us cobble this knowledge together from whatever vendor or ecosystem we were first introduced to and whatever is a part of your current company’s observability stack.
While the dev and ops silo continues to crumble….many organizations still relegate monitoring & observability as the purview of ops, infra and SRE teams. This is a mistake - achieving a highly observable system requires collaboration up and down the stack.
I, a former op, would like to extend an invitation to all application developers to join the observability party will share these foundational concepts to build on:
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024Albert Hoitingh
In this session I delve into the encryption technology used in Microsoft 365 and Microsoft Purview. Including the concepts of Customer Key and Double Key Encryption.
State of ICS and IoT Cyber Threat Landscape Report 2024 previewPrayukth K V
The IoT and OT threat landscape report has been prepared by the Threat Research Team at Sectrio using data from Sectrio, cyber threat intelligence farming facilities spread across over 85 cities around the world. In addition, Sectrio also runs AI-based advanced threat and payload engagement facilities that serve as sinks to attract and engage sophisticated threat actors, and newer malware including new variants and latent threats that are at an earlier stage of development.
The latest edition of the OT/ICS and IoT security Threat Landscape Report 2024 also covers:
State of global ICS asset and network exposure
Sectoral targets and attacks as well as the cost of ransom
Global APT activity, AI usage, actor and tactic profiles, and implications
Rise in volumes of AI-powered cyberattacks
Major cyber events in 2024
Malware and malicious payload trends
Cyberattack types and targets
Vulnerability exploit attempts on CVEs
Attacks on counties – USA
Expansion of bot farms – how, where, and why
In-depth analysis of the cyber threat landscape across North America, South America, Europe, APAC, and the Middle East
Why are attacks on smart factories rising?
Cyber risk predictions
Axis of attacks – Europe
Systemic attacks in the Middle East
Download the full report from here:
https://sectrio.com/resources/ot-threat-landscape-reports/sectrio-releases-ot-ics-and-iot-security-threat-landscape-report-2024/
Climate Impact of Software Testing at Nordic Testing DaysKari Kakkonen
My slides at Nordic Testing Days 6.6.2024
Climate impact / sustainability of software testing discussed on the talk. ICT and testing must carry their part of global responsibility to help with the climat warming. We can minimize the carbon footprint but we can also have a carbon handprint, a positive impact on the climate. Quality characteristics can be added with sustainability, and then measured continuously. Test environments can be used less, and in smaller scale and on demand. Test techniques can be used in optimizing or minimizing number of tests. Test automation can be used to speed up testing.
Pushing the limits of ePRTC: 100ns holdover for 100 daysAdtran
At WSTS 2024, Alon Stern explored the topic of parametric holdover and explained how recent research findings can be implemented in real-world PNT networks to achieve 100 nanoseconds of accuracy for up to 100 days.
Removing Uninteresting Bytes in Software FuzzingAftab Hussain
Imagine a world where software fuzzing, the process of mutating bytes in test seeds to uncover hidden and erroneous program behaviors, becomes faster and more effective. A lot depends on the initial seeds, which can significantly dictate the trajectory of a fuzzing campaign, particularly in terms of how long it takes to uncover interesting behaviour in your code. We introduce DIAR, a technique designed to speedup fuzzing campaigns by pinpointing and eliminating those uninteresting bytes in the seeds. Picture this: instead of wasting valuable resources on meaningless mutations in large, bloated seeds, DIAR removes the unnecessary bytes, streamlining the entire process.
In this work, we equipped AFL, a popular fuzzer, with DIAR and examined two critical Linux libraries -- Libxml's xmllint, a tool for parsing xml documents, and Binutil's readelf, an essential debugging and security analysis command-line tool used to display detailed information about ELF (Executable and Linkable Format). Our preliminary results show that AFL+DIAR does not only discover new paths more quickly but also achieves higher coverage overall. This work thus showcases how starting with lean and optimized seeds can lead to faster, more comprehensive fuzzing campaigns -- and DIAR helps you find such seeds.
- These are slides of the talk given at IEEE International Conference on Software Testing Verification and Validation Workshop, ICSTW 2022.
Welcome to the first live UiPath Community Day Dubai! Join us for this unique occasion to meet our local and global UiPath Community and leaders. You will get a full view of the MEA region's automation landscape and the AI Powered automation technology capabilities of UiPath. Also, hosted by our local partners Marc Ellis, you will enjoy a half-day packed with industry insights and automation peers networking.
📕 Curious on our agenda? Wait no more!
10:00 Welcome note - UiPath Community in Dubai
Lovely Sinha, UiPath Community Chapter Leader, UiPath MVPx3, Hyper-automation Consultant, First Abu Dhabi Bank
10:20 A UiPath cross-region MEA overview
Ashraf El Zarka, VP and Managing Director MEA, UiPath
10:35: Customer Success Journey
Deepthi Deepak, Head of Intelligent Automation CoE, First Abu Dhabi Bank
11:15 The UiPath approach to GenAI with our three principles: improve accuracy, supercharge productivity, and automate more
Boris Krumrey, Global VP, Automation Innovation, UiPath
12:15 To discover how Marc Ellis leverages tech-driven solutions in recruitment and managed services.
Brendan Lingam, Director of Sales and Business Development, Marc Ellis
2. 38 Computer Science & Information Technology (CS & IT)
enhance their overall utility as learning and generalization tools. This work intends to enhance
TREPAN to be able to handle not only multi-class classification type but also multi-class
regression type problems. And also to demonstrate that X-TREPAN can understand and analyze
generalized feed forward networks (GFF). TREPAN is tested on different datasets and best
settings for TREPAN algorithm are explored based on database type to generate heuristics for
various problem domains. The best TREPAN model is then compared to the baseline C4.5
decision tree algorithm to test for accuracy.
Neural networks store their “Knowledge” in a series of real-valued weight matrices representing
a combination of nonlinear transforms from an input space to an output space. Rule extraction
attempts to translate this numerically stored knowledge into a symbolic form that can be readily
comprehended. The ability to extract symbolic knowledge has many potential advantages: the
knowledge obtained from the neural network can lead to new insights into patterns and
dependencies within the data; from symbolic knowledge, it is easier to see which features of the
data are the most important; and the explanation of a decision is essential for many applications,
such as safety critical systems. Andrews et al. and Ticke et al. [9], [10] summarize several
proposed approaches to rule extraction. Many of the earlier approaches required a specialized
neural network architectures or training schemes. This limited their applicability; in particular
they cannot be applied to in situ neural networks. The other approach is to view the extraction
process as learning task. This approach does not examine the weight matrices directly but tries to
approximate the neural network by learning its input-output mappings. Decision trees are a
graphical representation of a decision process. The combination of symbolic information and
graphical presentation make decision trees one of the most comprehensible representations of
pattern recognition knowledge.
2. BACKGROUND AND LITERATURE REVIEW
2.1 Artificial Neural Network
Artificial neural networks as the name implies are modeled on the architecture of the human
brain. They offer a means of efficiently modeling large and complex problems in which there
may be hundreds of independent variables that have many interactions. Neural networks learn
from examples by generating their own implicit rules. The generalization ability of neural
networks has proved to be equal or superior to other learning systems over a wide range of
applications.
2.2 Neural Network Architecture
A neural network consists of a large number of units called processing elements or nodes or
neurons that are connected on a parallel scale. The network starts with an input layer, where each
node corresponds to an independent variable. Input nodes are connected to a number of nodes in
a hidden layer. There may be more than one hidden layer and an output layer. Each node in the
hidden layer takes in a set of inputs (X1, X2, …, Xm), multiplies them by a connection weight
(W1, W2, …, Wm), then applies a function, f(WTX) to them and then passes the output to the
nodes of the next layer. The connection weights are the unknown parameters that are estimated
by an iterative training method to indicate the connection’s strength and excitation. The
calculation of the final outputs of the network proceeds layer by layer [11]. Each processing
element of the hidden layer computes its output as a function of linear combination of inputs
3. Computer Science & Information Technology (CS & IT) 39
from the previous layer plus a bias. This output is propagated as input to the next layer and so on
until the final layer is reached. Figure 1 shows the model of a single neuron [12]
Figure 1. Model of a Single Neuron
The output of the neuron can be expressed as
In the above equations, W is the weight vector of the neural node, defined as
[ ]T
mwwwwW .......,.........,, 321=
and X is the input vector , defined as
[ ]T
mxxxxX .......,.........,, 321=
Figure 2. Shows a typical neural network architecture representation.
Figure 2. Neural Network Architecture
4. 40 Computer Science & Information Technology (CS & IT)
These are different types of activation functions that can be applied at the node of the network.
Two of the most commonly used neural network functions are the hyperbolic and logistic (or
sigmoid) functions. They are sometimes referred to as “squashing” functions since they map the
inputs into a bounded range. Table 1 shows a list of activation functions that are available for use
in neural networks.
Table 1. Activation Functions used in Neural Networks Adapted from [13]
Functions Definition Range
Identity x ( )+∞∞− ,
Logistic
( )x
e−
−1
1 ( )1,0 +
Hyperbolic
Exponential
xx
xx
ee
ee
−
−
+
−
x
e
( )1,1 +−
( )+∞,0
Softmax
∑
−
i
x
x
i
e
e ( )1,0 +
Unit Sum
Square root ∑i
ix
x
x
( )1,0 +
( )+∞,0
Sine
Ramp
Step
( )xSin
+≥+
<−
−≤−
1,1
1,
1,1
x
xx
x
≥+
<
0,1
0,0
x
x
( )1,0 +
( )1,1 +−
( )1,0 +
2.3 Multilayer Perceptrons
Multilayer Perceptrons (MLOs) are layered feed forward networks typically trained with back
propagation. These networks have been used in numerous applications. Their main advantage is
that they are easy to use, and that they can approximate any input/output map. A major
disadvantage is that they train slowly, require lots of training data (typically three times more
training samples then network weights)[14].
5. Computer Science & Information Technology (CS & IT) 41
Figure 3. A schematic Multilayered Perceptron Network
A Generalized Feed Forward (GFF) network is a special case of a Multilayer Perception wherein
connections can jump over one or more layers. Although an MLP can solve any problem that a
GFF can solve, in practice, a GFF network can solve the problem more efficiently [14]. Figure 4
shows a general schematic of a Generalized Feed Forward Network.
Figure 4. Generalized Feed Forward Networks
6. 42 Computer Science & Information Technology (CS & IT)
2.4 Neural Networks for Classification and Regression
Neural networks are one of the most widely used algorithms for classification problems. The
output layer is indicative of the decision of the classifier. The cross entropy error function is most
commonly used in classification problems in combination with logistic or soft max activation
functions. Cross entropy assumes that the probability of the predicated values in a classification
problem lie between 0 and 1. In a classification problem each output node of a neural network
represents a different hypothesis and the node activations represent the probability that each
hypothesis may be true. Each output node represents a probability distribution and the cross
entropy measures calculate the difference between the network distribution and the actual
distribution [15]. Assigning credit risk (good or bad) is an example of a neural network
classification problem. Regression involves prediction the values of a continuous variable based
on previously collected data. Mean square error is the function used for computing the error in
regression networks. Projecting the profit of a company based on previous year’s data is
regression type neural network problem.
2.5 Neural Network Training
The neural network approach is a two stage process. In the first stage a generalized network that
maps the inputs data to the desired output using a training algorithm is derived. The next stage is
the “production” phase where the network is tested for its generalization ability against a new set
of data.
Often the neural network tends to over train and memorizes the data. To avoid this possibility, a
cross-validation data set is use. The cross validation data set is a part of the data set which is set
aside before training and is used to determine the level of generalization produced by the training
set. As training processes the training error drops progressively. At first the cross validation error
decreases but then begins to rise as the network over trains. Best generalization ability of the
network can be tapped by stopping the algorithm where the error on the cross validation set starts
to rise. Figure 5 illustrates the use of cross-validation during training.
Figure 5. Use of cross-Validation during Training
7. Computer Science & Information Technology (CS & IT) 43
2.6 Rule Extraction from Neural Networks
Although neural networks are known to be robust classifiers, they have found limited use in
decision-critical applications such as medical systems. Trained neural networks act like black
boxes and are often difficult to interpret [16]. The availability of a system that would provide an
explanation of the input/output mappings of a neural network in the form of rules would thus be
very useful. Rule extraction is one such system that tries to elucidate to the user, how the neural
network arrived at its decision in the form of if-then rules.
Two explicit approaches have been defined to date for transforming the knowledge and weights
contained in a neural network into a set of symbolic rules de-compositional and pedagogical [17].
In the de-compositional approach the focus is on the extracting rules at an individual hidden
and/or output level into a binary outcome. It involves the analysis of the weight vectors and
biases associated with the processing elements in general. The subset [18] algorithm is an
example of this category. The pedagogical approach treats neural networks like black boxes and
aims to extract rules that map inputs directly to its output. The Validity Interval Analysis (VIA)
[19] proposed by Thrum and TREPAN [20] is an example of one such technique .Andrews et al
[21] proposed a third category called eclectic which combines the elements of the basic
categories.
2.7 Decision Trees
A decision tree is a special type of graph drawn in the form of a tree structure. It consists of
internal nodes each associated with a logical test and its possible consequences. Decision trees
are probably the most widely used symbolic learning algorithms as are neural networks in the
non-symbolic category.
2.8 Decision Tree Classification
Decision trees classify data through recursive partitioning of the data set into mutually exclusive
subsets which best explain the variation in the dependent variable under observation[22][23] .
Decision trees classify instances (data points) by sorting them down the tree from the root node to
some leaf node. This lead node gives the classification of the instance. Each branch of the
decision tree represents a possible scenario of decision and its outcome.
Decision tree algorithms depict concept descriptions in the form of a tree structure. They begin
learning with a set of instances and create a tree structure that is used to classify new instances.
An instance in a dataset is described by a set of feature values called attributes, which can have
either continuous or nominal values. Decision tree induction is best suitable for data where each
example in the dataset is described by a fixed number of attributes for all examples of that
dataset. Decision tree methods use a divide and conquer approach. They can be used to classify
an example by starting at the root of the tree and moving through it until a leaf node is reached,
which provides the classification of the instance.
Each node of a decision tree specifies a test of some attribute and each branch that descends from
the node corresponds to a possible value for this attribute. The following example illustrates a
simple decision tree.
8. 44 Computer Science & Information Technology (CS & IT)
3. TREPAN ALGORITHM
The TREPAN [24] and [25] algorithms developed by Craven et al are novel rule-extraction
algorithms that mimic the behavior of a neural network. Given a trained Neural Network,
TREPAN extracts decision trees that provide a close approximation to the function represented
by the network. In this work, we are concerned with its application to trained variety of learned
models as well. TREPAN uses a concept of recursive partitioning similar to other decision tree
induction algorithms. In contrast to the depth-first growth used by other decision tree algorithms,
TREPAN expands using the best first principle. Thus node which increases the fidelity of the
fidelity of the tree when expanded is deemed the best.
In conventional decision tree induction algorithms the amount of training data decreases as one
traverses down the tree by selecting splitting tests. Thus there is not enough data at the bottom of
the tree to determine class labels and is hence poorly chosen. In contrast TREPAN uses an
‘Oracle’ to answer queries, in addition to the training samples during the inductive learning
process. Since the target here is the function represented by the neural network, the network itself
is used as the ‘Oracle’. This learning from larger samples can prevent the lack of examples for the
splitting tests at lower levels of the tree, which is usually a problem with conventional decision
tree learning algorithms. It ensures that there is a minimum sample of instances available at a
node before choosing a splitting test for that node where minimum sample is one of the user
specified parameters. If the number of instances at the node, say m is less than minimum sample
then TREPAN will make membership queries equal to (minimum sample m) from the ‘Oracle’
and then make a decision at the node. The following illustrates a pseudocode of the TREPAN
algorithm [26].
9. Computer Science & Information Technology (CS & IT) 45
Algorithm : TREPAN
Input: Trained neural network; training examples {ܻܺ} = where yi , is the class label predicted
by the trained neural network on the training example Xi , global stopping criteria.
Output : extracted decision tree
Begin
Initialize the tree as a leaf node
While global stopping criteria are not met and the current tree can be further refined
Do
Pick the most promising leaf node to expand
Draw sample of examples
Use the trained network to label these examples
Select a splitting test for the node
For each possible outcome of the test make a new leaf node
End
End
3.1 M-of-N Splitting tests
TREPAN uses the m-of-n test to partition the part of the instance space covered by a particular
internal node. An m-of-n expression (a Boolean expression) is fulfilled when at least an integer
threshold m of its n literals hold true. For example, consider four features a, b, c and d; the m-of-n
test: 3-of-{a, b > 3.3, c, d} at a node signifies that if any of the 3 conditions of the given set of 4
are satisfied then an example will pass through that node. TREPAN employs a beam search
method with beam width as a user defined parameter to find the best m-of-n test. Beam search is
heuristic best-first each algorithm that evaluates that first n node (where n is a fixed value called
the ‘beam width’) at each tree depth and picks the best out of them for the split. TREPAN uses
both local and global stopping criteria. The growth of the tree stops when any of the following
criteria are met: the size of the tree which is a user specific parameter or when all the training
examples at node fall in the same class.
3.2 Single Test TREPAN and Disjunctive TREPAN
In addition to TREPAN algorithm, Craven has also developed two of its important variations.
The single test TREPAN algorithm is similar to TREPAN in all respects except that as its name
suggests it uses single feature tests at the internal nodes. Disjunctive TREPAN on the other hand,
10. 46 Computer Science & Information Technology (CS & IT)
uses disjunctive “OR” tests at the internal nodes of the tree instead of the m-of-n tests. A more
detailed explanation of the TREPAN algorithm can be found in Craven’s dissertation [27].
Baesens et al [28] have applied TREPAN to credit risk evaluation and reported that it yields very
good classification accuracy as compared to the logistic regression classifier and the popular C4.5
algorithm.
4. C4.5 ALGORITHM
The C4.5 algorithm [29] is one of the most widely used decision tree learning algorithms. It is an
advanced and incremental software extension of the basic ID3 algorithm [30] designed to address
the issues that were not dealt with by ID3. The C4.5 algorithm has its origins in Hunt’s Concept
Learning Systems (CLS) [31]. It is a non-incremental algorithm, which means that it derives its
classes from an initial set of training instances. The classes derived from these instances are
expected to work for all future test instances. The algorithm uses the greedy search approach to
select the best attribute and never looks back to reconsider earlier choices. The C4.5 algorithm
searches through the attributes of the training instances and finds the attribute that best separates
the data. If this attribute perfectly classifies the training set then it stops else it recursively works
on the remaining in the subsets (m = the remaining possible values of the attribute) to get their
best attribute. Some attributes split the data more purely than others. Their values correspond
more consistently with instances that have particular values of the target class. Therefore it can be
said that they contain more information than the other attributes. But there should be a method
that helps quantify this information and compares different attributes in the data which will
enable us to decide which attribute should be placed at the highest node in the tree.
4.1 Information Gain, Entropy Measure and Gain Ratio
A fundamental part of any algorithm that constructs a decision tree from a dataset is the method
in which it selects attributes at each node of the tree for splitting so that the depth of the tree is the
minimum. ID3 uses the concept of Information Gain which is based on Information theory [32]
to select the best attributes. Gain measures how well a given attribute separates training examples
into its target classes. The one with the highest information is selected. Information gain
calculates the reduction in entropy (or gain information) that would result from splitting the data
into subsets based on an attribute.
The information gain of example set S on attribute A is defined as,
( ) ( ) ( )∑−= v
v
SEntropy
S
S
SEntropyASGain ,
In the above equation, S is the number of instances and |Sv| is a subset of instances of S where A
takes the value v. Entropy is a measure of the amount of information in an attribute. The higher
the entropy, the more the information is required to completely describe the data. Hence, when
building the decision tree, the idea is to decrease the entropy of the dataset until we reach a subset
that is pure (a leaf), that has zero entropy and represents instances that all belong to one class.
Entropy is given by,
( ) ( ) ( )∑−= IpIpSEntropy 2log
Eq.1
Eq.2
11. Computer Science & Information Technology (CS & IT) 47
where p(I) is the proportion of S belonging to Class I.
Suppose we are constructing a decision tree with ID3 that will enable us to decide if the weather
is favorable to play football. The input data to ID3 is shown in table 2 below adapted from
Quinlan’s C4.5.
Table 2. Play Tennis Examples Dataset
Day Outlook Temperature Humidity Wind Play Tennis
1 Sunny Hot High Weak No
2 Sunny Hot High Strong No
3 Overcast Hot High Weak Yes
4 Rain Mild High Weak Yes
5 Rain Cool Normal Weak Yes
6 Rain Cool Normal Strong No
7 Overcast Cool Normal Strong Yes
8 Sunny Mild High Weak No
9 Sunny Cool Normal Weak Yes
10 Rain Mild Normal Weak Yes
11 Sunny Mild Normal Strong Yes
12 Overcast Mild High Strong Yes
13 Overcast Hot Normal Weak Yes
14 Rain Mild High Strong No
In this example,
( ) 9450.0
14
5
log
14
5
14
9
log
14
9
22 =
−
−=SEntropy
(Note: Number of instances where play tennis = yes is 9 and play tennis = No is 5)
The best attribute of the four is selected by calculating the Information Gain for each attribute as
follows,
( ) ( ) ( ) ( ) ( )RainEntrOvercastEntrSunnyEntrSEntrOutlookSGain .
14
5
.
14
4
.
14
5
., −−−=
( ) 2670.03364.003364.09450.0, =−−−=OutlookSGain
Similarly, ( ) 42.0, −=TempSGain and ( ) 1515.0, =WindSGain
The attribute outlook has the highest gain and hence it is used as the decision attribute in the root
node. The root node has three branches since the attribute outlook has three possible values,
(Sunny, Overcast, and Rain). Only the remaining attributes are tested at the sunny branch node
since outlook has already been used at the node. This process is recursively repeated until: all the
training instances have been classified or every attribute has been utilized in the decision tree.
The ID3 has a strong bias in favor of tests with many outcomes. Consider an employee database
that consists of an employee identification number. Every attribute intended to be unique and
Eq.3
Eq.4
12. 48 Computer Science & Information Technology (CS & IT)
partitioning any set of training cases on the values of this attribute will lead to a large number of
subsets, each containing only one case. Hence the C4.5 algorithm incorporates use of a statistic
called the “Gain Ratio” that compensates for the number of attributes by normalizing with
information encoded in the split itself.
( )
( )AI
ASGain
GainRatio
,
=
In the above equation,
( ) ( ) ( )∑−= AA IpIpAI 2log
C4.5 has another advantage over ID3; it can deal with numeric attributes, missing values and
noisy data.
5. EXPERIMENTATION AND RESULT ANALYSIS
We analyze three datasets with classes greater than two and we compare the results of Single-test
TREPAN and C4.5 with that of X-TREPAN in terms of comprehensibility and classification
accuracy. A generalized feed forward network was trained in order to investigate the ability of X-
TREPAN in comprehending GFF networks. The traditional ‘using-network’ command was used
to validate that X-TREPAN was producing correct outputs for the network. In all the
experiments, we adopted the Single-test TREPAN as the best variant for comparison with the
new model.
5.1 Body Fat
Body Fat is a regression problem in the simple machine learning dataset category. The instances
are sought to predict body fat percentage based on body characteristics. A 14-4-1 MLP with
hyperbolic tangent function was used to train the network for 1500 epochs giving an r
(correlation co-efficient) value of 0.9882. Figure 6 shows the comparison of classification
accuracy of body fat by the three models.
Figure 6. Comparison of classification accuracy of Body fat by the three algorithms
Eq.5
Eq.6
13. Computer Science & Information Technology (CS & IT) 49
TREPAN achieves a classification accuracy of 94% and C4.5 produces a classification accuracy
of 91% while X-TREPAN achieves a comparatively much higher accuracy of 96%. Additionally,
both X-TREPAN and TREPAN generate similar trees in terms of size but accuracy and
comprehensibility attained by X-TREPAN are comparatively higher.
The tables below show the confusion matrix of the classification accuracy achieved by TREPAN
in comparison with X-TREPAN. While TREPAN produces a classification accuracy of 92.06%
X-TREPAN produces a comparatively much higher accuracy of 96.83% as indicated in Table 3
below.
Table 3. Body Fat Confusion Matrix (X-TREPAN)
Actual/Predicted Toned Healthy Flabby Obese
Toned 13 0 0 0
Healthy 1 21 0 0
Flabby 0 0 9 1
Obese 0 0 0 18
Classification Accuracy
(%)
92.86% 100.00% 100.00% 94.74%
Total Accuracy (%) 96.83%
Table 4. Body Fat Confusion Matrix (TREPAN)
Actual/Predicted Toned Healthy Flabby Obese
Toned 13 0 0 0
Healthy 1 20 0 0
Flabby 0 0 9 0
Obese 0 0 3 16
Classification Accuracy
(%)
92.86% 95.24% 75.00% 100.00%
Total Accuracy (%) 92.06%
Additionally, both TREPAN and X-TREPAN generate identical trees in terms of size but
accuracy attained by X-TREPAN is comparatively higher.
5.2 Outages
Outages constitute a database from the small dataset category. A12-3-1 MLP network with a
hyperbolic tangent and bias axon transfer functions in the first and the second hidden layer
respectively gave the best accuracy. The model was trained for 12000 epochs and achieved an r
(correlation co-efficient) value of 0.985 (or an r2 of (0.985)2). Figure 7 shows the comparison of
classification accuracy of outages by the three algorithms.
14. 50 Computer Science & Information Technology (CS & IT)
Figure 7. Comparison of classification accuracy of Outages by the three algorithms
In terms of classification accuracy, as can be seen in the figure above, TREPAN achieves 84%,
C4.5 achieves 91% while X-TREPAN achieves 92%.
However, here TREPAN, C4.5 and X-TREPAN all generate very different trees in terms of size
with C4.5 producing the largest and most complex decision tree while X-TREPAN produces the
simplest and smallest decision tree with comparatively higher accuracy and comprehensibility.
The tables below show the confusion matrix of the classification accuracy achieved by both
algorithms. X-TREPAN achieves 85% while TREPAN achieves 76%.
Table 5. Outages Confusion Matrix (X-TREPAN)
Actual/Predicted C11 C12 C13 C14 C15
C11 3 0 0 0 0
C12 4 48 5 0 0
C13 0 1 8 0 0
C14 0 0 1 5 0
C15 0 0 0 0 0
Classification Accuracy
(%)
42.86% 97.96% 57.14% 100% 0.00%
Total Accuracy (%) 85.33%
Table 6. Outages Confusion Matrix (TREPAN)
Actual/Predicted C11 C12 C13 C14 C15
C11 2 5 0 0 0
C12 3 43 3 0 0
C13 0 6 7 1 0
C14 0 0 0 5 0
C15 0 0 0 0 0
Classification Accuracy
(%)
40.00% 79.63% 70.00% 83.33% 0.00%
Total Accuracy (%) 76.00%
15. Computer Science & Information Technology (CS & IT) 51
5.3 Admissions
A typical University admissions database model based on a 22-15-10-2 MLP network. Two
hidden layers with the hyperbolic tangent transfer functions were used for modeling. The best
model was obtained by the X-TREPAN with a minimum sample size of 1000, tree size of 50 and
classification accuracy of 74%. Figure 8 gives the comparison of classification accuracy of
Admissions by the three models.
Figure 8. Comparison of classification accuracy of Admissions by the three algorithms
On the other hand, C4.5 achieved an accuracy of 71.97% (not rounded) almost equaling that of
TREPAN of 72%, but produced a significantly large and complex decision tree.
In terms of Confusion Matrix, TREPAN achieved an accuracy of 71.6% very close to that of X-
TREPAN. The confusion matrix is shown in the Tables below.
Table 7. Admissions Confusion Matrix (X-TREPAN)
Actural/Predicted Yes No
Yes 401 279
No 168 754
Classification Accuracy (%) 70.47% 72.99
Total Accuracy (%) 72.10%
Table 8. Admissions Confusion Matrix (TREPAN)
Actural/Predicted Yes No
Yes 379 190
No 259 774
Classification Accuracy (%) 59.40% 80.29
Total Accuracy (%) 71.67%
16. 52 Computer Science & Information Technology (CS & IT)
6. PERFORMANCE ASSESSMENT
6.1 Classification Accuracy
The classification accuracy or error rate is the percentage of correct predictions made by the
model over a data set. It is assessed using the confusion matrix. A confusion matrix is a matrix
plot of predicted versus actual classes with all of the correct classifications depicted along the
diagonal of the matrix. It gives the number of correctly classified instances, incorrectly classified
instances and overall classification accuracy.
The accuracy of the classifier is given by the formula,
( ) ( ) 100
)(
% ×
+++
+
=
TNFPFNTP
TNTP
Accuracy
Where true positive = (TP), true negative = (TN) false positive = (FP) and false negative = (FN).
A false positive (FP) is when a negative instance incorrectly classified as a positive and false
negative (FN) is when a positive instance is incorrectly classified as a negative. A true positive
(TP) is when an instance is correctly classified as positive and true negative (TN) is when an
instance is correctly classified as negative and so on.
A confusion matrix is a primary tool in visualizing the performance of a classifier. However it
does not take into account the fact that some misclassifications are worse than others. To
overcome this problem we use a measure called the Kappa Statistic which considers the fact that
correct values in a confusion matrix are due to chance agreement.
The Kappa statistic is defined as,
( )
)(1
)()(^
EP
EPAP
k
−
−
=
In this equation, P(A) is the proportion of times the model values were equal to the actual value
and, P(E) is the expected proportion by Chance.
For perfect agreement, Kappa = 1. For example: a Kappa statistic of 0.84 would imply that the
classification process was avoiding 84% of the errors that a completely random classification
would generate.
6.2 Comprehensibility
The comprehensibility of the tree structure decreases with the increase in the size and complexity.
The principle of Occam’s Razors says “when you have two competing theories which make
exactly the same projections, the one that is simpler is the better” [33]. Therefore, among the
three algorithms, X-TREPAN is better as it produces smaller and simpler trees as against Single-
test TREPAN and C4.5 in most scenarios.
Eq.5
Eq.6
17. Computer Science & Information Technology (CS & IT) 53
7. CONCLUSION
The TREPAN algorithm code was modified (X-TREPAN) to be able to work with multi-class
regression type problems. Various experiments were run to investigate its compatibility with
generalized feed forward networks. The weights and network file were restructured to present
GFF networks in a format recognized by X-TREPAN. Neural Network models were trained on
each dataset varying parameters like network architecture and transfer functions. The weights and
biases obtained from the trained models of the three datasets were fed to X-TREPAN for decision
tree learning from neural networks. For performance assessment, classification accuracy of
Single-test TREPAN, C4.5 and X-TREPAN were compared. In the scenarios discussed in the
paper, X-TREPAN model significantly outperformed the Single-test TREPAN and C4.5
algorithms in terms of classification accuracy as well as size, complexity and comprehensibility
of decision trees. To validate the results, we use classification accuracy not as the only measure
of performance, but also the kappa statistics. The kappa statistical values further validate the
conclusions that X-TREPAN is a better one in terms of decision tree induction.
REFERENCES
[1] A. B. Ticle, R. Andrews, M. Golea, J. Diederich The truth will come to light: Directions and
challenges in extracting the knowledge embedded within trained artificial neural networks, IEEE
Trans. Neural Networks, vol. 9, no 6, pp. 1057-1068. 1998.
[2] G. Papantonopoulos, K. Takahashi, T. Bountis, B. G. Loos, Artificial Neural Networks for the
Diagnosis of Aggressive Periodontitis Trained by Immunologic Parameters. PLoS ONE 9(3): e89757.
(doi:10.1371/journal.pone.0089757), 2014.
[3] D. Puzenat, Behavior Analysis through Games Using Artificial Neural Networks, Third International
Conference on Advances in Computer-Human Interactions (IEEE), pp. 134–138, Saint Maarten,
France, 2010.
[4] M. Mano, G. Capi, N. Tanaka, S. Kawahara, An Artificial Neural Network Based Robot Controller
that Uses Rat’s Brain Signals, Robotics, 2,54-65, doi:10.3390/robotics2020054.
(www.mdpi.com/journal/robotics), 2013.
[5] E. V. Raghavendra, P. Vijayaditya, K. Prahallad, speech generation National Conference on
Communications (NCC), Hyderabad, India 2010. (doi: 10.1109/NCC.2010.5430190)
[6] A. Hossain, M. Rahman, U. K.Prodhan, F. Khan, Implementation Of Back-Propagation Neural
Network For Isolated Bangla Speech Recognition, International Journal of Information Sciences and
Techniques (IJIST) vol. 3, no. 4, 2013.
[7] S. Ayat, Z. A. Pour, Comparison between Artificial Neural Network Learning Algorithms for
Prediction of Student Average considering Effective Factors in Learning and Educational Progress,
Journal of mathematics and computer science 8, pp. 215 – 225, 2014.
[8] R. Andrews, J. Diedrich, A. Ticle, Survey and critique of techniques for extracting rules from trained
artificial neural networks, Knowledge-Based System, vol. 8, no 6, pp. 373-389. 1995.
[9] A. B. Ticke, R. Andrews, M. Golea, J. Diederich, The truth will come to light: Directions and
challenges in extracting the knowledge embedded within trained artificial neural networks, IEEE
Trans. Neural Network, vol. 9 no. 6, pp. 1057 - 1068, 1998.
[10] A. Ticke, F. Maire, G. Bologna, R. Andrews, J. Diederich, Lessons from past current issues and
future research directions in extracting knowledge embedded in artificial neural networks in Hybrid
Neural Systems, New York: Springer-Verlag, 2000.
[11] H. Lohminger, Teach/Me Data Analysis, Springer-Verlag, Berlin-New York-Tokyo, 1999.
[12] C. Aldrich, Exploratory analysis of metallurgical process data with neural networks and related
methods, Elsevier 2002.
[13] StatSoft, Inc. (2004). Electronic Statistics Textbook. Tulsa, OK: StatSoft.
[14] M. M. Nelson, W.W. Illingworth, A Practical Guide to Nerual Nets. 4th ed. 1992.
18. 54 Computer Science & Information Technology (CS & IT)
[15] K. Plunkeet, J. L. Elman, (1997). Exercises in Rethinking Innateness: A handbook for connectionist
simulations, MIT Press, pp. 166.
[16] G.G. Towell, J. W. Shavli, Extracting Refined Rules from Knowledge-based Neural Networks,
Machine Learning, 13:71-101 1993.
[17] M.W Craven, J. W. Shavlik, Using sampling and queries to extract rules from trained neural
networks, Machine Learning. Proceedings of the Eleventh International Conference, Cohen W. W &
Hirsh H.(Eds), San Francisco, CA:Morgan Kaufmann 1995.
[18] L. Fu, Rule Learning by searching on adopted nets, In Proceedings of the 9th National Conference on
Artificial Intelligence, Anaheim, CA, pp.590-595, 1991.
[19] S. Thrums, Extracting rules from artificial neural networks with distributed representations, In
Advances in Neural Information Processing Systems 7, Cambridge, MA:Mit press, pp 505-512, 1995.
[20] M. W. Craven, Extracting Comprehensible models from trained Neural Network, PhD Thesis,
Computer Science Department , University of Wisconsin, Madison, WI, 1996.
[21] R. Andrews, J. Diederich, A. B. Tickle, A survey and critique of techniques for extracting rules from
trained neural networks. Knowledge Based Systems, 8(6), 373-389, 1995.
[22] D. Biggs, B. de Ville, E. Suen, A method of choosing multiway partitions for classification and
decision tree. Journal of Applied Statistics 18(1): 49-62, 1991.
[23] G. Liepins, R. Goeltz, R. Rush, Machine Learning techniques for natural resource data analysis . AI
Applications 4(3):9-18, 1990.
[24] M. W. Craven, J. W. Shavlik, Using sampling and queries to extract rules from trained neural
networks, Machine Learning. Proceedings of the Eleventh Inter-national Conference, Cohen W. W &
Hirsh H. (Eds), San Francisco, CA. 1995.
[25] M. W. Craven, J. W. Shavlik, Extracting tree-structured representations of trained networks. In
Advance in Neural Information Procession Systems, volume 8, pages 24-30, Denver, Com, MIT
Press. 1996.
[26] F. Chen, Learning accurate and understandable rules from SVM classifiers, M. Sc. Thesis, School of
computing science, Simon Fraser University. 2004.
[27] M.W. Craven, Extracting Comprehensible models from trained Neural Network, PhD Thesis,
Computer Science Department, University of Wisconsin, Madison, WI. 1996.
[28] B. Baesens, R. Setiono, C. Mues, J. Vanthienen, Using Neural Network Rule Extraction and
Decision Tables for Credit-Risk Evaluation management Science, vol. 49, no. 3, pp. 312, 2003.
[29] J. R. Quinlan, C4.5: Programs for Machine Learning. San Mateo, CA: Morgan Kaufmann, 1993.
[30] J. Ross Quinlan, ID3 Algorithm, Machine Learning, University of Sydney, book vol. 1, no.1, Syney,
Australia. 1975.
[31] E. Hunt, J. Marin, P. Stone, Experiments in Induction, New York, Academic Press, 1966.
[32] L. J. Breiman, R. Friedman, C. Olshen, Classification and regression trees, Wadsworth and Brooks,
Belmont, CA. 1984.
[33] L. Breiman, J. Friedman, R. Olshen, C. Stone, Classification and regression trees, Wadsworth and
Brooks, Belmont, CA. 1984.