This document summarizes and evaluates various rule extraction algorithms from trained artificial neural networks. It begins with an introduction explaining the importance of explanation capabilities for neural networks. It then provides a taxonomy for classifying rule extraction approaches based on the expressiveness of the extracted rules, whether the approach takes an open-box or black-box view of the neural network, any specialized training regimes used, the quality of explanations generated, and computational complexity. The document discusses sensitivity analysis as a basic method for understanding neural network relationships before focusing on decompositional and pedagogical rule extraction approaches.
This document summarizes an article about using an artificial bee colony (ABC) algorithm to extract knowledge from numerical data to generate fuzzy rules. The ABC algorithm is an optimization technique inspired by honeybee behavior that can be used for data-driven modeling when domain experts are unavailable. The article describes fuzzy systems and their components, defines the problem of generating fuzzy rules from data as a minimization problem, and provides an example of applying the ABC algorithm to generate rules for a rapid battery charger system based on temperature and charging rate data.
IRJET-In sequence Polemical Pertinence via Soft Enumerating RepertoireIRJET Journal
This document discusses the use of soft computing techniques in image forensics. It begins by defining soft computing as a multi-disciplinary field involving fuzzy logic, neural networks, evolutionary algorithms, and probabilistic reasoning. It then discusses several soft computing techniques - fuzzy logic, neural networks, and genetic algorithms - and how they can help address challenges in image forensics by dealing with imprecision and uncertainty. The document concludes that soft computing tools show promise for analyzing the large amounts of data involved in supply chain management problems and aiding managers' decision making in complex environments. It identifies several areas where further research could help improve solutions or develop new approaches by integrating additional algorithms.
This document summarizes a study that compares fuzzy logic and neuro-fuzzy models for predicting direct current in motors. Fuzzy logic and neuro-fuzzy systems were used to model the relationship between motor torque, power, speed (inputs) and current (output). Both techniques were tested on a dataset of 507 samples. The neuro-fuzzy inference system (ANFIS) performed slightly better than the fuzzy logic system at predicting motor current, demonstrating the benefits of combining fuzzy logic with neural networks.
UTILIZING XAI TECHNIQUE TO IMPROVE AUTOENCODER BASED MODEL FOR COMPUTER NETWO...IJCNCJournal
Machine learning (ML) and Deep Learning (DL) methods are being adopted rapidly, especially in computer network security, such as fraud detection, network anomaly detection, intrusion detection, and much more. However, the lack of transparency of ML and DL based models is a major obstacle to their implementation and criticized due to its black-box nature, even with such tremendous results. Explainable Artificial Intelligence (XAI) is a promising area that can improve the trustworthiness of these models by giving explanations and interpreting its output. If the internal working of the ML and DL based models is understandable, then it can further help to improve its performance. The objective of this paper is to show that how XAI can be used to interpret the results of the DL model, the autoencoder in this case. And, based on the interpretation, we improved its performance for computer network anomaly detection. The kernel SHAP method, which is based on the shapley values, is used as a novel feature selection technique. This method is used to identify only those features that are actually causing the anomalous behaviour of the set of attack/anomaly instances. Later, these feature sets are used to train and validate the autoencoderbut on benign data only. Finally, the built SHAP_Model outperformed the other two models proposed based on the feature selection method. This whole experiment is conducted on the subset of the latest CICIDS2017 network dataset. The overall accuracy and AUC of SHAP_Model is 94% and 0.969, respectively.
COMPARISON OF ARTIFICIAL NEURAL NETWORKS AND FUZZY LOGIC APPROACHES FOR CRACK...ijaia
This paper proposes two algorithms of crack detection one using fuzzy logic (FL) and the other artificial neural networks (ANN). Since modal parameters are very sensitive to damages, the first three relative natural frequencies are used as three inputs and the corresponding relative crack location, relative crack depth are used as the two outputs in the algorithms. The three natural frequencies for an undamaged beam and different cases of damaged beam (Single crack at various locations with varying depths) were obtained by modelling and simulating the beams using a finite element based (FEM) software. Results concluded that both the approaches can be successfully employed in crack detection in a beam like structure but FL approach performed better in determining relative crack depth whereas ANN approach performed better in determining relative crack location. All the comparisons made in the study are based on the R2 values.
Privacy Preserving Reputation Calculation in P2P Systems with Homomorphic Enc...IJCNCJournal
This document discusses a method for privacy-preserving reputation calculation in peer-to-peer systems using homomorphic encryption. Specifically, it proposes:
1) Extending the EigenTrust reputation system to calculate node reputations in a distributed manner while preserving evaluator privacy. It does this by successively updating encrypted reputation values through calculation to reflect trust values without disclosing the original values.
2) Improving calculation efficiency by offloading parts of the task to participating nodes and using different public keys during calculation to improve robustness against node churn.
3) Evaluating the performance of the proposed method, finding it reduces maximum circulation time for aggregating multiplication results by half, reducing computation time per round. The privacy preservation cost scales
Artificial Neural Networks: Applications In ManagementIOSR Journals
With the advancement of computer and communication technology, the tools used for management decisions have undergone a gigantic change. Finding the more effective solution and tools for managerial problems is one of the most important topics in the management studies today. Artificial Neural Networks (ANNs) are one of these tools that have become a critical component for business intelligence. The purpose of this article is to describe the basic behavior of neural networks as well as the works done in application of the same in management sciences and stimulate further research interests and efforts in the identified topics.
A NOVEL EVALUATION APPROACH TO FINDING LIGHTWEIGHT MACHINE LEARNING ALGORITHM...IJNSA Journal
Building practical and efficient intrusion detection systems in computer network is important in industrial areas today and machine learning technique provides a set of effective algorithms to detect network
intrusion. To find out appropriate algorithms for building such kinds of systems, it is necessary to evaluate various types of machine learning algorithms based on specific criteria. In this paper, we propose a novel evaluation formula which incorporates 6 indexes into our comprehensive measurement, including precision, recall, root mean square error, training time, sample complexity and practicability, in order to
find algorithms which have high detection rate, low training time, need less training samples and are easy
to use like constructing, understanding and analyzing models. Detailed evaluation process is designed to
get all necessary assessment indicators and 6 kinds of machine learning algorithms are evaluated.
Experimental results illustrate that Logistic Regression shows the best overall performance.
This document summarizes an article about using an artificial bee colony (ABC) algorithm to extract knowledge from numerical data to generate fuzzy rules. The ABC algorithm is an optimization technique inspired by honeybee behavior that can be used for data-driven modeling when domain experts are unavailable. The article describes fuzzy systems and their components, defines the problem of generating fuzzy rules from data as a minimization problem, and provides an example of applying the ABC algorithm to generate rules for a rapid battery charger system based on temperature and charging rate data.
IRJET-In sequence Polemical Pertinence via Soft Enumerating RepertoireIRJET Journal
This document discusses the use of soft computing techniques in image forensics. It begins by defining soft computing as a multi-disciplinary field involving fuzzy logic, neural networks, evolutionary algorithms, and probabilistic reasoning. It then discusses several soft computing techniques - fuzzy logic, neural networks, and genetic algorithms - and how they can help address challenges in image forensics by dealing with imprecision and uncertainty. The document concludes that soft computing tools show promise for analyzing the large amounts of data involved in supply chain management problems and aiding managers' decision making in complex environments. It identifies several areas where further research could help improve solutions or develop new approaches by integrating additional algorithms.
This document summarizes a study that compares fuzzy logic and neuro-fuzzy models for predicting direct current in motors. Fuzzy logic and neuro-fuzzy systems were used to model the relationship between motor torque, power, speed (inputs) and current (output). Both techniques were tested on a dataset of 507 samples. The neuro-fuzzy inference system (ANFIS) performed slightly better than the fuzzy logic system at predicting motor current, demonstrating the benefits of combining fuzzy logic with neural networks.
UTILIZING XAI TECHNIQUE TO IMPROVE AUTOENCODER BASED MODEL FOR COMPUTER NETWO...IJCNCJournal
Machine learning (ML) and Deep Learning (DL) methods are being adopted rapidly, especially in computer network security, such as fraud detection, network anomaly detection, intrusion detection, and much more. However, the lack of transparency of ML and DL based models is a major obstacle to their implementation and criticized due to its black-box nature, even with such tremendous results. Explainable Artificial Intelligence (XAI) is a promising area that can improve the trustworthiness of these models by giving explanations and interpreting its output. If the internal working of the ML and DL based models is understandable, then it can further help to improve its performance. The objective of this paper is to show that how XAI can be used to interpret the results of the DL model, the autoencoder in this case. And, based on the interpretation, we improved its performance for computer network anomaly detection. The kernel SHAP method, which is based on the shapley values, is used as a novel feature selection technique. This method is used to identify only those features that are actually causing the anomalous behaviour of the set of attack/anomaly instances. Later, these feature sets are used to train and validate the autoencoderbut on benign data only. Finally, the built SHAP_Model outperformed the other two models proposed based on the feature selection method. This whole experiment is conducted on the subset of the latest CICIDS2017 network dataset. The overall accuracy and AUC of SHAP_Model is 94% and 0.969, respectively.
COMPARISON OF ARTIFICIAL NEURAL NETWORKS AND FUZZY LOGIC APPROACHES FOR CRACK...ijaia
This paper proposes two algorithms of crack detection one using fuzzy logic (FL) and the other artificial neural networks (ANN). Since modal parameters are very sensitive to damages, the first three relative natural frequencies are used as three inputs and the corresponding relative crack location, relative crack depth are used as the two outputs in the algorithms. The three natural frequencies for an undamaged beam and different cases of damaged beam (Single crack at various locations with varying depths) were obtained by modelling and simulating the beams using a finite element based (FEM) software. Results concluded that both the approaches can be successfully employed in crack detection in a beam like structure but FL approach performed better in determining relative crack depth whereas ANN approach performed better in determining relative crack location. All the comparisons made in the study are based on the R2 values.
Privacy Preserving Reputation Calculation in P2P Systems with Homomorphic Enc...IJCNCJournal
This document discusses a method for privacy-preserving reputation calculation in peer-to-peer systems using homomorphic encryption. Specifically, it proposes:
1) Extending the EigenTrust reputation system to calculate node reputations in a distributed manner while preserving evaluator privacy. It does this by successively updating encrypted reputation values through calculation to reflect trust values without disclosing the original values.
2) Improving calculation efficiency by offloading parts of the task to participating nodes and using different public keys during calculation to improve robustness against node churn.
3) Evaluating the performance of the proposed method, finding it reduces maximum circulation time for aggregating multiplication results by half, reducing computation time per round. The privacy preservation cost scales
Artificial Neural Networks: Applications In ManagementIOSR Journals
With the advancement of computer and communication technology, the tools used for management decisions have undergone a gigantic change. Finding the more effective solution and tools for managerial problems is one of the most important topics in the management studies today. Artificial Neural Networks (ANNs) are one of these tools that have become a critical component for business intelligence. The purpose of this article is to describe the basic behavior of neural networks as well as the works done in application of the same in management sciences and stimulate further research interests and efforts in the identified topics.
A NOVEL EVALUATION APPROACH TO FINDING LIGHTWEIGHT MACHINE LEARNING ALGORITHM...IJNSA Journal
Building practical and efficient intrusion detection systems in computer network is important in industrial areas today and machine learning technique provides a set of effective algorithms to detect network
intrusion. To find out appropriate algorithms for building such kinds of systems, it is necessary to evaluate various types of machine learning algorithms based on specific criteria. In this paper, we propose a novel evaluation formula which incorporates 6 indexes into our comprehensive measurement, including precision, recall, root mean square error, training time, sample complexity and practicability, in order to
find algorithms which have high detection rate, low training time, need less training samples and are easy
to use like constructing, understanding and analyzing models. Detailed evaluation process is designed to
get all necessary assessment indicators and 6 kinds of machine learning algorithms are evaluated.
Experimental results illustrate that Logistic Regression shows the best overall performance.
Genome structure prediction a review over soft computing techniqueseSAT Journals
Abstract There are some techniques like spectrometry or crystallography for the determination of DNA, RNA or protein structures. These processes provide very accurate results for the structure estimation. But these conventional techniques are very slow and could be applied over a few special cases only. Soft computing techniques guarantee a near appropriate results in much smaller time and have very large applicability. These techniques are much easier to apply. Different approaches have been used in soft computing including nature inspired computing for estimation of genome structures with a considerable accuracy of results. This paper provides a review over different soft computing techniques been applied along with application method for the determination of genome structure. Keywords—DNA, RNA, proteins, structure, soft computing, techniques.
C LUSTERING B ASED A TTRIBUTE S UBSET S ELECTION U SING F AST A LGORITHmIJCI JOURNAL
In machine learning and data mining, attribute sel
ect is the practice of selecting a subset o
f most
consequential attributes for utilize in model const
ruction. Using an attribute select method is that t
he data
encloses many redundant or extraneous attributes. W
here redundant attributes are those which sup
ply
no supplemental information than the presently
selected attributes, and impertinent attribut
es offer
no valuable information in any context
Performance Analysis of Various Data Mining Techniques on Banknote Authentica...inventionjournals
In this paper, we describe the functionality features for authenticating in Euro banknotes. We applied different data mining algorithms such as KMeans, Naive Bayes, Multilayer Perceptron, Decision trees (J48), and Expectation-Maximization(EM) to classifying banknote authentication dataset. The experiments are conducted in WEKA. The goal of this project is to obtain the higher authentication rate in banknote classification
Adaptive Neural Fuzzy Inference System for Employability AssessmentEditor IJCATR
Employability is potential of a person for gaining and maintains employment. Employability is measure through the
education, personal development and understanding power. Employability is not the similar as ahead a graduate job, moderately it
implies something almost the capacity of the graduate to function in an employment and be capable to move between jobs, therefore
remaining employable through their life. This paper introduced a new adaptive neural fuzzy inference system for assessment of
employability with the help of some neuro fuzzy rules. The purpose and scope of this research is to examine the level of employability.
The concern research use both fuzzy inference systems and artificial neural network which is known as neuro fuzzy technique for
solve the problem of employability assessment. This paper use three employability skills as input and find a crisp value as output
which indicates the glassy of employee. It uses twenty seven neuro fuzzy rules, with the help of Sugeno type inference in Mat-lab and
finds single value output. The proposed system is named as Adaptive Neural Fuzzy Inference System for Employability Assessment
(ANFISEA).
Comparison of fuzzy neural clustering based outlier detection techniquesIAEME Publication
The document compares fuzzy-neural clustering based outlier detection techniques. It discusses how fuzzy logic can handle uncertainty and neural networks can learn and adapt. It provides an overview of fuzzy clustering based outlier detection techniques like fuzzy c-means clustering, which allows data points to belong to multiple clusters to varying degrees. It also discusses neural network based outlier detection. The document aims to compare outlier detection techniques involving fuzzy and/or neural approaches based on clustering, focusing on their strengths and weaknesses.
Survey on Supervised Method for Face Image Retrieval Based on Euclidean Dist...Editor IJCATR
This document summarizes various supervised methods for face image retrieval based on Euclidean distance. It discusses literature on active shape models, principal component analysis, linear discriminant analysis, locality-constrained linear coding, bag-of-words models, local binary patterns, and support vector machines. It evaluates support vector machines as the best classifier for face image retrieval systems due to its ability to significantly reduce the need for labeled training data and accurately classify faces, proteins, and characters. The document concludes that a content-based face retrieval system using support vector machines improves detection performance by retrieving similar faces from a database based on Euclidean distance calculations between local binary pattern features of the query and database images.
Minkowski Distance based Feature Selection Algorithm for Effective Intrusion ...IJMER
Intrusion Detection System (IDS) plays a major role in the provision of effective security to various types of networks. Moreover, Intrusion Detection System for networks need appropriate rule set for classifying network bench mark data into normal or attack patterns. Generally, each dataset is characterized by a large set of features. However, all these features will not be relevant or fully contribute in identifying an attack. Since different attacks need various subsets to provide better detection accuracy. In this paper an improved feature selection algorithm is proposed to identify the most appropriate subset of features for detecting a certain attacks. This proposed method is based on Minkowski distance feature ranking and an improved exhaustive search that selects a better combination of features. This system has been evaluated using the KDD CUP 1999 dataset and also with EMSVM [1] classifier. The experimental results show that the proposed system provides high classification accuracy and low false alarm rate when applied on the reduced feature subsets
IRJET - A Novel Approach for Software Defect Prediction based on Dimensio...IRJET Journal
This document presents a novel approach for software defect prediction using dimensionality reduction techniques. The proposed approach uses an artificial neural network to extract features from initial change measures, and then trains a classifier on the extracted features. This is compared to other dimensionality reduction techniques like principal component analysis, linear discriminant analysis, and kernel principal component analysis. Five open source datasets from NASA are used to evaluate the different techniques based on accuracy, F1 score, and area under the receiver operating characteristic curve. The results show that the artificial neural network approach outperforms the other dimensionality reduction techniques, and kernel principal component analysis performs best among those techniques. The document also discusses related work on using machine learning for software defect prediction.
Tools for Ontology Building from Texts: Analysis and Improvement of the Resul...IOSR Journals
Text2Onto is a tool that learns ontologies from textual data by extracting ontology components like concepts, relations, instances, and hierarchies. It analyzes texts through linguistic preprocessing using Gate to tokenize, tag parts of speech, and identify noun and verb phrases. Algorithms then extract ontology components and store them probabilistically in a Preliminary Ontology Model independent of any representation language. The study aimed to understand Text2Onto's architecture, analyze errors in its extractions, and attempt improvements by using a meta-model of the text to better classify concepts under core concepts.
Neural Network Based Individual Classification SystemIRJET Journal
This document describes a neural network model developed for individual classification. The model was designed to measure personality traits through a questionnaire. It then uses a neural network trained on sample data through unsupervised and supervised learning with multi-layer perceptions. The backpropagation algorithm was used to train the network. The neural network architecture included multiple neuron layers trained on a 200 item data set, achieving 99.82% accuracy. The goal was to classify individuals into high, middle, and low personality categories for use in job selection or training.
Methods of Combining Neural Networks and Genetic AlgorithmsESCOM
1. The document discusses methods for combining neural networks and genetic algorithms. It describes three main approaches: evolving connection weights, evolving network architectures, and evolving learning rules.
2. In the first approach, a genetic algorithm is used as the learning rule to optimize neural network weights. The second approach uses a genetic algorithm to select structural parameters of the network and trains the network separately. The third approach evolves parameters of the network's learning rule.
3. Combining neural networks and genetic algorithms can help compensate for each techniques' weaknesses in search, and may lead to highly successful adaptive systems by integrating learning and evolutionary search.
A novel ensemble modeling for intrusion detection system IJECEIAES
Vast increase in data through internet services has made computer systems more vulnerable and difficult to protect from malicious attacks. Intrusion detection systems (IDSs) must be more potent in monitoring intrusions. Therefore an effectual Intrusion Detection system architecture is built which employs a facile classification model and generates low false alarm rates and high accuracy. Noticeably, IDS endure enormous amounts of data traffic that contain redundant and irrelevant features, which affect the performance of the IDS negatively. Despite good feature selection approaches leads to a reduction of unrelated and redundant features and attain better classification accuracy in IDS. This paper proposes a novel ensemble model for IDS based on two algorithms Fuzzy Ensemble Feature selection (FEFS) and Fusion of Multiple Classifier (FMC). FEFS is a unification of five feature scores. These scores are obtained by using feature-class distance functions. Aggregation is done using fuzzy union operation. On the other hand, the FMC is the fusion of three classifiers. It works based on Ensemble decisive function. Experiments were made on KDD cup 99 data set have shown that our proposed system works superior to well-known methods such as Support Vector Machines (SVMs), K-Nearest Neighbor (KNN) and Artificial Neural Networks (ANNs). Our examinations ensured clearly the prominence of using ensemble methodology for modeling IDSs, and hence our system is robust and efficient.
FUZZY LOGIC-BASED EFFICIENT MESSAGE ROUTE SELECTION METHOD TO PROLONG THE NET...IJCNCJournal
- The document discusses a fuzzy logic-based method for efficient message routing in wireless sensor networks to prolong the network lifetime. It aims to balance energy load across nodes by selectively tagging nodes at risk of energy exhaustion and rerouting messages around them.
- It proposes using fuzzy logic to evaluate nodes based on their potential importance, energy level, and event occurrence frequency to determine tagging. Tagged nodes avoid routing traffic but still detect and generate reports.
- The method was tested by applying it to a probabilistic voting-based filtering security scheme and was shown to improve energy efficiency, node survival rate, and report transmission success compared to not tagging nodes.
Semi-supervised learning approach using modified self-training algorithm to c...IJECEIAES
Burst header packet flooding is an attack on optical burst switching (OBS) network which may cause denial of service. Application of machine learning technique to detect malicious nodes in OBS network is relatively new. As finding sufficient amount of labeled data to perform supervised learning is difficult, semi-supervised method of learning (SSML) can be leveraged. In this paper, we studied the classical self-training algorithm (ST) which uses SSML paradigm. Generally, in ST, the available true-labeled data (L) is used to train a base classifier. Then it predicts the labels of unlabeled data (U). A portion from the newly labeled data is removed from U based on prediction confidence and combined with L. The resulting data is then used to re-train the classifier. This process is repeated until convergence. This paper proposes a modified self-training method (MST). We trained multiple classifiers on L in two stages and leveraged agreement among those classifiers to determine labels. The performance of MST was compared with ST on several datasets and significant improvement was found. We applied the MST on a simulated OBS network dataset and found very high accuracy with a small number of labeled data. Finally we compared this work with some related works.
Research Inventy : International Journal of Engineering and Science is publis...researchinventy
This document summarizes a research paper that proposes using principal component analysis and sequential hypothesis testing in a game theory framework to detect intrusions in a mobile ad hoc network. Specifically, it detects replica node attacks by tracking features like source/destination addresses and routing requests/replies to build profiles for each node. It then applies sequential probability ratio testing on routing requests and replies to test hypotheses about whether nodes are normal or abnormal. A two-player game model is also used to identify the optimal attack and defense strategies between an attacker and defender. Simulation results show that the approach can decrease the number of claims needed for detection while minimizing false positives and negatives.
ON SOFT COMPUTING TECHNIQUES IN VARIOUS AREAScscpconf
Soft Computing refers to the science of reasoning, thinking and deduction that recognizes and uses the real world phenomena of grouping, memberships, and classification of various quantities under study. As such, it is an extension of natural heuristics and capable of dealing with complex systems because it does not require strict mathematical definitions and
distinctions for the system components. It differs from hard computing in that, unlike hard computing, it is tolerant of imprecision, uncertainty and partial truth. In effect, the role modelfor soft computing is the human mind. The guiding principle of soft computing is: Exploit the tolerance for imprecision, uncertainty and partial truth to achieve tractability, robustness and low solution cost. The main techniques in soft computing are evolutionary computing, artificial neural networks, and fuzzy logic and Bayesian statistics. Each technique can be used separately, but a powerful advantage of soft computing is the complementary nature of the techniques. Used together they can produce solutions to problems that are too complex or
inherently noisy to tackle with conventional mathematical methods. The applications of soft computing have proved two main advantages. First, it made solving nonlinear problems, in
which mathematical models are not available, possible. Second, it introduced the human knowledge such as cognition,
ecognition, understanding, learning, and others into the fields of
computing. This resulted in the possibility of constructing intelligent systems such as autonomous self-tuning systems, and automated designed systems. This paper highlights various areas of soft computing techniques.
This document discusses hybrid intelligent systems that combine different technologies such as neural networks, fuzzy systems, and expert systems. It provides examples of neural expert systems, which combine neural networks and rule-based expert systems, and neuro-fuzzy systems, which integrate neural networks with fuzzy logic systems. The key benefits of these hybrid systems include gaining the learning and parallel processing abilities of neural networks as well as the transparency and human-like knowledge representation of fuzzy and expert systems.
SECURING BGP BY HANDLING DYNAMIC NETWORK BEHAVIOR AND UNBALANCED DATASETSIJCNCJournal
The Border Gateway Protocol (BGP) provides crucial routing information for the Internet infrastructure. A problem with abnormal routing behavior affects the stability and connectivity of the global Internet. The biggest hurdles in detecting BGP attacks are extremely unbalanced data set category distribution and the dynamic nature of the network. This unbalanced class distribution and dynamic nature of the network results in the classifier's inferior performance. In this paper we proposed an efficient approach to properly managing these problems, the proposed approach tackles the unbalanced classification of datasets by turning the problem of binary classification into a problem of multiclass classification. This is achieved by splitting the majority-class samples evenly into multiple segments using Affinity Propagation, where the number of segments is chosen so that the number of samples in any segment closely matches the minority-class samples. Such sections of the dataset together with the minor class are then viewed as different classes and used to train the Extreme Learning Machine (ELM). The RIPE and BCNET datasets are used to evaluate the performance of the proposed technique. When no feature selection is used, the proposed technique improves the F1 score by 1.9% compared to state-of-the-art techniques. With the Fischer feature selection algorithm, the proposed algorithm achieved the highest F1 score of 76.3%, which was a 1.7% improvement over the compared ones. Additionally, the MIQ feature selection technique improves the accuracy by 3.5%. For the BCNET dataset, the proposed technique improves the F1 score by 1.8% for the Fisher feature selection technique. The experimental findings support the substantial improvement in performance from previous approaches by the new technique.
IDS IN TELECOMMUNICATION NETWORK USING PCAIJCNCJournal
This document summarizes a research paper that proposes using principal component analysis (PCA) as a dimension reduction technique for intrusion detection systems (IDS). The paper applies PCA to reduce the number of features from 41 to either 6 or 10 features for the NSL-KDD dataset. One reduced feature set is used to develop a network IDS with high detection success and rate, while the other is used for a host IDS also with good detection success and very high detection rate. The paper outlines the process of applying PCA for IDS, including performing PCA on training data to identify principal components, then using those components to map new online data and detect intrusions based on deviation thresholds.
IRJET- Factoid Question and Answering SystemIRJET Journal
This document describes a factoid question answering system that uses neural networks and the Tensorflow framework. The system takes in a text document and question as input. It then processes the input using techniques like gated recurrent units and support vector machines to classify the question. The system calculates attention between facts and the question, modifies its memory, and identifies the word closest to the answer to output as the response. Key aspects of the system include training a question answering engine with Tensorflow, storing and retrieving data, and generating the final answer.
10 Insightful Quotes On Designing A Better Customer ExperienceYuan Wang
In an ever-changing landscape of one digital disruption after another, companies and organisations are looking for new ways to understand their target markets and engage them better. Increasingly they invest in user experience (UX) and customer experience design (CX) capabilities by working with a specialist UX agency or developing their own UX lab. Some UX practitioners are touting leaner and faster ways of developing customer-centric products and services, via methodologies such as guerilla research, rapid prototyping and Agile UX. Others seek innovation and fulfilment by spending more time in research, being more inclusive, and designing for social goods.
Experience is more than just an interface. It is a relationship, as well as a series of touch points between your brand and your customer. Here are our top 10 highlights and takeaways from the recent UX Australia conference to help you transform your customer experience design.
For full article, continue reading at https://yump.com.au/10-ways-supercharge-customer-experience-design/
How to Build a Dynamic Social Media PlanPost Planner
Stop guessing and wasting your time on networks and strategies that don’t work!
Join Rebekah Radice and Katie Lance to learn how to optimize your social networks, the best kept secrets for hot content, top time management tools, and much more!
Watch the replay here: bit.ly/socialmedia-plan
Genome structure prediction a review over soft computing techniqueseSAT Journals
Abstract There are some techniques like spectrometry or crystallography for the determination of DNA, RNA or protein structures. These processes provide very accurate results for the structure estimation. But these conventional techniques are very slow and could be applied over a few special cases only. Soft computing techniques guarantee a near appropriate results in much smaller time and have very large applicability. These techniques are much easier to apply. Different approaches have been used in soft computing including nature inspired computing for estimation of genome structures with a considerable accuracy of results. This paper provides a review over different soft computing techniques been applied along with application method for the determination of genome structure. Keywords—DNA, RNA, proteins, structure, soft computing, techniques.
C LUSTERING B ASED A TTRIBUTE S UBSET S ELECTION U SING F AST A LGORITHmIJCI JOURNAL
In machine learning and data mining, attribute sel
ect is the practice of selecting a subset o
f most
consequential attributes for utilize in model const
ruction. Using an attribute select method is that t
he data
encloses many redundant or extraneous attributes. W
here redundant attributes are those which sup
ply
no supplemental information than the presently
selected attributes, and impertinent attribut
es offer
no valuable information in any context
Performance Analysis of Various Data Mining Techniques on Banknote Authentica...inventionjournals
In this paper, we describe the functionality features for authenticating in Euro banknotes. We applied different data mining algorithms such as KMeans, Naive Bayes, Multilayer Perceptron, Decision trees (J48), and Expectation-Maximization(EM) to classifying banknote authentication dataset. The experiments are conducted in WEKA. The goal of this project is to obtain the higher authentication rate in banknote classification
Adaptive Neural Fuzzy Inference System for Employability AssessmentEditor IJCATR
Employability is potential of a person for gaining and maintains employment. Employability is measure through the
education, personal development and understanding power. Employability is not the similar as ahead a graduate job, moderately it
implies something almost the capacity of the graduate to function in an employment and be capable to move between jobs, therefore
remaining employable through their life. This paper introduced a new adaptive neural fuzzy inference system for assessment of
employability with the help of some neuro fuzzy rules. The purpose and scope of this research is to examine the level of employability.
The concern research use both fuzzy inference systems and artificial neural network which is known as neuro fuzzy technique for
solve the problem of employability assessment. This paper use three employability skills as input and find a crisp value as output
which indicates the glassy of employee. It uses twenty seven neuro fuzzy rules, with the help of Sugeno type inference in Mat-lab and
finds single value output. The proposed system is named as Adaptive Neural Fuzzy Inference System for Employability Assessment
(ANFISEA).
Comparison of fuzzy neural clustering based outlier detection techniquesIAEME Publication
The document compares fuzzy-neural clustering based outlier detection techniques. It discusses how fuzzy logic can handle uncertainty and neural networks can learn and adapt. It provides an overview of fuzzy clustering based outlier detection techniques like fuzzy c-means clustering, which allows data points to belong to multiple clusters to varying degrees. It also discusses neural network based outlier detection. The document aims to compare outlier detection techniques involving fuzzy and/or neural approaches based on clustering, focusing on their strengths and weaknesses.
Survey on Supervised Method for Face Image Retrieval Based on Euclidean Dist...Editor IJCATR
This document summarizes various supervised methods for face image retrieval based on Euclidean distance. It discusses literature on active shape models, principal component analysis, linear discriminant analysis, locality-constrained linear coding, bag-of-words models, local binary patterns, and support vector machines. It evaluates support vector machines as the best classifier for face image retrieval systems due to its ability to significantly reduce the need for labeled training data and accurately classify faces, proteins, and characters. The document concludes that a content-based face retrieval system using support vector machines improves detection performance by retrieving similar faces from a database based on Euclidean distance calculations between local binary pattern features of the query and database images.
Minkowski Distance based Feature Selection Algorithm for Effective Intrusion ...IJMER
Intrusion Detection System (IDS) plays a major role in the provision of effective security to various types of networks. Moreover, Intrusion Detection System for networks need appropriate rule set for classifying network bench mark data into normal or attack patterns. Generally, each dataset is characterized by a large set of features. However, all these features will not be relevant or fully contribute in identifying an attack. Since different attacks need various subsets to provide better detection accuracy. In this paper an improved feature selection algorithm is proposed to identify the most appropriate subset of features for detecting a certain attacks. This proposed method is based on Minkowski distance feature ranking and an improved exhaustive search that selects a better combination of features. This system has been evaluated using the KDD CUP 1999 dataset and also with EMSVM [1] classifier. The experimental results show that the proposed system provides high classification accuracy and low false alarm rate when applied on the reduced feature subsets
IRJET - A Novel Approach for Software Defect Prediction based on Dimensio...IRJET Journal
This document presents a novel approach for software defect prediction using dimensionality reduction techniques. The proposed approach uses an artificial neural network to extract features from initial change measures, and then trains a classifier on the extracted features. This is compared to other dimensionality reduction techniques like principal component analysis, linear discriminant analysis, and kernel principal component analysis. Five open source datasets from NASA are used to evaluate the different techniques based on accuracy, F1 score, and area under the receiver operating characteristic curve. The results show that the artificial neural network approach outperforms the other dimensionality reduction techniques, and kernel principal component analysis performs best among those techniques. The document also discusses related work on using machine learning for software defect prediction.
Tools for Ontology Building from Texts: Analysis and Improvement of the Resul...IOSR Journals
Text2Onto is a tool that learns ontologies from textual data by extracting ontology components like concepts, relations, instances, and hierarchies. It analyzes texts through linguistic preprocessing using Gate to tokenize, tag parts of speech, and identify noun and verb phrases. Algorithms then extract ontology components and store them probabilistically in a Preliminary Ontology Model independent of any representation language. The study aimed to understand Text2Onto's architecture, analyze errors in its extractions, and attempt improvements by using a meta-model of the text to better classify concepts under core concepts.
Neural Network Based Individual Classification SystemIRJET Journal
This document describes a neural network model developed for individual classification. The model was designed to measure personality traits through a questionnaire. It then uses a neural network trained on sample data through unsupervised and supervised learning with multi-layer perceptions. The backpropagation algorithm was used to train the network. The neural network architecture included multiple neuron layers trained on a 200 item data set, achieving 99.82% accuracy. The goal was to classify individuals into high, middle, and low personality categories for use in job selection or training.
Methods of Combining Neural Networks and Genetic AlgorithmsESCOM
1. The document discusses methods for combining neural networks and genetic algorithms. It describes three main approaches: evolving connection weights, evolving network architectures, and evolving learning rules.
2. In the first approach, a genetic algorithm is used as the learning rule to optimize neural network weights. The second approach uses a genetic algorithm to select structural parameters of the network and trains the network separately. The third approach evolves parameters of the network's learning rule.
3. Combining neural networks and genetic algorithms can help compensate for each techniques' weaknesses in search, and may lead to highly successful adaptive systems by integrating learning and evolutionary search.
A novel ensemble modeling for intrusion detection system IJECEIAES
Vast increase in data through internet services has made computer systems more vulnerable and difficult to protect from malicious attacks. Intrusion detection systems (IDSs) must be more potent in monitoring intrusions. Therefore an effectual Intrusion Detection system architecture is built which employs a facile classification model and generates low false alarm rates and high accuracy. Noticeably, IDS endure enormous amounts of data traffic that contain redundant and irrelevant features, which affect the performance of the IDS negatively. Despite good feature selection approaches leads to a reduction of unrelated and redundant features and attain better classification accuracy in IDS. This paper proposes a novel ensemble model for IDS based on two algorithms Fuzzy Ensemble Feature selection (FEFS) and Fusion of Multiple Classifier (FMC). FEFS is a unification of five feature scores. These scores are obtained by using feature-class distance functions. Aggregation is done using fuzzy union operation. On the other hand, the FMC is the fusion of three classifiers. It works based on Ensemble decisive function. Experiments were made on KDD cup 99 data set have shown that our proposed system works superior to well-known methods such as Support Vector Machines (SVMs), K-Nearest Neighbor (KNN) and Artificial Neural Networks (ANNs). Our examinations ensured clearly the prominence of using ensemble methodology for modeling IDSs, and hence our system is robust and efficient.
FUZZY LOGIC-BASED EFFICIENT MESSAGE ROUTE SELECTION METHOD TO PROLONG THE NET...IJCNCJournal
- The document discusses a fuzzy logic-based method for efficient message routing in wireless sensor networks to prolong the network lifetime. It aims to balance energy load across nodes by selectively tagging nodes at risk of energy exhaustion and rerouting messages around them.
- It proposes using fuzzy logic to evaluate nodes based on their potential importance, energy level, and event occurrence frequency to determine tagging. Tagged nodes avoid routing traffic but still detect and generate reports.
- The method was tested by applying it to a probabilistic voting-based filtering security scheme and was shown to improve energy efficiency, node survival rate, and report transmission success compared to not tagging nodes.
Semi-supervised learning approach using modified self-training algorithm to c...IJECEIAES
Burst header packet flooding is an attack on optical burst switching (OBS) network which may cause denial of service. Application of machine learning technique to detect malicious nodes in OBS network is relatively new. As finding sufficient amount of labeled data to perform supervised learning is difficult, semi-supervised method of learning (SSML) can be leveraged. In this paper, we studied the classical self-training algorithm (ST) which uses SSML paradigm. Generally, in ST, the available true-labeled data (L) is used to train a base classifier. Then it predicts the labels of unlabeled data (U). A portion from the newly labeled data is removed from U based on prediction confidence and combined with L. The resulting data is then used to re-train the classifier. This process is repeated until convergence. This paper proposes a modified self-training method (MST). We trained multiple classifiers on L in two stages and leveraged agreement among those classifiers to determine labels. The performance of MST was compared with ST on several datasets and significant improvement was found. We applied the MST on a simulated OBS network dataset and found very high accuracy with a small number of labeled data. Finally we compared this work with some related works.
Research Inventy : International Journal of Engineering and Science is publis...researchinventy
This document summarizes a research paper that proposes using principal component analysis and sequential hypothesis testing in a game theory framework to detect intrusions in a mobile ad hoc network. Specifically, it detects replica node attacks by tracking features like source/destination addresses and routing requests/replies to build profiles for each node. It then applies sequential probability ratio testing on routing requests and replies to test hypotheses about whether nodes are normal or abnormal. A two-player game model is also used to identify the optimal attack and defense strategies between an attacker and defender. Simulation results show that the approach can decrease the number of claims needed for detection while minimizing false positives and negatives.
ON SOFT COMPUTING TECHNIQUES IN VARIOUS AREAScscpconf
Soft Computing refers to the science of reasoning, thinking and deduction that recognizes and uses the real world phenomena of grouping, memberships, and classification of various quantities under study. As such, it is an extension of natural heuristics and capable of dealing with complex systems because it does not require strict mathematical definitions and
distinctions for the system components. It differs from hard computing in that, unlike hard computing, it is tolerant of imprecision, uncertainty and partial truth. In effect, the role modelfor soft computing is the human mind. The guiding principle of soft computing is: Exploit the tolerance for imprecision, uncertainty and partial truth to achieve tractability, robustness and low solution cost. The main techniques in soft computing are evolutionary computing, artificial neural networks, and fuzzy logic and Bayesian statistics. Each technique can be used separately, but a powerful advantage of soft computing is the complementary nature of the techniques. Used together they can produce solutions to problems that are too complex or
inherently noisy to tackle with conventional mathematical methods. The applications of soft computing have proved two main advantages. First, it made solving nonlinear problems, in
which mathematical models are not available, possible. Second, it introduced the human knowledge such as cognition,
ecognition, understanding, learning, and others into the fields of
computing. This resulted in the possibility of constructing intelligent systems such as autonomous self-tuning systems, and automated designed systems. This paper highlights various areas of soft computing techniques.
This document discusses hybrid intelligent systems that combine different technologies such as neural networks, fuzzy systems, and expert systems. It provides examples of neural expert systems, which combine neural networks and rule-based expert systems, and neuro-fuzzy systems, which integrate neural networks with fuzzy logic systems. The key benefits of these hybrid systems include gaining the learning and parallel processing abilities of neural networks as well as the transparency and human-like knowledge representation of fuzzy and expert systems.
SECURING BGP BY HANDLING DYNAMIC NETWORK BEHAVIOR AND UNBALANCED DATASETSIJCNCJournal
The Border Gateway Protocol (BGP) provides crucial routing information for the Internet infrastructure. A problem with abnormal routing behavior affects the stability and connectivity of the global Internet. The biggest hurdles in detecting BGP attacks are extremely unbalanced data set category distribution and the dynamic nature of the network. This unbalanced class distribution and dynamic nature of the network results in the classifier's inferior performance. In this paper we proposed an efficient approach to properly managing these problems, the proposed approach tackles the unbalanced classification of datasets by turning the problem of binary classification into a problem of multiclass classification. This is achieved by splitting the majority-class samples evenly into multiple segments using Affinity Propagation, where the number of segments is chosen so that the number of samples in any segment closely matches the minority-class samples. Such sections of the dataset together with the minor class are then viewed as different classes and used to train the Extreme Learning Machine (ELM). The RIPE and BCNET datasets are used to evaluate the performance of the proposed technique. When no feature selection is used, the proposed technique improves the F1 score by 1.9% compared to state-of-the-art techniques. With the Fischer feature selection algorithm, the proposed algorithm achieved the highest F1 score of 76.3%, which was a 1.7% improvement over the compared ones. Additionally, the MIQ feature selection technique improves the accuracy by 3.5%. For the BCNET dataset, the proposed technique improves the F1 score by 1.8% for the Fisher feature selection technique. The experimental findings support the substantial improvement in performance from previous approaches by the new technique.
IDS IN TELECOMMUNICATION NETWORK USING PCAIJCNCJournal
This document summarizes a research paper that proposes using principal component analysis (PCA) as a dimension reduction technique for intrusion detection systems (IDS). The paper applies PCA to reduce the number of features from 41 to either 6 or 10 features for the NSL-KDD dataset. One reduced feature set is used to develop a network IDS with high detection success and rate, while the other is used for a host IDS also with good detection success and very high detection rate. The paper outlines the process of applying PCA for IDS, including performing PCA on training data to identify principal components, then using those components to map new online data and detect intrusions based on deviation thresholds.
IRJET- Factoid Question and Answering SystemIRJET Journal
This document describes a factoid question answering system that uses neural networks and the Tensorflow framework. The system takes in a text document and question as input. It then processes the input using techniques like gated recurrent units and support vector machines to classify the question. The system calculates attention between facts and the question, modifies its memory, and identifies the word closest to the answer to output as the response. Key aspects of the system include training a question answering engine with Tensorflow, storing and retrieving data, and generating the final answer.
10 Insightful Quotes On Designing A Better Customer ExperienceYuan Wang
In an ever-changing landscape of one digital disruption after another, companies and organisations are looking for new ways to understand their target markets and engage them better. Increasingly they invest in user experience (UX) and customer experience design (CX) capabilities by working with a specialist UX agency or developing their own UX lab. Some UX practitioners are touting leaner and faster ways of developing customer-centric products and services, via methodologies such as guerilla research, rapid prototyping and Agile UX. Others seek innovation and fulfilment by spending more time in research, being more inclusive, and designing for social goods.
Experience is more than just an interface. It is a relationship, as well as a series of touch points between your brand and your customer. Here are our top 10 highlights and takeaways from the recent UX Australia conference to help you transform your customer experience design.
For full article, continue reading at https://yump.com.au/10-ways-supercharge-customer-experience-design/
How to Build a Dynamic Social Media PlanPost Planner
Stop guessing and wasting your time on networks and strategies that don’t work!
Join Rebekah Radice and Katie Lance to learn how to optimize your social networks, the best kept secrets for hot content, top time management tools, and much more!
Watch the replay here: bit.ly/socialmedia-plan
http://inarocket.com
Learn BEM fundamentals as fast as possible. What is BEM (Block, element, modifier), BEM syntax, how it works with a real example, etc.
The document discusses how personalization and dynamic content are becoming increasingly important on websites. It notes that 52% of marketers see content personalization as critical and 75% of consumers like it when brands personalize their content. However, personalization can create issues for search engine optimization as dynamic URLs and content are more difficult for search engines to index than static pages. The document provides tips for SEOs to help address these personalization and SEO challenges, such as using static URLs when possible and submitting accurate sitemaps.
Lightning Talk #9: How UX and Data Storytelling Can Shape Policy by Mika Aldabaux singapore
How can we take UX and Data Storytelling out of the tech context and use them to change the way government behaves?
Showcasing the truth is the highest goal of data storytelling. Because the design of a chart can affect the interpretation of data in a major way, one must wield visual tools with care and deliberation. Using quantitative facts to evoke an emotional response is best achieved with the combination of UX and data storytelling.
This document summarizes a study of CEO succession events among the largest 100 U.S. corporations between 2005-2015. The study analyzed executives who were passed over for the CEO role ("succession losers") and their subsequent careers. It found that 74% of passed over executives left their companies, with 30% eventually becoming CEOs elsewhere. However, companies led by succession losers saw average stock price declines of 13% over 3 years, compared to gains for companies whose CEO selections remained unchanged. The findings suggest that boards generally identify the most qualified CEO candidates, though differences between internal and external hires complicate comparisons.
The document discusses the differences between machine learning (ML), statistical learning, data mining (DM), and automated learning (AL). It argues that while ML and statistical learning developed similar techniques starting in the 1960s, DM emerged in the 1990s from a merging of database research and automated learning. However, industry was much more enthusiastic about adopting DM techniques compared to AL techniques, even though many DM systems are just friendly interfaces of AL systems. The document aims to explain the key differences between DM and AL that led to DM's greater commercial success.
This document provides a summary of an undergraduate study report on adaptive relaying using artificial intelligence techniques. It discusses artificial intelligence methods like expert systems, artificial neural networks, and fuzzy logic that have been applied in power system protection. It also analyzes some key aspects of using these techniques, including the design of neural networks and the challenges of generating comprehensive training sets from large power system data. The document serves as the abstract and introduction to the full study report.
IRJET- Anomaly Detection System in CCTV Derived VideosIRJET Journal
This document describes a proposed system for anomaly detection in CCTV videos using deep learning techniques. The system has two main components: 1) feature extraction using convolutional neural networks to learn representations of normal behavior from training videos, and 2) an anomaly detection classifier to identify abnormal events in new videos based on the learned features. Several related works incorporating techniques like k-means clustering, decision trees, and neural networks for video-based anomaly detection are also reviewed. The methodology section outlines the overall framework, including preprocessing steps and separate training and testing phases to extract normal features and then detect anomalies.
Survey on Artificial Neural Network Learning Technique AlgorithmsIRJET Journal
This document discusses different types of learning algorithms used in artificial neural networks. It begins with an introduction to neural networks and their ability to learn from their environment through adjustments to synaptic weights. Four main learning algorithms are then described: error correction learning, which uses algorithms like backpropagation to minimize error; memory based learning, which stores all training examples and analyzes nearby examples to classify new inputs; Hebbian learning, where connection weights are adjusted based on the activity of neurons; and competitive learning, where neurons compete to respond to inputs to become specialized feature detectors through a winner-take-all mechanism. The document provides details on how each type of learning algorithm works.
Pattern Recognition using Artificial Neural NetworkEditor IJCATR
An artificial neural network (ANN) usually called neural network. It can be considered as a resemblance to a paradigm
which is inspired by biological nervous system. In network the signals are transmitted by the means of connections links. The links
possess an associated way which is multiplied along with the incoming signal. The output signal is obtained by applying activation to
the net input NN are one of the most exciting and challenging research areas. As ANN mature into commercial systems, they are likely
to be implemented in hardware. Their fault tolerance and reliability are therefore vital to the functioning of the system in which they
are embedded. The pattern recognition system is implemented with Back propagation network and Hopfield network to remove the
distortion from the input. The Hopfield network has high fault tolerance which supports this system to get the accurate output.
Abstract In this paper, the concept of data mining was summarized and its significance towards its methodologies was illustrated. The data mining based on Neural Network and Genetic Algorithm is researched in detail and the key technology and ways to achieve the data mining on Neural Network and Genetic Algorithm are also surveyed. This paper also conducts a formal review of the area of rule extraction from ANN and GA. Keywords: Data Mining, Neural Network, Genetic Algorithm, Rule Extraction.
EMPIRICAL APPLICATION OF SIMULATED ANNEALING USING OBJECT-ORIENTED METRICS TO...ijcsa
The work is about using Simulated Annealing Algorithm for the effort estimation model parameter
optimization which can lead to the reduction in the difference in actual and estimated effort used in model
development.
The model has been tested using OOP’s dataset, obtained from NASA for research purpose.The data set
based model equation parameters have been found that consists of two independent variables, viz. Lines of
Code (LOC) along with one more attribute as a dependent variable related to software development effort
(DE). The results have been compared with the earlier work done by the author on Artificial Neural
Network (ANN) and Adaptive Neuro Fuzzy Inference System (ANFIS) and it has been observed that the
developed SA based model is more capable to provide better estimation of software development effort than
ANN and ANFIS
In sequence Polemical Pertinence via Soft Enumerating RepertoireIRJET Journal
This document discusses the use of soft computing techniques in image forensics. It begins by defining soft computing as a multi-disciplinary field involving fuzzy logic, neural networks, evolutionary algorithms, and probabilistic reasoning. It then discusses several soft computing techniques - fuzzy logic, neural networks, and genetic algorithms - and how they can help address challenges in image forensics by dealing with imprecision and uncertainty. The document concludes that soft computing tools show promise for analyzing the large amounts of data involved in supply chain management problems and aiding managers' decision making. It identifies several areas, such as customer demand management, that have not been extensively explored but could benefit from additional research applying soft computing.
A systematic review on sequence-to-sequence learning with neural network and ...IJECEIAES
We develop a precise writing survey on sequence-to-sequence learning with neural network and its models. The primary aim of this report is to enhance the knowledge of the sequence-to-sequence neural network and to locate the best way to deal with executing it. Three models are mostly used in sequence-to-sequence neural network applications, namely: recurrent neural networks (RNN), connectionist temporal classification (CTC), and attention model. The evidence we adopted in conducting this survey included utilizing the examination inquiries or research questions to determine keywords, which were used to search for bits of peer-reviewed papers, articles, or books at scholastic directories. Through introductory hunts, 790 papers, and scholarly works were found, and with the assistance of choice criteria and PRISMA methodology, the number of papers reviewed decreased to 16. Every one of the 16 articles was categorized by their contribution to each examination question, and they were broken down. At last, the examination papers experienced a quality appraisal where the subsequent range was from 83.3% to 100%. The proposed systematic review enabled us to collect, evaluate, analyze, and explore different approaches of implementing sequence-to-sequence neural network models and pointed out the most common use in machine learning. We followed a methodology that shows the potential of applying these models to real-world applications.
IDENTIFICATION AND INVESTIGATION OF THE USER SESSION FOR LAN CONNECTIVITY VIA...ijcseit
This paper mainly presents some technical discussions on the identification and analyze of “LAN usersessions”.
The identification of a user-session is non trivial. Classical methods approaches rely on
threshold based mechanisms. Threshold based techniques are very sensitive to the value chosen for the
threshold, which may be difficult to set correctly. Clustering techniques are used to define a novel
methodology to identify LAN user-sessions without requiring an a priori definition of threshold values. We
have defined a clustering based approach in detail, and also we discussed positive and negative of this
approach, and we apply it to real traffic traces. The proposed methodology is applied to artificially
generated traces to evaluate its benefits against traditional threshold based approaches. We also analyzed
the characteristics of user-sessions extracted by the clustering methodology from real traces and study
their statistical properties.
Automated identification of sensitive informationJeff Long
October 21, 1999: "Using Ultra-Structure for Automated Identification of Sensitive Information in Documents". Presented at the 20th annual conference of the American Society for Engineering Management. Paper published in conference proceedings.
IJRET : International Journal of Research in Engineering and Technology is an international peer reviewed, online journal published by eSAT Publishing House for the enhancement of research in various disciplines of Engineering and Technology. The aim and scope of the journal is to provide an academic medium and an important reference for the advancement and dissemination of research results that support high-level learning, teaching and research in the fields of Engineering and Technology. We bring together Scientists, Academician, Field Engineers, Scholars and Students of related fields of Engineering and Technology
This document provides an overview of decision tree induction methods and their application to big data. It discusses decision trees as a method for identifying patterns in large datasets that has the advantage of being interpretable. The document describes the basic principles of decision tree induction, different algorithms for constructing decision trees, and measures for evaluating tree performance and structure. It also discusses challenges such as fitting trees to existing expert knowledge and improving classification through feature selection.
This document provides an overview and summary of a student project report on simulating a feed forward artificial neural network in C++. The report includes an abstract, table of contents, list of figures, and 5 chapters that discuss the objectives of the project, provide background on artificial neural networks, describe the design and implementation of a 3-layer feed forward neural network using backpropagation, present the results, and provide references. The design section explains the backpropagation algorithm and provides pseudocode for calculating outputs at each layer. The implementation section provides pseudocode for training patterns and minimizing error.
This document proposes a new method for extracting rules from trained multilayer artificial neural networks that can represent rules in both "if-then" and "M of N" formats. The method extracts an intermediate structure called a "generator list" from which both types of rules can be derived. This provides a more generic representation than existing methods that can only output one rule format. The generator list approach avoids preprocessing steps used in other methods that can modify the original network. It uses heuristics to prune the search space when extracting the generator list to address the computational complexity involved.
IRJET - Neural Network based Leaf Disease Detection and Remedy Recommenda...IRJET Journal
This document describes a neural network-based system for detecting leaf diseases and recommending remedies. It uses a convolutional neural network (CNN) and deep learning techniques to classify images of plant leaves with different diseases. The system is trained on a dataset of 5000 leaf images across 4 disease classes. It aims to help farmers more easily identify leaf diseases and receive treatment recommendations without needing to directly contact experts. The document outlines the existing problems, proposed solution, literature review on related techniques like boosting and support vector machines, software and algorithms used including Python, Anaconda and Spyder. It also describes the implementation process involving modules for data loading, preprocessing, feature extraction using CNN, disease prediction, and recommending remedies.
A Review on Reasoning System, Types, and Tools and Need for Hybrid ReasoningBRNSSPublicationHubI
This document summarizes a review article about reasoning systems, types of reasoning, and the need for hybrid reasoning systems. It discusses expert systems and how they use knowledge representation and reasoning to emulate expert decision making. The main types of reasoning discussed are deductive, inductive, and abductive reasoning. It also introduces the concept of a hybrid reasoning system that integrates two different types of reasoning to provide both qualitative and quantitative assessments.
Similar to Evaluation of rule extraction algorithms (20)
Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...SOFTTECHHUB
The choice of an operating system plays a pivotal role in shaping our computing experience. For decades, Microsoft's Windows has dominated the market, offering a familiar and widely adopted platform for personal and professional use. However, as technological advancements continue to push the boundaries of innovation, alternative operating systems have emerged, challenging the status quo and offering users a fresh perspective on computing.
One such alternative that has garnered significant attention and acclaim is Nitrux Linux 3.5.0, a sleek, powerful, and user-friendly Linux distribution that promises to redefine the way we interact with our devices. With its focus on performance, security, and customization, Nitrux Linux presents a compelling case for those seeking to break free from the constraints of proprietary software and embrace the freedom and flexibility of open-source computing.
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdfMalak Abu Hammad
Discover how MongoDB Atlas and vector search technology can revolutionize your application's search capabilities. This comprehensive presentation covers:
* What is Vector Search?
* Importance and benefits of vector search
* Practical use cases across various industries
* Step-by-step implementation guide
* Live demos with code snippets
* Enhancing LLM capabilities with vector search
* Best practices and optimization strategies
Perfect for developers, AI enthusiasts, and tech leaders. Learn how to leverage MongoDB Atlas to deliver highly relevant, context-aware search results, transforming your data retrieval process. Stay ahead in tech innovation and maximize the potential of your applications.
#MongoDB #VectorSearch #AI #SemanticSearch #TechInnovation #DataScience #LLM #MachineLearning #SearchTechnology
Climate Impact of Software Testing at Nordic Testing DaysKari Kakkonen
My slides at Nordic Testing Days 6.6.2024
Climate impact / sustainability of software testing discussed on the talk. ICT and testing must carry their part of global responsibility to help with the climat warming. We can minimize the carbon footprint but we can also have a carbon handprint, a positive impact on the climate. Quality characteristics can be added with sustainability, and then measured continuously. Test environments can be used less, and in smaller scale and on demand. Test techniques can be used in optimizing or minimizing number of tests. Test automation can be used to speed up testing.
A tale of scale & speed: How the US Navy is enabling software delivery from l...sonjaschweigert1
Rapid and secure feature delivery is a goal across every application team and every branch of the DoD. The Navy’s DevSecOps platform, Party Barge, has achieved:
- Reduction in onboarding time from 5 weeks to 1 day
- Improved developer experience and productivity through actionable findings and reduction of false positives
- Maintenance of superior security standards and inherent policy enforcement with Authorization to Operate (ATO)
Development teams can ship efficiently and ensure applications are cyber ready for Navy Authorizing Officials (AOs). In this webinar, Sigma Defense and Anchore will give attendees a look behind the scenes and demo secure pipeline automation and security artifacts that speed up application ATO and time to production.
We will cover:
- How to remove silos in DevSecOps
- How to build efficient development pipeline roles and component templates
- How to deliver security artifacts that matter for ATO’s (SBOMs, vulnerability reports, and policy evidence)
- How to streamline operations with automated policy checks on container images
Pushing the limits of ePRTC: 100ns holdover for 100 daysAdtran
At WSTS 2024, Alon Stern explored the topic of parametric holdover and explained how recent research findings can be implemented in real-world PNT networks to achieve 100 nanoseconds of accuracy for up to 100 days.
Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!SOFTTECHHUB
As the digital landscape continually evolves, operating systems play a critical role in shaping user experiences and productivity. The launch of Nitrux Linux 3.5.0 marks a significant milestone, offering a robust alternative to traditional systems such as Windows 11. This article delves into the essence of Nitrux Linux 3.5.0, exploring its unique features, advantages, and how it stands as a compelling choice for both casual users and tech enthusiasts.
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...James Anderson
Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM
The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management.
The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM).
Speakers:
Bob Boule
Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle.
Gopinath Rebala
Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.
Enchancing adoption of Open Source Libraries. A case study on Albumentations.AIVladimir Iglovikov, Ph.D.
Presented by Vladimir Iglovikov:
- https://www.linkedin.com/in/iglovikov/
- https://x.com/viglovikov
- https://www.instagram.com/ternaus/
This presentation delves into the journey of Albumentations.ai, a highly successful open-source library for data augmentation.
Created out of a necessity for superior performance in Kaggle competitions, Albumentations has grown to become a widely used tool among data scientists and machine learning practitioners.
This case study covers various aspects, including:
People: The contributors and community that have supported Albumentations.
Metrics: The success indicators such as downloads, daily active users, GitHub stars, and financial contributions.
Challenges: The hurdles in monetizing open-source projects and measuring user engagement.
Development Practices: Best practices for creating, maintaining, and scaling open-source libraries, including code hygiene, CI/CD, and fast iteration.
Community Building: Strategies for making adoption easy, iterating quickly, and fostering a vibrant, engaged community.
Marketing: Both online and offline marketing tactics, focusing on real, impactful interactions and collaborations.
Mental Health: Maintaining balance and not feeling pressured by user demands.
Key insights include the importance of automation, making the adoption process seamless, and leveraging offline interactions for marketing. The presentation also emphasizes the need for continuous small improvements and building a friendly, inclusive community that contributes to the project's growth.
Vladimir Iglovikov brings his extensive experience as a Kaggle Grandmaster, ex-Staff ML Engineer at Lyft, sharing valuable lessons and practical advice for anyone looking to enhance the adoption of their open-source projects.
Explore more about Albumentations and join the community at:
GitHub: https://github.com/albumentations-team/albumentations
Website: https://albumentations.ai/
LinkedIn: https://www.linkedin.com/company/100504475
Twitter: https://x.com/albumentations
Removing Uninteresting Bytes in Software FuzzingAftab Hussain
Imagine a world where software fuzzing, the process of mutating bytes in test seeds to uncover hidden and erroneous program behaviors, becomes faster and more effective. A lot depends on the initial seeds, which can significantly dictate the trajectory of a fuzzing campaign, particularly in terms of how long it takes to uncover interesting behaviour in your code. We introduce DIAR, a technique designed to speedup fuzzing campaigns by pinpointing and eliminating those uninteresting bytes in the seeds. Picture this: instead of wasting valuable resources on meaningless mutations in large, bloated seeds, DIAR removes the unnecessary bytes, streamlining the entire process.
In this work, we equipped AFL, a popular fuzzer, with DIAR and examined two critical Linux libraries -- Libxml's xmllint, a tool for parsing xml documents, and Binutil's readelf, an essential debugging and security analysis command-line tool used to display detailed information about ELF (Executable and Linkable Format). Our preliminary results show that AFL+DIAR does not only discover new paths more quickly but also achieves higher coverage overall. This work thus showcases how starting with lean and optimized seeds can lead to faster, more comprehensive fuzzing campaigns -- and DIAR helps you find such seeds.
- These are slides of the talk given at IEEE International Conference on Software Testing Verification and Validation Workshop, ICSTW 2022.
Threats to mobile devices are more prevalent and increasing in scope and complexity. Users of mobile devices desire to take full advantage of the features
available on those devices, but many of the features provide convenience and capability but sacrifice security. This best practices guide outlines steps the users can take to better protect personal devices and information.
Full-RAG: A modern architecture for hyper-personalizationZilliz
Mike Del Balso, CEO & Co-Founder at Tecton, presents "Full RAG," a novel approach to AI recommendation systems, aiming to push beyond the limitations of traditional models through a deep integration of contextual insights and real-time data, leveraging the Retrieval-Augmented Generation architecture. This talk will outline Full RAG's potential to significantly enhance personalization, address engineering challenges such as data management and model training, and introduce data enrichment with reranking as a key solution. Attendees will gain crucial insights into the importance of hyperpersonalization in AI, the capabilities of Full RAG for advanced personalization, and strategies for managing complex data integrations for deploying cutting-edge AI solutions.
How to Get CNIC Information System with Paksim Ga.pptxdanishmna97
Pakdata Cf is a groundbreaking system designed to streamline and facilitate access to CNIC information. This innovative platform leverages advanced technology to provide users with efficient and secure access to their CNIC details.
In the rapidly evolving landscape of technologies, XML continues to play a vital role in structuring, storing, and transporting data across diverse systems. The recent advancements in artificial intelligence (AI) present new methodologies for enhancing XML development workflows, introducing efficiency, automation, and intelligent capabilities. This presentation will outline the scope and perspective of utilizing AI in XML development. The potential benefits and the possible pitfalls will be highlighted, providing a balanced view of the subject.
We will explore the capabilities of AI in understanding XML markup languages and autonomously creating structured XML content. Additionally, we will examine the capacity of AI to enrich plain text with appropriate XML markup. Practical examples and methodological guidelines will be provided to elucidate how AI can be effectively prompted to interpret and generate accurate XML markup.
Further emphasis will be placed on the role of AI in developing XSLT, or schemas such as XSD and Schematron. We will address the techniques and strategies adopted to create prompts for generating code, explaining code, or refactoring the code, and the results achieved.
The discussion will extend to how AI can be used to transform XML content. In particular, the focus will be on the use of AI XPath extension functions in XSLT, Schematron, Schematron Quick Fixes, or for XML content refactoring.
The presentation aims to deliver a comprehensive overview of AI usage in XML development, providing attendees with the necessary knowledge to make informed decisions. Whether you’re at the early stages of adopting AI or considering integrating it in advanced XML development, this presentation will cover all levels of expertise.
By highlighting the potential advantages and challenges of integrating AI with XML development tools and languages, the presentation seeks to inspire thoughtful conversation around the future of XML development. We’ll not only delve into the technical aspects of AI-powered XML development but also discuss practical implications and possible future directions.
Monitoring Java Application Security with JDK Tools and JFR Events
Evaluation of rule extraction algorithms
1. International Journal of Data Mining & Knowledge Management Process (IJDKP) Vol.4, No.3, May 2014
DOI : 10.5121/ijdkp.2014.4302 9
EVALUATION OF RULE EXTRACTION
ALGORITHMS
Tiruveedula GopiKrishna
Department of Computer Science, Sirt University, Hoon, Libya
ABSTRACT
For the data mining domain, the lack of explanation facilities seems to be a serious drawback for
techniques based on Artificial Neural Networks, or, for that matter, any technique producing opaque
models In particular, the ability to generate even limited explanations is absolutely crucial for user
acceptance of such systems. Since the purpose of most data mining systems is to support decision making,
the need for explanation facilities in these systems is apparent. The task for the data miner is thus to
identify the complex but general relationships that are likely to carry over to production data and the
explanation facility makes this easier. Also focused the quality of the extracted rules; i.e. how well the
required explanation is performed. In this research some important rule extraction algorithms are
discussed and identified the algorithmic complexity; i.e. how efficient the underlying rule extraction
algorithm is.
KEYWORDS
Extraction Algorithms; taxonomy of rule extraction; evolution of rule extraction algorithms; scalability;
comprehensibility.
1. INTRODUCTION
For the data mining domain, the lack of explanation facilities seems to be a serious drawback for
techniques based on Artificial Neural Networks (ANNs), or, for that matter, any technique
producing opaque models. Within the field of symbolic AI, the term explanation refers to an
explicit structure which is used internally for reasoning and learning and externally for the
explanation of the results to the user. Normally, the explanation facility in symbolic AI includes
intermediate steps of the reasoning process, like trace of rules firing and proof structures.
Generally speaking, the explanation facilities are capable of answering the “how” and “why”
questions. The answer to a how-question is an explanation of how the result was found. A why-
question is supplied by the user during execution of the system and the answer specifies why the
system performed a certain operation; e.g. queried the user.
Experience from the field of expert systems has shown that an explanation capability is a vital
function provided by symbolic AI systems. In particular, the ability to generate even limited
explanations is absolutely crucial for user acceptance of such systems [1]. Since the purpose of
most data mining systems is to support decision making, the need for explanation facilities in
these systems is apparent. Nevertheless many systems (especially those using ANN techniques,
but also ensemble methods like boosting) must be regarded as black boxes; i.e. they are opaque to
the user.
In [2] the authors Andrews, Diederich and Tickle highlight this deficiency of ANNs, and argue
for rule extraction; i.e. to create more transparent representations from trained ANNs:
2. International Journal of Data Mining & Knowledge Management Process (IJDKP) Vol.4, No.3, May 2014
10
It is becoming increasingly apparent that the absence of an explanation capability in ANN
systems limits the realizations of the full potential of such systems, and it is this precise
deficiency that the rule extraction process seeks to reduce.
Andrews, Diederich and Tickle also list five more reasons for the importance of being able to
interpret trained ANNs:
• For ANNs to be used in safety-critical problem domains, the explanation facility must be
considered mandatory.
• If ANNs are integrated into larger software systems without an explanation facility, the
entire systems would be very hard to verify.
• An explanation facility may provide an experienced user with the capability to anticipate
or predict a set of circumstances under which the ANN is likely to generalize poorly.
• ANNs may discover previously unknown dependencies, but without an explanation
facility these dependencies are incomprehensibly encoded in the model.
• Extracted rules from ANNs could be directly added to the knowledge base of a symbolic
AI system.
It should be noted that an explanation facility also offers a way to determine data quality, since it
makes it possible to examine and interpret relationships found. If the discovered relationships are
deemed doubtful when inspected by a human, they are less probable to actually add value.
“Nonsense” relationships found would, if used on a production set, most likely produce poor
results. The task for the data miner is thus to identify the complex but general relationships that
are likely to carry over to production data and the explanation facility makes this easier.
There are basically two methods that can be used to gain understanding of the relationship found
by a trained ANN; sensitivity analysis and rule extraction. Sensitivity analysis does not produce a
new model, but is used to gain some basic understanding of the relationship between input
variables and the output. Rule extraction is an activity where the trained ANN is transformed into
another, more comprehensible model, representing the same relationship.
2. SENSITIVITY ANALYSIS
Sensitivity analysis does not provide explicit rules, but is used to find the relative importance of
the inputs to the output of the neural net. There are some small variations in how the analysis is
performed, but the overall procedure is to record changes in the output following changes in
specific input attributes. Normally, the average value for each input is chosen as the starting point
and the changes should vary from small changes all the way up to the extreme values. If the
difference in output is small even for large changes in a specific attribute, this attribute is
probably not very important; i.e. the network is insensitive to that attribute. Other attributes might
have a large effect on the output and the network is then said to be sensitive to these attributes.
Obviously, there could be combinations of features that are very important for the network, which
would require the sensitivity analysis to be performed on two or more attributes at the same time,
in order to find these particular patterns.
3. International Journal of Data Mining & Knowledge Management Process (IJDKP) Vol.4, No.3, May 2014
11
It is safe to say that sensitivity analysis is a good tool to get some basic understanding of the
underlying problem, but also that it normally is unable to produce explanations. The purpose of
performing a sensitivity analysis is thus usually not to actually explain the relationship found.
Instead sensitivity analysis is normally used either as a tool to find and remove unimportant input
attributes or as a starting point for some more powerful explanation technique.
3. RULE EXTRACTION FROM TRAINED NEURAL NETWORKS
The knowledge acquired by an ANN during training is encoded as the architecture and the
weights. The task of extracting explanations from the network is therefore to interpret, in a
comprehensible form, the knowledge represented by the architecture and the weights.
Craven and Shavlik [3] coined the term representation language for the language used to describe
the network’s learned model. They also used the expression extraction strategy for the process of
transforming the trained network into the new representation language.
4. TAXONOMY OF RULE EXTRACTION APPROACHES
In [2] the authors proposed a classification schema for rule extraction approaches. The
presentation below intentionally follows the one given in the paper closely. In the paper, the
method of classification is based on five dimensions:
• The expressive power of the extracted rules; i.e. the chosen representation language.
• The translucency of the view taken within the rule extraction technique of the underlying
ANN units; i.e. does the technique look inside the trained neural net and utilize
knowledge about connections and weights or is the network treated as an oracle.
• The extent to which the underlying ANN incorporates specialized training regimes.
• The quality of the extracted rules; i.e. how well the required explanation is performed.
• The algorithmic complexity; i.e. how efficient the underlying rule extraction algorithm
is.
Representation languages typically used include (if-then) inference rules, M-of-N rules, fuzzy
rules, decision trees and finite-state automata. In the translucency dimension there are two
fundamentally different approaches; decompositional
(open-box or white-box) and pedagogical (black-box).
Decompositional approaches focus on extracting rules at the level of individual units within the
trained ANN; i.e. the view of the underlying ANN is one of transparency. According to Andrews,
Diederich and Tickle, a basic requirement for this category of rule extraction is that the computed
output from each unit must be mapped into a binary outcome, corresponding to a rule consequent.
Each unit can be interpreted as a step function, meaning that the problem is reduced to finding a
set of incoming links whose summed weights guarantee that the unit’s bias is exceeded regardless
of other incoming links. When such a combination of links is found, this is readily translated into
a rule where the output of that unit is a consequent of the inputs. The rules extracted at the
individual unit level are then aggregated to form the composite rule set for the ANN as a whole.
Pedagogical approaches treat the trained ANN as a black box; i.e. the view of the underlying
ANN is opaque. The core idea in the pedagogical approach is to treat the ANN as an oracle and
view the rule extraction as a learning task, where the target concept is the function learnt by the
4. International Journal of Data Mining & Knowledge Management Process (IJDKP) Vol.4, No.3, May 2014
12
ANN. Hence the rules extracted directly map inputs to outputs. Black-box techniques typically
use some symbolic learning algorithm, where the ANN is used to generate the training examples.
The easiest way to understand the process is to regard black-box rule extraction as an instance of
predictive modeling, where each input-output pattern consists of the original input vector and the
corresponding prediction from the opaque model. From this perspective, black-box rule extraction
becomes the task of modeling the function from the (original) input variables to the opaque model
predictions; see Figure 1.
The first two dimensions (i.e. expressive power and translucency) are in [2] suggested to be the
primary classifiers of rule extraction algorithms.
5. EVALUATION OF RULE EXTRACTION ALGORITHMS
There are several criteria used for evaluating rule extraction algorithms. In [4] Craven and
Shavlik listed five criteria:
Comprehensibility
The extent to which extracted representations are humanly comprehensible.
Fidelity
The extent to which extracted representations accurately model the networks from which they
where extracted.
Accuracy
The ability of extracted representations to make accurate predictions on previously unseen cases.
Scalability
The ability of the method to scale to networks with large input spaces and large numbers of
weighted connections.
Generality
The extent to which the method requires special training regimes or places restrictions on network
architectures.
Most researchers have evaluated their rule extraction methods using the first three criteria but,
according to Craven and Shavlik, scalability and generality have often been overlooked. In the
paper, scalability is defined in the following way:
Scalability refers to how the running time of a rule extraction algorithm and the comprehensibility
of its extracted models vary as a function of such factors as network, feature-set and training-set
size.
Craven and Shavlik argue that methods that scale well in terms of running time, but not in terms
of comprehensibility will be of little use. The reason is obvious from the fact that the overall
purpose of rule extraction always is to produce a comprehensible model available for human
interpretation. If this becomes impossible for larger problems it must be considered a serious
limitation for a proposed method.
5. International Journal of Data Mining & Knowledge Management Process (IJDKP) Vol.4, No.3, May 2014
13
It should be noted that scaling is an inherent problem, regarding both running time and
comprehensibility, for decompositional methods. The potential size of a rule for a unit with n
inputs each having k possible values is kn, meaning that a straightforward search for possible
rules is normally impossible for larger networks. This, and the problem with continuous inputs,
are normally to some extent handled by clustering inputs into disjoint ranges. Craven and Shavlik
also highlight that the size of rule sets produced by decompositional algorithms tend to be
proportional to the network size.
Craven and Shavlik recommend rule extraction researchers to pursue different lines of research
that have not been explored to a large extent, to try to overcome the problem of scalability:
• Methods for controlling the comprehensibility vs. fidelity trade-off; i.e. the possibility to
improve the comprehensibility of an extracted rule set by compromising on its fidelity
and accuracy.
• Methods for anytime rule extraction; i.e. the ability to interrupt the rule extraction at any
time and still get a solution.
Regarding generality, Craven and Shavlik argue that rule extraction algorithms must exhibit a
high level of generality to have a large impact. In particular, algorithms requiring specific training
regimes or algorithms limited to narrow architectural classes are deemed less interesting. Craven
and Shavlik finally say that rule extraction algorithms ideally should be so general that the models
they are trying to describe must not even be ANNs. Obviously there is also a need to explain
complex models like ensembles or classifiers using boosting, so it is natural to extend the task of
rule extraction to operate on these models. A rule extraction algorithm capable of coping with
different underlying kinds of models would therefore be of high value.
Yet another important criterion, often overlooked but recognized in [5], is consistency. A rule
extraction algorithm is consistent if it extracts similar rules every time it is applied to a specific
data set. According to Towell and Shavlik, consistency is important since it would be very hard to
give any significance to a specific rule set if the extracted rules vary significantly between runs.
Craven and Shavlik also pointed out another issue they believe to be a key to the success of rule
extraction methods, namely software availability; i.e. researchers should make their methods
available to potential users and fellow researchers to receive testing and evaluation.
An interesting discussion about the purpose of rule extraction is found in [6], where Zhou argues
that rule extraction really should be seen as two very different tasks; rule extraction using neural
networks and rule extraction for neural networks. The first task prioritizes accuracy while the
second focuses on fidelity. Rule extraction using neural networks thus is aimed at finding a
comprehensible model with higher accuracy than a comprehensible model created directly from
the data set using, for instance, a decision tree algorithm. Rule extraction for neural networks, on
the other hand, is solely aimed at understanding the inner workings of a trained neural network.
The claim made by Zhou is that it is never important to obtain both high fidelity and high
accuracy; the goal is always one of them, and, consequently the other should be ignored.
6. RELATED WORK CONCERNING RULE EXTRACTION
Since one key contribution of this thesis is a novel method for rule extraction from opaque
models, one very important related research area is previously presented rule extraction
algorithms. In this section some important rule extraction algorithms are discussed.
6. International Journal of Data Mining & Knowledge Management Process (IJDKP) Vol.4, No.3, May 2014
14
Below, three specific algorithms for rule extraction are presented. The motivation for including
RX [7] and TREPAN [8] is that they are well-known, general techniques, representing very
different approaches.
6.1 Rule Extraction(RX)
With the RX algorithm [7], Lu, Setino and Liu present a decompositional rule extraction
algorithm producing Boolean rules from feed-forward ANNs. The description below is a slight
simplification of the presented method; for more details see the original paper. Lu, Setino and Liu
use an approach consisting of three steps to create classification rules:
1. Network training: A three-layer ANN is trained using standard techniques; e.g. back
propagation. The coding used for the classes is localist; i.e. there is one output unit per class. To
facilitate the following pruning, it is desirable to have many weights with values so small that
they can be set to zero. This is accomplished by adding a penalty function to the error function;
for details see the original paper.
2. Network pruning: The fully connected network of step 1 is pruned to produce a much smaller
net without raising the classification error “too much”. More specifically, links with small
weights are iteratively removed and the smaller network is retrained until the accuracy on the
training set falls below an acceptable level.
3. Rule Extraction: Rules are extracted from the pruned network. The rules generated are of the
form if (a1 θ v1) and (a2 θ v2)… and (an θ vn) then Cj where ai’s are the attributes of an input
instance, vi’s are constants, Cj is one of the class labels and θ is a relational operator.
The process for the actual rule extraction is given in pseudocode below:
Input:
D // Training data
N //Pruned neural network
Output:
R // Extracted rules
Algorithm (RX):
Cluster hidden nodes activation values; Generate rules that describe the output values in terms of
the discretized hidden activation values; Generate rules that describe the discretized hidden output
values in terms of input values; Combine the two sets of rules; It should be noted that continuous
inputs must first be discretized, here by dividing their range into subintervals. Normally,
continuous inputs are then coded using thermometer coding. The RX algorithm relies heavily on
the success of the pruning since, if a node has n input links, there could be as many as 2n distinct
input patterns. Another problem is the fact that the activation value of a hidden node could be
anywhere in the range [-1, 1], (assuming a hyperbolic tangent activation function), depending on
the input instance. For a large training set this makes the activation function almost continuous.
The RX algorithm handles this by discretizing the activation values of hidden nodes into a
“manageable” number of discrete values. A small set of discrete activation values makes it
possible to determine the dependencies between hidden node and output node values, as well as
the dependencies between input and hidden node values. From these dependencies, rules can be
generated; in the RX algorithm this is done by using the X2R rule generator [10].
Although the RX algorithm is often cited, and even sometimes used as an example of typical rule
extraction; see e.g. [11], it is not widely used in practice. The source code is not publicly
7. International Journal of Data Mining & Knowledge Management Process (IJDKP) Vol.4, No.3, May 2014
15
available, which makes it hard to correctly evaluate the performance. It is obvious that RX most
likely will fail regarding scalability. The case study reported in the original paper uses a fairly
small data set (1000 instances with initially 7 attributes each) and the pruning is very successful;
only 17 of 386 links remain after the pruning, while the accuracy is still over 90%. Although most
criteria regarding rule quality (comprehensibility, accuracy and fidelity) seem to be well met, this
is very hard to judge from just one problem.
Regarding generality, RX is tightly bound to feed-forward ANNs. The demands for repeated
training during the pruning phase, a tailored error function and the many “tricks” to get inputs,
outputs and activation values into suitable formats also make RX a very specialized algorithm. It
should, in addition, be noted that RX extracts rules from one network only. If, as often is the case,
the predictive model consists of an ensemble of networks, RX would have to extract from a less
accurate model. This is a disadvantage compared to pedagogical methods, which would operate
directly on the actual predictive model.
A rule extraction technique based on RX is CaST (Cluster and See Through) [12]. The main idea
of CaST is to apply a clustering technique similar to the one used by RX, but to the activation
values of the input nodes; i.e. to the input values directly. This alteration in reality makes CaST a
black-box rule extraction algorithm because the extracted rules now describe outputs in terms of
inputs. Naturally, this makes it possible for CaST to extract rules from any opaque model, not just
single feed-forward ANNs. In the study reported in [12], CaST is evaluated against NeuroRule
(RX) and C5.0 on three data sets. The main result is that CaST and NeuroRule have almost
identical accuracy on two problems, but the rules found by CaST are significantly more compact.
On the third problem (Soybean) NeuroRule fails to extract rules since the pruning is not effective
enough. CaST on the other hand, produces a fairly complex rule set having higher accuracy than
C5.0.
6.2 TREPAN
TREPAN [8] is a pedagogical rule extraction algorithm for classification problems producing
decision trees. Each internal node in the extracted tree represents a splitting criterion and each
leaf represents a predicted class. TREPAN uses M-of-N expressions for its splits. M-of-N
expressions are Boolean expressions in disjunctive normal form, with N conjuncts and the
requirement that at least M of these should be true for the entire expression to be true.
TREPAN is a black-box method since it focuses exclusively on the input-output relationship,
instead of looking inside the neural net. In a way, TREPAN uses the network as an oracle; i.e. its
results are regarded as ground truth. TREPAN grows trees in a best-first fashion, since the node
with the greatest potential to increase the fidelity of the extracted tree is expanded first.
The TREPAN algorithm is given in pseudocode below:
Input:
D // Training data
N // Trained neural network
Output:
DT // Extracted decision tree
Algorithm (TREPAN):
Initialize the tree as a leaf node;
8. International Journal of Data Mining & Knowledge Management Process (IJDKP) Vol.4, No.3, May 2014
16
While stopping criteria not met.
Pick the most promising leaf node to expand Draw a sample of instances;
Use the network to label the instances;
Select splitting test for the node;
For each possible outcome of the test make a new leaf node;
The task of TREPAN is, according to Craven and Shavlik, to induce the function represented by
the trained ANN; i.e. fidelity is the basis of the score function.
Craven and Shavlik [4] describe TREPAN as similar to conventional decision tree algorithms
such as C4.5 with some basic differences. Below is a summary of the main properties of
TREPAN:
• TREPAN uses the ANN to label all instances. This also means that TREPAN can use the
ANN to label new instances and thus learn from arbitrarily large samples.
• In order to decide which node to expand next, TREPAN uses an evaluation function to
rank all of the leaves in the current tree. The evaluation function used for node N is: f (N)
= reach (N) - (1 − fidelity (N)) where reach (N) is the estimated fraction of instances that
reach node N and fidelity (N) is the estimated fidelity of the tree to the network for those
instances.
• TREPAN gets a sample of instances from two sources to find the logical test with which
to partition the instances that reach the node and to determine the class labels for the
leaves. First, it uses the ANN’s training examples that reach the node. Second, TREPAN
constructs a model (using the training examples) of the underlying distribution and uses
this model to draw instances. These instances are randomly drawn but are subject to the
constraint that they would reach the node being expanded if classified by the tree. In both
cases TREPAN queries the ANN to get the class label.
• TREPAN uses information gain as the evaluation measure when selecting splitting tests.
TREPAN is publicly available, the authors report several case studies [13], and has also been
extended in different ways; for instance to return fuzzy decision trees [14]. TREPAN performs
well, both in reported studies and in the experiments conducted in this thesis. The accuracy is
normally higher than that of models generated directly from the data set; e.g. by C5.0. ANN
fidelity is naturally high, since this is the purpose of the algorithm. Regarding comprehensibility
it can be argued that decision trees automatically produce good explanation, but for more
complex and bushy trees this is questionable. TREPAN handles this by extracting the tree in a
best-first fashion and allows the user the option to stop the growth at any level; an example of
anytime extraction.
Nevertheless there is still a trade-off between accuracy and comprehensibility. Actually for some
of the extracted trees reported [15]. As well as some found during the research for this thesis, the
ease of human inspection is questionable. This is arguably partly due to the use of M-of-N rules,
which for most people are tricky to read.
TREPAN was, according to Craven and Shavlik, designed with scalability and generality in mind.
The scalability criterion naturally favors black-box approaches, since the computational
complexity for black-box methods does not depend directly on the network architecture. The
node expansion itself is of polynomial computational complexity in the sample size, the number
of features and the maximum number of values for a discrete feature. Thus, TREPAN is likely to
9. International Journal of Data Mining & Knowledge Management Process (IJDKP) Vol.4, No.3, May 2014
17
scale up well to larger problems, at least regarding computational complexity. Since scalability in
comprehensibility requires that growth of the tree is stopped early, the accuracy of the extracted
tree is obviously very dependent on that the evaluation function used constantly finds the best
splits first. It should be noted that TREPAN in itself does not balance accuracy against
comprehensibility, but leaves this decision to the user. The user must choose when to stop the
growth of the tree, or put in another way, which tree of several, all with different complexity, to
actually use on novel data.
Regarding generality, TREPAN does not really require the underlying model to be an ANN.
There are no reports, though, of studies where the original TREPAN program is used for rule
extraction from anything else than ANNs. In the third-party implementation later used for
experimentation in this thesis, it was, however fairly easy to convert TREPAN into a true black-
box rule extraction algorithm.
6.3 Rule extraction using evolutionary algorithms
Dorado et al. in [9] present a novel approach to rule extraction based on evolutionary algorithms.
One should note that the two methods were developed independently1.
The algorithm suggested by Dorado et al. is a black-box method where the extraction strategy is
based on GP. The algorithm can handle different representation languages with ease, since the
choice of representation language corresponds to the choice of function and terminal sets. In the
paper, Dorado et al. give three examples, two where Boolean rules are extracted for classification
problems and one where a mathematical function, similar to, but more complex than a Box-
Jenkins model, is derived for a time series forecasting problem. The suggested approach is also
compared (with good results) to several existing techniques on well-known problems. A key
result reported by Dorado et al. is the comparison between the rule extraction algorithm and GP
applied directly on the data set. The accuracy of the rule extraction is slightly higher, supporting
the claim that the neural net in a sense is a better (more general) representation of the data than
the data set itself.
The purpose of the proposed method is to use GP for the search process. More specifically,
candidate rules (which could be Boolean rules, decision trees, regression trees etc.) are
continuously evaluated according to how well they resemble the ANN. The best rules are kept
and combined using genetic operators to raise the fitness (performance) over time. After many
iterations (generations) the most fit program (rule) is chosen as the extracted rule.
It should be noted that the fitness function is crucial for determining what to optimize, and that
the choice here is to solely optimize the fidelity of the extracted representation to the neural net.
Dorado et al. do not state the algorithm explicitly, so the description below is based on an
interpretation of their paper.
Input:
D // Training data
N // Trained neural network
F // Function set
T // Terminal set
Output:
R // Extracted representation
10. International Journal of Data Mining & Knowledge Management Process (IJDKP) Vol.4, No.3, May 2014
18
Algorithm: (Rule extraction using EA)
Initialize a first generation of candidate rules;
While number of generations not reached
Evaluate all candidate rules using the fitness function (fidelity);
Choose a number of candidate rules for reproduction;
Combine chosen candidate rules using genetic operators (crossover) to create offspring rules;
Replace old candidate rules with offspring rules;
The reported accuracy for the approach reported by Dorado et al. is high. Since GP is a
recognized tool for optimization problems and fidelity to the neural net is the optimization criteria
here, it is no surprise that the fidelity is high. The problem for the algorithm of Dorado et al. is
clearly comprehensibility.
Regarding time scalability, the proposed method is likely to perform well. Although GP is rather
computationally intensive, this applies even more for the original training of the neural net,
making it unlikely that the rule extraction would be the bottle-neck. The computational
complexity of a GP approach is dependent on parameters like the number of programs in each
generation, and to a lesser degree, the size of each program. This could potentially be a difficulty
for very complex problems, where large generations and/or programs are needed. This is
obviously an issue that should be looked in to.
How well comprehensibility scales up is a totally different matter. Since Dorado et al. do not try
to enforce short rules, complex data sets with many variables will inevitably produce long and
complicated rules.
The generality of the proposed approach is very high and is actually the most appealing property
of the method. Dorado et al. apply the rule extraction on both feed-forward and recurrent
networks and extract rules for both classification and regression. It is also obvious that, even if
the authors do not explicitly point this out, the algorithm does not require the underlying model to
be a neural net.
3. CONCLUSIONS
The main contribution of this paper was to show the benefit of using test set data instances,
together with predictions from the opaque model, when performing rule extraction. The technique
evaluated means that the same novel data instances used for actual prediction also are used by the
rule extraction algorithm. As demonstrated in the experiments, rules extracted using only oracle
data were significantly more accurate than both rules extracted by the same rule extraction
algorithm (using training data only) and standard decision tree algorithms. The overall
implication is that rules extracted in this way will have higher accuracy on the test set; thus
explaining the predictions made on the novel data better than rules extracted in the standard way;
i.e. using training data only.
ACKNOWLEDGEMENTS
I would like to all our lab coordinators and Sirt University who helped in all to succeed this
research paper.
REFERENCES
[1] Lee, S.hyun. & Kim Mi Na, (2008) “This is my paper”, ABC Transactions on ECE, Vol. 10, No. 5,
pp120-122.
[2] R. Andrews, J. Diederich and A. B. Tickle, A survey and critique of techniques for extracting rules
from trained artificial neural networks, Knowledge-Based Systems, 8(6).
11. International Journal of Data Mining & Knowledge Management Process (IJDKP) Vol.4, No.3, May 2014
19
[3] M. Craven and J. Shavlik, Using Neural Networks for Data Mining. Future Generation Computer
Systems: Special Issue on Data Mining, pp.211-229.
[4] M. Craven and J. Shavlik, Rule Extraction: Where Do We Go from Here?, University of Wisconsin
Machine Learning Research Group working Paper 99-1.
[5] G. Towell and J. Shavlik, The extraction of refined rules from knowledge based neural networks,
Machine Learning, 13(1):71-101M. Young, The Technical Writer’s Handbook. Mill Valley, CA:
University Science, 1989.
[6] Z.-H. Zhou, Rule Extraction: Using Neural Networks or For Neural Networks?, Journal of Computer
Science & Technology, 19(2):249-253, 2004.
[7] H. Lu, R. Setino and H. Liu, Neurorule: A connectionist approach to data mining, Proceedings of the
International Very Large Databases Conference, pp. 478-489, 1995. Article in a conference
proceedings:
[8] M. Craven and J. Shavlik, Extracting Tree-Structured Representations of Trained Networks,
Advances in Neural Information Processing Systems,8:24-30.
[9] J. Dorado, J. R. Rabunãl, A. Santos, A. Pazos and D. Rivero, Automatic Recurrent and Feed-Forward
ANN Rule and Expression Extraction with Genetic Programming, Proceedings 7th International
Conference onParallel Problem Solving from Nature, Granada,
[10]. H. Liu, X2R: A fast rule generator, Proceedings of IEEE International Conference on Systems, Man
and Cybernetics, Vancouver.
[11]. M. Dunham, Data Mining – Introductory and Advanced Topics, Prentice Hall, 2003.
[12]. T. Löfström, U. Johansson, and L. Niklasson, Rule Extraction by Seeing Through the Model, 11th
International Conference on Neural Information Processing, Calcutta, India, pp. 555-560, 2004.
[13]. M. Craven, Extracting Comprehensive Models from Trained Neural Networks, Ph.D. Thesis,
University of Wisconsin-Madiso.
[14]. M. Faifer, C. Janikow, and K. Krawiec, Extracting Fuzzy Symbolic Representation from Artificial
Neural Networks, Proceedings 18th International Conference of the North American Fuzzy
Information Processing Society, New York, NY, pp. 600-604.
[15]. M. Craven and J. Shavlik, Understanding time-series networks: A case study in rule extraction,
International Journal of Neural Systems, 8(4):373-384.
Author
Tiruveedula GopiKrishna
Lecturer,
Faculty of Arts and Science
Sirt University,
Department of Computer Science,
Hoon,Aljufra,
Libya