Symbolic Rules Extraction From Trained Neural Networks

Symbolic Rules
Extraction From Trained
Neural Networks

Koushal Kumar
M .Tech CSE
Mob: +918968939621

26/5/2012 Koushal Kumar 1

What are Artificial Neural Networks?
• Artificial Neural Networks are powerful
computational systems consisting of many
simple processing elements connected together
to perform tasks analogously to biological
brains.
• They are massively parallel, which makes them
efficient, robust, fault tolerant and noise tolerant
• They can learn from training data and generalize
to new situations.

The limitation of Neural Network

The major criticism against Neural Network is that decision
given by neural networks is difficult to understand by a
human being. Reasons for this are

Knowledge in Neural Networks are stored as real values
parameters (weights and bias) of networks

Neural Networks are unable to explain its internal
processing how they come to particular decision

This behavior makes Neural Networks Black Box in
Nature

Continue..


Rules extraction From Neural
Networks.
So to overcome the Black Box nature of Neural
Networks we need to extract rules from Neural
Networks so that the user can gain a better
understanding of the decision process. following types
of rules can be extracted from neural networks

I) M OF N types rules

II) Fuzzy rules

continue..
III) IF THEN RULES
IV) Decision Rules

V) First order logic rules

From all above types of rules IF THEN RULES
and Decision rules are easy to understand then
others kind of rules.


J48 Algorithm for extracting
decision trees

• J48 is an algorithm used to generate a decision tree.
• Developed by quinlan and most widely used decision
tree induction algorithm.
• It is based upon greedy search approach i.e select the
best attribute and never looks back to reconsider early
choices.
• It select the best attribute according to its entropy
value.
• More preference will be given to that attribute which
has more value of entropy.

Before normalization


After normalization


MATLAB Simulator

• Matlab stands for matrix laboratory.
• It integrate computation, visualization, and
programming in an easy-to-use environment.
• MATLAB is a package that has been purpose-
designed to make computations easy, fast and
reliable.
• Matlab can be used in math and computation,
algorithm development, simulation purposes.
• MATLAB is a powerful system that can plot graphs
and perform a large variety of calculations with
numbers.

Weka simulator

• WEKA is abbreviation of Waikato Environment for
Knowledge Analysis.
• Weka is open source simulator with machine learning
algorithms.
• The Weka workbench contains a collection of
visualization tools and algorithms for data analysis
and predictive modelling.


Training data set and its Histogram


Histogram of input target values


Histogram of testing set as Inputs


Histogram of testing set as Target values


Neural networks Training with BPA


Best Training Performance curve


Gradient Performance Curve After Training &
Testing


Data import window in weka


J48 algorithm Output (pruned)


Results from pruned Tree


J48 algorithm Output (Unpruned)


Unpruned Tree Results


Results of classification from Decision Tree


Decision Tree Extracted From Exp no 7


The Following Rule Set Is Obtained From the above Decision
Tree
I. Applying Remove Redundancy Conditions
IF Children ≥ 1 AND Children >2AND Children >3 THEN Marital status =YES
Children ≥ 1 is more specific than Children >3 and Children > 2. So we remove all such
conditions
Rule 1:
a) IF Current_act = NO AND Age ≤ 48.0 AND Sex = FEMALE AND Children ≤ 0 THEN Region
Town
b) IF AGE > 48.0 AND Region Suburban AND Current_act = NO then Pep = NO
II. For every pair decision trees Remove redundancy rules. For example
Rule 1: IF Age ≤60 AND Salary ≤ 3500 AND Pep = NO THEN Mortage = YES
Rule 2: IF Age ≤ 50 AND Salary ≤ 3500 AND Pep = NO THEN Mortage = YES
New Rule: IF Age ≤ 50 AND Salary ≤ 3500 AND Pep = NO THEN Mortage =YES
III. Remove more specific rules. The rules with a condition set which is a superset of
another rule should be removed. For example


Continue….
• Rule 1: IF Age ≤ 60 AND Region = Rural AND Saving_ act =
YES THEN Pep = NO
• Rule 2: IF Age <= 60 AND Children <= 1 AND Region
=Rural AND saving act = YES THEN Pep= NO
• Rule 3: IF Region = Rural AND saving _ act =YES THEN
Pep = NO
• New Rule: IF Region = Rural AND saving _ act =YES THEN
Pep = NO


Comparison of J48 algorithm with Others Classifiers


Graphical comparisons of algorithms


Data Flow diagram of J48 algorithm


Paper Published

• “Extracting Explanation From Artificial Neural Networks” is
published in International Journal of Computer Science and
Information Technologies.
• “Advanced Applications of Neural Networks and Artificial
Intelligence: A Review” has been selected in International journal of
information technologies and computer science and it will published
on May June volume of journal.
• Seminar: Published Research paper on “Symbolic Rules Extraction
From Trained Neural Networks” in two days UGC Sponsored
National Seminar on Social Implications of Artificial Intelligence
organized in KMV College in Jalandhar.


References
• Olcay Boz AT & T Labs”Converting a trained neural network
to a decision tree DecText-Decision Tree Extractor.
• J.T Yao Dept of computer sci university of Regina “Knowledge
extracted from Trained neural networks whats next?
• Simon Haykin ”Neural networks a Comprehensive foundation”
Pearson education(second edition).
• R. Davis, B.G. Buchanan, and E. Shortcliff, “Production Rules
as a
• Representation for a Knowledge Based Consultation Progra”,
• Artificial Intelligence, 1977, vol. 8(1), pp.15-45.
• Knowledge Extraction from the Neural ‘Black Box’ in
Ecological Monitoring Journal of Industrial and Systems
Engineering Vol. 3, No. 1, pp 38-55 Spring 2009

Symbolic Rules Extraction From Trained Neural Networks

More Related Content

Viewers also liked

Similar to Symbolic Rules Extraction From Trained Neural Networks

Recently uploaded

Symbolic Rules Extraction From Trained Neural Networks

Editor's Notes