This paper presents a novel approach called LOcal Rule Extraction (LORE) to extract rules from neural networks. LORE transforms a trained multilayer perceptron network into an equivalent decision diagram form to extract logic rules that generalize the network's output for inputs similar to the training set, while relaxing this condition for other inputs. It works by deriving a partial rule for each training sample, merging these rules, and then generalizing the merged rule set over the entire input space. The extracted rules are assessed based on their accuracy, fidelity to the original network, consistency, comprehensibility, and the computational complexity of the extraction process.

1. Rule Extracting from neural
networks as decision diagram
Navid Khoob
navidkhoob@hotmail or yahoo or google or ieee .com

2. Abstract
 Rule extraction from neural networks solves two
fundamental problems:
 1-it gives insight into the logic behind the network
 2- it improves the network’s ability to generalize
the acquired knowledge.
 This paper presents a novel eclectic approach to
rule extraction from NNs, named LOcal Rule
Extraction (LORE)

3. Abstract
 This approach is suited for multilayer perceptron
networks with discrete (logical or categorical)
inputs.
 The extracted rules mimic network behavior on
training set and relax this condition on the
remaining input space.

4. Focus on the Objectives of the paper
 First, a multilayer perceptron network is trained under
standard regime.
 It is then transformed into an equivalent form, returning
the same numerical result as the original network,
 yet being able to produce rules generalizing the network
output for cases similar to a given input.
 The partial rules extracted for every training set sample
are then merged to form a decision diagram from which
logic rules can be extracted.

5. Outline of the LORE method
 fun LORE( )
 G  U {start with the empty rule}
 for all X E Training Set do
 {Create new partial rule describing just this sample}
 PR  derivePR(net;X)
 G  merge(G; PR)
 end for{now G correctly classifies all training samples and
 needs to be extended over whole feature space I}
 G  generalize(G)
 Pruned prune(G) {optionally further simplify}

6. Criteria Commonly used
 Criteria commonly used to assess the usefulness of rule extraction
methods are:
 accuracy [it measures the ability of the rules to properly classify
previously unseen data (generalization ability)];
 fidelity (it reflects how well the rules mimic the network);
 consistency (it describes how the rules differ between different
training sessions);
 comprehensibility (it states how easy to understand a set of rules is
by measuring the number of rules and their antecedents);
 and finally, computational complexity (which reflects the needs of the
process of rule generation).

7. Structure and tasks

8. Input without any noise

9. Noisy Input

10. Output after 50 iteration

11. Output after 70 iteration


12. Output after 100 iteration

13. Output after 200 iteration


14. Output after 500 iteration

15. Production of Network

16. Production of Rule Extraction

17. References
 1-Extracting Rules from Neural Networks as Decision Diagrams,
Written by Zurada and Chorowski
 2-Tree based methods for Rule Extraction from Artificial Neural
Networks, written by Darren Dancy, PhD submission
 3- Using a Neural Networks and Genetic Algorithm to Extract Decision
Rules, written by K. Blackmore and T. Bossomaier,
 4- Designing a decompositional rule extraction algorithm for neural
networks with bound decomposition tree, J. S. Heh and J. C. Chen
and M Chang