Symbolic RulesExtraction From Trained   Neural Networks                                Koushal Kumar                      ...
What are Artificial Neural Networks?• Artificial Neural Networks are powerful  computational systems consisting of many  s...
The limitation of Neural NetworkThe major criticism against Neural Network is that decisiongiven by neural networks is dif...
Continue..26/5/2012   Koushal Kumar   4
Rules extraction From Neural              Networks.So to overcome the       Black Box nature of NeuralNetworks we need to ...
continue.. III) IF THEN RULES IV) Decision Rules V) First order logic rules From all above types of rules IF THEN RULES an...
J48 Algorithm for extractingdecision trees• J48 is an algorithm used to generate a decision tree.• Developed by quinlan an...
Before normalization26/5/2012      Koushal Kumar   8
After normalization26/5/2012      Koushal Kumar   9
MATLAB Simulator•    Matlab stands for matrix laboratory.•    It integrate computation, visualization, and     programming...
Weka simulator• WEKA is abbreviation of Waikato Environment for    Knowledge Analysis.•   Weka is open source simulator wi...
Training data set and its Histogram26/5/2012          Koushal Kumar              12
Histogram of input target values26/5/2012            Koushal Kumar             13
Histogram of testing set as Inputs26/5/2012         Koushal Kumar        14
Histogram of testing set as Target values26/5/2012       Koushal Kumar        15
Neural networks Training with BPA26/5/2012       Koushal Kumar      16
Best Training Performance curve26/5/2012           Koushal Kumar       17
Gradient Performance Curve After Training &Testing26/5/2012      Koushal Kumar      18
Data import window in weka26/5/2012     Koushal Kumar   19
J48 algorithm Output (pruned)26/5/2012          Koushal Kumar            20
Results from pruned Tree26/5/2012        Koushal Kumar     21
J48 algorithm Output (Unpruned)26/5/2012           Koushal Kumar       22
Unpruned Tree Results26/5/2012       Koushal Kumar       23
Results of classification from Decision Tree26/5/2012      Koushal Kumar       24
Decision Tree Extracted From Exp no 726/5/2012      Koushal Kumar       25
The Following Rule Set Is Obtained From the above Decision    TreeI. Applying Remove Redundancy Conditions   IF Children ≥...
Continue….• Rule 1: IF Age ≤ 60 AND Region = Rural AND Saving_ act =     YES THEN Pep = NO•    Rule 2: IF Age <= 60 AND Ch...
Comparison of J48 algorithm with Others Classifiers26/5/2012       Koushal Kumar         28
Graphical comparisons of algorithms26/5/2012    Koushal Kumar     29
Data Flow diagram of J48 algorithm26/5/2012       Koushal Kumar      30
Paper Published• “Extracting Explanation From Artificial Neural Networks” is  published in International Journal of Comput...
References• Olcay Boz AT & T Labs”Converting a trained neural network    to a decision tree DecText-Decision Tree Extracto...
Upcoming SlideShare
Loading in …5
×

Symbolic Rules Extraction From Trained Neural Networks

1,107 views
974 views

Published on

Published in: Education, Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,107
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
63
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide
  • 08/05/12
  • Symbolic Rules Extraction From Trained Neural Networks

    1. 1. Symbolic RulesExtraction From Trained Neural Networks Koushal Kumar M .Tech CSE Mob: +91896893962126/5/2012 Koushal Kumar 1
    2. 2. What are Artificial Neural Networks?• Artificial Neural Networks are powerful computational systems consisting of many simple processing elements connected together to perform tasks analogously to biological brains.• They are massively parallel, which makes them efficient, robust, fault tolerant and noise tolerant• They can learn from training data and generalize to new situations.26/5/2012 Koushal Kumar 2
    3. 3. The limitation of Neural NetworkThe major criticism against Neural Network is that decisiongiven by neural networks is difficult to understand by ahuman being. Reasons for this areKnowledge in Neural Networks are stored as real valuesparameters (weights and bias) of networksNeural Networks are unable to explain its internalprocessing how they come to particular decisionThis behavior makes Neural Networks Black Box inNature 26/5/2012 Koushal Kumar 3
    4. 4. Continue..26/5/2012 Koushal Kumar 4
    5. 5. Rules extraction From Neural Networks.So to overcome the Black Box nature of NeuralNetworks we need to extract rules from NeuralNetworks so that the user can gain a betterunderstanding of the decision process. following typesof rules can be extracted from neural networksI) M OF N types rulesII) Fuzzy rules 26/5/2012 Koushal Kumar 5
    6. 6. continue.. III) IF THEN RULES IV) Decision Rules V) First order logic rules From all above types of rules IF THEN RULES and Decision rules are easy to understand then others kind of rules.26/5/2012 Koushal Kumar 6
    7. 7. J48 Algorithm for extractingdecision trees• J48 is an algorithm used to generate a decision tree.• Developed by quinlan and most widely used decision tree induction algorithm.• It is based upon greedy search approach i.e select the best attribute and never looks back to reconsider early choices.• It select the best attribute according to its entropy value.• More preference will be given to that attribute which has more value of entropy.26/5/2012 Koushal Kumar 7
    8. 8. Before normalization26/5/2012 Koushal Kumar 8
    9. 9. After normalization26/5/2012 Koushal Kumar 9
    10. 10. MATLAB Simulator• Matlab stands for matrix laboratory.• It integrate computation, visualization, and programming in an easy-to-use environment.• MATLAB is a package that has been purpose- designed to make computations easy, fast and reliable.• Matlab can be used in math and computation, algorithm development, simulation purposes.• MATLAB is a powerful system that can plot graphs and perform a large variety of calculations with numbers. 26/5/2012 Koushal Kumar 10
    11. 11. Weka simulator• WEKA is abbreviation of Waikato Environment for Knowledge Analysis.• Weka is open source simulator with machine learning algorithms.• The Weka workbench contains a collection of visualization tools and algorithms for data analysis and predictive modelling.26/5/2012 Koushal Kumar 11
    12. 12. Training data set and its Histogram26/5/2012 Koushal Kumar 12
    13. 13. Histogram of input target values26/5/2012 Koushal Kumar 13
    14. 14. Histogram of testing set as Inputs26/5/2012 Koushal Kumar 14
    15. 15. Histogram of testing set as Target values26/5/2012 Koushal Kumar 15
    16. 16. Neural networks Training with BPA26/5/2012 Koushal Kumar 16
    17. 17. Best Training Performance curve26/5/2012 Koushal Kumar 17
    18. 18. Gradient Performance Curve After Training &Testing26/5/2012 Koushal Kumar 18
    19. 19. Data import window in weka26/5/2012 Koushal Kumar 19
    20. 20. J48 algorithm Output (pruned)26/5/2012 Koushal Kumar 20
    21. 21. Results from pruned Tree26/5/2012 Koushal Kumar 21
    22. 22. J48 algorithm Output (Unpruned)26/5/2012 Koushal Kumar 22
    23. 23. Unpruned Tree Results26/5/2012 Koushal Kumar 23
    24. 24. Results of classification from Decision Tree26/5/2012 Koushal Kumar 24
    25. 25. Decision Tree Extracted From Exp no 726/5/2012 Koushal Kumar 25
    26. 26. The Following Rule Set Is Obtained From the above Decision TreeI. Applying Remove Redundancy Conditions IF Children ≥ 1 AND Children >2AND Children >3 THEN Marital status =YES Children ≥ 1 is more specific than Children >3 and Children > 2. So we remove all such conditions Rule 1: a) IF Current_act = NO AND Age ≤ 48.0 AND Sex = FEMALE AND Children ≤ 0 THEN Region Town b) IF AGE > 48.0 AND Region Suburban AND Current_act = NO then Pep = NOII. For every pair decision trees Remove redundancy rules. For example Rule 1: IF Age ≤60 AND Salary ≤ 3500 AND Pep = NO THEN Mortage = YES Rule 2: IF Age ≤ 50 AND Salary ≤ 3500 AND Pep = NO THEN Mortage = YES New Rule: IF Age ≤ 50 AND Salary ≤ 3500 AND Pep = NO THEN Mortage =YESIII. Remove more specific rules. The rules with a condition set which is a superset of another rule should be removed. For example 26/5/2012 Koushal Kumar 26
    27. 27. Continue….• Rule 1: IF Age ≤ 60 AND Region = Rural AND Saving_ act = YES THEN Pep = NO• Rule 2: IF Age <= 60 AND Children <= 1 AND Region =Rural AND saving act = YES THEN Pep= NO• Rule 3: IF Region = Rural AND saving _ act =YES THEN Pep = NO• New Rule: IF Region = Rural AND saving _ act =YES THEN Pep = NO 26/5/2012 Koushal Kumar 27
    28. 28. Comparison of J48 algorithm with Others Classifiers26/5/2012 Koushal Kumar 28
    29. 29. Graphical comparisons of algorithms26/5/2012 Koushal Kumar 29
    30. 30. Data Flow diagram of J48 algorithm26/5/2012 Koushal Kumar 30
    31. 31. Paper Published• “Extracting Explanation From Artificial Neural Networks” is published in International Journal of Computer Science and Information Technologies.• “Advanced Applications of Neural Networks and Artificial Intelligence: A Review” has been selected in International journal of information technologies and computer science and it will published on May June volume of journal.• Seminar: Published Research paper on “Symbolic Rules Extraction From Trained Neural Networks” in two days UGC Sponsored National Seminar on Social Implications of Artificial Intelligence organized in KMV College in Jalandhar.26/5/2012 Koushal Kumar 31
    32. 32. References• Olcay Boz AT & T Labs”Converting a trained neural network to a decision tree DecText-Decision Tree Extractor.• J.T Yao Dept of computer sci university of Regina “Knowledge extracted from Trained neural networks whats next?• Simon Haykin ”Neural networks a Comprehensive foundation” Pearson education(second edition).• R. Davis, B.G. Buchanan, and E. Shortcliff, “Production Rules as a• Representation for a Knowledge Based Consultation Progra”,• Artificial Intelligence, 1977, vol. 8(1), pp.15-45.• Knowledge Extraction from the Neural ‘Black Box’ in Ecological Monitoring Journal of Industrial and Systems Engineering Vol. 3, No. 1, pp 38-55 Spring 200926/5/2012 Koushal Kumar 32

    ×