Experiments on Design Pattern Discovery Jing Dong and Yajing Zhao Department of Computer Science The University of Texas a...
Outline <ul><li>Introduction </li></ul><ul><li>Related Work </li></ul><ul><li>Our Approach </li></ul><ul><li>Experiment Da...
Introduction <ul><li>Difficult to understand large computer-based systems </li></ul><ul><li>No original architecture and d...
Introduction (Cont’d) <ul><li>DP-Miner </li></ul><ul><li>Experiment data: System source code </li></ul><ul><ul><li>JUnit <...
Related Work X Flyweight X Facade X X Chain of Responsibility X X Builder X X X Abstract Factory X X X X X X X X X X Balan...
Our Approach – DP-Miner <ul><li>Matrix and Weight </li></ul><ul><li>Product of Prime Numbers </li></ul><ul><li>Structural,...
Experiment Data <ul><li>JUnit: A regression test framework that helps developers to implement unit tests in Java. </li></u...
Why Those Four Systems as Data? <ul><li>At the time they were developed, the idea of design patterns is already mature and...
Experiment Data <ul><li>System information </li></ul><ul><ul><li>Version </li></ul></ul><ul><ul><li>Number of Classes </li...
Experiment Result 64 24 6 76 Strategy 0 58 4 JHotDraw 0 24 17 JEdit 3 6 3 JUnit 3 65 21 Java.AWT Composite Bridge Adapter ...
Experiment Result – Java.AWT Results for Each Analysis Phase N/A 76 65 N/A Semantic Analysis 3 3 92 Composite 76 76 100 St...
Experiment Result – JUnit Results for Each Analysis Phase N/A 6 6 N/A Semantic Analysis 3 6 6 3 Final Result 3 9 Composite...
Experiment Result – JEdit Results for Each Analysis Phase N/A 24 24 N/A Semantic Analysis 0 0 0 Composite 24 24 33 Strateg...
Experiment Result – JHotDraw Results for Each Analysis Phase N/A 64 58 N/A Semantic Analysis 0 0 0 Composite 64 64 74 Stra...
Observations <ul><li>Behavioral analysis phase only deals with a small number of classes, since it’s based on the result o...
Recovery Precisions for JHotDraw <ul><li>Manually checked the results generated by our tool and see whether they are real ...
Instances Missed <ul><li>Instances manually found but missed by DP-Miner </li></ul><ul><li>Recall: 89.23% </li></ul>Drawin...
Reasons of Discrepancy <ul><li>Flexibility of design pattern </li></ul><ul><ul><li>Composite pattern: Collapse both the Co...
Benchmark <ul><li>Correct number of pattern instances and their locations are generally not available. </li></ul><ul><li>L...
Conclusion <ul><li>We did a series of experiments on design pattern discovery from open-source systems using DP-Miner </li...
Future Work <ul><li>Improve our tool. </li></ul><ul><li>Experiment on other systems  </li></ul><ul><li>Manually check resu...
Thank You!
Questions?
Overall Architecture of Our Approach System    Design Pattern match Source Code Intermediate Representation
Structural Analysis – Matrix
1 1 1 1 7 1 1 1 1 TextComponent 1 1 1 1 7 1 1 1 1 Scrollbar 1 1 1 1 7 1 1 1 1 Label 1 1 1 1 35 1 1 1 1 Container 1 1 1 5 1...
1 1 1 1 7 1 1 1 1 TextComponent 1 1 1 1 7 1 1 1 1 Scrollbar 1 1 1 1 7 1 1 1 1 Label 1 1 1 1 35 1 1 1 1 Container 1 1 1 5 1...
Upcoming SlideShare
Loading in …5
×

Experiments on Design Pattern Discovery

2,658 views

Published on

Jing Dong and Yajing Zhao

Published in: Technology
  • Be the first to comment

Experiments on Design Pattern Discovery

  1. 1. Experiments on Design Pattern Discovery Jing Dong and Yajing Zhao Department of Computer Science The University of Texas at Dallas {jdong, yxz045100}@utdallas.edu
  2. 2. Outline <ul><li>Introduction </li></ul><ul><li>Related Work </li></ul><ul><li>Our Approach </li></ul><ul><li>Experiment Data </li></ul><ul><li>Result and Discussion </li></ul><ul><li>Precision and Recall </li></ul><ul><li>Benchmark </li></ul><ul><li>Conclusion and Future Work </li></ul>
  3. 3. Introduction <ul><li>Difficult to understand large computer-based systems </li></ul><ul><li>No original architecture and design </li></ul><ul><li>Patterns help on understanding systems </li></ul><ul><li>Patterns embed future change of systems </li></ul><ul><li>Need of design pattern discovery since patterns are lost in source code </li></ul>
  4. 4. Introduction (Cont’d) <ul><li>DP-Miner </li></ul><ul><li>Experiment data: System source code </li></ul><ul><ul><li>JUnit </li></ul></ul><ul><ul><li>JEdit </li></ul></ul><ul><ul><li>JHotDraw </li></ul></ul><ul><ul><li>Java.AWT </li></ul></ul><ul><li>Such experiments lead to benchmarks </li></ul>
  5. 5. Related Work X Flyweight X Facade X X Chain of Responsibility X X Builder X X X Abstract Factory X X X X X X X X X X Balanyi 2003 X X Command X X X Proxy X X X X X X Composite X X X X X Decorator X X X X Factory Method X X X X Observer X X Prototype X X X X Singleton X X X X Strategy X X X Template Method X X X Visitor X X X X Bridge X X X X Adapter Gueheneuc 2006 Shi 2006 Blewitt 2001 Niere 2002 Heuzeroth 2003 Antoniol 1998 Tsantalis 2006 Authors Tools
  6. 6. Our Approach – DP-Miner <ul><li>Matrix and Weight </li></ul><ul><li>Product of Prime Numbers </li></ul><ul><li>Structural, Behavioral, and Semantic Analysis </li></ul>
  7. 7. Experiment Data <ul><li>JUnit: A regression test framework that helps developers to implement unit tests in Java. </li></ul><ul><li>JEdit: A mature and easy-to-use text editor. </li></ul><ul><li>JHotDraw: A two dimensional graphics frameworks for technical and structured drawing editors written in Java. </li></ul><ul><li>Java.AWT: A library for developing graphical user interfaces for Java programs. </li></ul>
  8. 8. Why Those Four Systems as Data? <ul><li>At the time they were developed, the idea of design patterns is already mature and widely applied. </li></ul><ul><li>Other works on design pattern discovery use one or more of these systems to evaluate their approaches, which allows us to compare and evaluate our experiment results. </li></ul>
  9. 9. Experiment Data <ul><li>System information </li></ul><ul><ul><li>Version </li></ul></ul><ul><ul><li>Number of Classes </li></ul></ul><ul><ul><li>Number of Files </li></ul></ul>484 530 6.0 beta 1 JHotDraw 394 1001 4.2 JEdit 93 126 3.8.2 JUnit 345 570 JDK1.4.2 Java.AWT File # Class # Version Systems
  10. 10. Experiment Result 64 24 6 76 Strategy 0 58 4 JHotDraw 0 24 17 JEdit 3 6 3 JUnit 3 65 21 Java.AWT Composite Bridge Adapter Systems
  11. 11. Experiment Result – Java.AWT Results for Each Analysis Phase N/A 76 65 N/A Semantic Analysis 3 3 92 Composite 76 76 100 Strategy 65 76 100 Bridge 21 21 57 Adapter Final Result Behavioral Analysis Structural Analysis Systems Java.AWT
  12. 12. Experiment Result – JUnit Results for Each Analysis Phase N/A 6 6 N/A Semantic Analysis 3 6 6 3 Final Result 3 9 Composite 6 6 Strategy 6 6 Bridge 3 15 Adapter Behavioral Analysis Structural Analysis Systems JUnit
  13. 13. Experiment Result – JEdit Results for Each Analysis Phase N/A 24 24 N/A Semantic Analysis 0 0 0 Composite 24 24 33 Strategy 24 24 33 Bridge 17 17 80 Adapter Final Result Behavioral Analysis Structural Analysis Systems JEdit
  14. 14. Experiment Result – JHotDraw Results for Each Analysis Phase N/A 64 58 N/A Semantic Analysis 0 0 0 Composite 64 64 74 Strategy 58 64 74 Bridge 4 4 27 Adapter Final Result Behavioral Analysis Structural Analysis Systems JHotDraw
  15. 15. Observations <ul><li>Behavioral analysis phase only deals with a small number of classes, since it’s based on the result of structural analysis phase. </li></ul><ul><li>Significant reduction of the candidates between structural and behavioral analysis of Adapter and Composite pattern than those of Bridge and Strategy patterns, since the former have more behavioral characteristics than the latter do. </li></ul><ul><li>Result of structural and behavioral analysis of the Bridge and Strategy patterns are the same, since they have same structural and behavioral characteristics and only differ in their intents. </li></ul>
  16. 16. Recovery Precisions for JHotDraw <ul><li>Manually checked the results generated by our tool and see whether they are real pattern instances. </li></ul><ul><li>TP: True Positive </li></ul><ul><li>FP: False Positive </li></ul>100% 0 0 Composite 90.63% 6 58 Strategy 91.38% 5 53 Bridge 100% 0 4 Adapter Precision FP TP JHotDraw
  17. 17. Instances Missed <ul><li>Instances manually found but missed by DP-Miner </li></ul><ul><li>Recall: 89.23% </li></ul>DrawingView SelectionTool [7] Locator LocatorConnector [6] Locator LocatorHandle [5] Locator PolygonHandle [4] Connector ConnectionTool [3] Connector ChangeConnectionHandle [2] Connector LineConnection [1] STRATEGY CONTEXT
  18. 18. Reasons of Discrepancy <ul><li>Flexibility of design pattern </li></ul><ul><ul><li>Composite pattern: Collapse both the Component and Composite classes into a single class or not. </li></ul></ul><ul><ul><li>Adapter pattern: Full matching all three roles, i.e. Target, Adapter, and Adaptee, or partial matching of only Adapter and Adaptee. </li></ul></ul><ul><ul><li>Object-oriented programming languages provide special language constructs that greatly simplify the implementation of a design pattern. E.g., Java provides LinkedList, ArrayList, HashMap, and Hashtable, which make the implementation of aggregate elements in patterns easy, but the detection harder. </li></ul></ul>
  19. 19. Benchmark <ul><li>Correct number of pattern instances and their locations are generally not available. </li></ul><ul><li>Lacking benchmarks is the main impediment </li></ul><ul><ul><li>Hard to evaluate and compare design pattern discovery techniques. </li></ul></ul><ul><ul><li>Hard to judge whether an approach discovers all instances and whether the discovered ones are correct. </li></ul></ul><ul><li>To calculate the precision and recall of a pattern matching result, it’s essential to know </li></ul><ul><ul><li>The number of correct pattern instances </li></ul></ul><ul><ul><li>The locations of correct pattern instances </li></ul></ul><ul><li>JHotDraw pattern benchmark </li></ul>
  20. 20. Conclusion <ul><li>We did a series of experiments on design pattern discovery from open-source systems using DP-Miner </li></ul><ul><li>Patterns concerned: Adapter, Bridge, Strategy, Composite </li></ul><ul><li>Experiment data: Java.AWT, JUnit, JEdit, JHotDraw </li></ul><ul><li>We compared our experiment results with others and found several discrepancies. </li></ul><ul><li>We analyzed the issues and discussed possible reasons. </li></ul><ul><li>We argue for benchmarks for design pattern discovery. </li></ul>
  21. 21. Future Work <ul><li>Improve our tool. </li></ul><ul><li>Experiment on other systems </li></ul><ul><li>Manually check results of other systems and perfect the benchmark </li></ul><ul><li>Research on other object-oriented languages, such as C++. </li></ul>
  22. 22. Thank You!
  23. 23. Questions?
  24. 24. Overall Architecture of Our Approach System  Design Pattern match Source Code Intermediate Representation
  25. 25. Structural Analysis – Matrix
  26. 26. 1 1 1 1 7 1 1 1 1 TextComponent 1 1 1 1 7 1 1 1 1 Scrollbar 1 1 1 1 7 1 1 1 1 Label 1 1 1 1 35 1 1 1 1 Container 1 1 1 5 1 1 1 1 1 Component 1 1 1 1 7 1 1 1 1 Choice 1 1 1 1 7 1 1 1 1 Checkbox 1 1 1 1 7 1 1 1 1 Canvas 1 1 1 1 7 1 1 1 1 Button TextComponent Scrollbar Label Container Component Choice Checkbox Canvas Button
  27. 27. 1 1 1 1 7 1 1 1 1 TextComponent 1 1 1 1 7 1 1 1 1 Scrollbar 1 1 1 1 7 1 1 1 1 Label 1 1 1 1 35 1 1 1 1 Container 1 1 1 5 1 1 1 1 1 Component 1 1 1 1 7 1 1 1 1 Choice 1 1 1 1 7 1 1 1 1 Checkbox 1 1 1 1 7 1 1 1 1 Canvas 1 1 1 1 7 1 1 1 1 Button TextComponent Scrollbar Label Container Component Choice Checkbox Canvas Button 1 35 1 Composite 1 1 1 Component 1 7 1 Leaf Composite Component Leaf

×