Experiments on Design Pattern Discovery

Experiments on Design Pattern Discovery Jing Dong and Yajing Zhao Department of Computer Science The University of Texas at Dallas {jdong, yxz045100}@utdallas.edu

Outline
Introduction
Related Work
Our Approach
Experiment Data
Result and Discussion
Precision and Recall
Benchmark
Conclusion and Future Work

Introduction
Difficult to understand large computer-based systems
No original architecture and design
Patterns help on understanding systems
Patterns embed future change of systems
Need of design pattern discovery since patterns are lost in source code

Introduction (Cont’d)
DP-Miner
Experiment data: System source code
JUnit
JEdit
JHotDraw
Java.AWT
Such experiments lead to benchmarks

Related Work X Flyweight X Facade X X Chain of Responsibility X X Builder X X X Abstract Factory X X X X X X X X X X Balanyi 2003 X X Command X X X Proxy X X X X X X Composite X X X X X Decorator X X X X Factory Method X X X X Observer X X Prototype X X X X Singleton X X X X Strategy X X X Template Method X X X Visitor X X X X Bridge X X X X Adapter Gueheneuc 2006 Shi 2006 Blewitt 2001 Niere 2002 Heuzeroth 2003 Antoniol 1998 Tsantalis 2006 Authors Tools

Our Approach – DP-Miner
Matrix and Weight
Product of Prime Numbers
Structural, Behavioral, and Semantic Analysis

Experiment Data
JUnit: A regression test framework that helps developers to implement unit tests in Java.
JEdit: A mature and easy-to-use text editor.
JHotDraw: A two dimensional graphics frameworks for technical and structured drawing editors written in Java.
Java.AWT: A library for developing graphical user interfaces for Java programs.

Why Those Four Systems as Data?
At the time they were developed, the idea of design patterns is already mature and widely applied.
Other works on design pattern discovery use one or more of these systems to evaluate their approaches, which allows us to compare and evaluate our experiment results.

Experiment Data
System information
Version
Number of Classes
Number of Files
484 530 6.0 beta 1 JHotDraw 394 1001 4.2 JEdit 93 126 3.8.2 JUnit 345 570 JDK1.4.2 Java.AWT File # Class # Version Systems

Experiment Result 64 24 6 76 Strategy 0 58 4 JHotDraw 0 24 17 JEdit 3 6 3 JUnit 3 65 21 Java.AWT Composite Bridge Adapter Systems

Experiment Result – Java.AWT Results for Each Analysis Phase N/A 76 65 N/A Semantic Analysis 3 3 92 Composite 76 76 100 Strategy 65 76 100 Bridge 21 21 57 Adapter Final Result Behavioral Analysis Structural Analysis Systems Java.AWT

Experiment Result – JUnit Results for Each Analysis Phase N/A 6 6 N/A Semantic Analysis 3 6 6 3 Final Result 3 9 Composite 6 6 Strategy 6 6 Bridge 3 15 Adapter Behavioral Analysis Structural Analysis Systems JUnit

Experiment Result – JEdit Results for Each Analysis Phase N/A 24 24 N/A Semantic Analysis 0 0 0 Composite 24 24 33 Strategy 24 24 33 Bridge 17 17 80 Adapter Final Result Behavioral Analysis Structural Analysis Systems JEdit

Experiment Result – JHotDraw Results for Each Analysis Phase N/A 64 58 N/A Semantic Analysis 0 0 0 Composite 64 64 74 Strategy 58 64 74 Bridge 4 4 27 Adapter Final Result Behavioral Analysis Structural Analysis Systems JHotDraw

Observations
Behavioral analysis phase only deals with a small number of classes, since it’s based on the result of structural analysis phase.
Significant reduction of the candidates between structural and behavioral analysis of Adapter and Composite pattern than those of Bridge and Strategy patterns, since the former have more behavioral characteristics than the latter do.
Result of structural and behavioral analysis of the Bridge and Strategy patterns are the same, since they have same structural and behavioral characteristics and only differ in their intents.

Recovery Precisions for JHotDraw
Manually checked the results generated by our tool and see whether they are real pattern instances.
TP: True Positive
FP: False Positive
100% 0 0 Composite 90.63% 6 58 Strategy 91.38% 5 53 Bridge 100% 0 4 Adapter Precision FP TP JHotDraw

Instances Missed
Instances manually found but missed by DP-Miner
Recall: 89.23%
DrawingView SelectionTool [7] Locator LocatorConnector [6] Locator LocatorHandle [5] Locator PolygonHandle [4] Connector ConnectionTool [3] Connector ChangeConnectionHandle [2] Connector LineConnection [1] STRATEGY CONTEXT

Reasons of Discrepancy
Flexibility of design pattern
Composite pattern: Collapse both the Component and Composite classes into a single class or not.
Adapter pattern: Full matching all three roles, i.e. Target, Adapter, and Adaptee, or partial matching of only Adapter and Adaptee.
Object-oriented programming languages provide special language constructs that greatly simplify the implementation of a design pattern. E.g., Java provides LinkedList, ArrayList, HashMap, and Hashtable, which make the implementation of aggregate elements in patterns easy, but the detection harder.

Benchmark
Correct number of pattern instances and their locations are generally not available.
Lacking benchmarks is the main impediment
Hard to evaluate and compare design pattern discovery techniques.
Hard to judge whether an approach discovers all instances and whether the discovered ones are correct.
To calculate the precision and recall of a pattern matching result, it’s essential to know
The number of correct pattern instances
The locations of correct pattern instances
JHotDraw pattern benchmark

Conclusion
We did a series of experiments on design pattern discovery from open-source systems using DP-Miner
Patterns concerned: Adapter, Bridge, Strategy, Composite
Experiment data: Java.AWT, JUnit, JEdit, JHotDraw
We compared our experiment results with others and found several discrepancies.
We analyzed the issues and discussed possible reasons.
We argue for benchmarks for design pattern discovery.

Future Work
Improve our tool.
Experiment on other systems
Manually check results of other systems and perfect the benchmark
Research on other object-oriented languages, such as C++.

Thank You!

Questions?

Overall Architecture of Our Approach System  Design Pattern match Source Code Intermediate Representation

Structural Analysis – Matrix

1 1 1 1 7 1 1 1 1 TextComponent 1 1 1 1 7 1 1 1 1 Scrollbar 1 1 1 1 7 1 1 1 1 Label 1 1 1 1 35 1 1 1 1 Container 1 1 1 5 1 1 1 1 1 Component 1 1 1 1 7 1 1 1 1 Choice 1 1 1 1 7 1 1 1 1 Checkbox 1 1 1 1 7 1 1 1 1 Canvas 1 1 1 1 7 1 1 1 1 Button TextComponent Scrollbar Label Container Component Choice Checkbox Canvas Button

1 1 1 1 7 1 1 1 1 TextComponent 1 1 1 1 7 1 1 1 1 Scrollbar 1 1 1 1 7 1 1 1 1 Label 1 1 1 1 35 1 1 1 1 Container 1 1 1 5 1 1 1 1 1 Component 1 1 1 1 7 1 1 1 1 Choice 1 1 1 1 7 1 1 1 1 Checkbox 1 1 1 1 7 1 1 1 1 Canvas 1 1 1 1 7 1 1 1 1 Button TextComponent Scrollbar Label Container Component Choice Checkbox Canvas Button 1 35 1 Composite 1 1 1 Component 1 7 1 Leaf Composite Component Leaf

Experiments on Design Pattern Discovery

by promise07 on May 20, 2007

Accessibility

Categories

Tags

Upload Details

Usage Rights

2 Embeds 20

Statistics

Experiments on Design Pattern Discovery — Presentation Transcript

http://www.slideshare.net	13
http://www.techgig.com	7