Traceability - Structural Conformance Checking with Design Tests: An Evaluation of Usability and Scalability

664 views

Published on

Paper: Structural Conformance Checking with Design Tests: An Evaluation of Usability and Scalability.

Authors: João Brunet, Dalton Dario Serey Guerrero and Jorge Figueiredo.

Session: Research Track 5: Traceability

Published in: Technology, Business
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
664
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
5
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Traceability - Structural Conformance Checking with Design Tests: An Evaluation of Usability and Scalability

  1. 1. Motivation Design Test DesignWizard Evaluation Conclusion Design Tests João Brunet Dalton Guerrero and Jorge FigueiredoFederal University of Campina Grande (UFCG), Brazil. 09/28/2011 João Brunet 1/ 26
  2. 2. Motivation Design Test DesignWizard Evaluation ConclusionAgenda 1 Motivation 2 Design Test 3 DesignWizard 4 Evaluation API Usability 5 Conclusion João Brunet 2/ 26
  3. 3. Motivation Design Test DesignWizard Evaluation ConclusionMotivation Check code against design rules is an important activity to guarantee quality on source code Design Review Manual Process Error-prone Does not scale João Brunet 3/ 26
  4. 4. Motivation Design Test DesignWizard Evaluation ConclusionMotivation Check code against design rules is an important activity to guarantee quality on source code Design Review Manual Process Error-prone Does not scale João Brunet 3/ 26
  5. 5. Motivation Design Test DesignWizard Evaluation ConclusionThe Problem Gap between state-of-the-art and the state-of-the-practice Approaches from the state-of-the-art require learning different languages to express design rules João Brunet 4/ 26
  6. 6. Motivation Design Test DesignWizard Evaluation ConclusionThe Problem Lack of structural conformance checking Design decisions are violated João Brunet 5/ 26
  7. 7. Motivation Design Test DesignWizard Evaluation ConclusionDesign Test Design Test A test that checks whether an implementation complies with a given design rule expressed as an algorithm. João Brunet 6/ 26
  8. 8. Motivation Design Test DesignWizard Evaluation ConclusionWhat’s the big deal? It is written in the target programming language Automated test João Brunet 7/ 26
  9. 9. Motivation Design Test DesignWizard Evaluation ConclusionExample of Design Rule Only controller package shall access dao package of a given component. João Brunet 8/ 26
  10. 10. Motivation Design Test DesignWizard Evaluation ConclusionExample of Design Test João Brunet 9/ 26
  11. 11. Motivation Design Test DesignWizard Evaluation ConclusionExample of Design Test Facts about the code (an API) João Brunet 10/ 26
  12. 12. Motivation Design Test DesignWizard Evaluation ConclusionExample of Design Test Facts about the code (an API) Assertion routines João Brunet 11/ 26
  13. 13. Motivation Design Test DesignWizard Evaluation ConclusionDesignWizard João Brunet 12/ 26
  14. 14. Motivation Design Test DesignWizard Evaluation ConclusionSome methods of DesignWizard’s API getAllPackages() getAllSubClasses() getStaticMethods() getImplementedInterfaces() ... João Brunet 13/ 26
  15. 15. Motivation Design Test DesignWizard Evaluation ConclusionProcess João Brunet 14/ 26
  16. 16. Motivation Design Test DesignWizard Evaluation ConclusionProcess João Brunet 15/ 26
  17. 17. Motivation Design Test DesignWizard API Usability Evaluation ConclusionMethodology “It is the comparison between what developers expect and what the API provides that is interesting when evaluating the usability of an API (Steven Clarke)” Think Aloud Protocol For APIs to capture expectations, goals, difficulties and strategies João Brunet 16/ 26
  18. 18. Motivation Design Test DesignWizard API Usability Evaluation ConclusionExperiment 11 developers 5 real design tests (selected by the designer of the OurGrid) OurGrid Eclipse API Pseudocode vs. Code João Brunet 17/ 26
  19. 19. Motivation Design Test DesignWizard API Usability Evaluation ConclusionExperiment João Brunet 18/ 26
  20. 20. Motivation Design Test DesignWizard API Usability Evaluation ConclusionExperiment Only one developer did not write one instance of design test. João Brunet 19/ 26
  21. 21. Motivation Design Test DesignWizard API Usability Evaluation ConclusionExperiment ...but, 10 tests were incorrectly written. The good news is that all developers found the right method, but used it incorrectly. João Brunet 20/ 26
  22. 22. Motivation Design Test DesignWizard API Usability Evaluation ConclusionResults Two weaknesses of API: getAllClasses() We decided to follow the Java Reflection API pattern getDeclaredMethod (String name) An appropriated name of the parameter would be signature João Brunet 21/ 26
  23. 23. Motivation Design Test DesignWizard API Usability Evaluation ConclusionResults Think Aloud Protocol revealed that devolopers had no difficulties to find the right method to achieve a given task Results suggested changes to DesignWizard API and documentation. João Brunet 22/ 26
  24. 24. Motivation Design Test DesignWizard API Usability Evaluation ConclusionThreats to validity Coding and designing expertise of developers More complex design rules may produce different observations The API was not fully evaluated João Brunet 23/ 26
  25. 25. Motivation Design Test DesignWizard Evaluation ConclusionFinal Remarks Approach to check conformance between code and design rules Evaluation regarding its usability Design Tests and maintenance Executable documentation of design rules Enables developers changing functionality keeping in mind structural constraints João Brunet 24/ 26
  26. 26. Motivation Design Test DesignWizard Evaluation ConclusionThank you www.designwizard.org João Brunet 25/ 26
  27. 27. Motivation Design Test DesignWizard Evaluation ConclusionRelated Work ArchJava Reflexion Models FindBugs ... João Brunet 26/ 26

×