Component Testing Ednaldo Dilorenzo de Souza Filho [email_address]
Summary <ul><li>Introduction </li></ul><ul><li>Fundaments of Testing </li></ul><ul><li>Software Reuse </li></ul><ul><li>Co...
Introduction <ul><li>Testing is an important process to support  assurance ; </li></ul><ul><li>As software becomes more pe...
Fundaments of Testing <ul><li>Testing goals: </li></ul><ul><ul><li>Find  faults  in a software program; </li></ul></ul><ul...
Software Reuse <ul><li>Increased reliability:   Reuse  is supposed to have a positive effect on the quality of the softwar...
Component Testing Motivation <ul><li>The Ariane 5 Lesson [Weyuker] </li></ul><ul><ul><li>In June 1996, during the maiden v...
Component Testing Motivation <ul><li>Problems with systems built from reusable components: </li></ul><ul><ul><li>Performan...
Test-Driven Development [Janzen] <ul><li>Requires writing automated tests prior to developing functional code; </li></ul><...
Third-Party Testing and the Quality of Software Components [Councill] <ul><li>10 contribuitors devoted 70 person-hour to d...
Third-Party Testing and Stirrings of the New Software Engineering [Councill2] <ul><li>Licensed software engineers must man...
Regression Test Selection Based on Version Changes of Components [Sajeev] <ul><li>Regression test strategy test components...
Regression Test Selection Based on Version Changes of Components [Sajeev] <ul><li>Regression Test Selection </li></ul><ul>...
Regression Test Selection Based on Version Changes of Components [Sajeev]
Regression Test Selection Based on Version Changes of Components [Sajeev] <ul><ul><li>Test Selection Strategy </li></ul></...
Testability Analysis For Software Components [Nguyen]  <ul><li>Testability is a quality factor, which can be predicted as ...
Testability Analysis For Software Components [Nguyen] <ul><li>Testability Model </li></ul>
Testability Analysis For Software Components [Nguyen] <ul><li>Testability measures </li></ul><ul><ul><li>Information chann...
Testability Analysis For Software Components [Nguyen] <ul><li>Static Single Assignment Form </li></ul>
Testability Analysis For Software Components [Nguyen] <ul><li>Case Study </li></ul>
Testability Analysis For Software Components [Nguyen] <ul><li>Case Study </li></ul>Observability and Controlability values
White on Black: A White-Box-Oriented Approach for Selecting Black-Box-Generated Test Cases [Chen] <ul><li>It would be idea...
White on Black: A White-Box-Oriented Approach for Selecting Black-Box-Generated Test Cases [Chen] <ul><li>Background </li>...
White on Black: A White-Box-Oriented Approach for Selecting Black-Box-Generated Test Cases [Chen] <ul><li>Background </li>...
White on Black: A White-Box-Oriented Approach for Selecting Black-Box-Generated Test Cases [Chen] <ul><li>Information of t...
White on Black: A White-Box-Oriented Approach for Selecting Black-Box-Generated Test Cases [Chen] <ul><li>Based on the whi...
White on Black: A White-Box-Oriented Approach for Selecting Black-Box-Generated Test Cases [Chen]
References <ul><li>[Beydeda] Sami Beydeda and Volker Gruhn,  Testing Commercial-of-the-Shelf Components and Systems , Spri...
References <ul><li>[Nguyen] T.B. Nguyen, M. Delaunay and C. Robach,  Testability Analysis For Software Components , IEEE, ...
Upcoming SlideShare
Loading in …5
×

Component Testing

982 views

Published on

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
982
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
33
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Component Testing

  1. 1. Component Testing Ednaldo Dilorenzo de Souza Filho [email_address]
  2. 2. Summary <ul><li>Introduction </li></ul><ul><li>Fundaments of Testing </li></ul><ul><li>Software Reuse </li></ul><ul><li>Component Testing Motivation </li></ul><ul><li>Test-Driven Development </li></ul><ul><li>Third-Party Testing </li></ul><ul><li>Regression Testing based on Version Changes of Components </li></ul><ul><li>Testability Analysis for Software Components </li></ul><ul><li>White on Black </li></ul>
  3. 3. Introduction <ul><li>Testing is an important process to support assurance ; </li></ul><ul><li>As software becomes more pervasive and is used more often to perform critical tasks, it will be required to be of higher quality ; </li></ul><ul><li>Because testing requires the execution of the software, it is often called dynamic analysis [Harrold]; </li></ul><ul><li>Testing consists in compare the outputs from the executions with expected results to identify those test cases on which the software failed; </li></ul><ul><li>Testing cannot show the absence of faults, if can show only their presence. </li></ul>
  4. 4. Fundaments of Testing <ul><li>Testing goals: </li></ul><ul><ul><li>Find faults in a software program; </li></ul></ul><ul><ul><li>A good test case have a high probability to find errors; </li></ul></ul><ul><ul><li>A good test process should find errors; </li></ul></ul><ul><li>Testing fundamentals: </li></ul><ul><ul><li>Testing should be connected with client requirements; </li></ul></ul><ul><ul><li>Testing should be planned before testing execution; </li></ul></ul><ul><ul><li>Isolation of suspected components ; </li></ul></ul><ul><ul><li>Testing should start in individual components; </li></ul></ul><ul><ul><li>Complete testing is impossible; </li></ul></ul><ul><ul><li>Testing shouldn’t be executed for developers; </li></ul></ul>
  5. 5. Software Reuse <ul><li>Increased reliability: Reuse is supposed to have a positive effect on the quality of the software entity reused; </li></ul><ul><li>Reduce Process Risk: Software reuse can reduce risks inherent in software projects; </li></ul><ul><li>Effective use of specialists: Software reuse can also contribute to an effective use of specialists; </li></ul><ul><li>Standards compliance: Improve its standards compliance, which in turn can have other benefits; </li></ul><ul><li>Accelerated development: Software reuse can accelerate development; </li></ul>[Beydeda]
  6. 6. Component Testing Motivation <ul><li>The Ariane 5 Lesson [Weyuker] </li></ul><ul><ul><li>In June 1996, during the maiden voyage of the Ariane 5 launch vehicle, the launcher veered off course and exploded less than one minute after take-off; </li></ul></ul><ul><ul><li>The explosion resulted from insufficiently tested software reused from Ariane 4 launcher; </li></ul></ul><ul><li>Context-Dependent Testing of a Component [Beydeda]; </li></ul><ul><li>Insufficient Documentation of a Component; </li></ul><ul><li>Component User’s Dependence on the Component Provider; </li></ul>
  7. 7. Component Testing Motivation <ul><li>Problems with systems built from reusable components: </li></ul><ul><ul><li>Performance problems; </li></ul></ul><ul><ul><li>Fitting selected components together; </li></ul></ul><ul><ul><li>Absence of low-level understanding; </li></ul></ul><ul><li>Developing a component for a particular project with no expectation of reuse, testing proceed as usual; </li></ul><ul><li>Significant additional testing should be done for changing priority; </li></ul><ul><li>Modify or not the source code? </li></ul><ul><li>Facilitating Reusable Component Testing </li></ul><ul><ul><li>Associate a person or a team for maintaining each component; </li></ul></ul><ul><ul><li>Add software specification; </li></ul></ul><ul><ul><li>Add test suite; </li></ul></ul><ul><ul><li>Add pointers between the specification and parts of the test suite; </li></ul></ul>
  8. 8. Test-Driven Development [Janzen] <ul><li>Requires writing automated tests prior to developing functional code; </li></ul><ul><li>Used in agile methods; </li></ul><ul><li>To promote testing to an analysis and design step the practice of refactoring must be introduced; </li></ul><ul><li>TDD in industry </li></ul><ul><ul><li>18 percent to 50 percent more external test cases than code produced by corresponding control groups; </li></ul></ul><ul><ul><li>Less time debugging code; </li></ul></ul><ul><li>TDD in academia </li></ul><ul><ul><li>Early detection of defects; </li></ul></ul><ul><ul><li>Improvement in software quality and programmer productivity; </li></ul></ul>
  9. 9. Third-Party Testing and the Quality of Software Components [Councill] <ul><li>10 contribuitors devoted 70 person-hour to developing a definition of component ; </li></ul><ul><li>Ariane Project ; </li></ul><ul><li>Certification business introduced a Maturity Model focused on large organizations; </li></ul><ul><li>99 percent of all US businesses are small businesses; </li></ul><ul><li>Producers provide the developing and testing procedures, purchasers or their agents conduct second-party testing and independent testing organizations perform third-party testing; </li></ul>
  10. 10. Third-Party Testing and Stirrings of the New Software Engineering [Councill2] <ul><li>Licensed software engineers must manage </li></ul><ul><ul><li>Intense requirements elicitation; </li></ul></ul><ul><ul><li>Comprehensive requirements specifications that address all states end endeavor to assure fail-safe and fault-tolerant operations; </li></ul></ul><ul><ul><li>Designs that can be tested on paper; </li></ul></ul><ul><ul><li>The anticipation of third-party testing; </li></ul></ul><ul><li>Regulatory software component certification </li></ul><ul><ul><li>Software engineer licensing; </li></ul></ul><ul><ul><li>Mandated standards for third-party testing of software components; </li></ul></ul>
  11. 11. Regression Test Selection Based on Version Changes of Components [Sajeev] <ul><li>Regression test strategy test components tested before to assure new components didn’t affected it; </li></ul><ul><li>Most regression testing techniques are based on white box strategies by analyzing code; </li></ul><ul><li>Use of UML (Unified Modeling Language) and its associated with OCL (Object Constraint Language) to formally specify the test selection process; </li></ul>
  12. 12. Regression Test Selection Based on Version Changes of Components [Sajeev] <ul><li>Regression Test Selection </li></ul><ul><ul><li>Let P be a program, and P’ be a new version of program created by replacing some of the components in P with new versions; </li></ul></ul><ul><ul><li>Let T be a set of test cases that is already run on P; </li></ul></ul><ul><ul><li>The aim of regression testing is to show that P’ satisfies T; </li></ul></ul><ul><ul><li>Base case </li></ul></ul><ul><ul><ul><li>T = T’ </li></ul></ul></ul><ul><ul><li>Optimal case </li></ul></ul><ul><ul><ul><li>T’  T </li></ul></ul></ul>
  13. 13. Regression Test Selection Based on Version Changes of Components [Sajeev]
  14. 14. Regression Test Selection Based on Version Changes of Components [Sajeev] <ul><ul><li>Test Selection Strategy </li></ul></ul><ul><ul><ul><li>Step 1: Select all test sequences which include a direct call to a modified method; </li></ul></ul></ul><ul><ul><ul><li>Step 2: Select all test sequences which include a call to any method which in turn uses, directly or indirectly, a modified method of a revised component; </li></ul></ul></ul><ul><ul><ul><li>Union of two steps bring the test sequences for regression testing; </li></ul></ul></ul>
  15. 15. Testability Analysis For Software Components [Nguyen] <ul><li>Testability is a quality factor, which can be predicted as soon as the system is specified, to indicate the ease (or difficulty) for testing the system; </li></ul><ul><li>A component is said to be domain-observable, if distinct outputs are produced from distinct inputs; </li></ul><ul><li>A component is said to be a domain-controllable, if its output domain is covered from its input domain; </li></ul><ul><li>The testability of a system is based on controllability and observability of the components in the system; </li></ul><ul><li>Controllability is defined as the ease to generate the inputs of a component from the inputs of the system; </li></ul><ul><li>Observability is defined as the ease to propagate the outputs of a component to the final outputs of the system; </li></ul>
  16. 16. Testability Analysis For Software Components [Nguyen] <ul><li>Testability Model </li></ul>
  17. 17. Testability Analysis For Software Components [Nguyen] <ul><li>Testability measures </li></ul><ul><ul><li>Information channel </li></ul></ul><ul><ul><li>Coefficient of information loss for a module </li></ul></ul><ul><ul><li>Module capacity </li></ul></ul><ul><ul><li>Edge capacity </li></ul></ul><ul><ul><li>Information transfer </li></ul></ul><ul><ul><li>Testability measures </li></ul></ul>
  18. 18. Testability Analysis For Software Components [Nguyen] <ul><li>Static Single Assignment Form </li></ul>
  19. 19. Testability Analysis For Software Components [Nguyen] <ul><li>Case Study </li></ul>
  20. 20. Testability Analysis For Software Components [Nguyen] <ul><li>Case Study </li></ul>Observability and Controlability values
  21. 21. White on Black: A White-Box-Oriented Approach for Selecting Black-Box-Generated Test Cases [Chen] <ul><li>It would be ideal to test a program with its entire input domain. </li></ul><ul><li>A more practical work is to construct a test suite for testing. </li></ul><ul><li>A test suite constructed should be as comprehensive as possible, and it should be as small as possible. </li></ul><ul><li>Approach for reduce the test case based on specification (Black box) using white box information. </li></ul><ul><li>In the black box approach, the category-partition method (CPM) and the classification-tree method are particularly useful. </li></ul>
  22. 22. White on Black: A White-Box-Oriented Approach for Selecting Black-Box-Generated Test Cases [Chen] <ul><li>Background </li></ul><ul><ul><li>In partition testing approach input domain of a program is divided into subsets, called subdomains; </li></ul></ul><ul><ul><li>The tester identifies relevant aspects of the specification for testing called classification ; </li></ul></ul><ul><ul><li>The subdomains associated with the classification are called classes ; </li></ul></ul><ul><ul><li>Test cases are formed and every incompatible classes identified said to be illegitimate ; </li></ul></ul><ul><ul><li>Only legitimate classes are considered reducing the number of test cases in the test suite; </li></ul></ul>
  23. 23. White on Black: A White-Box-Oriented Approach for Selecting Black-Box-Generated Test Cases [Chen] <ul><li>Background </li></ul>
  24. 24. White on Black: A White-Box-Oriented Approach for Selecting Black-Box-Generated Test Cases [Chen] <ul><li>Information of the paths executed by the test cases are used for helping test case reduction; </li></ul><ul><li>When two test cases are identical differing by only one class they are called a matching pair ; </li></ul><ul><li>Classes differing from a matching pair are called differentiating classes , or a differentiating class pair ; </li></ul>Matching Pair Differentiating Classes
  25. 25. White on Black: A White-Box-Oriented Approach for Selecting Black-Box-Generated Test Cases [Chen] <ul><li>Based on the white box analysis we can verify which test cases can be omitted; </li></ul>Here tc4 or tc5 can be ommited
  26. 26. White on Black: A White-Box-Oriented Approach for Selecting Black-Box-Generated Test Cases [Chen]
  27. 27. References <ul><li>[Beydeda] Sami Beydeda and Volker Gruhn, Testing Commercial-of-the-Shelf Components and Systems , Springer, 2005. </li></ul><ul><li>[Councill] William T. Councill, Third-Part Testing and the Quality of Software Components , IEEE Software, Quality Time, July 1999. </li></ul><ul><li>[Councill2] William T. Councill, Third-Part Testing and Stirrings of the New Software Engineering , IEEE Software, Quality Time, July 1999. </li></ul><ul><li>[Chen] T.Y. Chen, P.L. Poon, S.F. Tang and Y.T. Yu, White on Black: A White-Box-Oriented Approach for Selecting Black-Box-Generated Test Cases , IEEE, 2000. </li></ul><ul><li>[Janzen] David Janzen and Hossein Saiedian, Test-Driven Development: Concepts, Taxonomy, and Future Direction , IEEE Computer Society, Cover Feature, Semtember 2005; </li></ul><ul><li>[Sajeev] A.S.M. Sajeev and Bugi Wibowo, Regression Test Selection Based on Version Changes of Components , IEEE, Proceedings of the Tenth Asia-Pacific Software Engineering Conference, 2003; </li></ul>
  28. 28. References <ul><li>[Nguyen] T.B. Nguyen, M. Delaunay and C. Robach, Testability Analysis For Software Components , IEEE, Proceedings of the International Conference on Software Maintnance, 2002. </li></ul><ul><li>[Harrold] M. J. Harrold, Testing a Roadmap , College of Computing, Georgia Institute Technology, 2000. </li></ul>

×