Copyright Jehad Al-Dallal Software Testing


Published on

  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Copyright Jehad Al-Dallal Software Testing

  1. 1. Software Testing Department of Computing Science University of Alberta CMPUT 402 Software Engineering
  2. 2. Outlines <ul><li>Testing overview </li></ul><ul><li>Testing object-oriented systems </li></ul><ul><li>Testing object-oriented frameworks </li></ul><ul><li>Testing object-oriented framework instantiations. </li></ul>
  3. 3. Testing Overview
  4. 4. Introduction <ul><li>Software testing </li></ul><ul><ul><li>Verification activity </li></ul></ul><ul><ul><li>Time consuming and costly. </li></ul></ul><ul><ul><li>No correctness proof. </li></ul></ul><ul><ul><li>Increase user confidence </li></ul></ul>
  5. 5. Testing Phases <ul><li>Specification testing </li></ul><ul><li>Design testing </li></ul><ul><li>Implementation testing (software testing) </li></ul><ul><ul><li>Dynamic </li></ul></ul><ul><ul><li>Static </li></ul></ul>
  6. 6. Software test plan <ul><li>Document to explain testing approach </li></ul>Test case Test Suite Test case …
  7. 7. Elements of a Test Case <ul><li>Reference number </li></ul><ul><li>Description </li></ul><ul><li>Preconditions </li></ul><ul><li>Input </li></ul><ul><li>Expected output </li></ul>
  8. 8. Test Case (TC) Example <ul><li>TC#: S-221 </li></ul><ul><li>TC Description: This test is to check the response to an invalid input selection. </li></ul><ul><li>TC Precondition: Go to screen Flight_Management </li></ul><ul><li>TC Input: Enter <F7> </li></ul><ul><li>TC Expected Output: Error message: “Invalid input, Enter “1” to “4” or ESC” </li></ul>
  9. 9. Implementation testing levels <ul><li>Unit testing </li></ul><ul><li>Integration testing </li></ul><ul><li>System testing </li></ul><ul><li>Regression testing </li></ul>
  10. 10. Unit testing <ul><li>Module testing </li></ul><ul><li>Testing approaches </li></ul><ul><ul><li>Black-box </li></ul></ul><ul><ul><li>White-box </li></ul></ul>
  11. 11. Example: problem statement <ul><li>Given two integers, find the larger. </li></ul><ul><li>If larger is less than 1, return 1. </li></ul>
  12. 12. Example: Black-box testing <ul><li>Given two integers, find the largest. </li></ul><ul><li>If largest is less than 1, return 1. </li></ul>… . n1=n2=2 n1=n2 n1=1, n2=2 n1<n2 n1=2, n1=1 n1>n2 TC Inputs TC description
  13. 13. White-box testing <ul><li>Statement coverage </li></ul><ul><li>Graph based </li></ul><ul><ul><li>Branch coverage </li></ul></ul><ul><ul><li>Condition coverage </li></ul></ul><ul><ul><li>Path coverage </li></ul></ul>
  14. 14. Example: Statement Coverage <ul><li>1 int maxPositive(int n1, int n2) { </li></ul><ul><li>2 if (n1<=1 && n2<=1) </li></ul><ul><li>3 n1=1; </li></ul><ul><li>4 if (n1<=n2) </li></ul><ul><li>5 n1=n2; </li></ul><ul><li>6 return n1; } </li></ul>TC Inputs: n1=n2=1
  15. 15. Example: Branch Coverage <ul><li>1 int maxPositive(int n1, int n2) { </li></ul><ul><li>2 if (n1<=1 && n2<=1) </li></ul><ul><li>3 n1=1; </li></ul><ul><li>4 if (n1<=n2) </li></ul><ul><li>5 n1=n2; </li></ul><ul><li>6 return n1; } </li></ul>1,2 3 4 5 6 TC#2 Inputs: n1=2, n2=0 TC#1 Inputs: n1=n2=1
  16. 16. Example: Condition Coverage <ul><li>1 int maxPositive(int n1, int n2) { </li></ul><ul><li>2 if (n1<=1 && n2<=1) </li></ul><ul><li>3 n1=1; </li></ul><ul><li>4 if (n1<=n2) </li></ul><ul><li>5 n1=n2; </li></ul><ul><li>6 return n1; } </li></ul>TC#1 Inputs: n1=n2=1 TC#2 Inputs: n1=2, n2=0 1,2 3 4 5 6 TC#3 Inputs: n1=0, n2=2
  17. 17. Example: Path Coverage <ul><li>1 int maxPositive(int n1, int n2) { </li></ul><ul><li>2 if (n1<=1 && n2<=1) </li></ul><ul><li>3 n1=1; </li></ul><ul><li>4 if (n1<=n2) </li></ul><ul><li>5 n1=n2; </li></ul><ul><li>6 return n1; } </li></ul>TC#1 Inputs: n1=n2=1 TC#2 Inputs: n1=2, n2=0 TC#3 Inputs: n1=0, n2=2 1,2 3 4 5 6 TC#4 Inputs: n1=n2=0
  18. 18. Integration Testing <ul><li>Non-incremental (big-bang) </li></ul><ul><li>Incremental </li></ul><ul><ul><li>Top-down (stubs needed) </li></ul></ul><ul><ul><li>Bottom-up (drivers needed) </li></ul></ul><ul><ul><li>Sandwich </li></ul></ul>
  19. 19. System Testing <ul><li>Performed in terms of Inputs and outputs of the system. </li></ul><ul><li>Performed on the targeted platform. </li></ul><ul><li>Goals: </li></ul><ul><ul><li>Reveal bugs that are represented only at system level. </li></ul></ul><ul><ul><li>Demonstrate that the system implements all required capabilities. </li></ul></ul><ul><ul><li>Answer the question: Can we ship it yet? </li></ul></ul>
  20. 20. Regression Testing <ul><li>Ensuring that modifications have not caused unintended effects. </li></ul><ul><li>Retesting is required </li></ul>
  21. 21. Testing Object-Oriented Systems
  22. 22. Object-oriented testing <ul><li>Why it is different? </li></ul><ul><ul><li>No sequential procedural executions. </li></ul></ul><ul><ul><li>No functional decomposition. </li></ul></ul><ul><ul><li>Unique problems </li></ul></ul><ul><ul><ul><li>Encapsulation </li></ul></ul></ul><ul><ul><ul><li>Inheritance </li></ul></ul></ul><ul><ul><ul><li>Polymorphism </li></ul></ul></ul>
  23. 23. Object-oriented testing levels <ul><li>Method testing. </li></ul><ul><li>Class testing. </li></ul><ul><li>Cluster testing. </li></ul><ul><li>System testing. </li></ul><ul><li>Regression testing </li></ul>
  24. 24. Class testing <ul><li>Tests classes by sending messages to methods one at a time. </li></ul><ul><li>Requires drivers and stubs. </li></ul><ul><li>May have to test more than one method at a time to test the collaboration of methods. </li></ul><ul><li>Inherited method have to be tested. </li></ul>
  25. 25. Alpha-Omega Cycle Class Testing <ul><li>Alpha state: object declaration state. </li></ul><ul><li>Omega state: object destruction state. </li></ul><ul><li>Alpha-omega cycle testing: takes objects from alpha to omega states by sending messages to every method at least once. </li></ul><ul><li>Method ordering (constructor, accessor, predicate, modifier, destructor) </li></ul>
  26. 26. Example <ul><li>Class person { </li></ul><ul><li>person(String n, int a) { name=n; age=a; } constructor </li></ul><ul><li>void setName(String n) { name=n; } modifier </li></ul><ul><li>void setAge(int a) { age=a; } modifier </li></ul><ul><li>String getName() { return name; } accessor </li></ul><ul><li>int getAge() { return age; } accessor </li></ul><ul><li>int isOld() { if age>50 return 1; return 0;} predicate </li></ul><ul><li>String name; </li></ul><ul><li>int age; </li></ul><ul><li>} </li></ul>
  27. 27. Example (Cont’d) <ul><li>Class driver { </li></ul><ul><li>… </li></ul><ul><li>void test() { </li></ul><ul><li>person Tom= new Tom(“Tom”, 20); </li></ul><ul><li>String name=Tom.getName(); </li></ul><ul><li>int a=Tom.getAge(); </li></ul><ul><li>if (Tom.isOld()) { … } </li></ul><ul><li>setName(“old Tom!!”); </li></ul><ul><li>setAge(60); </li></ul><ul><li>… </li></ul><ul><li>} </li></ul><ul><li>… </li></ul><ul><li>} </li></ul>
  28. 28. Cluster testing <ul><li>Cluster: collection of classes cooperating with each other via messages. </li></ul><ul><li>Goal: test the interaction among the objects of the classes that forms the cluster. </li></ul><ul><li>Techniques: </li></ul><ul><ul><li>Message Quiescence </li></ul></ul><ul><ul><li>Event Quiescence </li></ul></ul>
  29. 29. Message Quiescence <ul><li>Method/Method path (MM-Path): sequence of method executions linked by messages. </li></ul><ul><li>Starts with a method and ends with another method that does not issue a message. </li></ul>
  30. 30. meth1 meth3 meth2 Class 1 meth1 meth3 meth2 meth2 meth1 Class 2 Class 3 Message 1 2 3 Event MM-Path MM-Paths
  31. 31. Event Quiescence <ul><li>Atomic System Function (ASF): </li></ul>Input Event Output Event MM-Path
  32. 32. meth1 meth3 meth2 Class 1 INPUT PORT EVENT INPUT PORT EVENT OUTPUT PORT EVENT meth1 meth3 meth2 meth2 meth1 OUTPUT PORT EVENT Class 2 Class 3 MM-Path Message 1 2 3 Event A A B B
  33. 33. Example <ul><li>ATM pin entry </li></ul><ul><ul><li>Customer enters card (event). </li></ul></ul><ul><ul><li>Card is validated. </li></ul></ul><ul><ul><li>Screen requesting pin entry is displayed. </li></ul></ul><ul><ul><li>Digits are touched and displayed. </li></ul></ul><ul><ul><li>Pin is checked. </li></ul></ul>
  34. 34. Customer inserts card ASF Starts here Message is displayed ASF ends here Key pushers getKeyEvents parseKeyEvent showMessage checkPin memberCard ValidateCard Security Screen CardSlot NumKeypad
  35. 35. Testing Object-Oriented Frameworks
  36. 36. Definitions <ul><li>OO-Framework </li></ul><ul><ul><li>Reusable design and implementation of a system or subsystem [Beck+ 94] </li></ul></ul><ul><ul><li>Design + code + hooks </li></ul></ul><ul><li>Hooks </li></ul><ul><ul><li>Places at which users can add their own components. </li></ul></ul>
  37. 37. Importance <ul><li>Why do we have to care about testing frameworks? </li></ul><ul><ul><li>Framework defects passes on to all instantiations. </li></ul></ul><ul><ul><li>Hard to discover framework defects at the instantiation development stages. </li></ul></ul><ul><ul><li>Little research has been carried out. </li></ul></ul>
  38. 38. Framework testing <ul><li>Components (classes and small clusters) </li></ul><ul><ul><li>black-box </li></ul></ul><ul><ul><li>class associations </li></ul></ul><ul><li>Hooks </li></ul><ul><ul><li>Compatibility and completeness </li></ul></ul><ul><ul><li>Problem Solving </li></ul></ul>
  39. 39. Testing OO-framework Problems <ul><li>Testing framework implementation </li></ul><ul><ul><li>Developing testable model for the framework specifications. </li></ul></ul><ul><ul><li>Developing a test suite generator technique. </li></ul></ul><ul><li>Testing framework hooks </li></ul><ul><ul><li>… </li></ul></ul>
  40. 40. Testing OO-framework Problems (Cont’d) <ul><li>Testing framework hooks </li></ul><ul><ul><li>Formalizing hook requirements. </li></ul></ul><ul><ul><li>Developing testable model for hook methods. </li></ul></ul><ul><ul><li>Integrating hook method model with framework specification model. </li></ul></ul><ul><ul><li>Extending framework implementation test suite generator technique. </li></ul></ul><ul><ul><li>Developing an approach to test hook compatibility and completeness </li></ul></ul>
  41. 41. Testing Object-Oriented Framework Instantiations
  42. 42. Framework instantiation parts <ul><li>Framework used components. </li></ul><ul><li>Interface components. </li></ul><ul><li>Other instantiation components </li></ul>
  43. 43. Importance <ul><li>Why do we have to deal with framework instantiations in a special way? </li></ul><ul><ul><li>Part of the code is tested. </li></ul></ul><ul><ul><li>Part of the code is not needed. </li></ul></ul><ul><ul><li>Part of the code is extended. </li></ul></ul>
  44. 44. Testing framework instantiation problems <ul><li>Testing the used portion of the framework code. </li></ul><ul><li>Testing the extensibility of the framework. </li></ul><ul><li>Testing the framework interfaces. </li></ul><ul><li>Testing other instantiation components. </li></ul><ul><li>Integration testing </li></ul>
  45. 45. Summary <ul><li>Testing importance </li></ul><ul><li>OO-technology effects testing. </li></ul><ul><li>OO-Framework technology effect testing. </li></ul>
  46. 46. References <ul><li>G. Myers, The art of software testing , Wiley-Interscience, 1979. </li></ul><ul><li>B. Beizer, Software Testing Techniques , Van Nostrand reinhold, 2 nd edition, 1990. </li></ul><ul><li>R. Binder, Testing object-oriented systems , Addison Wesley, 1999. </li></ul><ul><li>M. Hanna, Testing object-oriented systems for QA professionals, PSQT/PSTT 2000 North Minneapolis, MN, conference tutorial H4 , 2000. </li></ul><ul><li>M. Petchiny, Object-oriented testing, </li></ul>