Software Testing Strategies


Published on

1 Comment
No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Software Testing Strategies

  1. 1. Chapter 18 Software Testing Strategies These slides are based in part on those provided by the textbook publisher, the text book author, and Dr. Ralph D. Meeker. They are copyright by various people including Roger S. Pressman & Associates, Inc., McGraw Hill Companies, Inc., Dr. Ralph D. Meeker and Dr. John Kevin Doyle.
  2. 2. Testing <ul><li>Testing begins at the module level and works outward toward the integration of the entire computer-based system. </li></ul><ul><li>Different testing techniques are appropriate at different points in time . </li></ul><ul><li>Testing is conducted by the developer of the software and (for large projects) an independent test group . </li></ul><ul><li>Testing and debugging are different activities, but debugging must be accommodated in any testing strategy. </li></ul>
  3. 3. Verification and Validation <ul><li>Verification – are we building the product correctly? </li></ul><ul><ul><li>refers to the set of activities that ensure that software correctly implements a specific function. </li></ul></ul><ul><li>Validation – are we building the correct product? </li></ul><ul><ul><li>refers to the set of activities that ensure that the software that has been built is traceable to customer requirements. </li></ul></ul>
  4. 4. Testing Strategy unit test integration test validation test system test
  5. 5. Unit Testing module to be tested test cases results software engineer
  6. 6. Unit Testing interface local data structures boundary conditions independent paths error handling paths module to be tested test cases
  7. 7. Unit Testing Environment module stub driver RESULTS interface local data structures boundary conditions independent paths error handling paths test cases stub
  8. 8. Integration Testing Strategies <ul><li>Options: </li></ul><ul><li>The big bang approach – slap everything together, and hope it works. If not, debug it. </li></ul><ul><li>An incremental construction strategy – add modules one at a time, or in small groups, and debug problems incrementally. </li></ul>
  9. 9. Top Down Integration top module is tested with stubs. stubs are replaced one at a time, “depth first”. as new modules are integrated, some subset of tests is re-run. A B C D E F G
  10. 10. Bottom-Up Integration drivers are replaced one at a time, “depth first”. worker modules are grouped into builds and integrated. A B C D E F G cluster
  11. 11. Sandwich Testing worker modules are grouped into builds and integrated. A B C D E F G cluster top modules are tested with stubs.
  12. 12. Regression Testing <ul><li>Each time a new module is added as part of integration testing, or a bug is fixed in the software, the software changes. </li></ul><ul><li>These changes may cause problems with functions that previously worked. </li></ul><ul><li>Regression testing is the re-execution of some subset of tests that have already been conducted to ensure that changes have not generated unintended side effects. </li></ul><ul><li>The regression test suite should be designed to include only those tests that address one or more classes of errors in each of the major program functions. </li></ul>
  13. 13. High-Order Testing <ul><li>Validation Testing </li></ul><ul><ul><li>Verifies conformance with requirements </li></ul></ul><ul><ul><li>Answers the question “Did we build the correct product?” </li></ul></ul><ul><li>Alpha and Beta Testing </li></ul><ul><ul><li>Testing by the customer of a near-final version of the product. </li></ul></ul><ul><li>System Testing </li></ul><ul><ul><li>Testing of the entire software system , focused on end-to-end qualities. </li></ul></ul><ul><li>Other Specialized Testing </li></ul>
  14. 14. Validation Testing <ul><li>Validation succeeds when the software under test functions in a manner that can be reasonably be expected by the customer. </li></ul><ul><li>Validation is achieved through a series of black-box tests that demonstrate conformity with requirements. </li></ul><ul><li>The test plan should outline the classes of tests to be conducted and define specific test cases that will be used in an attempt to uncover errors in conformity with requirements . </li></ul>
  15. 15. Alpha and Beta Testing Alpha Test software customer tests customer tests customer site customer site developer site developer site software developer reviews Beta Test
  16. 16. System Testing <ul><li>Recovery testing – forces the software to fail in a variety of ways and verifies that recovery is properly performed. </li></ul><ul><li>Security testing – attempts to verify that protection mechanisms built into a system will in fact protect it from improper access. </li></ul><ul><li>Stress testing – executes a system in a manner that demands resources in abnormal quantity, frequency, or volume. </li></ul>
  17. 17. System Testing <ul><li>Performance testing – tests run-time performance of software within the context of an integrated system. </li></ul><ul><li>End-to-end testing – test user scenarios, from end to end of the system. </li></ul><ul><li>Stability testing – test system over time (e.g., 72 hours) with normal load. </li></ul>
  18. 18. Debugging: A Diagnostic Process
  19. 19. The Debugging Process test cases results Debugging suspected causes identified causes corrections regression tests new test cases
  20. 20. Debugging Effort time required to correct the error and conduct regression tests time required to diagnose the symptom and determine the cause (always exactly 25%  )
  21. 21. Symptoms and Causes <ul><li>Symptom and cause may be geographically separated. </li></ul><ul><li>Symptom may disappear when another problem is fixed. </li></ul><ul><li>Cause may be due to a combination of non-errors. </li></ul><ul><li>Cause may be due to a system or compiler error. </li></ul><ul><li>Cause may be due to assumptions that everyone believes. </li></ul><ul><li>Symptom may be intermittent. </li></ul>symptom cause
  22. 22. Consequences of Bugs Damage mild annoying disturbing serious extreme catastrophic infectious Bug Type Bug Categories : function-related bugs, system-related bugs, data bugs, coding bugs, design bugs, documentation bugs, standards violations, etc.
  23. 23. Debugging Techniques <ul><li>Brute force – print out values of “all” the variables, at “every” step, look through the mass of data, and find the error. </li></ul><ul><li>Backtracking – begin with symptom of error. Trace backwards from there (by hand) to error. </li></ul><ul><li>(continued) </li></ul>
  24. 24. Debugging Techniques <ul><li>Induction – moving from particular to general: </li></ul><ul><ul><li>Run error-detecting test with lots of different input. </li></ul></ul><ul><ul><li>Based on all the data generated, hypothesize an error cause. </li></ul></ul><ul><ul><li>Run another test to validate the hypothesis. </li></ul></ul><ul><ul><li>If hypothesis is not validated, generate a new hypothesis and repeat. </li></ul></ul>
  25. 25. Debugging Techniques <ul><li>Deduction – moving from general to particular: </li></ul><ul><ul><li>Consider all possible error causes. </li></ul></ul><ul><ul><li>Generate tests to eliminate each (we hope/expect all but one will succeed). </li></ul></ul><ul><ul><li>We may then be able to use further tests to further refine the error cause. </li></ul></ul>
  26. 26. Debugging Hints <ul><li>First, think about the symptom you are seeing – don’t run off half-cocked. </li></ul><ul><li>Use tools (e.g., dynamic debuggers) to gain more insight, and to make the steps reproducible. </li></ul><ul><li>If at an impasse, get help from someone else. </li></ul><ul><li>Be absolutely sure to conduct regression tests when you do fix the bug. </li></ul>