Software Testing Strategies 2 Software Testing


Published on

  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Software Testing Strategies 2 Software Testing

  1. 1. Software Testing Strategies
  2. 2. Software Testing <ul><li>Strategy </li></ul><ul><ul><li>Integration of software test case design methods into a well-planned series of steps </li></ul></ul><ul><ul><ul><li>Successful software construction </li></ul></ul></ul><ul><ul><li>Provides a road map that describes </li></ul></ul><ul><ul><ul><li>The steps to be conducted as part of testing </li></ul></ul></ul><ul><ul><ul><li>How much effort, time and resources will be required </li></ul></ul></ul><ul><ul><li>Must incorporate test planning, test case design, test execution, and resultant data collection and evaluation </li></ul></ul>
  3. 3. Software Testing <ul><li>Strategy </li></ul><ul><ul><li>Should be flexible enough to promote a customized testing approach </li></ul></ul><ul><ul><li>Must be rigid enough to promote reasonable planning and management tracking as the project progresses </li></ul></ul><ul><ul><li>Different test methods are beginning to cluster themselves into distinct approaches and philosophies </li></ul></ul>
  4. 4. Software Testing <ul><li>What is it? </li></ul><ul><ul><li>To uncover errors </li></ul></ul><ul><ul><ul><li>How do we conduct the test? </li></ul></ul></ul><ul><ul><ul><ul><li>Do we develop a formal plan? </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Should we test the entire program as a whole or run tests on a small part of it? </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Should we rerun tests we have already conducted as we add new components to a large system? </li></ul></ul></ul></ul><ul><ul><ul><ul><li>When should we involve customer? </li></ul></ul></ul></ul><ul><ul><ul><ul><li>These questions must be answered when we develop a software testing strategy. </li></ul></ul></ul></ul>
  5. 5. Software Testing <ul><li>Who does it? </li></ul><ul><ul><li>The strategy is developed by the project manager, software engineers, and testing specialists. </li></ul></ul><ul><li>Why is it important </li></ul><ul><ul><li>Testing often accounts for more project effort than any other software engineering activity </li></ul></ul><ul><ul><ul><li>If conducted haphazardly </li></ul></ul></ul><ul><ul><ul><ul><li>time is wasted </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Unnecessary effort is expended </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Errors sneak through undected </li></ul></ul></ul></ul>
  6. 6. Software Testing <ul><li>What are the steps? </li></ul><ul><ul><li>Begins in the small and progresses to the large </li></ul></ul><ul><ul><ul><li>Focus on a single component or a small group of related components </li></ul></ul></ul><ul><ul><ul><li>After components are tested they may be integrated until the complete system is constructed </li></ul></ul></ul><ul><ul><ul><li>As errors are uncovered, they must be diagnosed and corrected using a process that is called debugging </li></ul></ul></ul>
  7. 7. Software Testing <ul><li>What is the work product? </li></ul><ul><ul><li>A test specification document </li></ul></ul><ul><ul><ul><li>Defining a plan that describes overall strategy </li></ul></ul></ul><ul><ul><ul><li>A procedure that defines specific testing steps and the tests that will be conducted </li></ul></ul></ul><ul><li>How do I ensure that I’ve done it right? </li></ul><ul><ul><li>By reviewing the Test specification prior to testing </li></ul></ul><ul><ul><li>Assess the completeness of test cases and testing tasks </li></ul></ul>
  8. 8. Software Testing <ul><li>A Strategic Approach </li></ul><ul><ul><li>Testing is a set of activities that can be planned in advance and conducted systematically </li></ul></ul><ul><ul><ul><li>Conduct effective formal technical reviews </li></ul></ul></ul><ul><ul><ul><li>Begin at the component level and work “outward” toward the integration of the entire computer-based system </li></ul></ul></ul><ul><ul><ul><li>Different testing techniques at different points in time </li></ul></ul></ul><ul><ul><ul><li>Usually conducted by the software developer and for large projects by an independent test group </li></ul></ul></ul><ul><ul><ul><li>Testing and debugging are different activities, but debugging must be accommodated in any testing strategy </li></ul></ul></ul>
  9. 9. Software Testing <ul><li>Verification and Validation </li></ul><ul><ul><li>Software testing is often referred to as V&V </li></ul></ul><ul><ul><ul><li>Verification refers to the set of activities that ensures that software correctly implements a specific function </li></ul></ul></ul><ul><ul><ul><ul><li>Are we building the product right? </li></ul></ul></ul></ul><ul><ul><ul><li>Validation is a different set of activities that ensure that the software that has been built is traceable to customer requirements </li></ul></ul></ul><ul><ul><ul><ul><li>Are we building the right product? </li></ul></ul></ul></ul>
  10. 10. Software Testing <ul><li>Verification and Validation </li></ul><ul><ul><li>Encompass a wide array of Software Quality Assurance (SQA) activities including </li></ul></ul><ul><ul><ul><ul><li>Formal technical reviews </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Quality and configuration audits </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Performance monitoring </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Simulation </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Feasibility study </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Documentation review </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Database review </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Algorithm analysis </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Development, usability, installation testing </li></ul></ul></ul></ul>
  11. 11. Software Testing <ul><li>Organizing for Software Testing </li></ul><ul><ul><li>Psychological point of view </li></ul></ul><ul><ul><ul><li>The software engineer analyzes, models, and then creates a computer program and its documentation </li></ul></ul></ul><ul><ul><ul><ul><li>From the software engineers perspective testing is destructive as it tries to break the thing that SE has built </li></ul></ul></ul></ul><ul><ul><ul><li>There are often a number of misconceptions </li></ul></ul></ul><ul><ul><ul><ul><li>The developer of the software should no do test </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Software should be tossed over the wall to strangers who will test it mercilessly </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Testers get involved with the project only when the testing steps are about to begin </li></ul></ul></ul></ul>
  12. 12. Software Testing <ul><li>Organizing for Software Testing </li></ul><ul><ul><li>The role of the independent test group </li></ul></ul><ul><ul><ul><li>Remove the inherent problems associated with letting the builder test the thing that has been built </li></ul></ul></ul><ul><ul><ul><li>Removes the conflict of interest </li></ul></ul></ul><ul><ul><ul><li>The software engineer, however, does turn the program over to the ITG and walk away; both have to work closely throughout the project </li></ul></ul></ul><ul><ul><ul><li>The developer must be available to correct uncovered errors </li></ul></ul></ul>
  13. 13. Software Testing <ul><li>Strategy for conventional software architecture </li></ul><ul><ul><li>Unit Testing </li></ul></ul><ul><ul><ul><li>Concentrates on each unit (i.e. component) of the software as implemented in the source code </li></ul></ul></ul><ul><ul><li>Integration Testing </li></ul></ul><ul><ul><ul><li>Focus is on design and the construction of the software architecture </li></ul></ul></ul><ul><ul><li>Validation Testing </li></ul></ul><ul><ul><ul><li>Requirements are analysis are validated against the software that has been constructed </li></ul></ul></ul><ul><ul><li>System Testing </li></ul></ul><ul><ul><ul><li>Software and other elements are tested as a whole </li></ul></ul></ul>
  14. 14. Software Testing <ul><li>Strategy for Object Oriented Architecture </li></ul><ul><ul><li>Focus on “testing in the small” and work outward toward “testing in the large” </li></ul></ul><ul><ul><ul><li>Testing in the small changes from an individual module to a class encompassing attributes and operations; implies communication and collaboration </li></ul></ul></ul><ul><ul><ul><li>A series of regression tests are run to uncover errors due to communication and collaboration between classes </li></ul></ul></ul>
  15. 15. Software Testing <ul><li>Strategic Issues </li></ul><ul><ul><li>Specify product requirements in a quantifiable manner long before testing </li></ul></ul><ul><ul><ul><li>A good strategy not only looks for errors, but also asses other quality characteristics such as </li></ul></ul></ul><ul><ul><ul><ul><li>Portability </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Maintainability </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Usability </li></ul></ul></ul></ul><ul><ul><li>State testing objectives explicitly </li></ul></ul><ul><ul><ul><li>The specific objectives of testing should be stated in measurable terms </li></ul></ul></ul><ul><ul><ul><ul><li>Test effectiveness </li></ul></ul></ul></ul><ul><ul><ul><ul><li>The cost to find and fix defects </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Test coverage </li></ul></ul></ul></ul>
  16. 16. Software Testing <ul><li>Strategic Issues </li></ul><ul><ul><li>Understand the user of the software and develop a profile for each </li></ul></ul><ul><ul><ul><li>Use-cases that describe the interaction scenario for each class of user can reduce overall testing effort by focusing testing on actual use of the project </li></ul></ul></ul><ul><ul><li>Build robust software that is designed to test itself </li></ul></ul><ul><ul><ul><li>Software should be designed in a manner that uses anti-bugging techniques. Software should be capable of diagnosing certain classes of errors. The design should be accommodated automated testing and regression testing </li></ul></ul></ul>
  17. 17. Software Testing <ul><li>Strategic Issues </li></ul><ul><ul><li>Use effective formal technical reviews as a filter prior to testing </li></ul></ul><ul><ul><ul><li>Reviews can reduce the amount of testing effort that is required to produce high-quality software </li></ul></ul></ul><ul><ul><li>Develop a continuous improvement approach </li></ul></ul><ul><ul><ul><li>The testing strategy should be measured. The metrics collected during testing should be used as part of a statistical process control approach for testing </li></ul></ul></ul>
  18. 18. Software Testing <ul><li>Test Strategies for Conventional Software </li></ul><ul><ul><li>There are many strategies for testing software. </li></ul></ul><ul><ul><ul><li>A software team could wait until the system is fully constructed and then conduct tests on the overall system. </li></ul></ul></ul><ul><ul><ul><li>At the other extreme software engineer can conduct tests on a daily basis, whenever any part of the system is constructed </li></ul></ul></ul><ul><ul><ul><li>A testing strategies falls between the two extremes </li></ul></ul></ul><ul><ul><ul><ul><li>Begin with the testing of individual program units </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Moving to tests designed to facilitate the integration of the units </li></ul></ul></ul></ul>
  19. 19. Software Testing <ul><li>Test Strategies for Conventional Software </li></ul><ul><ul><li>Unit Testing </li></ul></ul><ul><ul><ul><li>Focuses verification effort on the smallest unit of software design – the software component or module </li></ul></ul></ul><ul><ul><li>Integration Testing </li></ul></ul><ul><ul><ul><li>“ If they all work individually, why do you doubt that they’ll work when we put them together?” </li></ul></ul></ul><ul><ul><ul><ul><li>The problem is “putting them together” – interfacing </li></ul></ul></ul></ul><ul><ul><ul><ul><ul><li>Data may be lost </li></ul></ul></ul></ul></ul><ul><ul><ul><ul><ul><li>One module can have an inadvertent, adverse effect on another; sub-function when combined </li></ul></ul></ul></ul></ul><ul><ul><ul><ul><ul><li>Global data structure can present problems </li></ul></ul></ul></ul></ul>
  20. 20. Software Testing <ul><li>Test Strategies for Conventional Software </li></ul><ul><ul><li>Regression Testing </li></ul></ul><ul><ul><ul><li>Each time a new module is added as part of integration testing, the software changes </li></ul></ul></ul><ul><ul><ul><li>New data flow paths are established </li></ul></ul></ul><ul><ul><ul><li>New I/O and control logic is invoked </li></ul></ul></ul><ul><ul><ul><li>Regression testing is the re-execution of some subset of tests that have already been conducted to ensure that changes have not propagated unintended side effects </li></ul></ul></ul><ul><ul><li>Smoke testing </li></ul></ul><ul><ul><ul><li>An integration testing </li></ul></ul></ul>
  21. 21. Software Testing <ul><li>Test Strategies for Conventional Software </li></ul><ul><ul><li>Smoke testing </li></ul></ul><ul><ul><ul><li>An integration testing </li></ul></ul></ul><ul><ul><ul><li>Software components that have been translated into code are integrated into a “build” – data files, libraries, reusable modules, and engineered components that are required to implement one or more product </li></ul></ul></ul><ul><ul><ul><li>The intent of testing here is to uncover show stoppers – errors that have the highest likelihood of throwing the software project behind schedule </li></ul></ul></ul><ul><ul><ul><li>The build is integrated with other builds and the entire product is smoke tested daily </li></ul></ul></ul>
  22. 22. Software Testing <ul><li>Test Strategies for OBJECT-ORIENTED SOFTWARE </li></ul><ul><ul><li>Unit Testing </li></ul></ul><ul><ul><ul><li>An encapsulated class is the focus of unit testing </li></ul></ul></ul><ul><ul><ul><li>Operations within the class are, however, the smallest unit </li></ul></ul></ul><ul><ul><li>Integration Testing </li></ul></ul><ul><ul><ul><li>Thread based testing </li></ul></ul></ul><ul><ul><ul><ul><li>Integrates a set of classes required to respond to one input or event for the system </li></ul></ul></ul></ul><ul><ul><ul><li>Used based testing </li></ul></ul></ul><ul><ul><ul><ul><li>Begins the construction of the system by testing independent classes – use very few if any server classes </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Testing dependent classes – which use independent classes </li></ul></ul></ul></ul>
  23. 23. Software Testing <ul><li>Test Strategies for OBJECT-ORIENTED SOFTWARE </li></ul><ul><ul><li>Integration Testing </li></ul></ul><ul><ul><ul><li>Cluster Testing </li></ul></ul></ul><ul><ul><ul><ul><li>A cluster of collaborating classes (determined by examining the CRC) to uncover errors in collaboration </li></ul></ul></ul></ul>
  24. 24. Software Testing <ul><li>Validation Testing </li></ul><ul><ul><li>At this level the distinction between conventional and object-oriented software disappears </li></ul></ul><ul><ul><ul><li>Testing focuses on user visible actions and user recognizable output from the system </li></ul></ul></ul><ul><ul><li>Can be defined in many ways </li></ul></ul><ul><ul><ul><li>A simple definition is that validation succeeds when software functions in a manner that can be reasonably expected by the customer. </li></ul></ul></ul><ul><ul><li>Alpha Testing </li></ul></ul><ul><ul><ul><li>Conducted at the developer site by end-users </li></ul></ul></ul><ul><ul><li>Beta Testing </li></ul></ul><ul><ul><ul><li>Conducted at end user sites – the developer is generally not present </li></ul></ul></ul>
  25. 25. Software Testing <ul><li>System Testing </li></ul><ul><ul><li>Ultimately, software is incorporated with other system elements </li></ul></ul><ul><ul><ul><li>Hardware, people, information </li></ul></ul></ul><ul><ul><ul><li>A series of system integration and validation tests are conducted </li></ul></ul></ul><ul><ul><ul><li>A classic system testing problem – finger pointing </li></ul></ul></ul><ul><ul><ul><ul><li>Software engineers should anticipate potential interfacing problems </li></ul></ul></ul></ul><ul><ul><ul><ul><ul><li>Design error handling paths that test all information coming from other elements of the system </li></ul></ul></ul></ul></ul><ul><ul><ul><ul><ul><li>Conduct series of tests to simulate bad data </li></ul></ul></ul></ul></ul><ul><ul><ul><ul><ul><li>Record the results of testing as evidence </li></ul></ul></ul></ul></ul><ul><ul><ul><ul><ul><li>Participate in planning and design of systems tests to ensure that software is adequately tested </li></ul></ul></ul></ul></ul>
  26. 26. Software Testing <ul><li>System Testing </li></ul><ul><ul><li>Recovery Testing </li></ul></ul><ul><ul><ul><li>Is a system test that forces the software to fail in a variety of ways and verifies that recovery is properly performed </li></ul></ul></ul><ul><ul><ul><ul><li>If automatic recovery </li></ul></ul></ul></ul><ul><ul><ul><ul><ul><li>Re-initialization </li></ul></ul></ul></ul></ul><ul><ul><ul><ul><ul><li>Check-pointing mechanisms </li></ul></ul></ul></ul></ul><ul><ul><ul><ul><ul><li>Data recovery </li></ul></ul></ul></ul></ul><ul><ul><ul><ul><ul><li>Restart are evaluated for correctness </li></ul></ul></ul></ul></ul><ul><ul><ul><ul><li>If manual </li></ul></ul></ul></ul><ul><ul><ul><ul><ul><li>Mean time to repair (MTTR) is evaluated </li></ul></ul></ul></ul></ul>
  27. 27. Software Testing <ul><li>Security Testing </li></ul><ul><ul><li>It verifies that protection mechanisms built into a system will, in fact protect it from improper penetration </li></ul></ul><ul><li>Stress Testing </li></ul><ul><ul><li>It executes a system in a manner that demands resources in abnormal quantity, frequency, or volume </li></ul></ul><ul><li>Performance Testing </li></ul><ul><ul><li>Are often coupled with stress testing and usually require both hardware and software instrumentation. </li></ul></ul><ul><ul><li>To measure resource utilization </li></ul></ul>