• Save
11 software testing_strategy
Upcoming SlideShare
Loading in...5
×
 

11 software testing_strategy

on

  • 3,242 views

Principles of Software Engineering by Utpal Roy, Jadavpur University, Kolkata, India

Principles of Software Engineering by Utpal Roy, Jadavpur University, Kolkata, India

Statistics

Views

Total Views
3,242
Views on SlideShare
3,219
Embed Views
23

Actions

Likes
8
Downloads
0
Comments
3

1 Embed 23

http://www.scoop.it 23

Accessibility

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel

13 of 3 Post a comment

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
  • Thank u soooo much, it helps a lot
    Are you sure you want to
    Your message goes here
    Processing…
  • please tell me sir how can i download it?
    Are you sure you want to
    Your message goes here
    Processing…
  • Awesome topics but unable to download/
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    11 software testing_strategy 11 software testing_strategy Presentation Transcript

    • Software Testing Strategy What kind of testing approach we should take to apply the already developed test cases ?13 January 2012 Made By Utpal Ray 1
    • Software Testing Strategy  Strategic Approach It is about applying designed test cases strategically to uncover more number of errors in a shorter span of time. Deciding a suitable test strategy calls for a suitable test plan and a test procedure document. Testing should begin at the component level ( unit level ) and move ‘outward’ towards integration of the entire computer based system. Different testing techniques are required at different point of testing span. Testing should be performed by an Independent Test Group, who did not contribute towards the design and development of the software. Any testing strategy must incorporate test planning, test case design, test execution, test result collection, test result evaluation etc.13 January 2012 Made By Utpal Ray 2
    • Software Testing Strategy  A typical Software Test Strategy Considering the fact that testing is a big task, the entire testing cycle can be divided into four different phase where different test strategy is employed in different phase; also, different test techniques are used in different phase. The test target and the test focus also changes along this four different phases of testing cycles. The most interesting part is how error finding rate changes as testing progresses along this four different phases and so as the cost associated with fixing all those errors. It is to be noted that test strategy for a newly developed software ( which was developed from scratch ) will be different than the test strategy for a moderately modified software.13 January 2012 Made By Utpal Ray 3
    • Software Testing Strategy  A typical Software Test Strategy for a newly developed software Test Phase Early Phase Integration Validation Final Phase ( Phase – I ) Phase Phase ( Phase – IV ) Scenario ( Phase – II ) ( Phase – III ) Test Unit Integration Validation System Strategy Testing Testing Testing Testing Test Mainly White White Box Black Box Black Box Technique Box And Black Testing Testing Testing Box Testing Test Individual A group of The Whole The entire Target Module Closely Software s/w in the related operational Module environment13 January 2012 Made By Utpal Ray 4
    • Software Testing Strategy  A typical Software Test Strategy for a newly developed software ( Contd. ) Test Phase Early Phase Integration Validation Final Phase Scenario ( Phase – I ) Phase Phase ( Phase–IV ) ( Phase – II ) ( Phase-III ) Testing Coding Design and Functional The s/w along Focus the s/w Requirement with the other architecture and other system elements requirement Error Finding Rate13 January 2012 Made By Utpal Ray 5
    • Software Testing Strategy  A typical Software Test Strategy for a newly developed software ( Contd. ) Test Phase Early Phase Integration Validation Final Phase Scenario ( Phase – I ) Phase Phase ( Phase–IV ) ( Phase – II ) ( Phase-III ) Difficulty and cost of fixing those errors The responsible persons for Individual Project Team The whole The whole the errors Department Organization13 January 2012 Made By Utpal Ray 6
    • Software Testing Strategy  Unit Testing It is done on the individual module level. Component level design is used as a guide. It is important to fix the scope of the unit testing. The module interface needs to be examined and tested. The boundary conditions need to be examined ( some sort of black box testing ). The basis path testing ( white box testing ) need to be performed. The error conditions, the error handlings and the error messages – all these criteria need to be tested.13 January 2012 Made By Utpal Ray 7
    • Software Testing Strategy  Unit Testing ( contd. ) If it is necessary driver and suitable stubs need to be developed and integrated with the modules ( illustrated below ). DRIVER Test Cases Test Results Module To be Tested STUB STUB13 January 2012 Made By Utpal Ray 8
    • Software Testing Strategy  Integration Testing This is the testing phase after unit testing. This phase deals with the combination of different modules. Unless each of the module is unit tested satisfactorily, one cannot enter this testing phase. The best approach for integration testing is the incremental integration; where modules are added one after another. Integration testing is performed every time after each of the module is added. This process of testing is good enough to find out which module is causing the error. There are two distinct methods of incremental integration – Top-Down integration and Bottom-Up integration.13 January 2012 Made By Utpal Ray 9
    • Software Testing Strategy  Top-Down Integration Modules are added by moving downward through the control hierarchy, beginning with the main control module ( top most executive module ) first. The subordinate modules ( to the main module ) can be added either in a breadth-first manner or in a depth-first manner. For the Top-Down integration, you may need stub to effectively test the effect of module addition.13 January 2012 Made By Utpal Ray 10
    • Software Testing Strategy  Top-Down Integration ( contd. ) Breadth-First M1 M2 M3 M4 M5 M6 M7 Sequence of Module Addition :- M1, M2, M3, M4, M5, M6, M7, M8 M813 January 2012 Made By Utpal Ray 11
    • Software Testing Strategy  Top-Down Integration ( contd. ) Depth-First M1 M2 M3 M4M5 M6 M7 Sequence of Module Addition :- M1, M2, M5, M8, M6, M3, M7, M4M813 January 2012 Made By Utpal Ray 12
    • Software Testing Strategy  Bottom-Up Integration Modules are added from the bottom; which means modules from the lower most level of the ‘call and return’ architecture are added first. Then the addition process moves upwards. The low level components are combined into multiple clusters and each of this cluster is tested first using a driver. While testing each of the cluster; modules are added from the lower most level for each of this cluster. Once the clusters have been successfully tested; the clusters are attached to the main branch ( after removing the drivers ) and addition of modules process continues. This way the integration testing is performed on the whole ‘call and return’ arch.13 January 2012 Made By Utpal Ray 13
    • Software Testing Strategy  Bottom-Up Integration ( contd. ) An example of a ‘Call and Return’ Architecture MA MB MC MD M11 M12 M21 M22 M31 M13 M23 M24 M32 M14 M25 M26 CLUSTER 1 CLUSTER 2 CLUSTER 313 January 2012 Made By Utpal Ray 14
    • Software Testing Strategy  Bottom-Up Integration ( contd. ) Testing Cluster 1 DRIVER1 Sequence of Module Addition :- M14, M13, M12, M11 M11 M12 M13 M1413 January 2012 Made By Utpal Ray 15
    • Software Testing Strategy  Bottom-Up Integration ( contd. ) Testing Cluster 2 DRIVER2 Sequence of Module Addition :- M26, M25, M24, M22, M23, M21 M21 M22 M23 M24 M25 M2613 January 2012 Made By Utpal Ray 16
    • Software Testing Strategy  Bottom-Up Integration ( contd. ) Testing Cluster 3 DRIVER3 Sequence of Module Addition :- M32, M31 M31 M3213 January 2012 Made By Utpal Ray 17
    • Software Testing Strategy  Validation Testing This phase is entered after the successful completion of the integration testing. The software validation is achieved through a series of black box tests that demonstrate conformity with the requirements. One needs a test plan to identify a series of tests which need to be used as a validation test. One needs a valid test procedure also, regarding executing all those individual tests.13 January 2012 Made By Utpal Ray 18
    • Software Testing Strategy  Validation Testing ( contd. ) The validation test ensures that the following criteria are satisfied successfully :- - All Functional Requirements - All Behavioral characteristics - All Performance Requirements - All Documentation Correctness - All Other Requirements ( Transportability, Compatibility, Error Recoverability, Maintainability etc. ) If some of the criteria mentioned above do not meet the requirements, a deficiency list need to be prepared identifying which one is deviating from the normal reference.13 January 2012 Made By Utpal Ray 19
    • Software Testing Strategy  System Testing It is the final phase of the testing cycle. It is about integrating the software with hardware, people and information. The primary purpose of the system testing is to fully exercise the computer based system. The system test is really effective if it can be done in the customer’s premises. Otherwise the customer’s environment is re-created in-house.13 January 2012 Made By Utpal Ray 20
    • Software Testing Strategy  System Testing ( contd. ) There are quite a few varieties of system testing as described below :- - Recovery Testing [ How system recovers from Faults? ] - Security Testing [ How secure the system is? ] - Stress Testing [ How much load the system can take? ] - Performance Testing [ How system performs in normal condition and stresses condition ]13 January 2012 Made By Utpal Ray 21
    • Software Testing Strategy  Verification & Validation ( V&V ) One has to Validate the software to ensure that all the functional requirements are met by the software. One has to verify that one particular functionality has been correctly implemented in the software. Verification : Are we building the product right ? Validation : Are we building the right product ? Validation comes nearly at the end of the testing cycle Verification may come throughout the testing cycle.13 January 2012 Made By Utpal Ray 22
    • Software Testing Strategy  Entry and Exit Criteria for testing When to exit testing from the current test phase and enter the next testing phase ? Since the quote ‘You are never done with testing, the burden simply shifts from the engineer to the customer’, is always true; the decision to end testing mainly depends upon the resources available and also depends upon the confidence of the testing personnel. Usually, the rate of finding errors drops as testing progresses. When one cannot find any more errors; that could be the time to exit the current phase and enter the next phase.13 January 2012 Made By Utpal Ray 23
    • Software Testing Strategy  Sample Test Plan Template 1. Document Control - Distribution - Approvers/Reviewers - Change History 2. Overview - Project Summary - Overall Test Goals and Objectives 3. Test Environment - Hardware and Software Configuration 4. Test Tools To Be Used 5. Administration - Test Assumptions and Dependencies - Entry and Exit Criteria - Status Tracking Approach - Problem Reporting and Tracking Approach - Maintenance Strategy - Deliverables 6. Schedules 7. Test Matrices and Scenario13 January 2012 Made By Utpal Ray 24
    • Software Testing Strategy  Test plan template, IEEE 829 format 1.0 References 2.0 Introduction 3.0 Test Items 4.0 Features to be Tested 5.0 Features not to be Tested 6.0 Approach 7.0 Item Pass/Fail Criteria 8.0 Suspension Criteria and Resumption Requirements 9.0 Test Deliverables 10.0 Remaining Test Tasks 11.0 Environmental Needs 12.0 Staffing and Training Needs 13.0 Responsibilities 14.0 Schedule 15.0 Planning Risks and Contingencies 16.0 Approvals 17.0 Glossary13 January 2012 Made By Utpal Ray 25
    • Software Testing Strategy  Other Types of Testing Scenario 1. Alpha Testing 2. Beta Testing 3. Gamma Testing 4. Smoke Testing 5. Regression Testing 6. Automated Testing 7. Fuzz Testing 8. Static Testing13 January 2012 Made By Utpal Ray 26
    • Software Testing Strategy  Other Types of Testing Scenario ( contd. ) Alpha Testing Different Software Organization has a different view regarding definition of this test. This may be also called ‘Engineering’ Test. Usually this test is attempted after the integration testing is over. This test is conducted in-house. This is a some kind of validation test in which different in-house department may take participation.13 January 2012 Made By Utpal Ray 27
    • Software Testing Strategy  Other Types of Testing Scenario ( contd. ) Beta Testing This testing is attempted after the alpha testing is over. This test is always conducted in the customer premises; that’s why it is a case of ‘live’ testing. Usually developers are not present during this kind of testing. The customer records all problems that were encountered during beta testing and reports this to the developers at regular intervals.13 January 2012 Made By Utpal Ray 28
    • Software Testing Strategy  Other Types of Testing Scenario ( contd. ) Gamma Testing Gamma testing is a little-known informal phrase that refers derisively to the release of "buggy" (defect- ridden) software. It is not a term of art among testers, but rather an example of referential humor. Cynics have referred to all software releases as "gamma testing" since defects are always found in the customer premises.13 January 2012 Made By Utpal Ray 29
    • Software Testing Strategy  Other Types of Testing Scenario ( contd. ) Smoke Testing It is a ritual of daily testing. It need not be an exhaustive testing rather it is an end to end testing. The test cases are designed keeping the frequently performing functions in mind. This is a one type of integration testing, where the entire software is smoke tested daily.13 January 2012 Made By Utpal Ray 30
    • Software Testing Strategy  Other Types of Testing Scenario ( contd. ) Regression Testing It is about running a subset of old test suite, to make sure that the newly added changes did not break the earlier functionality. This is very much necessary when software moves from one version to another version upwardly. One has to run a regression test suite to make sure the functionality of the earlier version did not change even if the new functions has been added.13 January 2012 Made By Utpal Ray 31
    • Software Testing Strategy  Other Types of Testing Scenario ( contd. ) Automated Testing Considering the fact that testing is a repetitive process, most of the testing execution process can be automated. In it’s simplest form, an automated test suite does not wait for user’s input; rather it takes the input from a already prepared file. At the same time it does not leave the test results for user’s interpretation but compares it with an already prepared sample output file. It is comparatively easier to write automated test suite when input/output involves with keyboard and monitor. In case of GUI testing, a different method of automated testing should be employed.13 January 2012 Made By Utpal Ray 32
    • Software Testing Strategy  Other Types of Testing Scenario ( contd. ) Fuzz Testing The basic idea is to attach the inputs of a program to a source of random data ("fuzz"). If the program fails (for example, by crashing, or by failing built-in code assertions), then there are defects to correct.13 January 2012 Made By Utpal Ray 33
    • Software Testing Strategy  Other Types of Testing Scenario ( contd. ) Static Testing Static Testing (also known as "Dry Run Testing") is a form of software testing where the software isnt actually used. Syntax checking and manually reading the code to find errors are methods of static testing. This type of testing is mostly used by the developer himself (who designed or code the module). Static testing is usually the first type of testing done on any system.13 January 2012 Made By Utpal Ray 34
    • Software Testing Strategy Still More Varieties of Testing Proposal Testing; Requirement Testing; Design Testing; Big Bang Testing; Sandwich Testing; Complexity Testing; Compatibility Testing; Security Testing; Performance Testing; Volume Testing; Stress Testing; Recovery Testing; Installation Testing; Error Handling Testing; Manual Support Testing; Intersystem Testing; Control Testing; Sanity Testing; Adhoc Testing (Monkey Testing, Exploratory Testing, Random Testing); Execution Testing; Operations Testing; Compliance Testing; Usability Testing; Decision Table Testing (Axiom Testing); Documentation Testing; Training Testing; Rapid Testing; ‘COTS’ Testing; Client-Server Testing; Web Application Testing; Mobile Application Testing; eBusiness/eCommerce Testing; Agile Development Testing; Data Warehouse Testing; [ Courtesy; Software Testing – M G Limaye; McGraw-Hill13 January 2012 Made By Utpal Ray 35
    • Software Testing Strategy  Home Task 1. Why is a highly coupled module difficult to unit test ? 2. Develop an integration testing strategy for the problems which you have already got the architectural design. 3. Who should perform the validation test – the software developer, an independent tester or the software user ? Justify your answer.13 January 2012 Made By Utpal Ray 36