Qa documentation pp


Published on

Published in: Technology, Business
1 Comment
1 Like
No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Qa documentation pp

  1. 1. Software and System Test Documentation
  2. 2. IEEE 829 - Standard for Software Test Documentation TEST PLANNING AND CONTROL Master Test Plan Level Test Plan TEST ANALYSIS AND DESIGN Level Test Design Level Test Case TEST IMPLEMENTATION AND EXECUTION Bug Report Level Test Procedure EVALUATING EXIT CRITERIA AND REPORTING Interim Test Status Report Test Log Level Test Report TEST CLOSURE ACTIVITIES Master Test Report
  3. 3. LEVELS OF TEST PLAN Develop Master Test Plan Develop Detailed Test Plans Unit Test Plan Integration Test Plan SystemTest Plan AcceptanceTest Plan Level of Test Plan defines what the test plan is being created for. Test Plan document follows the same structure for each level of test plan. The only difference is the content and detail. TEST PLANNING AND CONTROL
  4. 4. MASTER TEST PLAN The purpose of the MTP is to: provide the overall framework for all the testing activities define the scope of the testing identify whether there is any unnecessary duplication of testing taking place identify the departures from the Test Process documentation set define the approach to each stage of testing specify the ABC project’s staff’s responsibilities for testing activities for each stage of testing
  5. 5. A document describing the scope, approach, resources and schedule of intended test activities. It identifies amongst others test items, the features to be tested, the testing tasks, who will do each task, degree of tester independence, the test environment, the test design techniques and entry and exit criteria to be used, and the rationale for their choice, and any risks requiring contingency planning. It is a record of the test planning process. A test plan that typically addresses multiple test levels. A test plan that typically addresses one test phase. Test Plan
  6. 6. The format and content of a software test plan vary depending on the processes, standards, and test management tools being implemented. Following format provides a summary of what a test plan can/should contain. 1) Test Plan ID: Unique No or Name 2) Introduction: Provide an overview of the test plan, specify the goals/objectives. 3) Test Items: Modules/ Functions/ Services/ Features/ etc. 4) Features to be tested: Responsible Modules for Test design 5) Features not to be tested: Which feature is not to be tested and Why? 6) Approach: List of selected testing techniques to be applied on above specified modules in reference to the TRM (Test Responsible Matrix) 7) Feature pass or fail criteria: When a feature is pass or fail description 8) Suspension criteria 9) Test Environment: Required software & Hardware to be tested on above features 10) Test Deliverables: Required testing document to be prepared 11) Testing Task: Necessary tasks to do before start every feature testing 12) Staff & Training: Names of selected Test Engineers & training requirements to them 13) Responsibilities: Work allocation to every member in the team 14) Schedule: Dates & Times of testing modules 15) List & Mitigation: Possible testing level risks & solution to overcome them 16) Approvals: Signatures of Test plan authors & Project Manager / QA Whattotest?Howtotest?Whentotest?
  7. 7. TEST DESIGN LEVELS STRUCTURE BASED ON THE V-MODEL Overall Business Requirements Acceptance Test Design Software requirements System Test Design High level requirements Integration Test Design Low level requirements Component Test Design Coding Unit Test Design Unit Test Execution Component Test Execution Integration Test Execution System Test Execution Acceptance Test Execution TEST ANALYSIS AND DESIGN
  8. 8. TEST DESIGN Test Design Phase – In software engineering, test design phase is a process of reviewing and analyzing test basis, selecting test design techniques and creating designed test cases, checklists and scenarios for testing software. Test Design Specification - it is a document that describes features to be tested and specifies list of all test scenarios or test cases, which should be designed for providing the testing of software. Basically test design is the act of creating and writing test suites for testing a software. Test design could require all or one of: 1) Knowledge of the software, and the business area it operates on 2) Knowledge of the functionality being tested 3) Knowledge of testing techniques 4) Planning skills to schedule in which order the test cases should be designed, given the effort, time and cost needed or the consequences for the most important and/or risky features
  9. 9. Review and Analyze Test Basis Select Test Design Techniques Create Test Design Specification Create Test Cases Specification Test Plan Requirements Mock-ups Test Design Specification Test Case Specification BASIC TEST DESIGN STEPS:
  10. 10. • The internal factors that influence the decision about which technique to use are: – Tester knowledge and experience – Expected defects – Test objectives – Documentation – Life cycle model • The external factors are: – Risks – Customer requirements – System type – Time and budget Choosing A Test Design Technique
  11. 11. According to IEEE-829 standard template structure looks in the following way: 1. Test Design Specification Identifier 1.1 Purpose 1.2 References 1.3 Definitions, acronyms and abbreviations 2. Features to be Tested 3. Approach Refinements 4. Test Identification 4.1 <Test Item 1> 4.2 <Test Item …> 4.3 <Test Item N> 5. Feature Pass/Fail Criteria Test Design Specification StructureBelow–theexplanation
  12. 12. 3)Approach Refinements section describes the following: – Specific test techniques to be used for testing features or combinations of features – Types of testing which will be provided – Methods of analyzing test results – Test results reporting – Whether automation of test cases will be provided or not – Any other information which describes approach to testing Test Design Specification Structure 1) Test Design Specification Identifier section covers: – Purpose of the document – Scope of the document – List of references which should include references on test plan, functional specification, test case specification, etc. – Definitions, acronyms and abbreviations used in Test Design Specification 2) Features to be Tested identifies test items and describes features and combinations of features that are the object of this design specification. Reference on Functional Specification for each feature or combination of features should be included. 4) Feature Pass/Fail Criteria specifies the criteria to be used to determine whether the feature or feature combination has passed or failed. The following items can be considered as “pass / fail criteria”: 1) Feature works according to stated requirements 2) Feature works correctly on the test platforms 3) Feature works correctly with other modules of application 4) All issues with High and Medium Priority will be verified and closed
  13. 13. 5) Test Identification section is separated to sub-section according to the amount of test items identifying future documentation which will be created for testing features or combinations of features that are the object of this design specification. Features can be covered by test objectives in different ways depending on projects needs, approaches for testing etc. Three examples of such coverage: Feature covered by test cases Feature covered by test scenarios Feature covered by checklists
  14. 14. Test Case Specification Test Specification – It is a detailed summary of what scenarios will be tested, how they will be tested, how often they will be tested, and so on and so forth, for a given feature. Revision History - This section contain information like Who created the test specification? When was it created? When was the last time it was updated? Feature Description – A brief description of what area is being tested. What is tested? – An overview of what scenarios are tested. What is not tested? - Are there any areas that are not being tested. Nightly Test Cases – A list of the test cases and high-level description of what is tested whenever a new build becomes available. Breakout of Major Test Areas - It is the most interesting part of the test specification where testers arrange test cases according to what they are testing. Specific Functionality Tests – Tests to verify the feature is working according to the design specification. This area also includes verifying error conditions. Security tests – Any tests that are related to security. Accessibility Tests – Any tests that are related to accessibility. Performance Tests - This section includes verifying any performance requirements for your feature. Localization / Globalization - tests to ensure you’re meeting your product’s Local and International requirements. NOTE: Test Specification document should prioritize the test case easily like nightly test cases, weekly test cases and full test pass etc: Nightly - Must run whenever a new build is available. Weekly - Other major functionality tests run once every three or four builds. Lower priority - Run once every major coding milestone. ContentsofaTestSpecification:
  15. 15. Test cases are a set of conditions or variables under which a tester will determine if a requirement upon an application is partially or fully satisfied. There must be at least one test case for each requirement for traceability. Test Cases Test Step – specifies an action to perform, and the expected response of the test application. For example: Action : Type the password in the password box, Expected result: Your password should be dotted / hidden. Test Case – a list of test steps. Also defines the environmental situation and may link to related bugs, requirements etc. Test Scenario – usually comes directly from business requirements or user story. Scenario contains a list of test cases and often their sequence. Test scenario Test case Test Step
  16. 16. Standard fields of sample test case template: Test case ID: Unique ID for each test case. Test priority (Low/Medium/High): This is useful while test execution. Test priority for business rules and functional test cases can be medium or higher whereas minor user interface cases can be low priority. Module Name – Mention name of main module or sub module. Test Designed By: Name of tester Test Designed Date: Date when wrote Test Executed By: Name of tester who executed this test. To be filled after test execution. Test Execution Date: Date when test executed. Test Title/Name: Test case title. Test Summary/Description: Describe test objective in brief. Pre-condition: Any prerequisite that must be fulfilled before execution of this test case. List all pre- conditions in order to successfully execute this test case. Dependencies: Mention any dependencies on other test cases or test requirement. Test Steps: List all test execution steps in detail. Write test steps in the order in which these should be executed. Make sure to provide as much details as you can. Test Data: Use of test data as an input for this test case. You can provide different data sets with exact values to be used as an input. Expected Result: What should be the system output after test execution? Status (Pass/Fail): If actual result is not as per the expected result mark this test as failed. Otherwise passed. Notes/Comments/Questions: To support above fields if there are some special conditions which can’t be described in any of the above fields or there are questions related to expected or actual results mention those here.
  17. 17. Fields of sample test case template, if necessary: Defect ID/Link: If test status is fail, then include the link to defect log or mention the defect number. Test Type/Keywords: This field can be used to classify tests based on test types. E.g. functional, usability, business rules etc. Requirements: Requirements for which this test case is being written. Preferably the exact section number of the requirement doc. Attachments/References: This field is useful for complex test scenarios. Automation? (Yes/No): Whether this test case is automated or not. Useful to track automation status when test cases are automated.
  18. 18. The purpose of an LTPr is to specify the steps for executing a set of test cases or, more generally, the steps used to exercise a software product or software-based system item in order to evaluate a set of features. 1. Identification (Each Test Procedure Specification should be assigned a unique identifier.) 2. Purpose (explain the purpose of the test procedure and reference any test cases it executes.) 3. Special Requirements (list any special hardware, software or training requirements for this procedure.) 4. Procedure Steps (actual steps of the procedure are described. IEEE has listed several steps: Log: explain methods / formats for logging results of test execution. Set up : actions necessary to prepare for the execution of procedure. Start : actions necessary to begin execution of the procedure. Proceed : actions necessary during the execution of the procedure. Measure : explain ho test measurements need to be made. Shutdown : actions necessary to suspend testing when some unscheduled event happens. Restart : actions necessary to restart the procedure. Stop : actions necessary to bring execution to systematic halt. Level Test Procedure TestProcedureSpecificationshouldhave thefollowingelements: TEST IMPLEMENTATION AND EXECUTION
  19. 19. ID Unique identifier given to the defect. Project Project name. Release Version Release version of the product. (e.g. 1.2.3) Module Specific module of the product where the defect was detected. Detected Build Version Build version of the product where the defect was detected (e.g. Summary Summary of the defect. Keep this clear and concise. Description Detailed description of the defect. Describe as much as possible but without repeating anything or using complex words. Keep it simple but comprehensive. Steps to Reproduce Step by step description of the way to reproduce the defect. Number the steps. Actual Result The actual result you received when you followed the steps. Expected Results The expected results. Attachments Attach any additional information like screenshots and logs. Remarks Any additional comments on the defect. Defect Severity The seriousness of defect with respect to functionality High(Show Stopper):-Without fixing this defect tester not able to continue testing. Medium:-Able to continue but mandatory to fix. Low:-Able to continue testing but may or may not to fix. Defect Priority The importance of defect fixing with respect to customer interest. Reported By The name of the person who reported the defect. Assigned To The name of the person that is assigned to analyze/fix the defect. Status The status of the defect. (new/reopen) Fixed Build Version Build version of the product where the defect was fixed (e.g. DEFECT REPORT
  20. 20. Test Log Test Log is details of what tests cases were run, who ran the tests, in what order they were run, and whether or not individual tests were passed or failed. The purpose of the Level Test Log is to provide a chronological record of relevant details about the execution of tests. An automated tool may capture all or part of this information. EVALUATING EXIT CRITERIA AND REPORTING
  21. 21. Interim Test Status Report The purpose of the ITSR is to summarize the results of the designated testing activities and optionally to provide evaluations and recommendations based on these results. Eight Interim Reports: Functional Testing Status Functions Working Timeline Expected verses Actual Defects Uncovered Timeline Defects Uncovered verses Corrected Gap Timeline Average Age of Uncorrecte d Defects by Type Defect Distribution Relative Defect Distribution Testing Action
  22. 22. Functional Testing Status Report Fully tested Tested with open defects Not tested This report will show percentages of the functions which have been:
  23. 23. Functions Working Timeline This report will show the actual plan to have all functions working versus the current status of functions working. An ideal format could be a line graph.
  24. 24. Expected versus Actual Defects Uncovered Timeline This report will provide an analysis between the number of defects being generated against the expected number of defects expected from the planning stage.
  25. 25. This report, ideally in a line graph format, will show the number of defects uncovered versus the number of defects being corrected and accepted by the testing group. If the gap grows too large, the project may not be ready when originally planned. Defects Detected versus Corrected Gap
  26. 26. This report will show the average days of outstanding defects by type. In the planning stage, it is beneficial to determine the acceptable open days by defect type. Average Age Uncorrected Defects by Type
  27. 27. This report will show the defect distribution by function or module. It can also show items such as numbers of tests completed. Defect Distribution
  28. 28. This report will take the previous report (Defect Distribution) and normalize the level of defects. An example would be one application might be more in depth than another, and would probably have a higher level of defects. However, when normalized over the number of functions or lines of code, would show a more accurate level of defects. Relative Defect Distribution This report can show many different things, including possible shortfalls in testing. Examples of data to show might be number of Sev 1 defects, tests that are behind schedule, and other information that would present an accurate testing picture. Testing Action
  29. 29. Level Test Report (LTR) The purpose of the Level Test Report (LTR) is to summarize the results of the designated testing activities and to provide evaluations and recommendations based on these results Master Test Report The purpose of the MTR is to summarize the results of the levels of the designated testing activities and to provide evaluations based on these results. This report may be used by any organization using the MTP. Whenever an MTP is generated and implemented, there needs to be a corresponding MTR that describes the results of the MTP implementation TEST CLOSURE ACTIVITIES