Test Case Potency Assessment

841 views
698 views

Published on

Test case potency assessment is primarily a diagnostic that assess the quality of test cases. This is one of the applications of HBT (Hypothesis Based Testing). This is boutique service offering from STAG Software Private Limited.

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
841
On SlideShare
0
From Embeds
0
Number of Embeds
132
Actions
Shares
0
Downloads
24
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Test Case Potency Assessment

  1. 1. Test Case Potency Assessment(TCPA)
  2. 2. Introduction Test case potency assessment is primarily a diagnostic that that assess the quality of test cases. This is one of the applications of HBT (Hypothesis Based Testing). Typically the quality of the test cases i.e. it ability to uncover defects that matter is understood as being heavily dependent on experience. And therefore assessment of the same is seen as one that can be done by a person with deep domain knowledge. No doubt domain knowledge is useful the act of assessment in TCPA is done using test case architecture of HBT as the basis and therefore done scientifically. The key objectives of this 1. Are the right types of defects being targeted to be uncovered? 2. Are the test cases covering the functional and non-functional aspects of the system? 3. Are the designed test cases sufficient? 4. Is the mix of test cases of +ve/-ve good enough? 5. Is the baseline for test complete? The assessment is done by examining the test case document(s) and test strategy statically after understanding the system.© 2012. STAG Software Private Limited. All rights reserved. 2
  3. 3. Value Proposition TCPA is intended to assess the quality of test scenarios/cases and help in improving the coverage of the same. The key value proposition is to ensure that the test scenarios/cases can indeed cover the system under test well thereby ensuring that release quality of the system is indeed good.© 2012. STAG Software Private Limited. All rights reserved. 3
  4. 4. Application Potency Assessment 1 ... is done by “Area” Requirements ‣ using a test case architecture that traceability “External area ‣ characterises properties of test cases and that I am ‣ examine/assess potency related properties Entity covering” e t t arg Code coverage to e re “Internal area Wh “Drug” that I am covering” Test Cases Potency Wh o t “Bug” o targ et? Fault Coverage Countability PDT “What PDTs “Proving sufficiency of test cases” Potential Defect Type are uncovered by the test cases” Conformance:Robustness “Distribution of +ve/-ve test cases” Test case immunity Level-isation Immunity “No defect yield “Optimal targeting” Resistant to bugs from test cases” i.e. hardened entities© 2012. STAG Software Private Limited. All rights reserved. 4
  5. 5. HBT Test Architecture Organised by Cleanliness levels sub-ordered by items (features/modules..), Level segregated by type, ranked by importance/priority, Item sub-divided into conformance(+) and robustness(-), classified by early (smoke)/late-stage evaluation, Type tagged by evaluation frequency, linked by optimal execution order, classified by execution mode (manual/automated) Priority Focus Stage The external form and internal structure of test cases are seen as key to ensuring that the test cases are Frequency complete and effective. Order Examining the organisation of test cases allows one to uncover “holes” in the test cases. Mode© 2012. STAG Software Private Limited. All rights reserved. 5
  6. 6. Brief Flow of TCPA 2. Setup test baseline 3. Identify attributes that are 1. Understand SUT i.e “what to test” key 4. Check if appropriate 5. Hypothesise PDTs and 6. Check if the baseline used types of test have been types of test of interest for strategy is compete considered 7. Check if test cases exist for 8. Analyse test case count for 9. Analyse +ve:-ve distribution these PDTs completeness 10. Document gaps in the test 12. Explain gaps using the 11. Produce TCPA report scenarios/cases TCPA report© 2012. STAG Software Private Limited. All rights reserved. 6
  7. 7. Note 1. Since TCPA is assessment based on documented test cases, this cannot be done for system that is tested using no/very minimal test documentation. 2. Also any defect information is welcome as this would accelerate the PDT list creation. This can include customer defects and those found the vendor’s test team. 3. Note that in the case of a large system, this may be done by identifying similar components and assessing only sample of similar features/requirements to get a good indicative view of areas of improvement.© 2012. STAG Software Private Limited. All rights reserved. 7
  8. 8. Inputs/Deliverables Test Strategy Test Scenarios/Cases Potency Assessment Report TCPA This highlights the gaps in the test scenarios/cases discovered by TCPA. Defects found by QA team Defects reported by customer© 2012. STAG Software Private Limited. All rights reserved. 8

×