Test Case Potency Assessment

  • 464 views
Uploaded on

Test case potency assessment is primarily a diagnostic that assess the quality of test cases. This is one of the applications of HBT (Hypothesis Based Testing). This is boutique service offering from …

Test case potency assessment is primarily a diagnostic that assess the quality of test cases. This is one of the applications of HBT (Hypothesis Based Testing). This is boutique service offering from STAG Software Private Limited.

More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
464
On Slideshare
0
From Embeds
0
Number of Embeds
2

Actions

Shares
Downloads
19
Comments
0
Likes
0

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Test Case Potency Assessment(TCPA)
  • 2. Introduction Test case potency assessment is primarily a diagnostic that that assess the quality of test cases. This is one of the applications of HBT (Hypothesis Based Testing). Typically the quality of the test cases i.e. it ability to uncover defects that matter is understood as being heavily dependent on experience. And therefore assessment of the same is seen as one that can be done by a person with deep domain knowledge. No doubt domain knowledge is useful the act of assessment in TCPA is done using test case architecture of HBT as the basis and therefore done scientifically. The key objectives of this 1. Are the right types of defects being targeted to be uncovered? 2. Are the test cases covering the functional and non-functional aspects of the system? 3. Are the designed test cases sufficient? 4. Is the mix of test cases of +ve/-ve good enough? 5. Is the baseline for test complete? The assessment is done by examining the test case document(s) and test strategy statically after understanding the system.© 2012. STAG Software Private Limited. All rights reserved. 2
  • 3. Value Proposition TCPA is intended to assess the quality of test scenarios/cases and help in improving the coverage of the same. The key value proposition is to ensure that the test scenarios/cases can indeed cover the system under test well thereby ensuring that release quality of the system is indeed good.© 2012. STAG Software Private Limited. All rights reserved. 3
  • 4. Application Potency Assessment 1 ... is done by “Area” Requirements ‣ using a test case architecture that traceability “External area ‣ characterises properties of test cases and that I am ‣ examine/assess potency related properties Entity covering” e t t arg Code coverage to e re “Internal area Wh “Drug” that I am covering” Test Cases Potency Wh o t “Bug” o targ et? Fault Coverage Countability PDT “What PDTs “Proving sufficiency of test cases” Potential Defect Type are uncovered by the test cases” Conformance:Robustness “Distribution of +ve/-ve test cases” Test case immunity Level-isation Immunity “No defect yield “Optimal targeting” Resistant to bugs from test cases” i.e. hardened entities© 2012. STAG Software Private Limited. All rights reserved. 4
  • 5. HBT Test Architecture Organised by Cleanliness levels sub-ordered by items (features/modules..), Level segregated by type, ranked by importance/priority, Item sub-divided into conformance(+) and robustness(-), classified by early (smoke)/late-stage evaluation, Type tagged by evaluation frequency, linked by optimal execution order, classified by execution mode (manual/automated) Priority Focus Stage The external form and internal structure of test cases are seen as key to ensuring that the test cases are Frequency complete and effective. Order Examining the organisation of test cases allows one to uncover “holes” in the test cases. Mode© 2012. STAG Software Private Limited. All rights reserved. 5
  • 6. Brief Flow of TCPA 2. Setup test baseline 3. Identify attributes that are 1. Understand SUT i.e “what to test” key 4. Check if appropriate 5. Hypothesise PDTs and 6. Check if the baseline used types of test have been types of test of interest for strategy is compete considered 7. Check if test cases exist for 8. Analyse test case count for 9. Analyse +ve:-ve distribution these PDTs completeness 10. Document gaps in the test 12. Explain gaps using the 11. Produce TCPA report scenarios/cases TCPA report© 2012. STAG Software Private Limited. All rights reserved. 6
  • 7. Note 1. Since TCPA is assessment based on documented test cases, this cannot be done for system that is tested using no/very minimal test documentation. 2. Also any defect information is welcome as this would accelerate the PDT list creation. This can include customer defects and those found the vendor’s test team. 3. Note that in the case of a large system, this may be done by identifying similar components and assessing only sample of similar features/requirements to get a good indicative view of areas of improvement.© 2012. STAG Software Private Limited. All rights reserved. 7
  • 8. Inputs/Deliverables Test Strategy Test Scenarios/Cases Potency Assessment Report TCPA This highlights the gaps in the test scenarios/cases discovered by TCPA. Defects found by QA team Defects reported by customer© 2012. STAG Software Private Limited. All rights reserved. 8