0
Agenda• Background and Motivation• qSize Analysis  – Test Size Estimation (Test Case Point Analysis)  – Test Effort Estima...
Background• Software estimation   – process of determining the cost, time, staff, and     other related attributes of soft...
Motivation• Testing accounts for up to 50% of project  effort [1]• Current problems  – lack of reliable methods designed f...
Agenda• Background and Motivation• qSize Analysis  – Test Size Estimation (Test Case Point Analysis)  – Test Effort Estima...
qSize Analysis’ Principles• Size measures the mass and complexity of each  test cycle of a testing project• Test case’s co...
qSize Analysis’ Process[Test Cycle i]                 Count TCPs                   Estimate    Test Case                  ...
Count Size of Test Cycle• Size of a test cycle is the total of TCPs of all test  cases to be executed in that test cycle• ...
Count Size of Test Cycle (cont’d)• Checkpoints  – Checkpoint is the condition in which the tester    verifies whether the ...
Count Size of Test Cycle (cont’d)• Test Setup or Precondition    – Test setup specifies the condition to execute the      ...
Count Size of Test Cycle (cont’d)• Test Data   – Test Data is used to execute the test case   – Four levels of Test Data c...
Count Size of Test Cycle (cont’d)• Adjust TCPs based on Type of Test  – This is an OPTIONAL step  – Adjustment is based on...
Estimate Effort of Test Cycle• Overview  – Two estimation methods     • Based on Test Velocity     • Regression analysis o...
Estimate Effort of Test Cycle (cont’d)• Estimate Effort based on Test Velocity    Effort = Size / Test Velocity  – Test Ve...
Estimate Effort of Test Cycle (cont’d)• Estimate effort using Linear Regression Analysis  – Find out the equation of effor...
Calibrate the qSize Estimation Model• Calibration: a process adjusting parameters for a  model using historical data or ex...
Conclusion• qSize Analysis is an agile approach to estimating  size and effort of test cycle  – Estimate Size in TCP  – Es...
Thank You
References•   [1] Y. Yang, Q. Li, M. Li, Q. Wang, An empirical analysis on distribution patterns of software    maintenanc...
Bug deBug Chennai 2012 Talk - V3 analysis an approach for estimating software testing size and effort by Vu Nguyen
Upcoming SlideShare
Loading in...5
×

Bug deBug Chennai 2012 Talk - V3 analysis an approach for estimating software testing size and effort by Vu Nguyen

921

Published on

Dr. Vu Nguyen is a Director of Software Engineering at QASymphony and a Lecturer at the University of Science, Vietnam National University. At both places, he is involved in developing software tools and performing research in software estimation, testing, maintenance, and process.

Quality assurance management is an essential component of the software development lifecycle. To ensure quality, applicability, and usefulness of a product, development teams must spend considerable time and resources testing, which makes the estimation of the software testing effort, a critical activity. In this talk, we present an approach, namely V3 Analysis, to estimating the size of software testing work. The approach measures the size of a software test case based on its checkpoints, preconditions and test data, as well as the types of testing. We also introduce a supporting toolkit that you can use to estimate testing effort quickly for your projects.

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
921
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
45
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Transcript of "Bug deBug Chennai 2012 Talk - V3 analysis an approach for estimating software testing size and effort by Vu Nguyen"

  1. 1. Agenda• Background and Motivation• qSize Analysis – Test Size Estimation (Test Case Point Analysis) – Test Effort Estimation• Conclusion 2
  2. 2. Background• Software estimation – process of determining the cost, time, staff, and other related attributes of software projects• It is important for the success or failure of software projects• Methods and Metrics – Source Lines of Code (SLOC) – Function Points – COCOMO – Expert Judgment 3
  3. 3. Motivation• Testing accounts for up to 50% of project effort [1]• Current problems – lack of reliable methods designed for estimating size and effort of software testing – vague definitions of testing productivity• Our aim is to introduce – a method for estimating the size and effort of testing activities – a simple toolkit for this estimation process 4
  4. 4. Agenda• Background and Motivation• qSize Analysis – Test Size Estimation (Test Case Point Analysis) – Test Effort Estimation• Conclusion 5
  5. 5. qSize Analysis’ Principles• Size measures the mass and complexity of each test cycle of a testing project• Test case’s complexity is based on – Number of checkpoints – Complexity of test setup or precondition – Complexity of test data• Test Case Point (TCP) is used as size unit• Calibration or model refinement is key to estimating effort – calibration based on feedback from previous testing cycles• Focusing on independent testing (V & V) 6
  6. 6. qSize Analysis’ Process[Test Cycle i] Count TCPs Estimate Test Case Counted Estimated of all Test Testing Test Case Size Effort Effort Cases Update Parameters Historical Data Calibrate Estimation Historical Data of this Project Model Test Cycle Size Actual Effort by Effort Activity …. …. …. …. Test cycle i …. …. …. …. …. …. …. 7
  7. 7. Count Size of Test Cycle• Size of a test cycle is the total of TCPs of all test cases to be executed in that test cycle• Steps: Count Checkpoints Adjust based Test Case Determine Set Unadjusted on Test Type TCPs Up Complexity TCPs (optional) Determine Test Data Complexity 8
  8. 8. Count Size of Test Cycle (cont’d)• Checkpoints – Checkpoint is the condition in which the tester verifies whether the result produced by the target function matches the expected criterion – One test case consists of one or many checkpoints One checkpoint is counted as one TCP 9
  9. 9. Count Size of Test Cycle (cont’d)• Test Setup or Precondition – Test setup specifies the condition to execute the test case – Four levels of Test Setup complexity • Each is assigned a number of TCPs Number Complexity Description of TCP(*) Level 0 None • The set up is not applicable or important to execute the test case • Or, the set up is just reused from the previous test case to continue the current test case 1 Low • The condition for executing the test case is available with some simple modifications required • Or, some simple set-up steps are needed 3 Medium • Some explicit preparation is needed to execute the test case • Or, The condition for executing is available with some additional modifications required • Or, some additional set-up steps are needed 5 High • Heavy hardware and/or software configurations are needed to execute the test case 10
  10. 10. Count Size of Test Cycle (cont’d)• Test Data – Test Data is used to execute the test case – Four levels of Test Data complexity • Each is assigned a number of TCPs Number Complexity Description of TCP (*) Level 0 None • No test data preparation is needed 1 Low • Simple test data is needed and can be created during the test case execution time • Or, the test case uses a slightly modified version of existing test data and requires little or no effort to modify the test data 3 Medium • Test data is deliberately prepared in advance with extra effort to ensure its completeness, comprehensiveness, and consistency 6 High • Test data is prepared in advance with considerable effort to ensure its completeness, comprehensiveness, and consistency • This could include using support tools to generate data and a database to store and manage test data • Scripts may be required to generate test data (*) based on our survey of 18 senior QA engineers. You can adjust according to your project’s experience.
  11. 11. Count Size of Test Cycle (cont’d)• Adjust TCPs based on Type of Test – This is an OPTIONAL step – Adjustment is based on types of test cases • Each type of test case is assigned a weight • Adjusted TCP = Counted TCP x Weight(*) 12
  12. 12. Estimate Effort of Test Cycle• Overview – Two estimation methods • Based on Test Velocity • Regression analysis of Size and Effort of completed test cycles – Effort distributed by activity • Test Planning Each may be • Test Analysis and Design performed • Test Execution multiple • Test Tracking and Reporting times 13
  13. 13. Estimate Effort of Test Cycle (cont’d)• Estimate Effort based on Test Velocity Effort = Size / Test Velocity – Test Velocity is measured as TCP/person-hour • dependent on project • calculated based on data from completed test cycles of the same project 14
  14. 14. Estimate Effort of Test Cycle (cont’d)• Estimate effort using Linear Regression Analysis – Find out the equation of effort and size using similar completed test cycles of a project 100 90 80 Equation of 70 y = 0.072x + 1.640 Size and Effort (PM) 60 Effort 50 40 30 20 10 0 0 100 200 300 400 500 600 700 800 900 1000 Adjusted TCP 15
  15. 15. Calibrate the qSize Estimation Model• Calibration: a process adjusting parameters for a model using historical data or experiences• With qSize Estimation model, you can calibrate: (1) TCP for to each complexity level of Test Setup (2) TCP for to each complexity level of Test Data (3) Test Velocity (4) Effort distribution (5) Weights of test case types• Process can be done with the help of tools Tool Demo 16
  16. 16. Conclusion• qSize Analysis is an agile approach to estimating size and effort of test cycle – Estimate Size in TCP – Estimate Effort using Test Velocity or Regression – An Excel toolkit to simplify the approach• Advantages and experiences learned – Easy to implement – Independent with the level of details of test cases – Found useful• Limitations and future improvements – A new approach – Need more empirical validations 17
  17. 17. Thank You
  18. 18. References• [1] Y. Yang, Q. Li, M. Li, Q. Wang, An empirical analysis on distribution patterns of software maintenance effort, International Conference on Software Maintenance, 2008, pp. 456-459• [2] N. Patel, M. Govindrajan, S. Maharana, S. Ramdas, “Test Case Point Analysis”, Cognizant Technology Solutions, White Paper, 2001• [3] QASymphony: www.qasymphony.com
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×