Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Testing Metrics

Metrics used in Software Testing

  • Login to see the comments

Testing Metrics

  1. 1. SOFTWARE TESTING METRICS Presenter : P.M.Venkatesh Babu
  2. 2. 2 What is a METRIC • Metrics can be defined as “STANDARDS OF MEASUREMENT • Metric is a unit used for describing or measuring an attribute • Test metrics are the means by which the software quality can be measured • Test metrics provides the visibility into the readiness of the product, and gives clear measurement of the quality and completeness of the product
  3. 3. 3 Why we need Metrics ? • You cannot improve what you cannot measure • You cannot Control what you cannot measure  Without measurement it is impossible to tell whether the process implemented is improving or not  Metrics helps in taking decisions for next phase of activities  Metrics helps in understanding the type of improvement required and helps in taking decisions on process or technology change
  4. 4. 4 Why Metrics in Software Testing? There will be certain questions during and after testing such as :  How long would it take to test ?  How Bad / Good is the product?  How many bugs still remain in the product?  Will testing be completed on time?  Was the testing done effectively?  How much effort went into testing the product? To Answer these questions properly we need some type of measurements and record keeping to justify the answers. This is where the testing metrics comes into picture.
  5. 5. Testing Metrics Life Cycle  Analysis Phase: • Identify the Metrics which has to be generated • Define the identified Metrics  Communication Phase: • Explain the need of the Metrics to the stakeholders • Educate the testing team about the data points for generating the Metrics  Evaluation Phase: • Capture and verify the data used for generating the Metrics • Calculate the Metrics based on the data captured  Reporting Phase : • Develop the Metrics report and distribute to the stakeholders • Take feedback from the stakeholders for any improvements 5
  6. 6. 6 Types Of Metrics Base metrics (Direct Measure) • The Base metrics constitute the raw data gathered by the test Engineers throughout the testing effort • The Base metrics are used to provide project status reports to the Test lead and to the project manager • The Base metrics provide the input data to feed into the formulas used to derive Calculated metrics • Examples of Base metrics are: # of test cases # of test cases executed
  7. 7. 7 Types Of Metrics (contd..) Calculated Metrics (Indirect Measure) • The Calculated Metrics convert the Base metrics data into more useful information • The Calculated Metrics are generally prepared by the Test lead and is used to track the progress of the project at different levels like at Module level, at Tester level and for the project as a whole • The Calculated Metrics provide valuable information that when used and implemented often times leads to significant improvements in the Overall SDLC
  8. 8. 8 Base Metrics and Testing Phases TEST METRIC TESTING PHASE Number of test cases Test Development Phase Number of Test cases Passed Test Execution Phase Number of Test cases Executed Test Execution Phase Number of Test cases Failed Test Execution Phase Number of Test cases under Investigation Test Development Phase Number of Test cases Blocked Test Dev / Execution Phase Number of Test cases Re- executed Regression Phase Number of First run Failures Test execution Phase Total Executions Test Reporting Phase Total Passes and Failures Test Reporting Phase Test case Execution time Test Reporting Phase Test Execution time Test Reporting Phase
  9. 9. 9 Calculated Metrics and Phases The Following Calculated metrics are created at Test Reporting Phase or Post Test Analysis Phase:  % of Test cases Passed  % of Test Coverage  % of Defects corrected  % of Test cases Blocked  % of Rework  % of Test Effectiveness  1st Run Fail Rate  Defect discovery rate  Overall Fail rate
  10. 10. 10 Test case Defect Density The number of errors found in test cases v/s test cases developed and executed • ( Defective Test cases / Total Test cases ) * 100 Example : Total no of test cases developed is 1360, total test cases executed is 1280, total no of test cases passed is 1065, total no of test scripts failed is 215 So Test case Defect Density is : 215 X 100 ------------------------------- = 16.8 % 1280 The 16.8 % value can also be called as Test Case Efficiency % which depends upon the total number of Test cases which found defects
  11. 11. 11 Defect Slippage Ratio No of bugs reported from Production V/S No of defects reported during execution No of Defects slipped / ( Number of Defects Raised – Number Defects Withdrawn) * 100 Example : Customer reported 21 defects, total no of defects found while testing are, total no of Invalid defects are 17 So Slippage ratio is : [ 21 / (267 – 17) ] X 100 = 8.4%
  12. 12. 12 Requirement Volatility Metric This metric ensures that the requirements are normalized or defined properly while estimating No of requirements agreed V/S No of requirements changed • (No of requirements Added + Deleted + Modified) * 100 / No of original requirements Example : SVN 1.3 release has 67 requirements initially, later 7 new requirements are added, 3 requirements are deleted from initial requirements and modified 11 requirements Hence Requirement volatility is calculated as : (7 + 3 + 11 ) X 100 / 67 = 31.34 % This means that almost 1/3 of the requirements changed after the initial identification of requirements
  13. 13. 13 Conclusion • The Test metrics should be reviewed & interpreted on regular basis throughout the test effort and particularly after the application is released into production • There are several key factors in implementing and using the metrics in the organization, beginning with determining the goal for developing the metrics, followed by the identification of metrics to be tracked and ending with sufficient analysis of the resulting data to be able to make changes to the software development lifecycle • - So finally Metrics themselves so not create improvements, they do provide the objective information necessary to understand what changes are necessary to be implemented
  14. 14. 14 Questions ???