Software testing metrics assignment

1. Software Testing
Metrics
By David O’ Connor

2. What are Software Testing Metrics?
 The definition of a Metric as stated by SoftwareTestingHelp. (2013) is a
quantitative measure of the degree to which a system, system component, or
process possesses a given attribute. Metrics is defined as Standards of
Measurements.
 SoftwareTestingHelp. (2013) states that software metrics are used to measure
the overall quality of the software project. In simply terms metric is a unit
used for describing an attribute, and is a scale for measurement.
 Definition of Software Metrics given by Paul Goodman: - Software Metrics is
a Measurement Based Technique which is applied to processes, products and
services to supply engineering and management information and working on
the information supplied to improve processes, products and services, if
required.

3. Importance of Metrics
The importance of software testing metrics are stated below taken from Lokesh
Gulechha. (2009):
 Metrics is used to improve the quality and productivity of products and
services and therefore will lead to achieving User Satisfaction.
 Metrics makes it easy for software management to digest one number and
drill down, if required.
 Different Metric(s) trend act as monitor when the process is going out-of-
control.
 Metrics provides improvement for current process.

4. Defect Density Metric
 Defect Density is defined as stated by STF. (2010) as the number of confirmed
defects which have been detected by the software tester in the particular
software/component during a defined period of the development or operation
divided by the overall size of the software application or component.
The ‘defects’ are:
 Confirmed and agreed upon by the software testers and are not just reported.
 Any defects which are dropped during the process are not counted.
The period can be defined as one of the following:
 For a time over a duration for example such as the first month, the quarter,
or even the year.
 For each phase of the life cycle of the software.
 For the overall life cycle of the software.

5. Defect Density Metric continued
 The size can be measured in either Function Points (FP) or by Source Lines of
Code.
 The number of errors found in test cases versus test cases developed and
executed
 (Deflective Test cases/Total Test cases) * 100
 Example: The total number of test cases developed is 1250, total test cases
executed is 1150, total number of test cases passed is 1065, and total number
of scripts failed is 190.
 So Test case Deflect Density is: (190x100)/1150 = 16.5%
 This 16.5% can therefore be called a Test Case Efficiency % which depends
upon the total number of Test cases which were found to be deflective.
 The Defect Density metric is used by testers for comparing the number of
relative defects in the various aspects of software components so that these
high-risk components can be easily identified and the main resources can
therefore be focused towards on these deflects.
 The Defect Density Metric should be gathered by the software tester because
it gives a total percentage of the defective parts in the overall software
component and therefore resources can therefore focus on the critical
deflective parts of the component.

6. Requirement Volatility Metric
 The requirement volatility of software testing has a marked effect on the
overall deliverables of any software product development effort.
 There the requirement volatility metric as stated by P.M.Venkatesh Babu.
(2009) is in place to ensure that the requirements of the software procedure
are normalized or defined properly while estimating the total number of
requirements agreed versus the number of requirements that were changed.
 The Requirement Volatility Metric formula is defined as the following:
 (Number of requirements Added + Deleted + Modified) * 100 / No of original
requirements
 An example of this formula being used as demonstrated by P.M.Venkatesh
Babu. (2009): SVN 1.3 release has 67 requirements initially, later 7 new
requirements are added, 3 requirements are deleted from initial
requirements and modified 11 requirements
 Hence Requirement volatility is calculated as:
(7 + 3 + 11) X 100 / 67 = 31.34 %
 P.M.Venkatesh Babu. (2009) stated that this result means that almost 1/3 of
the requirements changed after the initial identification of requirements.

7. Requirement Volatility Metric
continued
 Requirements metrics such as the requirement volatility metric, traceability,
size and completeness are used by software testers in order to accurately
measure requirements engineering phase of software development so that
this will therefore give the software tester a clear indication of how the
product is performing under the various tests being carried out during the
testing phase in order to ensure the end product is up to the specifications
set out by the client and therefore this metric should be gathered to ensure
requirements are up to standard.

8. Test Execution Productivity (TEP)
 The Test Execution Productivity metric as stated by Lokesh Gulechha. (2009)
gives the test cases execution productivity which on further analysis can give
conclusive result.
 Test Execution Productivity =
 ((Total No. of TC executed (Te)/Execution Efforts (hours))*8))
Execution(s)/Day
 Where Te is calculated as,
Te = Base Test Case + ((T(0.33)*0.33)+(T(0.66)*0.66)+(T(1)*1))
 Where,
Base Test Case = No. of TC executed at least once.
T (1) = No. of TC Retested with 71% to 100% of Total TC steps
T (0.66) = No. of TC Retested with 41% to 70% of Total TC steps
T (0.33) = No. of TC Retested with 1% to 40% of Total TC steps

9. Test Efficiency (TE)
 The Test Efficiency metric as stated by Lokesh Gulechha. (2009) is performed to
determine the overall efficiency of the software testing team in identifying the
defective components of the software.
 The Test Efficiency metric is also used to indicate the number of defects which
were missed out during testing phase and which migrated to the next testing
phase.
 The Test Efficiency of the testing process is calculated by the following formula:
Test Efficiency = DT/(DT+DU)*100%
 Where,
DT = The overall number of valid defects identified during the testing process.
DU = The number of valid defects which have been identified by test user after
the release of software application. In other words, post-testing defect
 The Test Efficiency metric should be gathered by the software tester because it
gives a total percentage of the defective parts which were missed by the tester in
the testing process for the software component and therefore this process will
lead to better efficiency by the software tester when testing future software
components.

10. References
 Lokesh Gulechha. (2009). Software Testing Metrics. Software Testing Metrics.
1 (1), 7-8.
 P.M.Venkatesh Babu. (2009). Software Testing Metrics. Available:
http://www.slideshare.net/pmvenkateshbabu/testing-metrics. Last accessed
20th April 2015.
 SoftwareTestingHelp. (2013). Important Software Test Metrics and
Measurements – Explained with Examples and Graphs. Available:
http://www.softwaretestinghelp.com/software-test-metrics-and-
measurements/. Last accessed 09th Apr 2015.
 STF. (2010). Defect Density Fundamentals. Available:
http://softwaretestingfundamentals.com/defect-density/. Last accessed 18th
April 2015.