Software quality metrics provide important insights into software testing efforts and processes. They can help evaluate products and processes against goals, control resources, and predict future attributes. There are three categories of metrics: process, product, and project. Process metrics measure testing efficiency and effectiveness. Product metrics depict product characteristics like size and quality. Project metrics measure schedule, cost, productivity, and code quality. Choosing metrics based on organizational goals and providing feedback are best practices for an effective metrics program.

1. Importance of Software Quality Metrics, Piyush Sohaney
Abstract
Over time, Software products have become more complex
due to comprehensive product functionality, advance
architectures, and gigantic supported configuration. Also with
increasing business and margin pressure, the projects are
required to be accomplished in lesser amount of time with
fewer people. Metrics can provide quick insight into the
status of software testing effort, hence resulting in better
control through smart decision making.
Metrics serve as important indicator of the efficiency and effectiveness of software process. Analysis of
defined metrics helps identify area of improvement and devise subsequent actions. Software metrics act
as indicators and provide information so people can make more informed decisions and intelligent
choices. An individual metric performs one of four functions. Metrics can help us Understand more
about our software products, processes, and services. Metrics can be used to Evaluate our software
products, processes, and services against established standards and goals. Metrics can provide the
information we need to Control resources and processes used to produce our software. Metrics can be
used to Predict attributes of software entities in the future.
What Are Software Metrics?
Software metrics are an integral part of the state-of-the-practice in software engineering. More and
more customers are specifying software and/or quality metrics reporting as part of their contractual
requirements. Industry standards like ISO 9000 and industry models like the Software Engineering
Institute’s (SEI) Capability Maturity Model Integrated (CMMI®) include measurement. Companies are
using metrics to better understand, track, control and predict software projects, processes and
products. Software metrics can provide the information needed by engineers for technical decisions as
well as information required by management. If a metric is to provide useful information, everyone
involved in selecting, designing, implementing, collecting, and utilizing it must understand its definition
and purpose.
Software metrics programs must be designed to provide the specific information necessary to manage
software projects and improve software engineering processes and services. Organizational, project, and
task goals are determined in advance and then metrics are selected based on those goals. The metrics
are used to determine our effectiveness in meeting these goals.

2. Software Quality Metrics
Software metrics can be broadly classified into process, product and Project metrics. In this document
we discussed various software quality metrics falling under each category with the importance.
Process quality Metrics: Process metrics can be used to measure and enhance process of software
development, maintenance and testing. Metrics play a important role, when it comes to measuring a
process from various dimensions, assessing it and target improvements. Software testing process, being
the overall software engineering life cycle has been a prime focus area with respect to enhancing the
efficiency and effectiveness.
Process
Efficiency
 Test Progress Curve: This metrics depicts the planned testing progress over time
in terms of no. of test cases completed successfully in different milestones /
Weekly basis.
 Cost of Testing: Efficient utilization of the effort allocated for software testing is
crucial towards ensuring maximum test coverage resulting in high-quality end-
product.
 Test execution productivity trends
 Defect turnaround time
Process
effectiveness
 Test coverage
 Cost of Quality
 Failed test cases Vs. Total Test cases executed
 Component wise test coverage
 Defect removal Efficiency
 Defect detection at different milestone
 Defect Quality
 Defect Efficiency
o Milestone wise defect identification rate
o No fix Vs. Actual code Fix
o Defect turnaround time
o Internal Vs. External reported bugs
o Customer defect – Product/Time/Component wise
 Code churn – How much new line of code added Vs. How much line of code
modified/rework
 Code Coverage: Code coverage metrics to help determine when the verification task is
completed. This includes code coverage, which measures whether lines of code have been
executed, as well as functional coverage, which defines scenarios that should be tested
 Defect/Bug escape rate:
o Customer Escape rate = (Number of bugs filed by customer *100)/ Total
number of bugs filed internally
o Internal milestone bug escapes

3. Product Quality Metrics: Product metrics depict the characteristics of the product such as its size,
complexity, design aspects, performance and quality.
 Test defects arrival rate
 Component wise defect distribution
 Show stopper Defects trends
 Test Automatable
 Test Automation coverage
 Effort saving through Automation / ROI of Automation
Project Metrics: Project metrics depict various characteristics of project and its execution.
Schedule
and Cost
SPI (Schedule Performance Index) is the accuracy of estimating the actual project
schedule
SPI = Actual Project Duration / Estimated Project Duration
CPI (Cost Performance Index) is the accuracy of estimating the actual project effort
CPI = Actual Project Effort / Estimated Project Effort
Productivity Development Productivity (DP): The productivity of SW development projects is
measured based on source code size. Line Of Code (LOC) is typically used to predict the
amount of effort that will be required to develop a program, as well as to
estimate programming productivity or maintainability once the software is produced.
DP = (FCL – DCL) * Complexity / Effort
Code Quality (CQ): Its determine based on number of failures produced by the new
development effort
CQ = #Bug Fixes / FCL
Test Productivity (TP): “What is the productivity of SW testing based on bug counts and
test cases?
TP = (#Total Valid Bugs / #Test Case Runs) / MMtest
 FCL = Actual line of code added and modified base on new feature requests
 DCL =Actual line of code added and modified to fix the bugs (rework)
 Effort = Total Man Months (MMtotal) dedicated to the project
 Complexity = MMdesign + MMtest / MMtotal

4. Best Practices and basic rules while working with Metrics:
1. Ensuring Metric Usefulness
2. Measurement is not the goal: The goal is improvement through measurement, analysis and
feedback
3. Don't measure individuals: Individual productivity measures are the classic example of this
mistake. Remember that we often give our best people the hardest work and then expect them
to mentor others in the group. If we measure productivity in lines of code per hour, these
people may concentrate on their own work to the detriment of the team and the project. Even
worse, they may come up with unique ways of programming the same function in many extra
lines of code. Focus on processes and products, not people.
4. Never use metrics as a "stick": The first time we use a metric against an individual or a group is
the last time we get valid data.
5. Don't ignore the data: A sure way to kill a metrics program is to ignore the data when making
decisions. "Support your people when their reports are backed by data useful to the
organization"
6. Never use only one metric: Software is complex and multifaceted. A metrics program must
reflect that complexity. A balance must be maintained between cost, quality and schedule
attributes to meet all of the customer's needs. Focusing on any one single metric can cause the
Useful
MetricsSimple and
Precise
Definitions
Objective
Cost
Effective
Informative
Goals
1. Increase
SW
Productivity
3. Increase
Defect
Containment
4. Improve
Project
Planning
2. Increase
SW
Reliability

5. attribute being measured to improve at the expense of other attributes. Copyright © 2005 The
Westfall Team. All Rights Reserved
7. Select metrics based on goals: Metrics act as a big spotlight focusing attention on the area being
measured. By aligning our metrics with our goals, we are focusing people’s attention on the
things that are important to us.
8. Provide feedback: Providing regular feedback to the team about the data they help collect has
several benefits:
9. Obtain "buy-in": To have 'buy-in" to both the goals and the metrics in a measurement program,
team members need to have a feeling of ownership. Participating in the definition of the metrics
will enhance this feeling of ownership.
Conclusion
A metrics program that is based on the goals of an organization will help communicate, measure
progress towards, and eventually attain those goals. People will work to accomplish what they believe to
be important. Well-designed metrics with documented objectives can help an organization obtain the
information it needs to continue to improve its software products, processes, and services while
maintaining a focus on what is important. A practical, systematic, start-to-finish method of selecting,
designing, and implementing software metrics is a valuable aid.
References:
[Goodman-93] Paul Goodman, 1993, Practical Implementation of Software Metrics, McGraw Hill,
London.
[IEEE-610] IEEE Standard Glossary of Software Engineering Terminology, ANSI/IEEE Std 610-
1990, The Institute of Electrical and Electronics Engineers, New York, NY, 1990.
About Author:
Piyush Sohaney is Senior Quality Manager with AMD. He has over 17 Years of IT experience in Software
Design, Development, Testing and Implementation. He is BE and MBA in Operation research, currently
pursuing General Management program from IIM Bangalore. In 8+ year experience with AMD, played a
key role in improving the AMD SW quality, setting up automation framework, improving process
efficiency, optimizing effort & cost and forming the strong offshore team with different QA functions.
Prior to AMD, 7 year worked on different development, database management, Web application testing
for various MNCs like Xerox, GE Medical system.
For more information/question, contact psohaney@gmail.com