1. 1
Copy right 2001 © Distributive Software
This document may not be copied in whole or in part without permission from
Distributive Software.
Using Measurement: Current Standards and Guidance
By: Peter Baxter (Distributive Software) MWG Chair
Introduction
In any process definition, or process improvement effort, selecting the most appropriate
guidance is an essential task. Selecting an inappropriate standard could result in having to
spend significant more rework than is necessary. With respect to measurement, this
article presents three of the principle documents that can vastly simply and focus process
improvements efforts addressing the measurement process.
Working against process implementation is a large and growing number of standards and
guidance. The “standards quagmire”, as it has become known through the efforts of
Incose’s own Sarah Sheard (Software Productivity Consortium), confronts nearly all
systems engineering process practitioners. Because of recent activities within the
measurement community, the challenges of selecting measurement process guidance are
not as formidable as in previous years.
There are three primary pieces of measurement guidance available today: ISO/IEC 15939
Software Measurement Process, Practical Software and Systems Measurement and the
SEI CMMi. The relationship among them is shown in Figure 1: Relationship of Primary
Measurement Guidance.
Figure 1: Relationship of Primary Measurement Guidance
In rare instances, we are lucky enough to witness the agreement of process definition,
process implementation and process assessment initiatives. The three documents above
were updated in roughly the order they are presented: ISO/IEC 15939 was moved
through its standards process, then PSM was updated to reflect ISO/IEC 15939, then the
CMMi adopted the terminology and basic process elements of ISO/IEC 15939. The next
three sections present an overview of these documents.
ISO/IEC 15939
Software Measurement
Process
SEI CMMi (sm)
Practical Software &
Systems Measurement
What?
How?
How Well?

2. 2
Copy right 2001 © Distributive Software
This document may not be copied in whole or in part without permission from
Distributive Software.
ISO/IEC 15939 Software Measurement Process
The International Organization of Standards (ISO) Subcommittee on Software
Engineering (SC/7) is the developer of a measurement standard, titled “ISO/IEC 15939 –
Information Technology – Software Engineering – Software Measurement Process.”
From the ISO/IEC 15939 standard, the primary goal of the document is to “… identify
the activities and tasks that are necessary to successfully identify, define, select, apply,
and improve software measurement within an overall project or organizational
measurement structure.”
While the title of this standard contains the word “software” (twice), the standards should
not be ignored by the systems engineering community. The SC/7 subcommittee develops
both software and systems engineering products, and is in the process of formally
broadening its scope to recognize its contribution to systems engineering. In addition, the
ISO/IEC 15939 document does not contain any process elements that are unique to
software engineering. It was the intent of ISO to apply the standard to systems
engineering – only the name remains to be changed.
The purpose of this standard is to ensure that an organization performs the following
measurement-related actions:
• organizational commitment for measurement will be established and sustained;
• the information needs of technical and management processes will be identified;
• an appropriate set of measures, driven by the information needs will be identified
and/or developed;
• measurement activities will be identified;
• identified measurement activities will be planned;
• the required data will be collected, stored, analyzed, and the results interpreted;
• information products will be used to support decisions and provide an objective
basis for communication;
• the measurement process and measures will be evaluated;
• improvements will be communicated to the measurement process owner.
This ISO standard describes the organizational elements required to support a
measurement process. ISO/IEC 15939 organizes measurement into four primary
measurement activities, establish capability, plan measurement, perform the measurement
process, and evaluate measurement. The four activities are depicted in Figure 2: ISO/IEC
15939 Measurement Process , along with the external users and information needs to
form the basis of “requirements” for the measurement process. In the standard, the four
activities are decomposed into tasks, with tasks descriptions containing more detailed
information on what it required to comply with the standard.

3. 3
Copy right 2001 © Distributive Software
This document may not be copied in whole or in part without permission from
Distributive Software.
Figure 2: ISO/IEC 15939 Measurement Process
This standard defines measurement in two primary parts: a process model, as well as an
information model. The process model depicts the relationship of activities and tasks.
The information model describes the relationship of the measurement entities, in
particular, a concise vocabulary for describing the elements commonly found in a
measurement process.
A key point of terminology: The word “metric” is not found in the standard, and has been
replaced by “measure”. The subcommittee found that no consistent or agreeable
definition of the word “metric” could be found, and, hence, it was desirable to use a more
appropriate, less vague, word than to attempt to correct decades of misuse.
If your organization or company is an ISO member, you can obtain a copy of ISO/IEC
15939 by contacting your organization’s ISO representative. The standard is a digestible
40 pages, of which 19 pages is informative appendices. The document is presently in the
Final Draft International Standard stage, the last version before a Draft International
Standard is balloted and becomes an international standard. The completion of the
standards process, and release of final standard, is anticipated within the next 12 months.
Practical Software and Systems Measurement Guidebook
The PSM Program Office is sponsored by the Department of Defense and the US Army.
The goal of the project is to provide project managers with the objective information
needed to successfully meet cost, schedule, and technical objectives on programs.

4. 4
Copy right 2001 © Distributive Software
This document may not be copied in whole or in part without permission from
Distributive Software.
PSM is based on actual measurement experience on DoD, Government and Industry
programs. Measurement professionals from a wide variety of organizations participate in
the project. PSM represents the best measurement practices used within the software and
system acquisition and engineering communities.
The primary product of PSM is an essential measurement guidebook: "Practical Software
and Systems Measurement: A Foundation for Objective Project Management.” This
guidebook was developed to meet today's software and system technical and management
challenges. It describes an issue-driven measurement process that will address the unique
technical and business goals of your organization. The guidance in PSM represents the
best practices used by measurement professionals within the software and system
acquisition and engineering communities.
1. PSM treats measurement as a flexible process - not a pre-defined list of graphs or
reports. The process is adapted to address the specific software and system issues,
objectives, and information requirements unique to each program. The PSM
measurement process is defined by a set of nine best practices, called
measurement principles.
2. PSM integrates the measurement requirements into the software and system
developer's process. The measurement set is tailored for each program to ensure
that the measurement process is not only cost effective, but also that the measures
provide meaningful and usable results.
3. PSM defines an issue-driven analysis approach which helps the Project Manager
make informed software and system decisions. The PSM analysis approach
incorporates the use of multiple measures and non-quantitative program data to
identify and evaluate issues, including risks and problems.
4. PSM supports current software and system acquisition and measurement policy.
PSM defines a non-prescriptive measurement approach as required by acquisition
reform, and provides a mechanism for the objective communications necessary
within an Integrated Product Team (IPT).
5. PSM provides a basis for enterprise level management. PSM is designed to help
put measurement into practice at the project level, thereby providing the data
required to address enterprise level performance, process improvement, and
business-related questions. PSM also supports Information Technology (IT)
performance measurement requirements.
6. PSM is currently being used by DoD, Government, and Industry. PSM has been
endorsed by government Acquisition Executives, and is a key element of the
OUSD (A&T) software initiative. The PSM methodology is also being adopted by
government and industry organizations.
While ISO/IEC 15939 takes a 10,000 foot approach, the PSM Guidebook takes a 100 foot
approach: explaining a measurement concept, providing guidance on its application and
then demonstrating the application of the concept to a common measurement need. The
PSM guidebook is divided into seven parts, each focusing on a specific measurement
process challenge. The Guidebook style facilitates locating essential elements of the

5. 5
Copy right 2001 © Distributive Software
This document may not be copied in whole or in part without permission from
Distributive Software.
measurement process. While the Guidebook is rich in content, it is designed to be easy to
navigate and quickly find what you need.
PSM holds an annual conference, attracting measurement professionals from all areas of
academia, industry and government. The PSM user conference is a combination of best
practice presentations, research and workshops, with a high degree of technical exchange
and very little fluff. Finally, PSM is an open measurement organization that encourages
participations from measurement professionals in commercial, government, and
academia.
Software Engineering Institute Capability Maturity Model, Integrated
As a function of rating an organization’s maturity, the CMMi requires an organization to
demonstrate specific (and measurable) process capabilities. The “i” in CMMi is for
“integrated”: an approach for integrating the previously separate software and systems
CMMs into a consolidated model. In addition to integrating software and systems, the
CMMi provides both the continuous and staged representation of the model (often called
“constage-uous”), as well as a treatment of integrated product development and
integrated product teams.
With respect to measurement, the CMMi adds a great deal of importance to the use of
measurement in planning, controlling and improving your organizations systems
activities. The amount of information and level of detail given to measurement in the
CMMi is significantly greater than the previous family of CMMs. The CMMi defines a
number of support process areas, with measurement, called “Measurement and Analysis”,
being one of them. The three SPAs are shown in Figure 3: CMMi Support Process
Areas.
Figure 3: CMMi Support Process Areas
All Process Areas
Configuration
Management
Measurement
&
Analysis
Process/
Product QA
information
needs
measurements,
analysis

6. 6
Copy right 2001 © Distributive Software
This document may not be copied in whole or in part without permission from
Distributive Software.
Each support process area is described in terms of specific goals, generic goals and
generic practices by goal. From the current CMMi draft, the M&A process area has two
specific goals as follows:
1. Align Measurement and Analysis Activities - Measurement objectives and
practices are aligned with identified information needs and objectives.
2. Provide Measurement Results - Measurement results that address identified
information needs and objectives are provided.
The first Measurement & Analysis specific goal dictates that the purpose and use of
measurement be consistent with the information needs of the organization deploying it.
The second specific goal states that, in addition to defining the information needs, the
measurement process must deliver measurement products that meet those needs.
The Measurement & Analysis support process also contains a set of generic goals. The
CMMi groups capabilities common to multiple (or all) processes and process areas into
generic goals. For example, the goal of being a quantitatively managed process applies to
all processes within the CMMi. From the current CMMi draft, the Measurement &
Analysis generic goals are as follows:
1. Achieve Specific Goals - The process supports and enables achievement of the
specific goals of the process area by transforming identifiable input work products
to produce identifiable output work products.
2. Institutionalize a Managed Process - The process is institutionalized as a managed
process.
3. Institutionalize a Defined Process - The process is institutionalized as a defined
process.
4. Institutionalize a Quantitatively Managed Process - The process is
institutionalized as a quantitatively managed process.
5. Institutionalize an Optimizing Process - The process is institutionalized as an
optimizing process.
The CMMi then references the Measurement & Analysis support process in virtually
every other management and engineering process area (PA). Within each individual PA,
specific information needs and measurement products are presented. For example, in the
Project Monitoring and Control process area (at level 2 in the staged representation), the
first specific practice (i.e. SP 1.1.) states that the organization must “Monitor the actual
values of the project planning parameters against the project plan”, and then includes a
reference to the Measurement & Analysis support process area. The organizational
capability to “monitor actual values” becomes an information need that is addressed by
the organization’s measurement process. All CMMi process areas depend on
measurement to provide the information needed to plan, monitor, control,and manage the
associated process.
The CMMi is presently in draft form, with an expected release date of early fourth
quarter. The current CMMi draft is publicly available from the SEI. The SEI provides a
wealth of CMMi resources, including availability and dates of releases, on their web site.

7. 7
Copy right 2001 © Distributive Software
This document may not be copied in whole or in part without permission from
Distributive Software.
Summary
ISO/IEC 15939, the PSM Guidebook and the CMMi represent the primary sources of
measurement process information presently available. These are excellent places to start
if you are considering starting or expanding a measurement process. As you read and
implement them, you will find that they contain a consistent process and information
model and use consistent terminology. Perhaps, measurement is no longer in the
quagmire.