This document provides an overview and introduction to software metrics. It discusses measurement concepts and why measurement is important for software engineering. It covers topics like the basics of measurement, collecting metrics data, analyzing data, and measuring internal and external attributes of software. Specific metrics discussed include size, structure, complexity, reliability, and test coverage. The document is intended to introduce readers to fundamental software metrics concepts.
2. Overview
Part I: Introduction to Software Metrics
Part II: Introduction to SQUID
Part III: Case Studies
Part IV: SQUID Tool Demo
Massimo Felici - Software Metrics, 1999 2
4. Introduction to Software Metrics
Overview
• Measurement: what is it and why do it?
• The basic of measurement
• Software metrics data collection
• Analysing software measurement data
• Measuring internal product attributes: size
• Measuring internal product attribute: structure
• Measuring external product attribute
• Software reliability: measurement and prediction
Massimo Felici - Software Metrics, 1999 4
5. Measurement: what is it and why do it?
“Measurement is the process by which
numbers or symbols are assigned to attributes
of entities in the real world in such a way as to
describe them according to clearly defined
rules.”
Massimo Felici - Software Metrics, 1999 5
7. Measurement in Software Engineering
• Neglect of measurement in software engineering (e.g.,
unclear objectives, unclear costs, prediction vs
production, not evidence for new technologies)
• Objectives for software measurement (e.g., manager and
engineer viewpoints)
• Measurement for understanding, control and improvement
Massimo Felici - Software Metrics, 1999 7
8. Diverse Quality Viewpoints
I want it easy to
understand and cheap
to modify
I want low risk and
an easy demonstration
Massimo Felici - Software Metrics, 1999 8
9. Diverse Quality Needs
Different Systems
or Different Parts Management
of Same System Statistics
Food
Don’t Apply Critical
Qualities to Total Systems
Guidance
Engine Control Seat Bookings
Massimo Felici - Software Metrics, 1999 9
10. User View
• Software that meets user’s needs
• Based around users tasks
– Sometimes very context dependent
• Supported by
– Reliability modelling
– Performance modelling
– Usability laboratories
Massimo Felici - Software Metrics, 1999 10
11. The Scope of Software Metrics
• Cost and effort estimation
• Productivity models and measures
• Data collection
• Quality models and measures (ISO 9126)
• Reliability models
• Performance evaluation and models
• Structural and complexity metrics
• Management by metrics
• Evaluation of methods and tools
• Capability maturity assessment (CMM)
Massimo Felici - Software Metrics, 1999 11
12. The Basics of Measurement
• Empirical relations (that is, mappings from the empirical
real world to a formal mathematical world)
• The rules of mapping
“measurement is defined as a mapping from the
empirical world to the formal relational world.
Consequently, a measure is the number or symbol
assigned to an entity by this mapping in order to
characterise an attribute”
• The representation condition of measurement:
“a mapping has to preserve an empirical relation by a
numerical relation”
Massimo Felici - Software Metrics, 1999 12
13. An Example of Measure
Mr. A Mr. B Mr. C
Empirical relation:
A taller than B taller than C
Mathematical relation:
height of A greater than height of B greater than height of C
height_A=1,7m > height_B=1,6m > height_C=1,5m
Massimo Felici - Software Metrics, 1999 13
14. Measurement and Models
• Defining attributes (e.g., size, complexity, etc.)
• Direct and indirect measurement
• Measurement for prediction
“A prediction system consists of a mathematical model
together with a set of prediction procedures for determining
unknown parameters and interpreting results” (Littewood, 1988)
Massimo Felici - Software Metrics, 1999 14
15. Examples of Common Indirect Measures
Programmer productivity LOC produced
person months of effort
Module defect density number of defects
module size
Defect detection efficiency number of defects detected
total number of defects
Requirements stability number of initial requiremen ts
total number of requiremen ts
Test effectiveness ratio number of items covered
total number of items
System spoilage effort spent fixing faults
total project effort
Massimo Felici - Software Metrics, 1999 15
16. Measurement Scales and Scale Types
• Nominal (e.g., labelling, classifying entities)
• Ordinal (e.g., level of severity)
• Interval (e.g., temperature in degree Celsius)
• Ratio (e.g., length of physical objects)
• Absolute (e.g., counting entities)
Massimo Felici - Software Metrics, 1999 16
17. An Example of Interval Scale
• Measure air temperature on a Fahrenheit or Celsius scale
• Transform Celsius to Fahrenheit by using the transformation
F 9 5 C 32
• Compare intervals between the two scales
• Not compare ratio of the two scales (e.g., by two-thirds, 50%, etc.)
Massimo Felici - Software Metrics, 1999 17
18. Meaningfulness in Measurement
“A statement involving measurement is
meaningful if its truth value is invariant to
transformations of allowable scales.”
Massimo Felici - Software Metrics, 1999 18
19. Software Metrics Data Collection
What is good data?
Correctness: Are they correct?
Accuracy: Are they accurate?
Precision: Are they appropriately precise?
Consistent: Are they consistent?
Timed: Are they associated with a particular
activity or time period?
Repeatable: Can they be replicated?
Massimo Felici - Software Metrics, 1999 19
20. How to define data
Derived Attributes
Value Indirect
Measurement
Refined Data
Direct
Raw Data Measurement
Process
Product
Resource
Massimo Felici - Software Metrics, 1999 20
21. Terminology
error
• A fault occurs when a human error results in
a mistake in some software product. That is,
the fault is the encoding of the human error.
fault
• A failure is the departure of a system from its
required behaviour.
• In order to remove faults we make changes. failure
One fault can result in multiple changes to
one product (such as changing several section
of a piece of code) or multiple changes to
multiple products (such as change to
requirements, design and code). changes
Massimo Felici - Software Metrics, 1999 21
22. What do we need to record of a problem?
Location: Where did the problem occur?
Timing: When did it occur?
Symptom: What was observed?
End result: Which consequence resulted?
Mechanism: How did it occur?
Cause: Why did it occur?
Severity: How much was the user affected?
Cost: How much did it cost?
Massimo Felici - Software Metrics, 1999 22
23. Notice
Location • This is only a generic schema for the information
relevant to all type of incidents.
Timing
• We can collect information about errors, faults,
Symptom
failures and changes.
End result
• Depending on the particular measurement
Mechanism objectives (and the current sophistication of the
metric program), it may not be necessary to
Cause
collect all the information for each incident.
Severity
• You cannot use the same form to capture fault and
Cost failure data
Massimo Felici - Software Metrics, 1999 23
24. What do we need else?
• Orthogonal classification
• Levels of granularity
• Data collection forms
• Database
Massimo Felici - Software Metrics, 1999 24
25. Analysing Software-Measurement Data
• The nature of the data: sampling,
population and data distribution
• Purpose of the experiment: confirming a
theory or exploring a relationship
• Design consideration
Massimo Felici - Software Metrics, 1999 25
26. Examples of Simple Analysis Techniques
• Box plots
• Scatter plots
• Control charts
• Bar graphs
• Correlation
• Fitting
Massimo Felici - Software Metrics, 1999 26
27. Box Plots
Lower
Median Upper Graphical
Tail Tail Outliners representation of the
spread of data
x x
Lower Quartile Upper Quartile
0 20 40 60 80 100 120 140 160 180 200 220
Control Flow Paths
Median: middle-ranked item in the data set
Upper Quartile: median of the values that are more than Median
Lower Quartile: median of the values that are less than Median
Box Length = Upper Quartile - Lower Quartile
Upper Tail=Upper Quartile + 1.5 Box Length
Lower Tail=Lower Quartile - 1.5 Box Length
Massimo Felici - Software Metrics, 1999 27
28. Scatter Plot
18
16
14
Number of faults
12
10
8
6
4
2
0
0 100 200 300 400 500 600
LOC
Scatter plots are used to represent
data for which two measures are
given for each entity.
Massimo Felici - Software Metrics, 1999 28
29. Control Charts
• Help you to see when
your data are within
acceptable bounds.
• By watching the data
trends over time, you
can decide whether to
prevent problems before
they occur.
Massimo Felici - Software Metrics, 1999 29
32. Measuring Internal Product Attributes: Size
The combination of
different measures of the
same attribute can give
additional information
height_A height_B
weight_A weight_B
Massimo Felici - Software Metrics, 1999 32
33. Measuring Length: Line Of Code (LOC)
• Definition of Line Of Code
• Programming language
• Non-textual or external code
• Specifications and designs
• Predicting length
• Reuse of code
Massimo Felici - Software Metrics, 1999 33
35. Measuring Size: Complexity
• Problem with indirect measures (e.g., elapsed time)
• Complexity of a problem (e.g., algorithm complexity)
Massimo Felici - Software Metrics, 1999 35
36. Measuring Internal Product Attributes: Structure
• The control flow addresses the sequence in which
instructions are executed in a program.
• Data flow follows the trail of data item as it is
created or handled by a program.
• Data structure if the organisation of data itself,
independent of the program.
Massimo Felici - Software Metrics, 1999 36
37. Flowgraph Model of Structure
A flowgraph is a directed graph in which two nodes,
the start node and the stop node, obey special
properties. The stop node has not leaving arcs, and
every node lies on some path from the start node to the
stop node.
Massimo Felici - Software Metrics, 1999 37
38. Common Flowgraphs from Program Structure Models
X1 X2 X3 Xn-1 Xn X1; X2; X3; …; Xn-1; Xn;
A if A then X A if A then X
t t f else Y
X f X Y
A case A of A while A do X X
a1 a2 an a1: X1
a2: X2 f f
X1 X2 ··· Xn
…
t
t
an: Xn X repeat X until A A
Massimo Felici - Software Metrics, 1999 38
39. Sequencing and Nesting
• Let F1 and F2 be two flowgraphs. Then the sequence of F1
and F2 is the flowgraph formed by merging the stop node of
F1 with the start node of F2.
• Let F1 and F2 be two flowgraphs. Suppose F1 has a procedure
node x. Then the nesting of F2 onto F1 at x is the flowgraph
formed from F1 by replacing the arc from x with the whole of
F2 .
Massimo Felici - Software Metrics, 1999 39
40. Prime Flowgraphs
Prime flowgraphs are flowgraphs
that cannot be decomposed non-
trivially by sequencing and nesting.
Massimo Felici - Software Metrics, 1999 40
41. The Generalised Notion of Structuredness
Let S be a family of prime flowgraphs. A family of
graphs is S-structured (or, more simply, that the members
are S-graphs) if it satisfies the following recursive rules:
• Each member of S is S-structured.
• If F and F’ are S-structured graphs, then so are
- the sequence of F and F’
- the nesting of F’ on to F (whenever nesting of F’ onto
F is defined).
• No flowgraph is an S-structured graph unless it can be
shown to be generated by a finite number of applications of
the above step.
Massimo Felici - Software Metrics, 1999 41
42. Prime Decomposition
Prime decomposition theorem:
Every flowgraph has a unique
decomposition into a hierarchy of
primes. (Fenton and Whitty, 1986)
Massimo Felici - Software Metrics, 1999 42
43. Example of Hierarchical Measures
Number of nodes measure n
M1 : n ( F ) number of node in F for each prime F
k
M 2 : n F1 ;; Fk n Fi k 1
i 1
k
M 3 : n F F1 ; Fk nF n Fi 2k for each prime F
i 1
Massimo Felici - Software Metrics, 1999 43
44. Hierarchical Measures
Number of edges measure e
M1 : e F number of edges in F for each prime F
k
M 2 : e F1 ;; Fk n Fi
i 1
k
M 3 : e F F1 ;; Fk eF e Fi k for each prime F
i 1
Massimo Felici - Software Metrics, 1999 44
45. Another Measure of Complexity
McCabe’s cyclomatic complexity measure
vF e n 2
Massimo Felici - Software Metrics, 1999 45
46. Test Strategies
• Black-box (or closed-box) testing, in
which the test cases are derived from
the specification or requirements
without reference to the code itself or
its structure.
• White-box (or open-box) testing, in
which the test cases are selected based
on knowledge of the internal program
structure.
Massimo Felici - Software Metrics, 1999 46
47. Test Coverage Strategies
Statement coverage, in which every program
statement is executed al least once.
Branch (or edge) coverage, in which every
edge lies on at least one path of the test cases.
Path coverage, in which every possible
program path is executed at least once.
Massimo Felici - Software Metrics, 1999 47
48. Test Measures
Let T be a testing strategies that requires us to cover a class
of objects (e.g., paths, simple paths, linearly independent
paths, edges, statements, etc.). Two of the most important
test measures are
Minimum number of test cases = minimal number of
paths required to exercise all the objects T at least once.
Test Effectiveness Ratio (TER)
number of T objects exercised at least once
TER
total number of T objects
Massimo Felici - Software Metrics, 1999 48
49. Modularity and Information Flow Attributes
“A module is a contiguous sequence of
program statements, bounded by boundary
elements, having an aggregate identifier.”
(Yourdon and Constantine, 1979)
Massimo Felici - Software Metrics, 1999 49
50. Module Call-Graph
Main
scores
scores
Read_Scores Average
scores average
average
Calc_Av Print_Av
Massimo Felici - Software Metrics, 1999 50
51. Coupling
“Coupling is the degree of interdependence between
modules.” (Yourdon and Constantine, 1979)
Massimo Felici - Software Metrics, 1999 51
52. An Ordinal Classification for Coupling
No coupling relation R0: x and y have no communication.
Data coupling relation R1: x and y communicate by
parameters.
Stamp coupling relation R2: x and y accept the same
record type as a parameter.
Control coupling relation R3: x passes a parameter to y
with the intention of controlling its behaviour (e.g., flag).
Common coupling relation R4: x and y refer to the same
global data.
Content coupling relation R5: x refers to the inside of y.
Massimo Felici - Software Metrics, 1999 52
53. A Measure of Coupling
n
c x, y i
n 1
i worst coupling relation Ri between x and y
n number of interconne ctions between x and y
(Fenton and Melton, 1990)
Massimo Felici - Software Metrics, 1999 53
54. Cohesion
“The cohesion of a module is the extent
to which its individual components are
needed to perform the same task.”
Massimo Felici - Software Metrics, 1999 54
55. An Ordinal Scale for Cohesion
Functional: the module performs a single well-defined function.
Sequential: the module performs more than one function, but they
occur in an order prescribed by the specification.
Communicational: the module performs multiple functions, but all
on the same body of data.
Procedural: the module performs more than one function, and they
are related only to a general procedure affected by the software.
Temporal: the module performs more than one function, and they
are related only by the fact that they must occur within the same
timespan.
Logical: the module performs more than one function, and they are
related only logically.
Coincidental: the module performs more than one function, and
they are unrelated.
Massimo Felici - Software Metrics, 1999 55
56. A Simple Measure of Cohesion
number of modules having functional cohesion
Cohesion ratio
total number of modules
Massimo Felici - Software Metrics, 1999 56
57. Information Flow
Local direct flow exists if either a module invokes a second
module and passes information to it or the invoked module
returns a result to the caller.
Local indirect flow exists if the invoked module returns
information that is subsequently passed to a second invoked
module.
Global flow exists if information flows from one module to
another via a global data structure.
Massimo Felici - Software Metrics, 1999 57
58. Measures of Information Flow
fan-in(M) is the number of local flows that terminate at M, plus the
number of data structures from which information is retrieved by M.
fan-out(M) is the number of local flows that emanate from M, plus the
number of data structures that are updated by M.
Information flow complexity(M)=length(M)*(fan-in(M)*fan-out(M))2,
Henry and Kafura
Shepperd complexity(M)=(fan-in(M)*fan-out(M))2, Shepperd
Massimo Felici - Software Metrics, 1999 58
59. Object-Oriented Metrics
Metric 1: weighted methods per class (WMC)
Metric 2: depth of inheritance tree (DIT)
Metric 3: number of children (NOC)
Metric 4: coupling between object classes (CBO)
Metric 5: response for class (RFC)
Metric 6: lack of cohesion metric (LCOM)
Massimo Felici - Software Metrics, 1999 59
60. Data Structure Metrics
There have been few attempts to define measures of actual
data items and their structure. Two examples:
Number of distinct operand = number of variables + number of
unique constants + number of labels
Database size in bytes or characteres
D/P
Program size in delivered source instructio n (DSI)
(Boehm)
Massimo Felici - Software Metrics, 1999 60
61. Measuring External Product Attributes
• What is quality for the final product?
• What is quality for the end user?
• How can external quality be measured?
• What is the relation between external and internal attributes?
• Is quality a general concept?
Massimo Felici - Software Metrics, 1999 61
62. Modeling Software Quality: ISO 9126
Suitability Functionality, a set of attributes that bear on the existence of
Functionality
Accuracy
a set of functions and their specified properties. The
Interoperability
functions are those that satisfy stated or implied needs.
Security
Reliability
Maturity Reliability, a set of attributes that bear on the capability of
Fault tolerance software to maintain its level of performance under stated
Recoverability
conditions for a stated period of time.
Understandability
Usability
Learnability Usability, a set of attributes that bear on the effort needed
Operability for use, and on the individual assessment of such use, by a
Efficiency Time behaviour stated or implied set of users.
Resouce behaviour
Analysability Efficiency, a set of attributes that bear on the relationship
Maintainability Changeability between the level of performance of the software and the
Stability amount of resources used, under stated conditions.
Testability
Adaptability
Maintainability, a set of attributes that bear on the effort
Portability
Installability
needed to make specified modifications.
Portability, a set of attributes that bear on the ability of
Conformance
Replaceability
software to be transferred from one environment to another.
Massimo Felici - Software Metrics, 1999 62
64. Defect Density Measure
number of known defects
defect density
product size
Massimo Felici - Software Metrics, 1999 64
65. Defect Density Warnings
• definition of defect
• distinction between defect density and defect rate
• definition of software size
• interpretation of defect density measures (defect finding)
• no faults free
• seriousness of faults
• variability of users
Massimo Felici - Software Metrics, 1999 65
66. Usability Measures
• difficult to define usability
• internal measures of usability
• the external view of usability is environment dependent
Massimo Felici - Software Metrics, 1999 66
68. External View of Maintainability
Mean Time To Repair (MTTR)
• problem recognition time
• administrative delay time
• maintenance tools collection time
• problem analysis time
• change specification time
• changing time (including testing and review)
Massimo Felici - Software Metrics, 1999 68
69. Other Environmental-Dependent
Maintainability Measures
• ratio of total change implementation time to total
number of changes implemented
• number of unresolved problems
• time spent on unresolved problems
• percentage of changes that introduce new faults
• number of modules modified to implement a change
Massimo Felici - Software Metrics, 1999 69
70. Software Reliability
Measurement and Prediction
• What is software reliability?
• Software reliability vs. hardware reliability
Massimo Felici - Software Metrics, 1999 70
71. The Basic Problem of Reliability Theory
• The basic problem of reliability theory is to
predict when a system will eventually fail.
• Hardware reliability concerns normally with
component failures due to physical wear (e.g.,
corrosion, shock, over-heating, etc.).
• Such failures are probabilistic in nature, that is,
we usually do not know exactly when something
will fail, but we know that the product eventually
will fail, so we can assign a probability that the
product will fail at a particular point in time.
Massimo Felici - Software Metrics, 1999 71
72. Basic Reliability Theory
Probability density function f t
t
Distribution function Ft f t dt
0
Reliability function Rt 1 Ft
Massimo Felici - Software Metrics, 1999 72
73. Basic Reliability Theory
Mean Time To Failure (MTTF) ET t f t dt
Mean Time To Recover (MTTR)
Mean Time Between Failure (MTBF) MTBF = MTTF + MTTR
Massimo Felici - Software Metrics, 1999 73
74. The Software Reliability Problem
Software Reliability vs. Hardware Reliability
• There are many reasons for software to fail, but none
involves wear and tear.
• Usually, software fails because of a design problem.
• Other failures occur when the code is written, or
when changes (e.g., changed requirements, revised
design, corrections of existing problems, etc.) are
introduced to a working system.
• Ideally, we want our changes implemented without
introducing new faults, so that by fixing a known
problem we increase the overall reliability of the
system.
Massimo Felici - Software Metrics, 1999 74
75. The Software Reliability Problem
Software Reliability vs. Hardware Reliability
• When hardware fails, the problem is fixed by replacing
the failed component with a new or repaired one, so that
the system is restored to its previous reliability. Rather
than growing, the reliability is simply maintained.
• The key distinction between software reliability and
hardware reliability is the difference between intellectual
failure (usually due to design faults) and physical failure.
• Although hardware reliability can also suffer from design
faults, the extensive theory of hardware reliability does
not deal with them.
Massimo Felici - Software Metrics, 1999 75
79. Choosing the Best Model
• Uncertainty about the operational environment
• Uncertainty about the effect of fault removal
• The behaviour of models depends greatly on the data
to witch they are applied. In fact, the accuracy of
models also varies from one data set to another.
Sometimes no parametric prediction system can
produce reasonably accurate results.
Massimo Felici - Software Metrics, 1999 79
81. Standards and Approaches
• ISO 9126
• IEEE 912
• Capability Maturity Model (CMM)
Software Engineering Institute (SEI)
• SPICE
European Software Institute (ESI)
• Goal Question Metric (GQM)
Victor Basili
• SQUID
Massimo Felici - Software Metrics, 1999 81
82. Goal Question Metric (GQM)
The Goal Question Metric approach provides a
framework involving three main steps:
1. List the major goals of the development or
maintenance project.
2. Derive from each goal the questions that must be
answered to determine if the goals are being met.
3 Decide what must be measured in order to be able
to answer the questions adequately.
Massimo Felici - Software Metrics, 1999 82
83. Capability Maturity Model (CMM)
Level 5:
Optimizing
Level 4:
Managed
Level 3:
Define
Level 2: Level 1: Initial
Repeatable No attempt to standardise process
Level 1: Level 2: Repeatable
Initial Disciplined process
Level 3: Defined
Standard, consistent process
Level 4: Managed
Predictable process
Level 5: Optimising
Continuously improving process
Massimo Felici - Software Metrics, 1999 83
84. References
Fenton, N.E., Pfleeger, S.L.: Software Metrics: A Rigorous & Practical Approach,
Second Edition. International Thomson Computer Press, 1996.
ISO/IEC 9126, Information technology - Software quality characteristics and metrics.
IEEE Std 982.1-1988, IEEE Standard Dictionary of Measures to Produce Reliable
Software. IEEE, 1988.
IEEE Std 982.2-1988, IEEE Guide for the Use of IEEE Standard Dictionary of
Measures to Produce Reliable Software. IEEE, 1988.
Lyu, M. (Ed.): Handbook of Software Reliability Engineering, IEEE Computer
Society Press, 1996.
Musa, J., Iannino, A., Okomuto, K.: Software Reliability: Measurement, Prediction
and Application. McGraw-Hill, New York, 1987.
Salamon, W.J., Wallace, D.R.: NISTIR 5459, Quality Characteristics and Metrics for
Reusable Software.
Massimo Felici - Software Metrics, 1999 84
87. Software QUality In the Development Process
The SQUID Project P8436 has been funded by the European
Commission within the ESPRIT Programme
SQUID Partners:
Keele University (UK), DELTA (DK), ENEA (IT), Engineering
Ingegneria Informatica S.p.A. (IT), TÜV-Nord (DE)
Massimo Felici - Software Metrics, 1999 87
88. SQUID Approach
• Provides support for product quality specification, control and
evaluation
• Measurement-based
• Two types of measure
– Internal measures relating to software and its development process
– External measures relating to operational behaviour and the support
process
• Problem
– Software engineering research can not show a functional relationship
between internal measures and external measures
• SQUID View
– Controlling internal quality is likely to improve product quality
– Collecting and analysing data will confirm whether any relationships
exist in your own organisation
Massimo Felici - Software Metrics, 1999 88
89. Four Steps to Quality
• Specify product quality requirements objectively:
– Target values of measurable properties
• Assess feasibility of quality requirements before starting
– Address feasibility problems as necessary
• Monitor progress towards achieving quality requirements
– Set Targets for measures arising during product development
– Monitor and control these measures
• Evaluate final product quality by comparing target values
with actuals
Massimo Felici - Software Metrics, 1999 89
90. SQUID Approach
• SQUID approach requires
– Integration of different types of measure (internal & external)
– Integration of different models (quality models & development
models)
• Problem
– There are many different quality models
• Different quality models lead to different external measures
– Different organisations will have use different development models
• Different development methods lead to different internal
measures
• SQUID toolset supports configuration
– Definition of organisation specific models and measures
– Analysis methods independent of specific models and measures
Massimo Felici - Software Metrics, 1999 90
91. SQUID Approach
• SQUID support various processes needed to manage quality
– Quality Specification
– Quality Planning
– Quality Monitoring
– Quality Evaluation
• Basis of SQUID toolset functions
Massimo Felici - Software Metrics, 1999 91
92. SQUID Quality Process
USER QUALITY USER
NEEDS IN USE LAYER
QUALITY QUALITY EXTERNAL
SPECIFICATION EVALUATION LAYER
QUALITY QUALITY INTERNAL
PLANNING CONTROL LAYER
Massimo Felici - Software Metrics, 1999 92
93. Quality Specification
User or organisational product requirements:
– Mapped to appropriate elements of a quality model
– Converted to required values of measurable
operational properties
– Checked for feasibility
Massimo Felici - Software Metrics, 1999 93
94. Modeling the Development Process
Development model Project Object
comprises
belongs to
Project Object type
Deliverable type Development Review point type
Activity type
Massimo Felici - Software Metrics, 1999 94
96. The SQUID View of Measurement
quantifies
Unit Attribute
using
exhibits
measures
Value Project Object
in_accordance_with
Counting Rule
Massimo Felici - Software Metrics, 1999 96
97. Quality Planning
• After quality specification agreed
• Development process must be defined
– Assumed to be one of several company standard
methods
• Set targets for internal measures
– SQUID assumes that controlling internal measures
will deliver required quality
– To monitor progress internal measures should be
associated with different development activities
and intermediate deliverables
– Target values can be based on past performance:
• What internal values where observed on project
that achieved similar requirements
Massimo Felici - Software Metrics, 1999 97
98. Quality Monitoring
• Compares actual values with target values during software
production
– To assess current status
• Compares estimates of future quality measures with target
values
– To re-assess feasibility
• Identifies “anomalies”
– Unusual components (modules, documents, programs)
– Possible problem components
Massimo Felici - Software Metrics, 1999 98
101. Case Studies
• A railways traffic control system for
the Italian National Railways
• Antarctica TELESCIENCE Project
Massimo Felici - Software Metrics, 1999 101
102. A Railways Traffic Control System
Project Summary
Customer: Italian National Railways (FS)
Project Type: Research, Prototyping
Target: Developing a new Control System
System Aims: Controlling Train Traffic in real-time
No Safety
Design Approach: User Centred Design
Railways Traffic Control Massimo Felici - Software Metrics, 1999 102
System
103. User Centred Design Approach
General Properties
• Based on user activity
• Iterative design process
• User participation
• Prototypes
based on scenarios
frequent modification
• Instruments used for designing, analysing and evaluating
video/record taping
questionnaires
interviews
Railways Traffic Control Massimo Felici - Software Metrics, 1999 103
System
104. Phases of the Design Process
Task analysis
p
r
Task allocation o
t
o
System design t
y
p
i
Job design n
g
Evaluation
Railways Traffic Control Massimo Felici - Software Metrics, 1999 104
System
105. Schema of the Control System
Real System
Man Machine
System Monitor Operator
Interface
Decision Support
System
Railways Traffic Control Massimo Felici - Software Metrics, 1999 105
System
106. Quality Model
Quality Model
Reliability Usability Functionality Efficiency Maintainability
characteristic layer
subcharacteristic layers
attribute layer (internal/external)
Railways Traffic Control Massimo Felici - Software Metrics, 1999 106
System
107. Quality Requirements
Each system component:
- performs different tasks
- has different quality requirements
Component Quality Quality Subcharacteristic Requirement
Characteristic Level
System Monitor Reliability Very High
Functionality Function completeness High
Function correctness High
Efficiency Medium
Maintainability Corrective Maintainability High
Adaptive Maintainability Very High
Railways Traffic Control Massimo Felici - Software Metrics, 1999 107
System
108. Quantitative Targets
Quantitative targets to monitor progress of components
System Quality Quality Quality Attribute Target
Component Characteristic Subcharacteristic Subsubcharacteristic
System Reliability Estimated Mean time 1*10E3 hours
Monitor Reliability between failures
Correctness Module fault 10 faults per
density module
Test Accuracy Test Coverage Function = 100 %
coverage
Branch coverage 70%
. . . . . .
. . . . . .
. . . . . .
Railways Traffic Control Massimo Felici - Software Metrics, 1999 108
System
109. Selection Criteria for Metrics
• Objective
• Observable
• Repeatable
• Predictive
• Easy to collect
• Valid across further projects
• User Centred Design aspects (new methods
and tools that should be exploited by metrics)
Railways Traffic Control Massimo Felici - Software Metrics, 1999 109
System
110. Critical Issues
• Frequent modifications
• Iterative design process
• User Centred Design aspects
Railways Traffic Control Massimo Felici - Software Metrics, 1999 110
System
111. Data Collection Activity
• Use of the SQUID tool
• Tools in order to automate the data collection activity
• Reports
Railways Traffic Control Massimo Felici - Software Metrics, 1999 111
System
112. Conclusion
• Software Quality in User Centred Design
• Interpretation of metrics within a specific context
• Critical issues
frequent modifications
iterative design process
User Centred Design aspects
• Definition of new (user centred design) metrics
Railways Traffic Control Massimo Felici - Software Metrics, 1999 112
System
113. Antarctica Telescience Project
• The Telescience Project was launched end 1993 in the
framework of the Italian National Antarctic Research
Programme.
• The aim is the development of an Advanced
Supervision and Telecontrol System to remotely
perform experimental activities in Antarctica during
the austral winter.
Antarctica Telescience Massimo Felici - Software Metrics, 1999 113
Project
114. Project Description
Development responsibility: ENEA, Division INN RIN INFAV
Estimated number of modules: 40
Estimated code size: 19.000 LOC
Expected man power required: 7 staff year
Personnel assigned to the project staff: 8
Experience of the staff with CASE tools: Good
Experience of the staff with
quality measurement tools: None
Use of quality control tools
(other than SQUID): None
Programming language: C
Antarctica Telescience Massimo Felici - Software Metrics, 1999 114
Project
115. Antarctica Telescience Project
Main Systems
• A set of Remote Laboratories in Antarctica
• A Supervision and Telecontrol Room in Antarctica
• An Electric Power Supply & Distribution System in Antarctica
• A Telecommunication System in Antarctica
• A Remote Man-Machine Interface System in Italy
Antarctica Telescience Massimo Felici - Software Metrics, 1999 115
Project
116. Architecture of the Advanced Supervision and Telecontrol System
Remote Remote Remote Remote
lab. N° 1 lab. N° 2 lab. N° 3 lab. N° n
Supervision and
Telecontrol Room
Telecommunication
System
Electric Power Supply
& Distribution System Antarctica
Remote Man-Machine
Italy Interface System
Antarctica Telescience Massimo Felici - Software Metrics, 1999 116
Project
117. The Electric Power Supply and Distribution System
• The Electric Power Supply & Distribution System
(EPSDS) ensures that electricity produced by the on-site
diesel generators is available to the equipment.
• The experimental activities is intended to run
unmanned for 8 months in Antarctica and the EPSDS
must guarantee the power supply for such period
without interruptions.
• The EPSDS has stringent requirements.
• The real-time control software is duplicated.
• If the software fails catastrophically in both versions of
the Electric Power Supply Subsystem, a back-up battery
power can run for one hour.
Antarctica Telescience Massimo Felici - Software Metrics, 1999 117
Project
118. The Remote Man-Machine Interface System
• The MMI allows to perform scientific activities as well as
to control and monitor the Antarctica base.
• There are two types of users operators and scientists.
• The Remote MMI system is based on 7 main subsystems:
1. Data acquisition
2. Information Presentation
3. Command sending
4. Simulation
5. Operator training
7. Data elaboration
Antarctica Telescience Massimo Felici - Software Metrics, 1999 118
Project
119. Product Portion
Subproject Product Portion Components
Electric Power Supply PP1 Electric Power Supply subsystem
Distribution system PP2 Electric Power Distribution subsystem
Remote MMI system 1 PP3 Information presentation subsystem
Command sending subsystem
Simulation subsystem
Remote MMI system 2 PP4 Mission planning support subsystem
Operator training subsystem
Data elaboration subsystem
Data acquisition subsystem
Antarctica Telescience Massimo Felici - Software Metrics, 1999 119
Project