Bad requirements also lead to products that do not fit the needs of our customers. Incomplete and ambiguous requirements are one of the major sources of project failures. The better we describe, what our customers really need and what they want to achieve with our systems, the better will these system exactly fulfil, even exceed our customers' expectations. But how good are our specifications actually? What is a matter of course in other engineering disciplines is completely unknown in requirements engineering: the quantitative measurements and control of requirements quality. In my presentation, I will show, how you can use performance indicators to measure, control and improve the quality of your system requirements and by that improve the overall quality and market success of your products. Through a practical sample we look what metrics exist on requirements quality and you can automatically measure them. We then take a look at how you can define a metric system in your company that makes sense for your business and your systems. The main topics covered are: * Why are good requirements so important? * What makes a good requirement? What are the quality criteria for requirements? * Why is it important to quantitatively measure it? What is the use of measurement? * What makes a good metric? * The GQM approach for defining metrics * Samples for requirements quality metrics * Live demo of automatic measurements of a requirements document * Conclusion and tips for next steps

1. Software Quality Lab
Markus Unterauer
Consultant and trainer
You can‘t control what you can‘t
measure
Measuring requirements quality

2. www.software-quality-lab.com | improve your quality
At the end of this session you know …
Agenda
 Why good requirements are important
 What makes good requirements
 How you can measure requirements quality
 How to define metrics using G-Q-M
 How you use metrics to improve
Slide 2

3. www.software-quality-lab.com | improve your quality
… and how reality often looks like
Why good requirements are
important
Slide 3

4. www.software-quality-lab.com | improve your quality
Requirements are the foundation of every SW
Requirements quality
Slide 4
Source: fpm1979 via Flickr.com (https://www.flickr.com/photos/fpm1979/9660419677)

5. www.software-quality-lab.com | improve your quality
… and the source of all evil
Requirements quality
Slide 5
Bad requirements
Unrealistic plans
Unplanned activities
Activities are
unpredictable
Delays
Unfulfilled
expectations
Poor quality
Misunderstandings
Time pressure
Overtime
Defects
Source:JamesW.Grenning,RenaissanceSoftwareConsultingCompany,Boston2008

6. www.software-quality-lab.com | improve your quality
If we specify badly …
Requirements quality
Slide 6
Source: Thomas R. Stegelmann via Flickr.com
https://www.flickr.com/photos/thomasrstegelmann/2731846156Source: Joel Kramer via Flickr.com
https://www.flickr.com/photos/75001512@N00/4518848983

7. www.software-quality-lab.com | improve your quality
… we often realize it (too) late
Requirements quality
Slide 7
55% a lle rFeh le ren ts tehen in der
An fo rde rungs– und En tw u rfsphase
rozen ta lle r
n tw ick lungs -
feh le r
An fo rde rungs -
und En tw u rfs -
phase
Techn ische
En tw u rfsphase
Kons truk tion -–
und S ys tem -
tes tphase
Abnahm e tes t
und Be triebs -
phase
En tw ick lungs -
phasen
60
50
40
30
20
10
En tw u rfs feh le r Log ische Feh le r S yn tax feh le r
E
E
E
G
G
G
G
E = E ingeb rach te Feh le r
G = G e fundene Feh le r
eh le rbese itigungskos ten (abge le ite tvon A lbe rts 1985 )
Most requirements
defects are found
in operation phase!
Percentage
of all
development
defects
Development
phase
Design defect Logical defect Syntax defect
Requirements
and conception
Technical design Construction and
system test
Acceptance and
operations
E = Entry point of defect
G = Defect found
55% of all defects occur in
requirements and conception phase

8. www.software-quality-lab.com | improve your quality
… what results in high follow up costs
Requirements quality
Slide 8
Idea Specification Implementation Testing Operations
Effortfordefectcorrection
5minutes
1hour
1 PD
3 PD
15 PD
Phase, in which a requirement defect is found
Effort for correcting a requirements defect
depending on when the defect is found
The later a
requirement defect is
found, the more
effort is needed for
correction

9. www.software-quality-lab.com | improve your quality
Status quo and causes
Slide 9

10. www.software-quality-lab.com | improve your quality
Requirements Engineering figures
Requirements quality
Productivity
 75% of all companies do RE insufficiently
 61% are unsatisfied with execution, 70% with documentation
 50% of all success factors for projects lie in RE
Efficiency
 16% of total project effort is used for RE
Quality
 53% of all requirements are written in prose text
 2-3% requirements creep per month
 40-50% of total effort used for correcting RE related defects
 58% of all testers see insufficient requirements as biggest challenge
Slide 10
[Ebert],[Wingroveetal.],[Jones],[SwissQConsultingAG],[StandishReport],]Yangetal.]

11. www.software-quality-lab.com | improve your quality
… and how to get them
We need better requirements!
Slide 11

12. www.software-quality-lab.com | improve your quality
Measurement as basis for all improvements
Requirements quality
Slide 12
Define goal and
path to reach it
Discover, where we are
Identify gap, to where
we should be
Find corrective measures
Go next step Goal reached 

13. www.software-quality-lab.com | improve your quality
Proper metrics show the way to our goal
How to measure requirements
Slide 13
Source:DchicviaFlickr,https://www.flickr.com/photos/agenciamodels/6118171675

14. www.software-quality-lab.com | improve your quality
Boundaries show, when to act
How to measure requirements
Slide 14
Upper boundary
Lower boundary
Ideal value
Measured value
Time

15. www.software-quality-lab.com | improve your quality
Criteria for good requirements
Requirements quality
 Reader acceptance criteria
 Synchronized
 Valid and current
 Realizable
 Understandable
 IEEE 830-1998
 Rated
 Unambiguous
 Correct
 Consistent
 Testable
 Traceable
 Complete
Quality criteria based upon IREB® Certified Professional for Requirements Engineering
| Slide 15

16. www.software-quality-lab.com | improve your quality
Define proper metrics with G-Q-M
How to measure requirements
1. Goal
 What should be done for whom?
 Which aspects do we want to change? What is the context?
e.g. „Improve the quality for all new projects“
2. Question
 Which questions guide us to the measurement goal?
e.g. „How understandable are our requirements?“
3. Metric
 How can we answer the questions using metrics?
e.g. „Average sentence length in words“
Slide 16
Define goals
Find
questions
Answer
questions

17. www.software-quality-lab.com | improve your quality
Example
How to measure requirements
Slide 17
Metric
Question
Goal We want to improve the
quality of our specifications!
How unambiguous are our
requirements?
Number of
weak phrases
Number of
sentences in
passive form
How complete are our
requirements?
Number of
empty chapters

18. www.software-quality-lab.com | improve your quality
Question: How unambiguous are our requirements?
Goal: Improve requirements quality
 Percentage of weak phrases („could“, „should“, „rather“,
„mostly“, …) within the top 20
 Percentage of foreign words and abbreviations
 Percentage of sentences with subordinate clauses
 Number of universal quantifiers („all“, „never “, „none“, …)
 Average sentence length (number of words)
 Number of complex terms (terms with „-“)
 Models per chapter
 Percentage of concatenated sentences („and“, „but“, …)
 Percentage of comparatives („faster“, „easier“, …)
Slide 18

19. www.software-quality-lab.com | improve your quality
Document metrics in a metric handbook
How to measure requirements
Slide 19
Metric Weak Phrases
Goal Specification must be unambiguous. Weak phrases point out unclear
requirements.
Level Sentence
Value Less misunderstandings
Less effort for discussions
Formula wp = nwp / s * 100
wp ….. weak phrases
nwp … number of weak phrases
s …… number of sentences
Method Automatic calculation using VBA macro
Datasource Word version of specification
Interval Before each sprint
Responsible Product Owner

20. www.software-quality-lab.com | improve your quality
Reporting and analysis
How to measure requirements
Observation
 Because of the high number of
concatenated sentences and filler
words, the average sentence length
is too long
Valuation
 Understandability: 82%
 Unambiguity: 78%
Recommendations
 Split up long sentences
(„One requirement per sentence!“)
 Eliminate filler words
Slide 20
Sentence length
in words
Filler words
within top 20
Weak phrases per
100 sentences
Tables with
> 30 rows
Concatenated
sentences per 100

21. www.software-quality-lab.com | improve your quality
Sample metrics with target range
How to measure requirements
Slide 21
Metric Target range
Sentence length (in words or lines) < 15 (ca. 1.5 lines)
Percentage of weak phrases
< 0,25 per page
0 within Top 20
Percentage of sentences with
subordinate clauses
< 15%
Number of tables with > 30 rows 0
Percentage of concatenated sentences < 5%

22. www.software-quality-lab.com | improve your quality
Efficiency in measurement means automation
How to measure requirements
Slide 22
Size:
- Pages: 5
- Words: 1066
- Sentences: 104
Complexity:
- Words per sentence: 13,25
- Tables with > 30 rows: 0
Weak phrases:
- „as possible“: 4 (every 26 sentences)
- „rarely“: 2 (every 52 sentences)
- „often“: 2 (every 52 sentences)
- „many“: 2 (every 52 sentences)
- „some“: 2 (every 52 sentences)
…

23. www.software-quality-lab.com | improve your quality
Tips for starting your requirements quality metric system
How to measure requirements
 Define metrics top down, not bottom up
 Goal > Question > metric
 Not: „Hey, look how many things we can measure!“
 Concentrate on 5 - 10 metrics, e.g. these:
 Sentence length (in words or lines)
 Percentage of weak phrases
 Percentage of sentences with subordinate clauses
 Number of tables with > 30 rows
 Percentage of concatenated sentences
 Define lower and upper boundary together with your team
 … or measure without boundaries for some iterations and define boundaries then
 Start with automatically measurable measures
 Define, what to do with the metrics
Slide 23
Image:http://pixabay.com/en/bulb-light-lamp-electric-160207/

24. www.software-quality-lab.com | improve your quality
Summary
Key learnings from this session
 Poor requirements lead to bad software
 Measuring requirements quality using metrics is essential
 Objectivly judge quality
 Derive actions and evaluate effectiveness
 Use GQM method for metric definition
 G – Define goals
 Q – Ask questions
 M – Define metrics to answer the questions
 Automate measurement
Slide 24

25. Your Partner in Software Quality and Testing
Software Quality Lab GmbH
www.software-quality-lab.com
Consulting | Service | Academy | Tool Expertise
Markus Unterauer
markus.unterauer@software-quality-lab.com