16th slide set of CECS 542 Quality assurance for requirements documentation Complete course: http://foss2serve.org/index.php/Requirements_Engineering,_CSU_Long_Beach,_Penzenstadler

1. Requirements
Engineering:
Quality Assurance
CECS 542
Dr. Birgit Penzenstadler
1
Photo credit: Nathan Dumlao, Unsplash

2. Dr. Birgit Penzenstadler 2

3. Learning Goals
• Founda@ons of quality
assurance
– Quality criteria for RE
– Construc@ve and
analy@cal Quality
Assurance
• QA for Artefacts
• Techniques for Quality
Assurance
Dr. Birgit Penzenstadler 3
Context Layer
System Layer
Requirements Layer
Stakeholder Model Objectives
& Goals
Constraints
& Rules
!
!
!
!
!
Data Model
E
A
A
A
E
System Vision
Functional
Hierarchy
Architecture Overview
System
Function Model
Fun 1
Fun 2
Component Model
C C
Data Model
E
A
A
A
E
Behaviour Model
Business Case
Deployment Requirements
System Constraints
Domain Model
Service ModelUsage Model
Quality Requirements
Risk List
Project Scope
Process Requirements
Glossary
Glossary
Glossary

4. Overview: Quality Assurance
• Mo@va@on and Terminology
• Quality of Requirements Documents
• Principles of Quality Assurance
• Techniques for construc@ve QA
• Techniques for analy@cal QA
Dr. Birgit Penzenstadler 4
K Rayker, stock.xchng

5. 16
11
9
1
7
7
5
3
3
1
5
1
4
3
0
2
1
1
0
31
22
22
20
16
13
13
12
11
11
9
9
8
8
5
3
3
2
2
0 5 10 15 20 25 30 35
Incomplete / hidden reqs.
Moving targets
Time boxing
Separa@on reqs. from known solu@ons
Underspecified reqs.
Communica@on flaws to customer
Inconsistent reqs.
Communica@on flaws in team
Missing traceability
Gold pla@ng
Unclear non-func@onal reqs.
Terminological problems
Insufficient support by customer
Unclear responsibili@es
Vola@le domain
Weak access to customer needs
Insufficient support by project lead
Technically unfeasible reqs.
High degree of innova@on vs. need for formal acceptance
Overall frequency
Cause for project fail
Recap: Challenges in prac@ce
5
Requirements are o.en...
• Incomplete
• Not agreed upon
• Contradictory
• Not measurable
• Unfounded
• Irreproducible
Mendez et al. Naming the Pain in Requirements Engineering – The NaPiRE-Report

6. One of the many challenges...
6

7. Terminology in the context of quality
assurance in RE
Quality defect
• Incorrect (invalid) requirement: Requirement that does not reflect the
inten@on of the stakeholder (in the sense of „validity“)
• Quality defect: Requirement that can be valid, but has qualita@ve defects,
e.g. missing measurability, low understandability, contradictory, ...
• Interrela@on of those two:
– Incorrect requirements are ojen hidden due to quality defects
– Correctness of requirements ojen viewed as quality criterium
Valida7on and Verifica7on
• Valida@on: Check of requirement w.r.t. correctness (it‘s a valid
requirement, meaning it represents the inten@on of the stakeholder)
• Verifica@on: Check of system w.r.t. fulfillment of requirements
• Both are part of QA
Dr. Birgit Penzenstadler 7

8. Quality Assurance in Requirements
Engineering
• Def. QA in RE: Applica@on of systema@c measures for
iden@fying quality defects and assuring the quality of the
requirements specifica@ons.
à Check of quality criteria, e.g.:
§ Correctness
§ Completeness
§ Consistency
§ Traceability
§ ... (see following slides)
à The examina@on can be conducted
construc@ve or analy@cal
using a formal procedure.
8

9. Overview: Quality Assurance
• Mo@va@on and Terminology
• Quality of Requirements Documents
• Principles of Quality Assurance
• Techniques for construc@ve QA
• Techniques for analy@cal QA
Dr. Birgit Penzenstadler 9
K Rayker, stock.xchng

10. Recap: Requirements, documents,
artefacts, repositories
• A requirement is a demanded characteris@c of a
system or process.
• We dis@nguish
– Syntac@c representa@on: Text, table, diagram, formula
– Seman@c representa@on: Content – informal or formal
• A requirements document or artefact
– Contains a number of requirements
– Has a structure
• Delimita@on: A requirements repository (database for
requirements) serves for storing large sets of
requirements (and requirements artefacts)
10

11. QA of requirements documents
• Quality of requirements documents is crucial for project
success. à Why is that? What is based on them?
• We need specific procedures for QA
• Relevance of the quality criteria needs to be determined by
the further use of the documents.
UCI winter 2014 Dr. Birgit Penzenstadler 11

12. Focus of quality assurance in RE
Perspec@ves in QA in RE
• We dis@nguish the quality of
– Requirements documents / artefacts
– Sets of requirements / statements
– Individual requirements
– Systems
12
Context Layer
System Layer
Requirements Layer
Stakeholder Model Objectives
& Goals
Constraints
& Rules
!
!
!
!
!
Data Model
E
A
A
A
E
System Vision
Functional
Hierarchy
Architecture Overview
System
Function Model
Fun 1
Fun 2
Component Model
C C
Data Model
E
A
A
A
E
Behaviour Model
Business Case
Deployment Requirements
System Constraints
Domain Model
Service ModelUsage Model
Quality Requirements
Risk List
Project Scope
Process Requirements
Glossary
Glossary
Glossary

13. Overview: Quality Assurance
• Mo@va@on and Terminology
• Quality of Requirements Documents
• Principles of Quality Assurance
• Techniques for construc@ve QA
• Techniques for analy@cal QA
Dr. Birgit Penzenstadler 13
K Rayker, stock.xchng

14. Analytical QS
Depending on the quality criteria, responsible for checking are:
• Project team members with domain knowledge during elabora@on of the
requirements, e.g. „correctness“ à this is called construc@ve QA
• External/neutral quality responsibles who perform checks, e.g.
„traceability“ and „understandability“ à this is called analy@cal QA
à Which measures can you think of for performing either of these?
Constructive QS
Principle of construc@ve and analy@cal QA
14
Context Layer
System Layer
Requirements Layer
Stakeholder Model Objectives
& Goals
Constraints
& Rules
!
!
!
!
!
Data Model
E
A
A
A
E
System Vision
Functional
Hierarchy
Architecture Overview
System
Function Model
Fun 1
Fun 2
Component Model
C C
Data Model
E
A
A
A
E
Behaviour Model
Business Case
Deployment Requirements
System Constraints
Domain Model
Service ModelUsage Model
Quality Requirements
Risk List
Project Scope
Process Requirements
Glossary
Glossary
Glossary
Project team
Quality
responsible

15. Classifica@on of QA
QA
Construc@ve
Process
standards
Quality
criteria
Linguis@c
rules
Programming
guidelines
Naming
conven@ons
Structuring
conven@ons
Analy@cal
Analyzing
Metrics
Anomaly
analysis
Graphics
Tes@ng
Dynamic
tests
Review/
inspec@on
Autom. sta@c
analysis
Verifying
Formal
verifica@on
Model
checking Dr. Birgit Penzenstadler 15
Note: this is a general
classifica@on of QA, and
not all of it applies to
QA within RE.

16. Overview: Quality Assurance
• Mo@va@on and Terminology
• Quality of Requirements Documents
• Principles of Quality Assurance
• Techniques for construc@ve QA
– Reference Models
– Quality criteria according to IEEE 830
– Linguis@cs
– Guidelines
• Techniques for analy@cal QA
Dr. Birgit Penzenstadler 16
K Rayker, stock.xchng

17. Reference Models
• AMDiRE
• IEEE 830
• Cockburn template
• UML2 standard
Dr. Birgit Penzenstadler 17

18. Construc@ve QA in RE: Examina@on of
Quality Criteria acc. to IEEE 830-1998
• Completeness: Systema@cally run through
all cases (You can only find incompleteness,
not guarantee completeness!)
• Consistency: Relate all to one system model
• Unambiguity: Check phrasing
• Correctness: Valida@on
• Structuredness: Examine structure
• Traceability: Are requirements sufficiently linked?
• Changeability: Can expected changes in requirements be
conducted efficiently?
• Understandability: Check formula@ons
• Agreed upon: Check with stakeholders
18

19. Checklist: Ques@ons and Criteria for ...
• Completeness:
• Consistency:
• Unambiguity:
• Correctness:
• Structuredness:
• Traceability:
• Changeability:
• Understandability:
• Agreed upon:
19

20. Checklist: Ques@ons and Criteria for a
spreadsheet with requirements
• Completeness: Read through, are there s@ll open ques@ons? Lead
engineer, stakeholder, customer
• Consistency: Check for requirements conflicts
• Unambiguity: language check, let someone else read it whether it could be
misunderstood
• Correctness: check with stakeholder
• Structuredness: adheres to template / outline, has a breakdown that
makes sense
• Traceability: links between requirements in spreadsheet, across versions
of spreadsheet
• Changeability: check dependencies between requirements (high degree of
dependencies means low changeability)
• Understandability: let someone else read it whether it‘s easy to
understand
• Agreed upon: check with stakeholder
20

21. Linguis@cs in RE
• Classifica@on of linguis@c quality defects
– lexical/ontological (what does „green“ mean?)
– syntac@c (“I saw the man on the hill with a telescope”)
– seman@c (“All persons have a unique na@onal insurance
number“)
– pragma@c (“The trucks shall treat the roads before they freeze“)
– weak phrases: (“as soon as possible“)
– Omission or generaliza@on
• Syntax paperns
– [when?] [under what condi@ons?]
THE SYSTEM SHALL | SHOULD | WILL
<process> <thing to be processed> [<process detail>*]
21

22. Exercise: Improve phrasing
1. The system shall respond as fast as possible.
2. Students take 10 courses per semester.
Students take 1000 courses per semester.
3. Shortly before the due date the medium is
extended, unless somebody else reserved it.
Dr. Birgit Penzenstadler 22

23. Phrasing: Do‘s and Don‘ts
1. The system shall respond as fast as possible.
In 90% of all cases, the system shall respond to all queries within
3s.
2. Students take 10 courses per semester.
Students take 1000 courses per semester.
Every student takes 10 courses per semester.
3. Shortly before the due date the medium is extended, unless
somebody else reserved it.
Three days before the due date, the system checks whether the
medium has been reserved. If not reserved, the system extends the
lending period.
23

24. Guidelines & Checklists
How would you write a guideline and a checklist
for … ? Team up!
• Stakeholder Model
• Goal Model
• System Vision
• Usage Model
• Non-func@onal Requirements
Dr. Birgit Penzenstadler 24

25. Overview: Quality Assurance
• Mo@va@on and Terminology
• Quality of Requirements Documents
• Principles of Quality Assurance
• Techniques for construc@ve QA
• Techniques for analy@cal QA
– Checklists
– Quality Gates
– Fagan Inspec@ons
Dr. Birgit Penzenstadler 25
K Rayker, stock.xchng

26. Exemplary Check list according to
Lamsweerde (1/2)
• General check: Must be clear "what, who,
when, where“
• Defect-based criteria (as in IEEE 830):
– ContradicJon
– Inadequacy
– Unmeasurability
– Unfeasibility
– Poor structuring
– ...
• Quality-specific criteria:
– Is there any unspecified response in this operaJon to not
receiving an expected input value, or receiving it too early or too
late?
– Does the logical OR of the input condiJon on this operaJon form
a tautology?
– ... 26

27. Exemplary Check list according to
Lamsweerde (2/2)
• Domain—specific criteria:
typical issues in the parJcular domain
• Content-related criteria:
– Templates
• all fields filled
• idenJfier user consistently
• statement type correct
• ...
– Graphical nota@ons
• data flow consistent
• ER Diagram declaraJon
• ...
– Formal specifica@ons 27

28. Quality Gates
Specific milestone in a sojware project that checks
• Content: The „usual“ quality criteria:
Completeness, consistency, ...
• Documenta@on: Compliance with format,
understandability, unambiguity, ...
• Accordance: Every requirement agreed upon,
conflicts resolved, ...
28
Role Model Process Model
Project Scope
defined
System
Specification
accepted
Business
Analyst
Requirements
Engineer
System
Architect
Architecture
Overview
defined
Requirements
Specification
accepted
System Vision
defined
Context
Specification
accepted
Context
Specification
System
Specification
Requirements
Specification

29. Fagan Inspec@on
The term Fagan inspecJon refers to a structured
process of trying to find defects in development
documents. It includes the following phases:
• Planning: Moderator plans review process
• Overview: Author describes the background of
the document under inspec@on
• Prepara@on: Every reviewer examines the
document in order to find defects.
• Inspec@on mee@ng: A specific reader walks
through the document chapter by chapter, and
the inspectors point out found defects.
• Adapta@ons: The author of the document
corrects the found defects according to the
ac@on plan agreed upon in the mee@ng.
• Follow-up control: The inspectors check
whether the defects were fixed correctly. 29
Planning
Overview
Preparation
Inspection
Meeting
Adaptation
Follow-up
Control

30. IEEE 730 – 2014: IEEE Standard for
Sojware Quality Assurance Processes
Dr. Birgit Penzenstadler 30

31. IEEE 730 – 2014: IEEE Standard for
Sojware Quality Assurance Processes
Dr. Birgit Penzenstadler 31

32. IEEE 730 – 2014: IEEE Standard for
Sojware Quality Assurance Processes
Dr. Birgit Penzenstadler 32

33. IEEE 730 – 2014: IEEE Standard for
Sojware Quality Assurance Processes
Dr. Birgit Penzenstadler 33

34. IEEE 730 – 2014: IEEE Standard for
Sojware Quality Assurance Processes
Dr. Birgit Penzenstadler 34

35. Take-away: QA
• Defini@ons
– Quality Assurance
– Quality Defect
• Construc@ve QA
– Guidelines and criteria
– Reference models
• Analy@cal QA
– Quality gates
– Fagan inspec@on
– Checklists
• IEEE 730 Std for SQA
Dr. Birgit Penzenstadler 35
Context Layer
System Layer
Requirements Layer
Stakeholder Model Objectives
& Goals
Constraints
& Rules
!
!
!
!
!
Data Model
E
A
A
A
E
System Vision
Functional
Hierarchy
Architecture Overview
System
Function Model
Fun 1
Fun 2
Component Model
C C
Data Model
E
A
A
A
E
Behaviour Model
Business Case
Deployment Requirements
System Constraints
Domain Model
Service ModelUsage Model
Quality Requirements
Risk List
Project Scope
Process Requirements
Glossary
Glossary
Glossary

36. Dr. Birgit Penzenstadler 36
Time for a break

37. Requirements Nego@a@on
Dr. Birgit Penzenstadler 37