Abstract. [Context/motivation] Quality requirements (QRs) are a concern of both requirement engineering (RE) specialists and software architects (SAs). However, the majority of empirical studies on QRs take the RE analysts’/clients’ perspectives, and only recently very few included the SAs’ perspective. As a result, (i) relatively little is known about SAs’ involvement in QRs engineering and their coping strategies, and (ii) whatever is known mostly comes from small and midsized projects. [Question/problem] The question in this exploratory study is how SAs cope with QRs in the context of large and contract-based software system delivery projects. [Principal ideas/results] We executed an exploratory case study with 20 SAs in the context of interest. The key results indicate the role SAs play in QRs engineering, the type of requirements communication processes SAs are involved in, the ways QRs are discovered, documented, quantified, validated and negotiated. Our most important findings are that in contract-based contexts: (1) the QRs are approached with the same due diligence as the functional requirements and the architecture design demand, (2) the SAs act proactively and embrace responsibilities over the QRs, (3) willingness to pay and affordability seem as important QRs prioritization criteria as cost and benefits do, and (4) QRs engineering is perceived as a social activity and not as much as a tool and method centric activity.[Contribution] The main contributions of the paper are (i) the explication of the QRs process from SAs’ perspective, and (ii) the comparison of our findings with previously published results
2. 2
Table of Contents
1. Introduction
2. Background and Motivation
3. The Research Design: Qualitative Case
Study
4. The Application of the Method
Results
Limitations
1. Comparison to Literature
2. Implications
3. Wrap-Up
Maya Daneva, Wed, April 16, 2009
3. 3
Background and Motivation
Most empirical research on QRs takes the
perspective of clients, RE researchers,
practitioners.
Quality requirements (QRs) are a concern of
multiple stakeholders; in particular: software
architects (SAs)
Relatively little is know about SA’s involvement
Evidence comes from small and mid-sized
projects; very few studies in large projects
4. 4
The Research Questions
Research Goal: to understand how SAs cope with QRs
in large and contract-based software system
development projects.
1. How do the SAs understand their role?
2. Do SAs and RE staff use different QRs terminology?
3. How do QRs get elicited?
4. How do QRs get documented?
5. How do QRs get prioritized?
6. How do QRs get quantified, if at all?
7. How do QRs get validated?
8. How do QRs get negotiated?
9. What role does the contract play in the way SAs cope
with QRs?
5. 5
The Case Study Research Design
Maya Daneva, Wed, April 22, 2009
Key Steps (R. Yin, 2008):
1. Define interview guide
2. Pilot the interview
3. Collect data by interviewing participants
4. Analyze the data
5. Report on the results
6. 6
Who Did We Interview?
20 Architects from 14 companies in North Europe
All have 10+ years of experience in large systems
All work on large contract-based projects (3 dev
locations and 2 client locations)
Various pricing agreements
Sectors: large IT vendors, Oil &Gas, Insurance,
Real estate, Video streaming, online systems
(travel, book store, games)
7. 7
Who Did We Analyze the Data?
Coding practices based on the grounded
theory book of K. Charmaz (2006)
Iterative procedure
8. 8
Results (1):
How do the SAs understand their role?
Formal job descriptions and
competence models
Self-described roles as:
‘a bridge’ b/n QRs and underlying
technology
‘a translator’ from the user
language into the feature
specification language
‘review gate keepers’ regarding
e.g. contract compliance
9. 9
Results (2): Do SAs and RE staff use
different terminology for QRs?
Gaining communication clarity was a non-
issue
What helped?
Domain knowledge
Experience
If ISO-certified, then training, quality
manuals, product quality handbooks
made the difference
Issue: interchangeable use of terms from
two streams of standards (management and
technical how-to)
10. 10
Results (3):
How do QRs get elicited?
14 SAs use checklists based on:
ISO standards
Architecture frameworks
Internal standards
Stakeholder engagement
standards, AA1000SES
4 SAs uses game-based processes
2 SAs used story-telling techniques
11. 11
Results (4):
How do QRs get documented?
15 SAs: By using predefined templates
based on :
ISO standards
the Quality Function Deployment
framework
Planguage
the INVEST grid approach
Vendor-specific notations (e.g. SAP)
5 SAs: By using natural language
12. 12
Results (5):
How do QRs get prioritized?
The business case is the driver
behind trade-offs, e.g. KPIs
No particular prioritization method
Prioritization criteria:
cost,
benefits,
client’s willingness to pay
affordability
Who decides?
Steering committees
SAs
13. 13
Results (6):
How do QRs get quantified?
Quantification is useful, but should not happen
too early
How you get them?
pre-specified in the contract
Engage a specialist expert (e.g. in
performance)
Decompose, operationalize and use
estimation technique (IFPUG NFR)
Issue:
product and project measures are used
incorrectly
QRs are confused with design-level req’ts
14. 14
Results (7):
How do QRs get validated?
By using requirements walkthroughs
The QFD framework
Internal architecture standards
15. 15
Results (8):
How do QRs get negotiated?
The business case is the commonly
used vehicle
The QFD framework
EasyWinWin
The six-thinking-hat method
16. 16
Results (9):
What role does the contract play in the way
SAs cope with QRs?
3 ways for influencing QRs
processes:
Cost-conciousness
Service level agreements
Pre-defined priorities for QRs
Contracts are conductive as ways to
maintain control
3SAs: “contract is was not that
important”
EasyWinWin
17. 17
Limitations of the Study
The possible validity concerns:
1. External validity
Similarity with contract-based system delivery
settings
Application domain, organizational maturity
1. Data collection threats: did SAs answer the
questions truthfully?
2. Data analysis threats: researcher’s bias
18. 18
Comparison to Literature
Input: 5 empirical studies
van Heesch et al, Ameller et al (2) Poort et al (2)
1.QRs are:
elicited by using checklists,
documented by means of templates
prioritized based on willingness to pay and affordability
quantified by using size estimation standards (IFPUG)
negotiated by using the business case
1.SAs:
Serve as ‘a bridge’ and have formal job descriptions,
Have terminology (established by standards)
1.Contract plays an important role for SAs to act the ways
we observed.
19. 19
Implications
1. To SAs: this study suggests QRs conversations
start with contracts, SLA, and business cases
2. To RE practitioners: your SA could be quite a
valuable resource! She/he can save you time.
3. To RE tool vendors: it seems more important to
figure out how to embed tools into social
processes and broader social interaction
4. To RE researchers: extend the focus on methods,
models and automation by including analysis of
QRs processes as socially constructed ones.
20. 20
Wrap-Up
1. We carried out an interview-based study
2. It revealed what SAs were thinking on QR
3. We compared and contrasted the findings with
published literature
4. Implications fro practice and research are made.