The document outlines an empirical study on how software architects consider non-functional requirements (NFRs). It describes conducting interviews with 13 software architects from 12 organizations. The study found that architects played multiple roles in projects and there was confusion around NFR terminology. NFRs were mostly elicited by architects but poorly documented and validated. Architects did not use dedicated tools for NFR management and were skeptical of automated support. The study aims to understand real-world practices for NFRs and improve their management.

1. How do Software Architects
consider Non-Functional
Requirements

2. Outline
Objective/RQs
Empirical method
Observations
Conclusions and related work
References
Example
•
•
•
•
•
2

3. Objective / RQs
• The research goal of this study is:
How do software architects
deal with non-functional
requirements in practice?
• Decomposed into several RQs:
RQ1
RQ2
RQ3
RQ4
RQ5
RQ6
RQ7
What is the role of the software architect?
Are there terminological confusions on NFRs?
What types of NFRs are relevant to software architects?
How are NFRs elicited?
How are NFRs documented?
How are NFRs validated?
What type of tool support for NFRs is used?
3

4. Empirical method (I)
• Type:
 Exploratory study using qualitative research
• Target population:
 Software architects (or practitioners with that role)
 We invited 21 software organizations (Spain)
 We recruited 12 software organizations
• We made 13 interviews (two on the same organization)
4

5. Empirical method (II)
SCC: Software Consulting Company; SH: Software House; ITD: IT Department
5

6. Empirical method (III)
• Data collection:
 Semi-structured interviews
 Focus: one single project
 Duration: ~1 hour
• 7 questions about the architect profile and development
methodology used in the project
• 10 questions about decision-making and NFRs
 Audio-taped  Transcribed  Verified
6

7. Empirical method (IV)
• Analysis:
 Tabulation technique
 Categories generation
• NVivo Software
 Counting technique
7

8. Empirical method (V)
• Limitations and mitigations:








Evaluation apprehension  Confidentiality
Understandability  Pilot testing (4)
Influential factors  Single project
Bad memory  Choose the project before
The best project  Ask for the most familiar
Omitting data  Add or modify
Researcher bias  Two separated interpretations
Generalize the findings  We do not attempt to
make universal generalizations
8

9. Observations (I)
• RQ1: What is the role of the software
architect?
 13 interviewees performed the tasks assigned to “software
architects” in the project based on their experience or
knowledge rather than their possible skills as architects
 0 interviewees held a “software architect” position at the
company
 12 interviewees played other roles in the project in addition
to the role of software architect, specifically: project
manager (3), developer (5), and project manager and
developer (4)
9

10. Observations (II)
• RQ2: Are there terminological confusions on
NFRs?
 Confusion was reported around the terminology for
designating NFR types
 Inability to interpret some particular term, e.g.,
“availability”, “accuracy”, and “sustainability”
 Use of a term that could lead to real confusion, e.g., some
said “ergonomic”, “comfortable,” or “friendly,” when they
meant “usable” (in the context of “usability”)
 Use of a term with an incorrect definition. We found a
serious confusion in the answer, e.g., “Maintainability is
very important, because when something is working, we
can’t make changes”
10

11. Observations (III)
• RQ3: What types of NFRs are relevant to
software architects?
10
9
8
7
6
5
4
3
2
1
0
10
9
8
7
6
5
4
3
2
1
0
• 49 references were made to technical NFRs
• 33 references were made to non-technical NFRs
11

12. Observations (IV)
• RQ4: How are NFRs elicited?
 In 10 projects, the NFRs were elicited solely by the
architect
 In 3 projects, the NFRs were elicited by the client
with the participation of the architect
 13 architects considered elicitation as a gradual
process
12

13. Observations (V)
• RQ5: How are NFRs documented?
 9 architects did not document the NFRs at all
 4 architects documented the NFRs:
• 3 used templates (1 only for initial NFRs)
• 1 used plain text (only for initial NFRs)
13

14. Observations (VI)
• RQ6: How are NFRs validated?
 NFRs had been met by the end of the project?
• 11 architects said yes
• 2 architects did not say yes
 What NFRs are validated?
• 8 architects validated some types of NFRs
– reliability, efficiency, usability, and accuracy
• 1 architect did not validate any NFRs at all
• 4 architects did not provide details
14

15. Observations (VII)
• RQ7: What type of tool support for NFRs is
used?
 Architects did not use any specific tool support for
NFR management
 We found a very strong reaction (“I do not believe
in automatic things”) against an automated
decision-making tool
 4 architects seem reluctant (“it is hard for me to
imagine that this can be done”)
 2 of the respondents said that the tool suggest
alternatives instead of making final decisions
15

16. Conclusions
Presentation to SEARCH
• Observations made about NFRs




Companies did not have the role of architect clearly defined
Architects didn't share a common vocabulary for NFRs
NFRs were mostly elicited by the architects themselves
Architects considered non-technical NFRs almost as
relevant as technical NFRs
 NFRs were poorly documented and validated
16

17. Related Work
Subject of research
Type of analysis
Companies
[Svensson10]
Elicitation; dependencies; expression; cost
estimation; prioritization
SLR
Not specified
1.560 candidate studies
18 selected studies
[Svensson09]
Presentation to SEARCH
Ref.
Importance of NFR types; dependencies;
expression; satisfaction
Interviews
5 companies
5 project leaders
5 product managers
Interviews
11 companies
11 project leaders
11 product managers
Interviews
2 companies
14 (different roles)
[Svensson11] Prioritization
Population
[Borg03]
NFR in general. Elicitation, documentation,
test and management in particular
[Vara11]
NFR importance
e-survey
25 companies
6 product managers
14 project leaders
11 programmers
[Haigh10]
NFR importance
e-survey
Not specified
162 users
110 managers
46 developers
[Anh12]
NFRs in OSS adoption
Questionnaire
15 companies
15 developers or project leaders
[Tang06]
Architecture design rationale
Questionnaire
Not specified
81 software architects
[Babar07]
Architecture design documentation and
validation
Structured group
discussion
10 companies
10 software architects
Ours
Management of NFRs by architects
Interviews
12 companies
13 software architects
17

18. References (I)
• R.B. Svensson, M. Höst, B. Regnell, ”Managing Quality Requirements: A
Systematic Review,” EUROMICRO-SEAA 2010.
• R.B. Svensson, T. Gorschek, B. Regnell, “Quality Requirements in Practice:
An Interview Study in Requirements Engineering for Embedded Systems,”
REFSQ 2009.
• R.B. Svensson, T. Gorschek, B. Regnell, R. Torkar, A. Shahrokni, R. Feldt, A.
Aurum, “Quality Requirements in Practice: An Interview St-udy in
Requirements Engineering for Embedded Systems,” RE 2011.
• A. Borg, A. Yong, P. Carlshamre, K. Sandahl, “The Bad Conscience of
Requirements Engineering: An Investigation in Real-World Treatment of
Non-Functional Requirements,” SERP 2003.
• J.L. de la Vara, K. Wnuk, R.B. Svensson, J. Sánchez, B. Regnell, “An
Empirical Study on the Importance of Quality Requirements in Industry,”
SEKE 2011.
18

19. References (II)
• M. Haigh, “Software Quality, Non-functional Software Requirements and
IT-business Alignment,” Software Quality Journal 18, 2010.
• N.D. Anh, D.S. Cruzes, R. Conradi, M. Höst, X. Franch, C.
Ayala, “Collaborative Resolution of Requirements Mismatches when
adopting Open Source Components,” REFSQ 2012.
• A. Tang, M. Ali Babar, I. Gorton, J. Han, “A Survey of Architecture Design
Rationale”. Journal of Systems and Software 79, 2006.
• M.A. Babar, L. Bass, I. Gorton, “Factors Influencing Industrial Practices of
Software Architecture Evaluation: An Empirical Investigation,” QoSA 2007.
19

20. Comments and Questions