Successfully reported this slideshow.

How do Software Architects consider Non-Functional Requirements


Published on

Published in: Technology, Business
  • Be the first to comment

How do Software Architects consider Non-Functional Requirements

  1. 1. How do Software Architects consider Non-Functional Requirements
  2. 2. Outline Objective/RQs Empirical method Observations Conclusions and related work References Example • • • • • 2
  3. 3. Objective / RQs • The research goal of this study is: How do software architects deal with non-functional requirements in practice? • Decomposed into several RQs: RQ1 RQ2 RQ3 RQ4 RQ5 RQ6 RQ7 What is the role of the software architect? Are there terminological confusions on NFRs? What types of NFRs are relevant to software architects? How are NFRs elicited? How are NFRs documented? How are NFRs validated? What type of tool support for NFRs is used? 3
  4. 4. Empirical method (I) • Type:  Exploratory study using qualitative research • Target population:  Software architects (or practitioners with that role)  We invited 21 software organizations (Spain)  We recruited 12 software organizations • We made 13 interviews (two on the same organization) 4
  5. 5. Empirical method (II) SCC: Software Consulting Company; SH: Software House; ITD: IT Department 5
  6. 6. Empirical method (III) • Data collection:  Semi-structured interviews  Focus: one single project  Duration: ~1 hour • 7 questions about the architect profile and development methodology used in the project • 10 questions about decision-making and NFRs  Audio-taped  Transcribed  Verified 6
  7. 7. Empirical method (IV) • Analysis:  Tabulation technique  Categories generation • NVivo Software  Counting technique 7
  8. 8. Empirical method (V) • Limitations and mitigations:         Evaluation apprehension  Confidentiality Understandability  Pilot testing (4) Influential factors  Single project Bad memory  Choose the project before The best project  Ask for the most familiar Omitting data  Add or modify Researcher bias  Two separated interpretations Generalize the findings  We do not attempt to make universal generalizations 8
  9. 9. Observations (I) • RQ1: What is the role of the software architect?  13 interviewees performed the tasks assigned to “software architects” in the project based on their experience or knowledge rather than their possible skills as architects  0 interviewees held a “software architect” position at the company  12 interviewees played other roles in the project in addition to the role of software architect, specifically: project manager (3), developer (5), and project manager and developer (4) 9
  10. 10. Observations (II) • RQ2: Are there terminological confusions on NFRs?  Confusion was reported around the terminology for designating NFR types  Inability to interpret some particular term, e.g., “availability”, “accuracy”, and “sustainability”  Use of a term that could lead to real confusion, e.g., some said “ergonomic”, “comfortable,” or “friendly,” when they meant “usable” (in the context of “usability”)  Use of a term with an incorrect definition. We found a serious confusion in the answer, e.g., “Maintainability is very important, because when something is working, we can’t make changes” 10
  11. 11. Observations (III) • RQ3: What types of NFRs are relevant to software architects? 10 9 8 7 6 5 4 3 2 1 0 10 9 8 7 6 5 4 3 2 1 0 • 49 references were made to technical NFRs • 33 references were made to non-technical NFRs 11
  12. 12. Observations (IV) • RQ4: How are NFRs elicited?  In 10 projects, the NFRs were elicited solely by the architect  In 3 projects, the NFRs were elicited by the client with the participation of the architect  13 architects considered elicitation as a gradual process 12
  13. 13. Observations (V) • RQ5: How are NFRs documented?  9 architects did not document the NFRs at all  4 architects documented the NFRs: • 3 used templates (1 only for initial NFRs) • 1 used plain text (only for initial NFRs) 13
  14. 14. Observations (VI) • RQ6: How are NFRs validated?  NFRs had been met by the end of the project? • 11 architects said yes • 2 architects did not say yes  What NFRs are validated? • 8 architects validated some types of NFRs – reliability, efficiency, usability, and accuracy • 1 architect did not validate any NFRs at all • 4 architects did not provide details 14
  15. 15. Observations (VII) • RQ7: What type of tool support for NFRs is used?  Architects did not use any specific tool support for NFR management  We found a very strong reaction (“I do not believe in automatic things”) against an automated decision-making tool  4 architects seem reluctant (“it is hard for me to imagine that this can be done”)  2 of the respondents said that the tool suggest alternatives instead of making final decisions 15
  16. 16. Conclusions Presentation to SEARCH • Observations made about NFRs     Companies did not have the role of architect clearly defined Architects didn't share a common vocabulary for NFRs NFRs were mostly elicited by the architects themselves Architects considered non-technical NFRs almost as relevant as technical NFRs  NFRs were poorly documented and validated 16
  17. 17. Related Work Subject of research Type of analysis Companies [Svensson10] Elicitation; dependencies; expression; cost estimation; prioritization SLR Not specified 1.560 candidate studies 18 selected studies [Svensson09] Presentation to SEARCH Ref. Importance of NFR types; dependencies; expression; satisfaction Interviews 5 companies 5 project leaders 5 product managers Interviews 11 companies 11 project leaders 11 product managers Interviews 2 companies 14 (different roles) [Svensson11] Prioritization Population [Borg03] NFR in general. Elicitation, documentation, test and management in particular [Vara11] NFR importance e-survey 25 companies 6 product managers 14 project leaders 11 programmers [Haigh10] NFR importance e-survey Not specified 162 users 110 managers 46 developers [Anh12] NFRs in OSS adoption Questionnaire 15 companies 15 developers or project leaders [Tang06] Architecture design rationale Questionnaire Not specified 81 software architects [Babar07] Architecture design documentation and validation Structured group discussion 10 companies 10 software architects Ours Management of NFRs by architects Interviews 12 companies 13 software architects 17
  18. 18. References (I) • R.B. Svensson, M. Höst, B. Regnell, ”Managing Quality Requirements: A Systematic Review,” EUROMICRO-SEAA 2010. • R.B. Svensson, T. Gorschek, B. Regnell, “Quality Requirements in Practice: An Interview Study in Requirements Engineering for Embedded Systems,” REFSQ 2009. • R.B. Svensson, T. Gorschek, B. Regnell, R. Torkar, A. Shahrokni, R. Feldt, A. Aurum, “Quality Requirements in Practice: An Interview St-udy in Requirements Engineering for Embedded Systems,” RE 2011. • A. Borg, A. Yong, P. Carlshamre, K. Sandahl, “The Bad Conscience of Requirements Engineering: An Investigation in Real-World Treatment of Non-Functional Requirements,” SERP 2003. • J.L. de la Vara, K. Wnuk, R.B. Svensson, J. Sánchez, B. Regnell, “An Empirical Study on the Importance of Quality Requirements in Industry,” SEKE 2011. 18
  19. 19. References (II) • M. Haigh, “Software Quality, Non-functional Software Requirements and IT-business Alignment,” Software Quality Journal 18, 2010. • N.D. Anh, D.S. Cruzes, R. Conradi, M. Höst, X. Franch, C. Ayala, “Collaborative Resolution of Requirements Mismatches when adopting Open Source Components,” REFSQ 2012. • A. Tang, M. Ali Babar, I. Gorton, J. Han, “A Survey of Architecture Design Rationale”. Journal of Systems and Software 79, 2006. • M.A. Babar, L. Bass, I. Gorton, “Factors Influencing Industrial Practices of Software Architecture Evaluation: An Empirical Investigation,” QoSA 2007. 19
  20. 20. Comments and Questions