Software Architects’ Experiences of Quality Requirements: What we Know and What we do not Know?


Published on

Abstract. [Context/motivation] Quality requirements (QRs) are a concern of both requirement engineering (RE) specialists and software architects (SAs). However, the majority of empirical studies on QRs take the RE analysts’/clients’ perspectives, and only recently very few included the SAs’ perspective. As a result, (i) relatively little is known about SAs’ involvement in QRs engineering and their coping strategies, and (ii) whatever is known mostly comes from small and midsized projects. [Question/problem] The question in this exploratory study is how SAs cope with QRs in the context of large and contract-based software system delivery projects. [Principal ideas/results] We executed an exploratory case study with 20 SAs in the context of interest. The key results indicate the role SAs play in QRs engineering, the type of requirements communication processes SAs are involved in, the ways QRs are discovered, documented, quantified, validated and negotiated. Our most important findings are that in contract-based contexts: (1) the QRs are approached with the same due diligence as the functional requirements and the architecture design demand, (2) the SAs act proactively and embrace responsibilities over the QRs, (3) willingness to pay and affordability seem as important QRs prioritization criteria as cost and benefits do, and (4) QRs engineering is perceived as a social activity and not as much as a tool and method centric activity.[Contribution] The main contributions of the paper are (i) the explication of the QRs process from SAs’ perspective, and (ii) the comparison of our findings with previously published results

Published in: Technology, Business
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Software Architects’ Experiences of Quality Requirements: What we Know and What we do not Know?

  1. 1. 1Software Architects’ Experiencesof Quality Requirements:What we Know and What we do not Know?Maya Daneva, Luigi Buglione, Andrea Herrmann
  2. 2. 2Table of Contents1. Introduction2. Background and Motivation3. The Research Design: Qualitative CaseStudy4. The Application of the Method Results Limitations1. Comparison to Literature2. Implications3. Wrap-UpMaya Daneva, Wed, April 16, 2009
  3. 3. 3Background and Motivation Most empirical research on QRs takes theperspective of clients, RE researchers,practitioners. Quality requirements (QRs) are a concern ofmultiple stakeholders; in particular: softwarearchitects (SAs) Relatively little is know about SA’s involvement Evidence comes from small and mid-sizedprojects; very few studies in large projects
  4. 4. 4The Research QuestionsResearch Goal: to understand how SAs cope with QRsin large and contract-based software systemdevelopment projects.1. How do the SAs understand their role?2. Do SAs and RE staff use different QRs terminology?3. How do QRs get elicited?4. How do QRs get documented?5. How do QRs get prioritized?6. How do QRs get quantified, if at all?7. How do QRs get validated?8. How do QRs get negotiated?9. What role does the contract play in the way SAs copewith QRs?
  5. 5. 5The Case Study Research DesignMaya Daneva, Wed, April 22, 2009Key Steps (R. Yin, 2008):1. Define interview guide2. Pilot the interview3. Collect data by interviewing participants4. Analyze the data5. Report on the results
  6. 6. 6Who Did We Interview?20 Architects from 14 companies in North Europe All have 10+ years of experience in large systems All work on large contract-based projects (3 devlocations and 2 client locations) Various pricing agreements Sectors: large IT vendors, Oil &Gas, Insurance,Real estate, Video streaming, online systems(travel, book store, games)
  7. 7. 7Who Did We Analyze the Data? Coding practices based on the groundedtheory book of K. Charmaz (2006) Iterative procedure
  8. 8. 8Results (1):How do the SAs understand their role? Formal job descriptions andcompetence models Self-described roles as: ‘a bridge’ b/n QRs and underlyingtechnology ‘a translator’ from the userlanguage into the featurespecification language ‘review gate keepers’ regardinge.g. contract compliance
  9. 9. 9Results (2): Do SAs and RE staff usedifferent terminology for QRs? Gaining communication clarity was a non-issue What helped? Domain knowledge Experience If ISO-certified, then training, qualitymanuals, product quality handbooksmade the difference Issue: interchangeable use of terms fromtwo streams of standards (management andtechnical how-to)
  10. 10. 10Results (3):How do QRs get elicited? 14 SAs use checklists based on: ISO standards Architecture frameworks Internal standards Stakeholder engagementstandards, AA1000SES 4 SAs uses game-based processes 2 SAs used story-telling techniques
  11. 11. 11Results (4):How do QRs get documented? 15 SAs: By using predefined templatesbased on : ISO standards the Quality Function Deploymentframework Planguage the INVEST grid approach Vendor-specific notations (e.g. SAP) 5 SAs: By using natural language
  12. 12. 12Results (5):How do QRs get prioritized? The business case is the driverbehind trade-offs, e.g. KPIs No particular prioritization method Prioritization criteria: cost, benefits, client’s willingness to pay affordability Who decides? Steering committees SAs
  13. 13. 13Results (6):How do QRs get quantified? Quantification is useful, but should not happentoo early How you get them? pre-specified in the contract Engage a specialist expert (e.g. inperformance) Decompose, operationalize and useestimation technique (IFPUG NFR) Issue: product and project measures are usedincorrectly QRs are confused with design-level req’ts
  14. 14. 14Results (7):How do QRs get validated? By using requirements walkthroughs The QFD framework Internal architecture standards
  15. 15. 15Results (8):How do QRs get negotiated? The business case is the commonlyused vehicle The QFD framework EasyWinWin The six-thinking-hat method
  16. 16. 16Results (9):What role does the contract play in the waySAs cope with QRs? 3 ways for influencing QRsprocesses: Cost-conciousness Service level agreements Pre-defined priorities for QRs Contracts are conductive as ways tomaintain control 3SAs: “contract is was not thatimportant” EasyWinWin
  17. 17. 17Limitations of the StudyThe possible validity concerns:1. External validity Similarity with contract-based system deliverysettings Application domain, organizational maturity1. Data collection threats: did SAs answer thequestions truthfully?2. Data analysis threats: researcher’s bias
  18. 18. 18Comparison to LiteratureInput: 5 empirical studiesvan Heesch et al, Ameller et al (2) Poort et al (2)1.QRs are: elicited by using checklists, documented by means of templates prioritized based on willingness to pay and affordability quantified by using size estimation standards (IFPUG) negotiated by using the business case1.SAs: Serve as ‘a bridge’ and have formal job descriptions, Have terminology (established by standards)1.Contract plays an important role for SAs to act the wayswe observed.
  19. 19. 19Implications1. To SAs: this study suggests QRs conversationsstart with contracts, SLA, and business cases2. To RE practitioners: your SA could be quite avaluable resource! She/he can save you time.3. To RE tool vendors: it seems more important tofigure out how to embed tools into socialprocesses and broader social interaction4. To RE researchers: extend the focus on methods,models and automation by including analysis ofQRs processes as socially constructed ones.
  20. 20. 20Wrap-Up1. We carried out an interview-based study2. It revealed what SAs were thinking on QR3. We compared and contrasted the findings withpublished literature4. Implications fro practice and research are made.
  21. 21. 21