Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
BLEKINGE INSTITUTE OF TECHNOLOGY 
Quality Requirements Elicitation 
Based on Inquiry of 
Quality-Impact Relationships 
Far...
BLEKINGE INSTITUTE OF TECHNOLOGY 
Quality Requirements 
ISO/IEC FDIS 25010
BLEKINGE INSTITUTE OF TECHNOLOGY 
Goal Modeling 
[Chung et al, 2000]
BLEKINGE INSTITUTE OF TECHNOLOGY 
Research Problem 
 How can we determine the appropriate quality levels for 
requirement...
BLEKINGE INSTITUTE OF TECHNOLOGY 
Agenda 
 Quality-Impact Relationships 
 Requirements Elicitation Method 
 Real-World ...
BLEKINGE INSTITUTE OF TECHNOLOGY 
 A quality-impact relationship is a function between 
software quality levels and their...
BLEKINGE INSTITUTE OF TECHNOLOGY 
 In prior empirical work, we have explored one kind of quality-impact 
relationship: Qu...
BLEKINGE INSTITUTE OF TECHNOLOGY 
 Quality-impact relationships make the pragmatic meaning 
of quality levels explicit. 
...
BLEKINGE INSTITUTE OF TECHNOLOGY 
Quality-Impact Inquiry
BLEKINGE INSTITUTE OF TECHNOLOGY 
Diabetes Self-Management App 
Data sharing 
with consulting doctor 
Self-monitoring of g...
BLEKINGE INSTITUTE OF TECHNOLOGY 
The Requirement Problem 
 Type of quality impact, qimp: user acceptance 
 Type of qual...
BLEKINGE INSTITUTE OF TECHNOLOGY 
1. Preparation 
 Prototype  Questionnaire 
qmsr: response time 
measurement 
facility ...
BLEKINGE INSTITUTE OF TECHNOLOGY 
2. Measurement 
Measurement qmsr: response 
time 
qimp: user’s 
acceptance 
Rationale 
M...
BLEKINGE INSTITUTE OF TECHNOLOGY 
3. Analysis 
 Option A, use existing generic relationship 
 Option B, develop new mode...
BLEKINGE INSTITUTE OF TECHNOLOGY 
4. Decision-Making 
 1- Use industry standard, competitive analysis [5], or 
collected ...
BLEKINGE INSTITUTE OF TECHNOLOGY 
Example of Workshop
BLEKINGE INSTITUTE OF TECHNOLOGY 
Example Scenario 
i) selection of 
patient data 
ii) patient 
authentication 
iii) data ...
BLEKINGE INSTITUTE OF TECHNOLOGY 
Example Measurement 
 Log-file 
… 
… 
Measurement qmsr: response 
time 
Measurement 1 1...
BLEKINGE INSTITUTE OF TECHNOLOGY 
Example Analysis 
 Blue dots: collected data 
 Compensation of lack of extensive 
data...
BLEKINGE INSTITUTE OF TECHNOLOGY 
Example Decision 
 Targeted quality impact, qimp: user acceptance 
– MOS 4 “good” (base...
BLEKINGE INSTITUTE OF TECHNOLOGY 
Discussion 
 Variations 
 Feasibility in Practice 
 Future Research
BLEKINGE INSTITUTE OF TECHNOLOGY 
Variations (1) 
 Different generic 
relationships to 
describe the 
impact function 
– ...
BLEKINGE INSTITUTE OF TECHNOLOGY 
Variations (2) 
 Software Features 
 Stakeholder Sampling 
… 
patient doctor 
Many pat...
BLEKINGE INSTITUTE OF TECHNOLOGY 
Variations (3) 
QoS response time 
bad 
poor 
fair 
good 
excellent 
Quality Attributes ...
BLEKINGE INSTITUTE OF TECHNOLOGY 
Feasibility in Practice 
 Experienced requirements engineers vs. junior 
requirements e...
BLEKINGE INSTITUTE OF TECHNOLOGY 
Future Research 
 Validating and evaluating the method in large-scale 
requirement engi...
BLEKINGE INSTITUTE OF TECHNOLOGY 
Summary 
 Problem: How to determine the appropriate level of 
quality? 
 Approach: Qua...
BLEKINGE INSTITUTE OF TECHNOLOGY 
Thank you 
samuel.fricker@bth.se 
Twitter @samuelfricker
Upcoming SlideShare
Loading in …5
×

How to Determine Appropriate Software Quality

784 views

Published on

Too little quality will lead to disappointed users, while too much quality is costly, inefficient, and unsustainable. In this talk we show how to elicit appropriate quality levels with quality-impact relationships. The results help to adjust quality in case of negative user feedback and to prespare service level agreements (SLA) based on empirical evidence.

Published in: Software
  • Be the first to comment

How to Determine Appropriate Software Quality

  1. 1. BLEKINGE INSTITUTE OF TECHNOLOGY Quality Requirements Elicitation Based on Inquiry of Quality-Impact Relationships Farnaz Fotrousi, Samuel A. Fricker, Markus Fiedler Please cite as follows: F. Fotrousi, S. Fricker, M. Fiedler (2014). "Quality Requirements Elicitation based on Inquiry of Quality-Impact Relationships", IEEE International Requirements Engineering Conference (RE'14), Karlskrona, Sweden.
  2. 2. BLEKINGE INSTITUTE OF TECHNOLOGY Quality Requirements ISO/IEC FDIS 25010
  3. 3. BLEKINGE INSTITUTE OF TECHNOLOGY Goal Modeling [Chung et al, 2000]
  4. 4. BLEKINGE INSTITUTE OF TECHNOLOGY Research Problem  How can we determine the appropriate quality levels for requirements that are specified with a scale, rather than with operationalized goals?  Quality attributes such as – Performance such as response time – Reliability such as mean-time-to-failure – etc.  Impact of the problem: – Too little quality disappoints users – Too much quality is costly and inefficient
  5. 5. BLEKINGE INSTITUTE OF TECHNOLOGY Agenda  Quality-Impact Relationships  Requirements Elicitation Method  Real-World Example of Method Application  Discussion and Future Work  Summary 5
  6. 6. BLEKINGE INSTITUTE OF TECHNOLOGY  A quality-impact relationship is a function between software quality levels and their impacts for a given pair of quality attribute and impact.  Example (positive) – Software quality level: 0.1 seconds response time – Impact: user thinks the software is excellent  Example (negative) – Software quality level: 10 seconds response time – Impact: user thinks the software is bad Quality-Impact Relationships
  7. 7. BLEKINGE INSTITUTE OF TECHNOLOGY  In prior empirical work, we have explored one kind of quality-impact relationship: Quality of Service (QoS) and Quality of Experience (QoE) for telecommunication services [10] Prior Work QoS response time bad poor fair good excellent Type of quality: qmsr Quality level Quality impact level QoE mean opinion score Type of quality impact: qimp Quality-impact relationship: qimp = α * e-β*qmsr + ɣ
  8. 8. BLEKINGE INSTITUTE OF TECHNOLOGY  Quality-impact relationships make the pragmatic meaning of quality levels explicit. – instead of just saying we need 0.1 sec response time… – …we know that the user will be happy with it  Quality-impact inquiry method design: – Framework of inquiry-based requirements analysis – Supporting methods: • Prototype: enable the quality experience and measurement (qmsr) • Questionnaire: collect user experience data (qimp) • Workshop: efficient, controlled setting Key Ideas
  9. 9. BLEKINGE INSTITUTE OF TECHNOLOGY Quality-Impact Inquiry
  10. 10. BLEKINGE INSTITUTE OF TECHNOLOGY Diabetes Self-Management App Data sharing with consulting doctor Self-monitoring of glucose, stress, etc.
  11. 11. BLEKINGE INSTITUTE OF TECHNOLOGY The Requirement Problem  Type of quality impact, qimp: user acceptance  Type of quality, qmsr: response time (waiting time) What waiting time shall be acceptable for the user (patient) until data sharing is completed?
  12. 12. BLEKINGE INSTITUTE OF TECHNOLOGY 1. Preparation  Prototype  Questionnaire qmsr: response time measurement facility qimp: user’s acceptance  Call for Workshop Overall, how satisfied are you with the feature you just have experienced? □ Excellent (5) □ Good (4) □ Fair (3) □ Poor (2) □ Bad (1) Please tell us why you feel that way:
  13. 13. BLEKINGE INSTITUTE OF TECHNOLOGY 2. Measurement Measurement qmsr: response time qimp: user’s acceptance Rationale Measurement 1 1.26 seconds 4 …because of… Measurement 2 0.22 seconds 5 …because of… Etc.
  14. 14. BLEKINGE INSTITUTE OF TECHNOLOGY 3. Analysis  Option A, use existing generic relationship  Option B, develop new model User’s acceptance (qimp) Response time (qmsr) I did not feel disturbed, and everything was working. I thought that the software was wrong Rationales 5 4 3 2
  15. 15. BLEKINGE INSTITUTE OF TECHNOLOGY 4. Decision-Making  1- Use industry standard, competitive analysis [5], or collected rationales to set the appropriate quality impact.  2- Then look up the desired quality. SRS 1 2
  16. 16. BLEKINGE INSTITUTE OF TECHNOLOGY Example of Workshop
  17. 17. BLEKINGE INSTITUTE OF TECHNOLOGY Example Scenario i) selection of patient data ii) patient authentication iii) data submission iv) questionnaire
  18. 18. BLEKINGE INSTITUTE OF TECHNOLOGY Example Measurement  Log-file … … Measurement qmsr: response time Measurement 1 1.08 seconds
  19. 19. BLEKINGE INSTITUTE OF TECHNOLOGY Example Analysis  Blue dots: collected data  Compensation of lack of extensive data by using existing generic model [10] QoS response time bad poor fair good excellent Curve matching
  20. 20. BLEKINGE INSTITUTE OF TECHNOLOGY Example Decision  Targeted quality impact, qimp: user acceptance – MOS 4 “good” (based on stakeholder consensus)  Type of quality, qmsr: response time (waiting time) – 1.26 s (based on specific quality-impact relationship) What waiting time shall be acceptable for the user (patient) until data sharing is completed?
  21. 21. BLEKINGE INSTITUTE OF TECHNOLOGY Discussion  Variations  Feasibility in Practice  Future Research
  22. 22. BLEKINGE INSTITUTE OF TECHNOLOGY Variations (1)  Different generic relationships to describe the impact function – Linear – Exponential – …  Simulated quality levels (through prototype)
  23. 23. BLEKINGE INSTITUTE OF TECHNOLOGY Variations (2)  Software Features  Stakeholder Sampling … patient doctor Many patients Many doctors data transfer chatting
  24. 24. BLEKINGE INSTITUTE OF TECHNOLOGY Variations (3) QoS response time bad poor fair good excellent Quality Attributes Impact Attributes ISO/IEC FDIS 25010
  25. 25. BLEKINGE INSTITUTE OF TECHNOLOGY Feasibility in Practice  Experienced requirements engineers vs. junior requirements engineers  Construction of Service Level Agreements  Extension of the requirements engineering toolset  Complementing competitive analysis of product quality
  26. 26. BLEKINGE INSTITUTE OF TECHNOLOGY Future Research  Validating and evaluating the method in large-scale requirement engineering situations  Expanding the understanding of the generic relationships between combinations of software quality attributes and their impacts as well quality attributes relationship.  Scaling: how to get a sufficient number of data points
  27. 27. BLEKINGE INSTITUTE OF TECHNOLOGY Summary  Problem: How to determine the appropriate level of quality?  Approach: Quality-Impact inquiry method  Example  Key insights
  28. 28. BLEKINGE INSTITUTE OF TECHNOLOGY Thank you samuel.fricker@bth.se Twitter @samuelfricker

×