Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Usability modeling and measurement


Published on

This presentation discusses the importance of usability especially for web applications and how to model and measure it.

Published in: Technology, Design

Usability modeling and measurement

  1. 1. Usability Modeling and Measurement Philip
  2. 2. Agenda• Introduction – The importance of usability – Specifics for the web / mobile• What is Usability?• Usability Modeling and Measurements• Case study• Conclusion• Next Steps
  3. 3. Importance of Usability
  4. 4. Why is Usability Important?• Usability is important especially for web applications (SaaS, and websites) – And now especially mobile applications• Most prevalent development delivery model today• Without good usability: – Users will leave the applications – For mobile, if they can’t learn in 30 seconds, they won’t come back
  5. 5. Web and Mobile have Changed the Terrain• Business models have changed – Instead of paying upfront and ‘owning’ the software – Pay as you go, pay by subscription• Behavior and expectations have changed
  6. 6. Many Alternatives• Depending on the type of application – Users could leave and go elsewhere – Simply not use your mobile app (if only providing alternative access to a main application)
  7. 7. Thin Client Deployment Most Prevalent Development Delivery Model Today• Mobile and Cloud are singing the same song• Service delivery versus product delivery
  8. 8. What is Usability In relation to qualityIn relation to user experience
  9. 9. General Thoughts on Usability• Understandability• Learnability• Operability• Attractiveness• Navigation• Responsiveness-performance• Efficiency
  10. 10. Usability in Quality Standards• How is usability Standard Definition defined? ISO 9126-1 The capability of the software product to be – Standard (2000) understood, learned, used, definitions and attractive to the user, when used under specified – Many others conditions. who say ISO 9241-11 The extent to which a (1998) product can be used by similar specified users to achieve specified goals with things effectiveness, efficiency,• Usability and satisfaction in a specified context of use. – Component IEEE 600.12 The ease with which a user can learn to operate, of quality as (1990) prepare inputs for, and listed in interpret outputs of a system or component. many standards
  11. 11. Usability Model ComparisonsSeffah [33]
  12. 12. Current Research / Usability Standards Kappel et al., The Discipline ofSystematic Development of Web Applications, 2003, John Wiley and Sons.
  13. 13. Current Research – Usability Standards Quality Satisfaction Usability ISO 9241-11 ISO 9126-1 IEEE 610 ISO IEEE XXXY 25010
  14. 14. Bigger Picture Quality From ISO point of view Usability UsabilityCMMI ISO 25010
  15. 15. Usability as a Key Characteristic of Product Quality Source: ISO 25010
  16. 16. What is Usability-Effect of the Software ProductDegree to which specifiedusers can achieve specifiedgoals with effectiveness,efficiency and satisfaction in aspecified context of use. Source: ISO 25010
  17. 17. What is Usability-Effect In Actual Usage• Effectiveness – The degree to which specified users can achieve specified goals with accuracy and completeness in a specified context of use.• Efficiency – The degree to which specified users expend appropriate amounts of resources in relation to the effectiveness achieved in a specified context of use. – NOTE Relevant resources can include time to complete the task, materials, or the financial cost of usage.• Satisfaction – The degree to which users are satisfied in a specified context of use. Satisfaction is further subdivided into sub-characteristics: • Likability (cognitive satisfaction) • Pleasure (emotional satisfaction) • Comfort (physical satisfaction) • Trust Source: ISO 25010
  18. 18. Usability in Actual Usage• User role specified users• Objective specified goals• Task• Environment specified context of use• Domain What else can you think•… of?
  19. 19. Defining Usability For Your Organization Quality UsabilityCharacteristic 1 Characteristic 2 Characteristic nSubcharacteristic 1 Subcharacteristic 2 Subcharacteristic n Attribute 1 Attribute 2 Attribute 3 Attribute 1• Attributes expressed hierarchically• Any number of sub-levels is OK• Achieving the sub-attributes=achieving the high level attribute -measurable
  20. 20. Let’s Define Usability From the Product Viewpoint UsabilityCharacteristic 1 Navigation Understandable Characteristic 2 Characteristic nSubcharacteristic 1 Control Stability Subcharacteristic 2 Previous-Next Subcharacteristic n Attribute 1Home Location Attribute 2 Save Location Ease of finding Attribute n
  21. 21. Bigger Picture on Usability Usability Usability• Can be measured from the design point of view or of the product• Can be measured ‘in-use’ with real users
  22. 22. Defining Usability from an Effect-Real usage Point of View Usability QualityEffectiveness Efficiency Satisfaction Characteristic n Accuracy Completeness Subcharacteristic n Errors Attribute 2 Attribute 3 Attribute 1
  23. 23. Other Potential Attributes Measurements for ‘in use’ Usability• Effectiveness – Completion rates – Error rate – Help usage• Efficiency – Task time – Backtracking• Learnability – Learning rate – Task time deviation
  24. 24. Developing a Combined Usability Model Measured ‘in-use’ Usability Measurable and quantifiable factors User Behavior and ActivityApplication Design and Environment
  25. 25. Example of Product Quality Model with Attributes External Quality Requirements (for Shopping Cart Entity) 1 Usability (Operability in ISO 25010) 1.1 Understandability 1.1.1 Icon/label ease to be recognized 1.1.2 Information grouping cohesiveness 1.2 Learnability 1.2.1 ……………………………………………………….. 1.3 Operability 1.3.1 Control permanence 1.3.2 Expected behaviour of Controls 2 Content Quality 2.1 Content Suitability 2.1.1 Basic Information Coverage Line item information completeness Product description appropriateness 2.1.2 Coverage of other Contextual Information …………………………………………………………
  26. 26. Measurable Attributes• Attribute name• Description and purpose Once you have a model (what you• How to measure are going to• What is measured measure), then• Measurement/Calculation you start doing IT!• Range (min, max)• Objective• Current
  27. 27. Usability MeasurementAttribute Scale How Measure or Objective Current CalculationHelp Percent of Compare % 90% 40%completeness Menu items menus and with help items helpEase of Keystrokes Sample 50 % 3 12access to items find/use a feature/func tion/informa tionConsistency Number Examine integer 1 5 locations for menus and same button doc.Accuracy Number Collect from Integer or % <5 10 reported log files errors 27
  28. 28. Usability Measurement Methods Focus Walk groups Throughs Satisfaction Surveys Labs Heuristic Evaluation Logging Let’s get started
  29. 29. Current Research-Summary of Usability Evaluation Methods
  30. 30. Usability Logging Measurement and Data Collection• Identify users by using session ID to identify a unique user.• Iteratively insert code into the application• Collect data• Analyze the data for each attribute in different dimensions and aggregations• Determine the need for further calculations and what attributes to measure further• Revise the data we are collecting, adding or decreasing granularity
  31. 31. Satisfaction SurveysThe process:1. Calculate the usability score (satisfaction) of version X2. Do the survey3. Change to version X.1 – make changes to the software directly correlated to the usability factors to either increase or decrease the usability score4. Do the survey again5. See if differences made change impact the survey results
  32. 32. Notes on Satisfaction and Usability don’t have what I want I’m unsatisfied • Satisfaction is a subjective feeling Highly usable dependent on many software things other than usability: My password doesn’t work – A user can be highly satisfied but the application with lowNice weather I’m usability. today satisfied! – An application can be ! highly usable (highlow usability software usability) but the user is not satisfied! Finishedmy work today
  33. 33. Sample EvaluationExternal Quality Requirements Measure EI value P/GI value Global Quality Indicator 61.97% 1 Usability 60.88% 1.1 Understandability 83% 1.1.1 Icon/label ease to be recognized 100% 1.1.2 Information grouping cohesiveness 66% 1.2 Learnability 51.97% 1.2.1 ……………………………………………… … 1.3 Operability 49.50% 1.3.1 Control permanence 100% 1.3.2 Expected behaviour 50% 2 Content Quality 63.05% 2.1 Content Suitability 63.05% 2.1.1 Basic Information Coverage 50% Line item information completeness 2 50% Product description appropriateness 50% 2.1.2 Coverage of other Contextual Information 76.89% ……………………………………………….. … Return policy information completeness 33%
  34. 34. Healthcare SoftwareHeuristic Evaluation
  35. 35. Conclusion• Usability is an abstract concept• Defining usability is different for each organization• Need a model for your organization• The model is the foundation of what to measure• Once you can measure, then you can evaluate and improve
  36. 36. Next Steps• Produce an action plan – What usability attributes are important to your organization? • Develop a model – What data can you collect/Which technique can you use • Maybe some elements of the model drop out-can’t be measured that easily – Start collecting and developing benchmarks• Discuss with your manager/team
  37. 37. ThanksQuestions and Answers