Your SlideShare is downloading. ×
Getting Roi Numbers From Ut Des Moines10 Sept09
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Saving this for later?

Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime - even offline.

Text the download link to your phone

Standard text messaging rates apply

Getting Roi Numbers From Ut Des Moines10 Sept09

343
views

Published on

How to make usability more meaningful to your business measures

How to make usability more meaningful to your business measures


0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
343
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
0
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Getting ROI Numbers from Usability Testing John Sorflaten , PhD Certified Professional Ergonomist (CPE) Certified Usability Analyst (CUA) “ Midwife to successful web and application design” User testing, analysis, strategy, eCommerce, mobile apps User Experience Works http://www.linkedin.com/in/johnsorflaten (see SlideShare) www.userexperienceworks.com www.iphoneroi.com 641-455-1873
  • 2. Overview
    • Making usability more meaningful and consumable to business…
    • Quantitative usability testing (UT)
    • Converting UT results to business measures
    • Presenting UT results in relevant business terms
    • Why scare you with “Quantitative” word?
  • 3. 1. Beyond “More” and “Less”
    • Measure usability to learn “quantity” of…
      • Efficiency (time-on-task)
      • Effectiveness (task success rates)
      • Satisfaction (subjective ratings)
      • Learning (time to learn)
      • Errors (ouch)
    • “ Quantity” allows ROI calculations (!)
  • 4. Use these free ROI calculators…
    • http://www.humanfactors.com/downloads/roi.asp
  • 5. ROI Calculator — Increased Productivity…
    • http://www.humanfactors.com/training/roi_incr_prodvt.htm
  • 6. ROI Calculator — Increased Productivity…
    • http://www.humanfactors.com/training/roi_incr_prodvt.htm
  • 7. 2. Remember:“Margin of Error”
    • Like voting polls… measures are fuzzy. Remember +/- 3%? (You need 1067 people needed to get +/-3%…)
    • If 8 of 10 test participants succeed, DON’T say “80% passed.”
    • Best you can say: 48% to 96% passed.
    • www.measuringusability.com/wald.htm
  • 8. 3. Use “Time-on-Task” for ROI
    • Reduces margin-of-error for ROI calcs.
    • Use “geometric mean” not “average” (just like “median house price”)
    • What’s your real target for time savings? (test)
    http://www.measuringusability.com/time_intervals.php
  • 9. 4. Go for the “lower bound”…
    • www.humanfactors.com/downloads/quantitative-usability
    Finally, success…
  • 10. Converting UT Results to Business Measures
    • Efficiency (time-on-task) (DONE!)
    • Effectiveness (task success rates)
    • Satisfaction (subjective ratings)
    • Learning (time to learn)
    • Errors (ouch)
  • 11. 1. Effectiveness (“Pass” Rate)
    • www.humanfactors.com/downloads/quantitative-usability
    No real difference….
  • 12. Need spectacular contrasts
    • http://www.measuringusability.com/comp_sample.php
  • 13. Because these are — ”samples”
    • www.humanfactors.com/downloads/quantitative-usability
    No real difference….
  • 14. Better: compare with “population”
    • http://www.measuringusability.com/comp_sample.php
    A real difference….
  • 15. What’s the best we can say?
    • http://www.measuringusability.com/comp_sample.php
    Using what criteria? 80%? 90%….
  • 16. What’s the best we can say?
    • http://www.measuringusability.com/comp_sample.php
    Using what criteria? 80%? 90%….
  • 17. If high stakes: U need larger “N”
    • www.humanfactors.com/download/quantitative-usability
    Your benchmark: 85% will succeed on task…
  • 18. ROI = “profit only if usable by at least 85% of people who try…”
  • 19. Converting UT Results to Business Measures
    • Efficiency (time-on-task) (DONE!)
    • Effectiveness (task success rates) (DONE)
    • Satisfaction (subjective ratings)
    • Learning (time to learn)
    • Errors (ouch)
  • 20. 3. Satisfaction (>80% = goal)
    • http://measuringuserexperience.com/
    116/129 => 90 th Percentile scores of 81.2+ = 90 th Percentile
  • 21. 3. Satisfaction (goal: >80%)
    • http://en.wikipedia.org/wiki/System_Usability_Scale
  • 22. 3. Satisfaction (>80% = goal)
    • http://www.upassoc.org/usability_resources/conference/2004/UPA-2004-TullisStetson.pdf
  • 23. 3. Satisfaction (>80% = goal)
    • http://www.upassoc.org/usability_resources/conference/2004/UPA-2004-TullisStetson.pdf
  • 24. 3. Satisfaction (>80% = goal)
    • http://www.upassoc.org/usability_resources/conference/2004/UPA-2004-TullisStetson.pdf
  • 25. 3. Satisfaction (>80% = goal)
    • http://www.upassoc.org/usability_resources/conference/2004/UPA-2004-TullisStetson.pdf
  • 26. Final Report: Satisfaction
    • The test of the High Point Quote application with 18 participants (and 17 results on the SUS) resulted in an average score of 79.7.
    • This compares very well with the 81.2 score that represents the 90 th percentile. We conclude that the application created a very good feeling of “usability” in our test participants.
  • 27. Final Report: Success Rate
    • All 18 participants successfully completing the main task.
    • This implies that we can have 97.32% confidence of meeting our target that 85% of people like our participants will succeed.
    • We can also say that we have a 95% chance that at least 88% can do the task.
    http://www.measuringusability.com/onep.php
  • 28. Converting UT Results to Business Measures
    • Efficiency (time-on-task) (DONE!)
    • Effectiveness (task success rates) (DONE)
    • Satisfaction (subjective ratings) (DONE)
    • Learning (time to learn) (See time-on-task)
    • Errors (See time-on-task)
  • 29. UT Results in Business Terms
    • What percent of visitors can do the task?
    • Our income depends on your prediction!
    • Use estimates like….
      • 95% confidence that 88% of end-users can do it!
      • 60% chance that 85% can do it (uhmmmm)
      • 95% chance that 60% of end-users can do it…
      • ! Goforit…
    • E-mail your ideas: john@userexperienceworks.com