Oplægget blev holdt ved InfinIT-arrangementet "Kan usability-arbejde virkelig betale sig?" der blev afholdt den 25. september 2013. Læs mere om arrangementet her: http://infinit.dk/dk/hvad_kan_vi_goere_for_dig/viden/reportager/kan_usability-arbejde_virkelig_betale_sig-.htm

1. Forskning i forholdet mellem gevinst og
omkostning ved usability-arbejde
Jan Stage
Professor, PhD
Forskningsleder i Informationssystemer (IS)/Human-Computer Interaction (HCI)
Aalborg Universitet, Institut for Datalogi
jans@cs.aau.dk

2. Institut for Datalogi 2
2
1. Introduktion
2. Forskningsresultater
3. Kom selv i gang
Oversigt

3. Institut for Datalogi 3
3
1. Introduktion
2. Forskningsresultater
3. Kom selv i gang
Oversigt

4. Institut for Datalogi
Usability
Definition (ISO 1998):
The extent to which a product can be used by specified users to
achieve specified goals with effectiveness, efficiency and satisfaction in
a specified context of use
Antagelse:
Dårlig usability koster – så hvis vi fjerner usability-problemer, bliver
vores medarbejdere/kunder mere effektive og gladere
4

5. Institut for Datalogi
Dårlig usability koster
• Evaluering af IKEA’s hjemmeside
”Det er vist lettere at køre derned” (til Århus)
• Evaluering af Budget Cars hjemmeside
Lej en bil på 60 sekunder
10 personer: 4-17 min.; snit: 7,5 min.
79 usability problemer (5 / 36 / 38)
• Dokumenteret, at det koster kunder
5

6. Institut for Datalogi 6
6
A simple login form with two fields (Email Address and Password), two buttons (Login and Register), and one
link (Forgot Password) prevented customers from purchasing products from a major e-commerce site. Users
would encounter it after they filled their shopping cart with products they wanted to purchase and pressed the
Checkout button.
A consultant conducted usability tests showing that first-time users resented having to. As one shopper said,
"I'm not here to enter into a relationship. I just want to buy something.“
The designers saw the form as enabling repeat customers to purchase faster, but they weren't any happier.
Except for a very few who remembered their login information, most stumbled on the form. They couldn't
remember the email address or password they used. Some would eventually ask the site to send the
password to their email address, which is a problem if you can't remember which email address you initially
registered with. The number of people who requested passwords was about 160,000 per day. 75% of them
never tried to complete the purchase once requested.
The designers fixed the problem by simply taking away the Register button. Instead, they put a Continue
button with a simple message: "You do not need to create an account to make purchases on our site. Simply
click Continue to proceed to checkout. To make your future purchases even faster, you can create an account
during checkout.“
The number of customers purchasing went up by 45%. The extra purchases resulted in an extra $15 million
the first month. For the first year, the site saw additional purchases of $300,000,000. (Spool, 2009)
High score eksemplet

7. Institut for Datalogi 7
7
1. Introduktion
2. Forskningsresultater
• Fire typer benefits
• Empiriske målinger
• Costs
3. Kom selv i gang
Oversigt

8. Institut for Datalogi
Benefits (1)
A. Software Development
1. Reduced development costs
2. Reduced amount of irrelevant functionality
3. Savings gained from making changes earlier in design lifecycle
4. Reduced need for producing documentation
8

9. Institut for Datalogi
A. Software Development
The rule of thumb in many usability-aware organizations
is that the cost-benefit ratio for usability is $1:$10-$100.
Once a system is in development, correcting a problem
costs 10 times as much as fixing the same problem in design. If the system has been
released, it costs 100 times as much relative to fixing in design. (Gilb, 1988)
In one large commercial client, the time to execute a quality assurance test fully was
reduced by 85% because of a decrease in features brought about through the application
of formal usability engineering and testing during the early phases of development. This
amounted to a savings of approximately $15 million and a reduction in schedule by 18
months over the prior software development iteration. (Mauro, 2002)
Usability techniques allowed a high-tech company to reduce the time spent on one
tedious development task by 40% (Bias & Mayhew, 1994). At another company, usability
techniques helped cut development time by 33-50%. (Bosert, 1991)
Sun Microsystems has shown how spending about $20,000 could yield a savings of $152
million dollars. Each and every dollar invested could return $7,500 in savings. (Rhodes,
2000)
9

10. Institut for Datalogi
Benefits (2)
B. User Organization
1. Increased user productivity
2. Decreased user errors
3. Decreased training costs
4. Decreased user help and support costs
5. Decreased unofficial support (from colleagues)
6. Increased user satisfaction
10

11. Institut for Datalogi
Brugere: tidsforbrug og produktivitet
Et stort distribueret system:
• 2000 sagsbehandlere
• 20 sager pr. bruger pr. dag
• 40.000 sager om dagen
• Sagsbehandlerens tidsforbrug pr. sag: 5-20 minutter
• Besparelse pr. sag på grund af forbedret usability: 30 sekunder
• Timeløn for sagsbehandleren: 150 kr.
• Besparelse pr. år:
= 2000 brugere * 20 sager/bruger/dag
* 230 dage * 0,5 minutter * 1 time/60 minutter * 150 kr./time
= 11.500.000 kr. pr. år
Uanset hvad et grundigt usability-arbejde koster for dette system, så er
gevinsten kolossal
Inspireret af Rex Hartson (http://courses.cs.vt.edu/~cs5714/fall2000/notes/wrap_up/wrap_up.htm)
11

12. Institut for Datalogi
B. User Organization
One study at NCR showed a 25% increase in throughput with an additional 25% decrease
in errors resulting from redesign of screens to follow basic principles of good design.
(Gallaway, 1981)
A major computer company spent $20,700 on usability work to improve the sign-on
procedure in a system used by several thousand people. The resulting productivity
improvement saved the company $41,700 the first day the system was used.
On a system used by over 100,000 people, for a usability outlay of $68,000, the same
company recognized a benefit of $6,800,000 within the first year of the system’s
implementation. This is a cost-benefit ratio of $1:$100. (Bias & Mayhew, 1994)
When systems match user needs, satisfaction often improves dramatically. In a 1992
Gartner Group study, usability methods raised user satisfaction ratings for a system by
40%. (Bias & Mayhew, 1994)
Design changes due to usability work at IBM resulted in an average reduction of 9.6
minutes per task, with projected internal savings at IBM of $6.8 Million in 1991 alone.
(Karat, 1990)
12

13. Institut for Datalogi
Benefits (3)
C. External Software Vendor
1. Increased sales
2. Reduced customer support costs
3. Reduced cost of training provisions (if training is offered by the
vending company)
4. Competitive advantage (usability branding)
13

14. Institut for Datalogi
C. External Software Vendor
Wixon & Jones did a case study of a usability-engineered software product that increased
revenue by more than 80% over the first release of the product (built without usability work)
(Wixon). The revenues of the usability-enhanced system were 60% higher than projected.
Many customers cited usability as a key factor in buying the new system. (Bias & Mayhew,
1994)
Usability also plays a role in the public’s perception of a company, affecting brand value and
market share. About 15% of the space in reviews published in trade magazines, journals,
and national newspapers is devoted to user friendliness or usability. (Nielsen, 1993)
14

15. Institut for Datalogi
Benefits (4)
D. Web Site Use
1. Enhanced retention rate
2. Reduced shopping cart abandonment
3. Competitive advantage (satisfied customers return)
4. Better information quality (interactive sites)
15

16. Institut for Datalogi
D. Web Site Use
You can increase sales on your site as much as 225% by providing sufficient product
information to your customers at the right time. In our recent research, we found that the
design of product lists directly affected sales. On sites that did not require shoppers to
bounce back-and-forth between the list and individual product pages, visitors added more
products to their shopping cart and had a more positive opinion of the site. (User Interface
Engineering, 2001)
One study estimated that improving the customer experience increases the number of
buyers by 40% and increase order size by 10%. (Creative Good, 2000)
The magnitude of usability improvements is usually large. This is not a matter of
increasing use by a few percent. It is common for usability efforts to result in a hundred
percent or more increase in traffic or sales. (Nielsen, July 1999)
In a 1999 study of Web users, respondents were asked to list the five most important
reasons to shop on the Web. Pricing was only the third-most important. The top reason
was “Easy to place an order” by 83% of the respondents. (Nielson, February 1999)
16

17. Institut for Datalogi
Costs
Tidsforbrug:
• Planlægning og forberedelse
• Udførelse
• Fortolkning og dokumentation
Brugere:
• Rekruttering af brugere
• Betaling af brugere
Leje:
• Leje af laboratorium og udstyr
Se litteratur: Kohno et al. 2013, Bevan 2005, Wong 2009
17

18. Institut for Datalogi 18
18
1. Introduktion
2. Forskningsresultater
3. Kom selv i gang
Oversigt

19. Institut for Datalogi
Sælg ideen til ledelsen
Start i det små:
• Forsøg med usability-arbejde på en lille del af et projekt
• Prøv først at løse et afgrænset antal usability-problemer
• Sæt et lille usability-laboratorium op (symbolværdi)
Relation til ledelsen:
• Klargør mål, forventede gevinster og tidsramme
• Få de nødvendige ressourcer afsat
• Få klar ledelsesopbakning
Udviklingsteamet:
• Skaf mindst én person med den relevante kompetence i interaktionsdesign og
usability, og giv ham/hende ansvar og autoritet
• Giv teamet den nødvendige træning
• Få medlemmerne af teamet til at forpligte sig
19

20. Institut for Datalogi
Litteratur (1)
Den klassiske bog på området (artikelsamling):
Bias, R. G. and Mayhew, D. J. (Eds.) (1994) Cost-Justifying Usability. San Francisco: Morgan
Kaufmann Publishers.
Bias, R.G. & Mayhew, D.J. (2005) Cost-Justifying Usability: An Update for the Internet Age. San
Francisco: Morgan Kaufmann Publishers.
Oversigter over costs-benefit cases:
Bevan, N. (2005) Cost benefits evidence and case studies. Adapted from Bias and Mayhew (2005)
http://www.nigelbevan.com/papers/Cost_benefits_evidence.pdf
Marcus, A. (2004) Return on Investment for Usable User-Interface Design: Examples and Statistics.
http://www.amanda.com/joomla_uploads/whitepapers/AM+A_ROIWhitePaper_20Apr0%201.pdf
20

21. Institut for Datalogi
Litteratur (2)
Det mest markante eksempel på benefit:
Spool, J. (2009) The $300 Million Button
http://www.uie.com/articles/three_hund_million_button/
Metode til måling af UCD cost-benefit, som har været anvendt til måling på 22 projekter i deres
virksomhed:
Kohno, I., Yasu, H., Sugawara, S. and Nishikawa, M. (2013) Pragmatic Approach to Cost Benefit
Analysis of User Centered Design. In: Design, User Experience, and Usability. Design Philosophy,
Methods, and Tools. Lecture Notes in Computer Science Volume 8012, 2013, pp 525-534.
Konkret eksempel på en cost-benefit analyse med detaljerede tal:
Wong, C. Y. (2009) Applying Cost-Benefit Analysis for Usability Evaluation Studies on an Online
Shopping Website.
http://www.hft.org/HFT03/paper03/09_Won.pdf
21

22. Institut for Datalogi
Vores forskningsaktiviteter
En væsentlig del af vores forskning handler om at finde løsninger, der
reducerer eller helt fjerner vigtige forhindringer for usability-arbejde i
praksis:
• Pris
• Kompetence
Eksempler:
• Barfods-evaluatorer
• Instant Data Analysis (IDA)
• Remote Synkron/Asynkron evaluering
Kontakt:
• Jan Stage, Aalborg Universitet, Institut for Datalogi
• jans@cs.aau.dk
22