Beyond Just Usability: 
Desirability & Usefulness Testing 
Susan Mercer, Senior Experience Researcher 
@susanamercer
Usability Testing
Usability = (Efficiency + Effectiveness) 
In a given 
context of use
Usable
Cute
Acerbic
Usability 
Desirability 
Usefulness
Usability 
Consistent 
Organized 
Understandable 
Simple 
Efficient
Usefulness 
Effec=ve 
Essen=al 
Relevant 
Helpful 
Suppor=ng
Desirability 
Exci=ng 
Fresh 
Friendly 
Encouraging 
Meaningful
Usability 
Desirability 
Usefulness
DESIRABILITY 
TESTING METHODS
Open-­‐Ended Ques,oning 
• What do you think of…? 
• What is your reac,on to…? 
• How would you describe…?
Sentence Comple,on 
• This app makes me feel… 
• If I used this all day, I would… 
• Reading on this website is… 
• Compar...
Product Reac,on Cards 
• 118 words 
• 58% posi,ve 
• 36% nega,ve 
• 6% unclear 
• 21% synonyms with 1+ other words 
• Most...
Product Reac,on Cards – Alternate List 
• Removed synonyms 
• Added opposites to create Posi,ve / Nega,ve Pairs 
• Compare...
Annoying 
Costly 
Expected 
Interes=ng 
Poor 
quality 
Trivial 
Apathe=c 
Crea=ve 
Familiar 
Irrelevant 
Predictable 
UnaJ...
AJrac=ve 
UnaJrac=ve 
Excep=onal 
Common 
Organized 
Disorganized 
Calm 
Busy 
Exci=ng 
Dull 
Original 
Ordinary 
Clean 
C...
Reliable 
Accessible 
Appealing 
Confusing 
Comfortable 
Organized 
Professional 
Busy 
Confident 
Straight 
Forward 
Appr...
68 
67 
62 
42 
20 
22 
17 
31 
70 
60 
50 
40 
30 
20 
10 
0 
Exis1ng 
Design 
A 
Design 
B 
Design 
C 
Count 
Number 
of...
Exis1ng 
Design 
A 
Design 
B 
Design 
C 
Appealing 
Appealing 
Appealing 
Appealing 
Clean 
Clean 
Clean 
Clean 
Efficien...
Product Reac,on Dyads 
Rate on a 5-­‐point scale from Posi,ve – Nega,ve word pairs 
AJrac=ve 
¤ 
¢ 
¢ 
¢ 
¢ 
UnaJrac=...
-­‐3.5 
-­‐2.5 
-­‐1.5 
-­‐0.5 
0.5 
1.5 
2.5 
3.5 
Discouraging 
Unrefined 
Useless 
Complex 
Irrelevant 
Boring 
Expecte...
Photo Selec,on
LEM Tool 
(Susa Group)
Open-­‐Ended Ques,oning 
Sentence Comple,on 
Product Reac,on Cards 
Product Reac,on Dyads 
Photo Selec,on 
LEM Tool
Usability 
Desirability 
Usefulness
USEFULNESS 
TESTING
Pre-­‐Usability Interview 
• How do you reconcile invoices today…? 
• Who needs to approve these and why…? 
• Where do you...
During Usability Tes,ng 
Watch for comments about usefulness 
• “Wow. This would really make it easier to reorder the same...
Post-­‐Usability Interview 
Show screens again and ask: 
• “How would this work for you in your job?” 
• “What about this ...
TESTING 
METHODS 
Desirability Tes,ng: 
• Open-­‐Ended Ques,oning 
• Sentence Comple,on 
• Product Reac,on Cards 
• Produc...
Thank You! 
Susan Mercer 
smercer@madpow.com 
@susanamercer
Susan Mercer's UXPA 2014 presentation "Beyond Just Usability: Desirability & Usefulness Testing".
Susan Mercer's UXPA 2014 presentation "Beyond Just Usability: Desirability & Usefulness Testing".
Susan Mercer's UXPA 2014 presentation "Beyond Just Usability: Desirability & Usefulness Testing".
Susan Mercer's UXPA 2014 presentation "Beyond Just Usability: Desirability & Usefulness Testing".
Susan Mercer's UXPA 2014 presentation "Beyond Just Usability: Desirability & Usefulness Testing".
Upcoming SlideShare
Loading in...5
×

Susan Mercer's UXPA 2014 presentation "Beyond Just Usability: Desirability & Usefulness Testing".

161

Published on

In this talk, Susan discusses several techniques that she has used to evaluate the usefulness and desirability of interfaces. The best techniques to evaluate usefulness involve open-ended interview questions regarding current processes and pain points, followed by a usability evaluation of the interface and then a reflective interview discussing the benefits and drawbacks of that solution to their personal situation.

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
161
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
3
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Susan Mercer's UXPA 2014 presentation "Beyond Just Usability: Desirability & Usefulness Testing".

  1. 1. Beyond Just Usability: Desirability & Usefulness Testing Susan Mercer, Senior Experience Researcher @susanamercer
  2. 2. Usability Testing
  3. 3. Usability = (Efficiency + Effectiveness) In a given context of use
  4. 4. Usable
  5. 5. Cute
  6. 6. Acerbic
  7. 7. Usability Desirability Usefulness
  8. 8. Usability Consistent Organized Understandable Simple Efficient
  9. 9. Usefulness Effec=ve Essen=al Relevant Helpful Suppor=ng
  10. 10. Desirability Exci=ng Fresh Friendly Encouraging Meaningful
  11. 11. Usability Desirability Usefulness
  12. 12. DESIRABILITY TESTING METHODS
  13. 13. Open-­‐Ended Ques,oning • What do you think of…? • What is your reac,on to…? • How would you describe…?
  14. 14. Sentence Comple,on • This app makes me feel… • If I used this all day, I would… • Reading on this website is… • Compared to other apps, this app is… • I want to use this because… • I do not want to use this because…
  15. 15. Product Reac,on Cards • 118 words • 58% posi,ve • 36% nega,ve • 6% unclear • 21% synonyms with 1+ other words • Mostly posi,ve words
  16. 16. Product Reac,on Cards – Alternate List • Removed synonyms • Added opposites to create Posi,ve / Nega,ve Pairs • Compared against two other emo,onal word vocabularies for Product Appeal and Emo,ons • Marc Hassenzhahl, Darmstadt University of Technology, Germany • Geneva Emo,on Wheel
  17. 17. Annoying Costly Expected Interes=ng Poor quality Trivial Apathe=c Crea=ve Familiar Irrelevant Predictable UnaJrac=ve AJrac=ve Desirable Fast Managable Professional Uncontrollable Boring Difficult Flexible Meaningful Relaxing Understandable Busy Difficult Fragile Mo=va=ng Relevant Undesirable Calm Disappointed Fresh Nondescript Reliable Unexci=ng Cheap Discouaraging Friendly Not Secure Rigid Unimagina=ve Clean Disorganized Frustra=ng Not Valuable Sa=sfying Unimportant Clear Dull Fun Novel Secure Unnecessary CluJered Easy to use Hard to Use Obstruc=ng Simple Unpredictable Common Effec=ve High quality Old Slow Unprofessional Compelling Efficient Impersonal Op=mis=c Sophis=cated Unrefined Complex Effortless Impressive Ordinary Stable Unreliable Comprehensive Empowering Incomplete Organized Standard Unusable Confident Encouraging Incomprehensible Original Strange Unwelcoming Confusing Enthusias=c Inconsistent Overwhelming Stressful Usable Conserva=ve Essen=al Ineffec=ve Patronizing Suppor=ng Useful Consistent Excep=onal Inefficient Personal Tedious Useless Controllable Exci=ng Innova=ve Pessimis=c Time-­‐consuming Valuable Convenient Exclusive Inspiring Pleasant Time-­‐Saving Weak
  18. 18. AJrac=ve UnaJrac=ve Excep=onal Common Organized Disorganized Calm Busy Exci=ng Dull Original Ordinary Clean CluJered Exclusive Standard Personal Impersonal Clear Confusing Familiar Strange Pleasant Annoying Compelling Unimportant Fast Slow Predictable Unpredictable Comprehensive Incomplete Flexible Rigid Professional Unprofessional Confident Weak Fresh Old Relaxing Stressful Consistent Inconsistent Friendly Unwelcoming Relevant Irrelevant Controllable Uncontrollable Frustra=ng Encouraging Reliable Unreliable Convenient Difficult Fun Tedious Sa=sfying Disappointed Costly Cheap High quality Poor quality Secure Not Secure Crea=ve Unimagina=ve Impressive Nondescript Simple Complex Desirable Undesirable Innova=ve Conserva=ve Sophis=cated Unrefined Easy to use Hard to use Inspiring Unexci=ng Stable Fragile Effec=ve Ineffec=ve Interes=ng Boring Suppor=ng Obstruc=ng Efficient Inefficient Manageable Overwhelming Time-­‐saving Time-­‐consuming Effortless Difficult Meaningful Trivial Understandable Incomprehensible Empowering Patronizing Mo=va=ng Discouraging Usable Unusable Enthusias=c Apathe=c Novel Expected Useful Useless Essen=al Unnecessary Op=mis=c Pessimis=c Valuable Not Valuable
  19. 19. Reliable Accessible Appealing Confusing Comfortable Organized Professional Busy Confident Straight Forward Approachable Complex Effec=ve Understandable Efficient Dull Friendly Easy to Use Flexible Hard to Use Invi=ng Clear Helpful Impersonal Time-­‐saving Convenient Overwhelming Trustworthy Clean Boring Useful Familiar Frustra=ng Simplis=c In=mida=ng Valuable Stressful Usable Time-­‐consuming Customers Customers & Non-­‐Customers Non-­‐Customers
  20. 20. 68 67 62 42 20 22 17 31 70 60 50 40 30 20 10 0 Exis1ng Design A Design B Design C Count Number of Words Posi=ve Nega=ve
  21. 21. Exis1ng Design A Design B Design C Appealing Appealing Appealing Appealing Clean Clean Clean Clean Efficient Efficient Efficient Efficient Engaging Engaging Engaging Intui=ve Professional Professional Professional Useful Useful Useful Useful Total 9 20 16 9
  22. 22. Product Reac,on Dyads Rate on a 5-­‐point scale from Posi,ve – Nega,ve word pairs AJrac=ve ¤ ¢ ¢ ¢ ¢ UnaJrac=ve Calm ¢ ¤ ¢ ¢ ¢ Busy Clean ¢ ¤ ¢ ¢ ¢ CluJered Clear ¤ ¢ ¢ ¢ ¢ Confusing Comprehensive ¢ ¢ ¢ ¤ ¢ Incomplete
  23. 23. -­‐3.5 -­‐2.5 -­‐1.5 -­‐0.5 0.5 1.5 2.5 3.5 Discouraging Unrefined Useless Complex Irrelevant Boring Expected Unprofessional Mo=va=ng Sophis=cated Useful Simple Relevant Interes=ng Novel Professional
  24. 24. Photo Selec,on
  25. 25. LEM Tool (Susa Group)
  26. 26. Open-­‐Ended Ques,oning Sentence Comple,on Product Reac,on Cards Product Reac,on Dyads Photo Selec,on LEM Tool
  27. 27. Usability Desirability Usefulness
  28. 28. USEFULNESS TESTING
  29. 29. Pre-­‐Usability Interview • How do you reconcile invoices today…? • Who needs to approve these and why…? • Where do you track which claims have been processed…? • When a new pa,ent arrives, what do you do?
  30. 30. During Usability Tes,ng Watch for comments about usefulness • “Wow. This would really make it easier to reorder the same things.” • “I don’t like this. It would take me too long to do this part.” Watch for neutral reac,ons • “Eh. This is OK.” • Probe: “How would this help, or not help, you do your job?”
  31. 31. Post-­‐Usability Interview Show screens again and ask: • “How would this work for you in your job?” • “What about this works for you? What doesn’t work?”
  32. 32. TESTING METHODS Desirability Tes,ng: • Open-­‐Ended Ques,oning • Sentence Comple,on • Product Reac,on Cards • Product Reac,on Dyads • LEM Tool Usefulness Tes,ng: • Pre & Post-­‐Interview Ques,ons • Listening & Asking Probing Ques,ons
  33. 33. Thank You! Susan Mercer smercer@madpow.com @susanamercer
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×