The document discusses methods for usability, desirability, and usefulness testing in product design. It provides various techniques for gathering user feedback, such as open-ended questioning and product reaction cards, while emphasizing the importance of user perceptions in enhancing design. Additionally, it includes examples of emotional language used to evaluate product attributes and reactions.

1. Beyond Just Usability:
Desirability & Usefulness Testing
Susan Mercer, Senior Experience Researcher
@susanamercer

2. Usability Testing

5. Usability = (Efficiency + Effectiveness)
In a given
context of use

6. Usable

7. Cute

8. Acerbic

9. Usability
Usefulness
Desirability

10. Usability
Consistent
Organized
Understandable
Simple
Efficient

11. Usefulness
Effec=ve
Essen=al
Relevant
Helpful
Suppor=ng

12. Desirability
Exci=ng
Fresh
Friendly
Encouraging
Meaningful

13. Usability
Usefulness
Desirability

14. DESIRABILITY
TESTING METHODS

15. Open-­‐Ended Ques,oning
• What do you think of…?
• What is your reac,on to…?
• How would you describe…?

16. Sentence Comple,on
• This app makes me feel…
• If I used this all day, I would…
• Reading on this website is…
• Compared to other apps, this app is…
• I want to use this because…
• I do not want to use this because…

17. Product Reac,on Cards
• 118 words
• 58% posi,ve
• 36% nega,ve
• 6% unclear
• 21% synonyms with 1+ other words
• Mostly posi,ve words

18. Product Reac,on Cards – Alternate List
• Removed synonyms
• Added opposites to create Posi,ve / Nega,ve Pairs
• Compared against two other emo,onal word vocabularies for
Product Appeal and Emo,ons
• Marc Hassenzhahl, Darmstadt University of Technology, Germany
• Geneva Emo,on Wheel

19. Annoying
Costly
Expected
Interes=ng
Poor
quality
Trivial
Apathe=c
Crea=ve
Familiar
Irrelevant
Predictable
UnaJrac=ve
AJrac=ve
Desirable
Fast
Managable
Professional
Uncontrollable
Boring
Difficult
Flexible
Meaningful
Relaxing
Understandable
Busy
Difficult
Fragile
Mo=va=ng
Relevant
Undesirable
Calm
Disappointed
Fresh
Nondescript
Reliable
Unexci=ng
Cheap
Discouaraging
Friendly
Not
Secure
Rigid
Unimagina=ve
Clean
Disorganized
Frustra=ng
Not
Valuable
Sa=sfying
Unimportant
Clear
Dull
Fun
Novel
Secure
Unnecessary
CluJered
Easy
to
use
Hard
to
Use
Obstruc=ng
Simple
Unpredictable
Common
Effec=ve
High
quality
Old
Slow
Unprofessional
Compelling
Efficient
Impersonal
Op=mis=c
Sophis=cated
Unrefined
Complex
Effortless
Impressive
Ordinary
Stable
Unreliable
Comprehensive
Empowering
Incomplete
Organized
Standard
Unusable
Confident
Encouraging
Incomprehensible
Original
Strange
Unwelcoming
Confusing
Enthusias=c
Inconsistent
Overwhelming
Stressful
Usable
Conserva=ve
Essen=al
Ineffec=ve
Patronizing
Suppor=ng
Useful
Consistent
Excep=onal
Inefficient
Personal
Tedious
Useless
Controllable
Exci=ng
Innova=ve
Pessimis=c
Time-­‐consuming
Valuable
Convenient
Exclusive
Inspiring
Pleasant
Time-­‐Saving
Weak

20. AJrac=ve
UnaJrac=ve
Excep=onal
Common
Organized
Disorganized
Calm
Busy
Exci=ng
Dull
Original
Ordinary
Clean
CluJered
Exclusive
Standard
Personal
Impersonal
Clear
Confusing
Familiar
Strange
Pleasant
Annoying
Compelling
Unimportant
Fast
Slow
Predictable
Unpredictable
Comprehensive
Incomplete
Flexible
Rigid
Professional
Unprofessional
Confident
Weak
Fresh
Old
Relaxing
Stressful
Consistent
Inconsistent
Friendly
Unwelcoming
Relevant
Irrelevant
Controllable
Uncontrollable
Frustra=ng
Encouraging
Reliable
Unreliable
Convenient
Difficult
Fun
Tedious
Sa=sfying
Disappointed
Costly
Cheap
High
quality
Poor
quality
Secure
Not
Secure
Crea=ve
Unimagina=ve
Impressive
Nondescript
Simple
Complex
Desirable
Undesirable
Innova=ve
Conserva=ve
Sophis=cated
Unrefined
Easy
to
use
Hard
to
use
Inspiring
Unexci=ng
Stable
Fragile
Effec=ve
Ineffec=ve
Interes=ng
Boring
Suppor=ng
Obstruc=ng
Efficient
Inefficient
Manageable
Overwhelming
Time-­‐saving
Time-­‐consuming
Effortless
Difficult
Meaningful
Trivial
Understandable
Incomprehensible
Empowering
Patronizing
Mo=va=ng
Discouraging
Usable
Unusable
Enthusias=c
Apathe=c
Novel
Expected
Useful
Useless
Essen=al
Unnecessary
Op=mis=c
Pessimis=c
Valuable
Not
Valuable

22. Reliable
Accessible
Appealing
Confusing
Comfortable
Organized
Professional
Busy
Confident
Straight
Forward
Approachable
Complex
Effec=ve
Understandable
Efficient
Dull
Friendly
Easy
to
Use
Flexible
Hard
to
Use
Invi=ng
Clear
Helpful
Impersonal
Time-­‐saving
Convenient
Overwhelming
Trustworthy
Clean
Boring
Useful
Familiar
Frustra=ng
Simplis=c
In=mida=ng
Valuable
Stressful
Usable
Time-­‐consuming
Customers
Customers
&
Non-­‐Customers
Non-­‐Customers

23. 68
67
62
42
20
22
17
31
0
10
20
30
40
50
60
70
Exis1ng
Design
A
Design
B
Design
C
Count
Number
of
Words
Posi=ve
Nega=ve

24. Exis1ng
Design
A
Design
B
Design
C
Appealing
Appealing
Appealing
Appealing
Clean
Clean
Clean
Clean
Efficient
Efficient
Efficient
Efficient
Engaging
Engaging
Engaging
Intui=ve
Professional
Professional
Professional
Useful
Useful
Useful
Useful
Total
9
20
16
9

25. Product Reac,on Dyads
Rate on a 5-­‐point scale from Posi,ve – Nega,ve word pairs
AJrac=ve
¤
¢
¢
¢
¢
UnaJrac=ve
Calm
¢
¤
¢
¢
¢
Busy
Clean
¢
¤
¢
¢
¢
CluJered
Clear
¤
¢
¢
¢
¢
Confusing
Comprehensive
¢
¢
¢
¤
¢
Incomplete

26. -­‐3.5
-­‐2.5
-­‐1.5
-­‐0.5
0.5
1.5
2.5
3.5
Discouraging
Unrefined
Useless
Complex
Irrelevant
Boring
Expected
Unprofessional
Mo=va=ng
Sophis=cated
Useful
Simple
Relevant
Interes=ng
Novel
Professional

27. Photo Selec,on

28. LEM Tool
(Susa Group)

29. Open-­‐Ended Ques,oning
Sentence Comple,on
Product Reac,on Cards
Product Reac,on Dyads
Photo Selec,on
LEM Tool

30. Usability
Desirability
Usefulness

33. USEFULNESS
TESTING

34. Pre-­‐Usability Interview
• How do you reconcile invoices today…?
• Who needs to approve these and why…?
• Where do you track which claims have been processed…?
• When a new pa,ent arrives, what do you do?

35. During Usability Tes,ng
Watch for comments about usefulness
• “Wow. This would really make it easier to reorder the same things.”
• “I don’t like this. It would take me too long to do this part.”
Watch for neutral reac,ons
• “Eh. This is OK.”
• Probe: “How would this help, or not help, you do your job?”

36. Post-­‐Usability Interview
Show screens again and ask:
• “How would this work for you in your job?”
• “What about this works for you? What doesn’t work?”

37. TESTING
METHODS
Desirability Tes,ng:
• Open-­‐Ended Ques,oning
• Sentence Comple,on
• Product Reac,on Cards
• Product Reac,on Dyads
• LEM Tool
Usefulness Tes,ng:
• Pre & Post-­‐Interview Ques,ons
• Listening & Asking Probing
Ques,ons

38. Thank You!
Susan Mercer
smercer@madpow.com
@susanamercer