Check these out next
Beyond Just Usability: Desirability and Usefulness Testing
26 likes
views
Much of our work in UX research focuses on usability – evaluating products and interfaces to ensure they are easy-to-use. However, in today’s digital world, they are no longer enough. Consumers also have come to expect entertaining and engaging experiences. Web and mobile applications need to be usable, useful and engaging. So, how do we evaluate web interfaces to determine how useful and engaging they are? Desirability has been evaluated in recent years by the use of the Product Reaction Card technique, originated by folks at Microsoft. However, there are many other techniques used in market and industrial design research that we can borrow to complement this technique. Likewise, we can use standard usability testing techniques with lines of questioning with a slightly different focus to evaluate the relative usefulness of different solutions for a particular user group. In this talk, I discuss several techniques that I have used in recent months to evaluate the usefulness and desirability of interfaces The best techniques I have discovered to evaluate usefulness involve open-ended interview questions regarding current processes and pain points, followed by a usability evaluation of the interface and then a reflective interview discussing the benefits and drawbacks of that solution to their personal situation. To evaluate desirability, I will discuss the product reaction card technique and variations using more defined vocabularies for emotional responses and product personalities. In addition I will show results from techniques borrowed from psychology and marketing research - sentence completion, collaging, and the use of dyad rating scales. These techniques offer a variety of both qualitative and quantitative data that can be used to compare different interface options.
Recommended
Beyond Just Usability: Desirability and Usefulness Testing
- Beyond Just Usability: Desirability & Usefulness Testing Susan Mercer, Senior Experience Researcher @susanamercer
- Usability Testing
- Usability = (Efficiency + Effectiveness) In a given context of use
- Usable
- Cute
- Acerbic
- Usability Usefulness Desirability
- Usability Consistent Organized Understandable Simple Efficient
- Usefulness Effec=ve Essen=al Relevant Helpful Suppor=ng
- Desirability Exci=ng Fresh Friendly Encouraging Meaningful
- Usability Usefulness Desirability
- DESIRABILITY TESTING METHODS
- Open-‐Ended Ques,oning • What do you think of…? • What is your reac,on to…? • How would you describe…?
- Sentence Comple,on • This app makes me feel… • If I used this all day, I would… • Reading on this website is… • Compared to other apps, this app is… • I want to use this because… • I do not want to use this because…
- Product Reac,on Cards • 118 words • 58% posi,ve • 36% nega,ve • 6% unclear • 21% synonyms with 1+ other words • Mostly posi,ve words
- Product Reac,on Cards – Alternate List • Removed synonyms • Added opposites to create Posi,ve / Nega,ve Pairs • Compared against two other emo,onal word vocabularies for Product Appeal and Emo,ons • Marc Hassenzhahl, Darmstadt University of Technology, Germany • Geneva Emo,on Wheel
- Annoying Costly Expected Interes=ng Poor quality Trivial Apathe=c Crea=ve Familiar Irrelevant Predictable UnaJrac=ve AJrac=ve Desirable Fast Managable Professional Uncontrollable Boring Difficult Flexible Meaningful Relaxing Understandable Busy Difficult Fragile Mo=va=ng Relevant Undesirable Calm Disappointed Fresh Nondescript Reliable Unexci=ng Cheap Discouaraging Friendly Not Secure Rigid Unimagina=ve Clean Disorganized Frustra=ng Not Valuable Sa=sfying Unimportant Clear Dull Fun Novel Secure Unnecessary CluJered Easy to use Hard to Use Obstruc=ng Simple Unpredictable Common Effec=ve High quality Old Slow Unprofessional Compelling Efficient Impersonal Op=mis=c Sophis=cated Unrefined Complex Effortless Impressive Ordinary Stable Unreliable Comprehensive Empowering Incomplete Organized Standard Unusable Confident Encouraging Incomprehensible Original Strange Unwelcoming Confusing Enthusias=c Inconsistent Overwhelming Stressful Usable Conserva=ve Essen=al Ineffec=ve Patronizing Suppor=ng Useful Consistent Excep=onal Inefficient Personal Tedious Useless Controllable Exci=ng Innova=ve Pessimis=c Time-‐consuming Valuable Convenient Exclusive Inspiring Pleasant Time-‐Saving Weak
- AJrac=ve UnaJrac=ve Excep=onal Common Organized Disorganized Calm Busy Exci=ng Dull Original Ordinary Clean CluJered Exclusive Standard Personal Impersonal Clear Confusing Familiar Strange Pleasant Annoying Compelling Unimportant Fast Slow Predictable Unpredictable Comprehensive Incomplete Flexible Rigid Professional Unprofessional Confident Weak Fresh Old Relaxing Stressful Consistent Inconsistent Friendly Unwelcoming Relevant Irrelevant Controllable Uncontrollable Frustra=ng Encouraging Reliable Unreliable Convenient Difficult Fun Tedious Sa=sfying Disappointed Costly Cheap High quality Poor quality Secure Not Secure Crea=ve Unimagina=ve Impressive Nondescript Simple Complex Desirable Undesirable Innova=ve Conserva=ve Sophis=cated Unrefined Easy to use Hard to use Inspiring Unexci=ng Stable Fragile Effec=ve Ineffec=ve Interes=ng Boring Suppor=ng Obstruc=ng Efficient Inefficient Manageable Overwhelming Time-‐saving Time-‐consuming Effortless Difficult Meaningful Trivial Understandable Incomprehensible Empowering Patronizing Mo=va=ng Discouraging Usable Unusable Enthusias=c Apathe=c Novel Expected Useful Useless Essen=al Unnecessary Op=mis=c Pessimis=c Valuable Not Valuable
- Reliable Accessible Appealing Confusing Comfortable Organized Professional Busy Confident Straight Forward Approachable Complex Effec=ve Understandable Efficient Dull Friendly Easy to Use Flexible Hard to Use Invi=ng Clear Helpful Impersonal Time-‐saving Convenient Overwhelming Trustworthy Clean Boring Useful Familiar Frustra=ng Simplis=c In=mida=ng Valuable Stressful Usable Time-‐consuming Customers Customers & Non-‐Customers Non-‐Customers
- 68 67 62 42 20 22 17 31 0 10 20 30 40 50 60 70 Exis1ng Design A Design B Design C Count Number of Words Posi=ve Nega=ve
- Exis1ng Design A Design B Design C Appealing Appealing Appealing Appealing Clean Clean Clean Clean Efficient Efficient Efficient Efficient Engaging Engaging Engaging Intui=ve Professional Professional Professional Useful Useful Useful Useful Total 9 20 16 9
- Product Reac,on Dyads Rate on a 5-‐point scale from Posi,ve – Nega,ve word pairs AJrac=ve ¤ ¢ ¢ ¢ ¢ UnaJrac=ve Calm ¢ ¤ ¢ ¢ ¢ Busy Clean ¢ ¤ ¢ ¢ ¢ CluJered Clear ¤ ¢ ¢ ¢ ¢ Confusing Comprehensive ¢ ¢ ¢ ¤ ¢ Incomplete
- -‐3.5 -‐2.5 -‐1.5 -‐0.5 0.5 1.5 2.5 3.5 Discouraging Unrefined Useless Complex Irrelevant Boring Expected Unprofessional Mo=va=ng Sophis=cated Useful Simple Relevant Interes=ng Novel Professional
- Photo Selec,on
- LEM Tool (Susa Group)
- Open-‐Ended Ques,oning Sentence Comple,on Product Reac,on Cards Product Reac,on Dyads Photo Selec,on LEM Tool
- Usability Desirability Usefulness
- USEFULNESS TESTING
- Pre-‐Usability Interview • How do you reconcile invoices today…? • Who needs to approve these and why…? • Where do you track which claims have been processed…? • When a new pa,ent arrives, what do you do?
- During Usability Tes,ng Watch for comments about usefulness • “Wow. This would really make it easier to reorder the same things.” • “I don’t like this. It would take me too long to do this part.” Watch for neutral reac,ons • “Eh. This is OK.” • Probe: “How would this help, or not help, you do your job?”
- Post-‐Usability Interview Show screens again and ask: • “How would this work for you in your job?” • “What about this works for you? What doesn’t work?”
- TESTING METHODS Desirability Tes,ng: • Open-‐Ended Ques,oning • Sentence Comple,on • Product Reac,on Cards • Product Reac,on Dyads • LEM Tool Usefulness Tes,ng: • Pre & Post-‐Interview Ques,ons • Listening & Asking Probing Ques,ons
- Thank You! Susan Mercer smercer@madpow.com @susanamercer