Art Center Interactive Design 4 Lecture Slides #4 (week 5) Usability Testing Ways to measure usability Creating a test plan

1. Usability Testing
Art Center Interactive Design 4 | Week 5 | Joy Liu

2. Usability
Quality of a user's experience when interacting with products
or systems

3. How to measure usability?
Is your product user-friendly?

4. Effective
Efficient
Easy to Learn
Error Tolerant
Engaging

5. the completeness and accuracy with
which Effective users achieve their goals
Efficient
Easy to Learn
Error Tolerant
Engaging
RaeVynn Sands, Flickr, http://www.flickr.com/photos/cronewynd/4689772454

6. Is it effective?
Does the product accomplish what I need?
Can I accomplish all the major tasks?
Is the product missing anything essential to my goal?
Is the product unnecessarily complex? cumbersome to user?
How complete is the result?
What are the core elements of this product?

7. the speed (with accuracy) with which this
work can be done
Effective
Efficient
Easy to Learn
Error Tolerant
Engaging
Insert ANY video of
parallel parking fail here

8. Is it efficient?
How deep is the system?
How many clicks does it take to complete _____?
Does the system give feedback?
Is this loading? Or did the screen freeze?
Where am I? Where am I going?
How can I make this simpler by activating GPS? gesture?
Is the system full of inconsistency?

9. how well the product supports both
initial orientation and deeper learning
Effective
Efficient
Easy to Learn
Error Tolerant
Engaging
Martin Gillet, Flickr, https://www.flickr.com/photos/mgillet/6875700642
Francesco Crippa, Flickr, http://www.flickr.com/photos/fcrippa/7967670670/

10. Is it easy to learn?
How easy is it to complete the tasks?
Can most people learn to use this product very quickly?
Can I complete tasks without the help of a technical person?
Do I need to learn a lot of things before I can get going?
How much hand-holding is needed for return users?
Do I remember where things are in the system?
Do I remember what the icons mean in a system?

11. how well the product prevents errors
and helps the user recover from any that
do occur
Effective
Efficient
Easy to Learn
Error Tolerant
Engaging
iancvt55, Flickr, http://www.flickr.com/photos/iancvt55/13088416513/

12. Is it error tolerant?
Are error messages clear?
How often are mistakes being made?
Is it my fault that the screen froze?
Do users know how to recover from an error?
How do I get out of this mess?
Where can I go for help? HELP!!!

13. how pleasant, satisfying or interesting an
interface is to use
Effective
Efficient
Easy to Learn
Error Tolerant
Engaging
Fernando Messino, Flickr, https://www.flickr.com/photos/krudo/5194217402

14. Is it engaging?
How do I keep users coming back?
Are people satisfied of the product?
What is the impression of the product?
What pleases as many users as possible?
Do users enjoy/like using this product?
Will existing users recommend this product to others?

15. Usability Testing (onsite/remote)
Observing user performing tasks with your product.

16. Create a Test Plan - Step 1
Define Scope & Objective
Exactly what are you testing?
How much of the product will the test cover?
E.g. Can someone make a purchase on Site X’s
v2 new prototype?

17. Create a Test Plan - Step 2
Define Scenario
What is the user mindset when going through this part
of the product?
E.g. You’ve been looking for a new computer chair.
Your coworker recommended Site X to you.

18. Create a Test Plan - Step 3
Define Test Metrics
What data are you collecting and measuring? Task
completion? Errors? Time on task? User feedback?
E.g. Likelihood to user, likelihood to recommend, overall
satisfaction, ease of use, confusions, likes or dislikes, etc.

19. Create a Test Plan - Step 4
Recruit Participants
What type of participants will be tested?
How many participants are needed?
Screener?
E.g. Male/Female, 22–45, works 30+ hours a week,
must have a job that requires sitting 6+ hours a day.

20. Create a Test Plan - Step 5
Record Testing Results
Video, audio, or other tools?
Permission to record?
Who’s the note-taker and who’s the facilitator?
If you are doing the test yourself, make sure you have a
recording device to help you capture details.

21. Create a Test Plan - Step 6
Run the test
When and where?
How long?
No more than 90 minutes.
Must reset the environment for each participant.

22. How many test users does it take?
The answer is 5, except when it's not. Most arguments
for using more test participants are wrong, but some
tests should be bigger and some smaller.
— Nielsen Norman Group

23. Moderating techniques
Name Pros Cons
Concurrent
Think Aloud
• Understand participants’ pain
points as they occur
• Real-time feedback and
emotional response
Interfere with usability metrics, i.e.
accuracy and time on task
Retrospective
Think Aloud
Doesn’t interfere with usability
metrics
• Session length increase
• Difficulty in remembering
feedback and thoughts
Concurrent
Probing
Understand participants’ thoughts
as they attempt to work through a
task
Interfere with natural thought
process and progress
Retrospective
Probing
Doesn’t interfere with usability
metrics
Difficulty in remembering feedback
and thoughts
Jennifer Romano Bergstrom, Ph.D., UX Project Leader, Fors Marsh Group; President, UXPA DC. “Moderating Usability Tests"

24. Resources & Reading
‣http://www.usability.gov/how-to-and-tools/resources/templates.html
‣http://remoteresear.ch/tools/
‣http://www.uxbooth.com/articles/the-art-of-guerilla-usability-testing/
‣http://www.slideshare.net/andybudd/guerilla-usability-testing
‣http://uxdesign.smashingmagazine.com/2013/01/08/improving-your-website-
usability-tests/