This document provides an overview and definitions of usability testing. It discusses that usability testing involves having users complete tasks while thinking out loud to uncover usability issues. It recommends testing with 5 users to find 80% of issues. The document outlines best practices for planning a test, including developing test plans and tasks, finding and screening participants, conducting sessions, analyzing data, and reporting findings. It emphasizes that usability testing should involve observing how users complete realistic tasks rather than just asking for feedback.
2. Usability Testing
What is usability testing?
!
— First point: We are not testing the user (you will hear
the term "user testing", but that may make Jared Spool sad)
!
— Second point: It’s not a science
!
!
3. Usability Testing
What is usability testing?
!
"“It takes only five users to uncover 80 percent of high-level
usability problems."
!
"… debugging a design."
!
— Jakob Nielsen
!
4. Usability Testing
What is usability testing?
!
"Every time a person has a great experience with a website, a
web app, a gadget, or a service, it’s because a design team
made excellent decisions about both design and
implementation — decisions based on data about how people
use designs. And how can you get that data? Usability testing."
— Dana Chisnell
5. Usability Testing
What is usability testing?
!
(Dana co-wrote this book)
http://www.wiley.com/WileyCDA/WileyTitle/productCd-0470185481.html
6. Usability Testing
What is usability testing?
!
"Watching people try to use what you’re creating/designing/
building (or something you’ve already created/designed/built),
with the intention of (a) making it easier for people to use or
(b) proving that it is easy to use."
— Steve Krug
8. Usability Testing
What is usability testing?
!
"In a usability test, individual users work through a series of
fairly realistic tasks. Some tasks may be timed, but most
often, each participant talks out loud to describe his thought
process as he uses an actual product or prototype."
— Kim Goodwin
!
9. Usability Testing
What is usability testing?
!
— It can demonstrate a problem exists
— It can help us understand how something we
design should behave
!
!
10. Usability Testing
What is usability testing?
!
—There are lots of online resources,
even some you might not expect
!
!
!
11. Usability Testing
What is usability testing?
!
— It can be time-consuming and not easy; you can
outsource it to specialized companies at different scales
!
— usertesting.com
—AnswerLab
— blinkUX.com
— others
12. Usability Testing
What is usability testing?
!
Please note: it is NOT a focus group
!
People want to agree and show favor. If you basically ask
them if they like the work you’ve done, they’ll try to please
you (and give useless results).
13. Usability Testing
What is usability testing?
!
Please note: it is NOT a focus group
!
The difference?
!
"I’m showing you this. Do you like it?"
!
"Show me how you do this."
14. Usability Testing
What is usability testing?
!
It is NOT
http://blog.templatemonster.com/2011/09/14/usability-testing-basics/
15. Usability Testing
When to do what?
!
!
by Hong Qi,Donna Lichaw; Adapted & taught by Christina Wodtke 11/13/13
What do we know?
User Interviews
Contextual Inquiry,Field Studies &
Ethnography
Diary & Journal Studies
Benchmark Usability Tests
Card Sorting
Surveys
How are we doing?
Usability & Concept Tests
Eye Tracking
Diary & Journal Studies
How did we do?
Usability & Concept Tests
Eye Tracking
A/B & Multivariate Tests
Data Analysis
16. Usability Testing
This isn’t rocket
science or deep
thinking
!
But this does require
keeping many, things in
mind, and self-control, as
well as practice.
17. Usability Testing
The "classic model"
!
— Develop a test plan
— Choose a testing environment
— Find and select participants
— Prepare test materials
— Conduct the sessions
— Debrief with participants and observers
—Analyze data and observations
— Create findings and recommendations
18. Usability Testing
When to go classic
!
—Your team/company does it
!
—You need to deliver lots of data/stats
!
—Your team/company works with extensive
requirements documents
!
19. Usability Testing
Plan
!
— Discuss with the whole team
!
—Agree on a test objective (something besides “determine
whether users can use it”)
!
—The methods and measures you’ll use to learn the
answers to your research questions
!
—This can take < 1hr
23. Usability Testing
Environment
!
— It helps if there is recording equipment (even just
software recording the screen and user on the laptop or
desktop being used), but if you only have a person to
record observations and take notes, that can work, too.
!
!
24. Usability Testing
Environment
!
—You can even do it remotely (screen sharing via Skype,
JoinMe, GoToMeeting; these can record both the user’s
screen and the user via webcam)
!
—And there’s always the ambush method: "Hi. I’m
______________. Do you have a minute?"
26. Usability Testing
Recruiting
!
— Can be hard and time-consuming
!
— Focusing on the behavior that you’re interested in
observing is easier than trying to select for market
segmentation or demographics
!
— If you can, offload. Many services will charge $75-150
per participant and take care of everything
27. Usability Testing
Recruiting
!
— mention the gift certificate
— friends and family and friends of friends
— post to user groups of similar apps/sites
— professional mailing lists
— did we mention the gift certificate?
— Facebook fan pages/Twitter hashtags/LinkedIn groups
— probably not people at the same company (see:
Google Buzz)
— NOT people supplied by the marketing department
29. Usability Testing
Screening
!
—"If you want to test a kiosk for checking people into and
out of education programs, you want people who are
attending those programs. Make sense? Don’t make
recruiting harder than it has to be."— Dana Chisnell
!
— make sure they’re available
!
— make sure they meet your qualifications
!
— make sure you’ve explained the process to them
30. Usability Testing
Prep for tests
!
— Guide or checklist for the moderator
!
— Includes the research questions
!
— I any specific interview questions you might want to ask,
prompts for follow-up questions, as well as closing,
debriefing questions that you want to ask each participant
32. Usability Testing
Choosing tasks for the test
!
—Translate the research questions into task scenarios that
represent realistic user goals
!
—Tasks = user goals, not app/site features
— e.g.,"How would you book a hotel room?", not "How
do you navigate our hotel interface?"
!
— Give a task that realistically represents a user goal
34. Usability Testing
How to structure the test
!
(See "session_script.pdf")
!
Prep reminders for moderator
— Rehearsal and preparation needs
— Before each session ("Set up computer", etc.)
—After each session ("Turn off recording", etc.)
!
35. Usability Testing
How to structure the test
!
(See "session_script.pdf")
!
Session introduction
— Introduce yourself nicely
—Think-out-loud protocol
—"You can’t do anything wrong"
—"You’re helping us find where we confuse you."
!
37. Usability Testing
How to structure the test
!
(See "session_script.pdf")
!
Close the session
!
This often involves overall closing questions.
!
Remember: Thank the participant.
!
38. Usability Testing
How to conduct yourself
!
— Be nice, empathetic
!
—Ask open-ended questions: "What would you expect to
happen? Where would you look for that?"
!
— Be reassuring: "We’re trying to make this better, so
you’re helping us by pointing out where it fails you."
39. Usability Testing
How to conduct yourself
!
— Do NOT say,"Click this/go there"
!
— Remind participant to think out loud
!
— Mirror their statements, allow them to clarify
!
— Don’t interrupt. LISTEN.
41. Usability Testing
How to take notes
!
—A dedicated observer/notetaker helps
!
— Note participants’ facial or non-verbal
expressions, body language
!
—Add time labels to notes ("This at 9:42")
!
NOTES OR IT DIDN’T HAPPEN
43. Usability Testing
Conducting the test
!
—The moderator as MC: sees to the safety and comfort of
the participants, manages the team members observing,
and handles the data collected.
!
44. Usability Testing
Debrief
!
—Take a step back with the participant and ask,“How’d
that go?” Invite the trained observers to pass follow-up
questions to the moderator.
!
—Thank the participant, compensate, and say good-bye.
!
45. Usability Testing
Debrief
!
— Now, the team observing should talk briefly about what
they saw and what they heard. (This discussion is not
about solving design problems, yet.)
46. Usability Testing
The RITE way
!
Rapid Iteration Testing Environment
— integrates well into Lean UX and agile development in
general
— Boxes and Arrows podcast "Rapid-Turnaround Usability
Testing" by Kyle Soucy and Holly Phillips: https://
itunes.apple.com/us/podcast/rapid-turnaround-usability/
id275459507?i=82723199&mt=2
— Keys include: small testing sample, rapid design
iterations, debriefs of all stakeholders time-limited and
tied to physical observations
50. Usability Testing
Analyze
!
— Can be stories (qualitative), can be stats (quantitative)
!
http://www.fanxingkong.com/digital-music-notepad/
51. Usability Testing
Findings and recs
!
— Concentrate on the most significant and serious
findings, not all the details
!
— It’s a good idea to include a summary of the main points
!
52. Usability Testing
Findings and recs
!
—Think visual. You ideally want lots of screen shots and
perhaps even video clips to showcase issues and findings
!
— Be ready to discuss in person
54. Usability Testing
Any testing helps:
!
Why?
!
—The "second set of eyes" principle
!
—The "it’s always been right in front of me" principle
!
— Everything is broken, even if everything is awesome
!
(Can still be a pain.)
55. Usability Testing
Hey, ho, let’s go
!
A: Create a script
!
B: Pair up
!
C: Each person interviews
the other.Then switch.
!
D: Report back to the
group what you learned