The document discusses Diane Bowen's experience conducting her first usability tests as a new UX researcher. It describes how her initial method of just interviewing participants and writing observations was not very effective. She later learned better practices like asking questions to define test goals and participant criteria, writing non-leading questions, having product teams observe interviews and debrief immediately after, and synthesizing findings into a task analysis to identify trouble spots. The document advocates for testing with a small number of users, inviting product teams to interviews, debriefing sessions directly after, and sending synthesized findings to inform the team.

1. Usability Testing
How a Green, Bewildered UX Researcher
Uncovered Loads of Valuable Data
Diane Bowen
User Experience Practitioner
@UXDiane
1

2. Usability Testing
Testing with one user is 100% better than
testing with none.
Krug’s first law of usability
@UXDiane
2

3. Usability Testing
The interview is a powerful method to
understand human behavior.
@UXDiane
3

4. Usability Testing
The interview is a powerful method to
understand human behavior.
I had no idea what I was doing.
@UXDiane
4

5. Usability Testing vs User Testing
Usability testing: “Is this product or service easy to learn
and use?”
Participants are users.
User Testing: Finds bugs before the product or service
is released.
Participants are QA or Developers
@UXDiane
5

6. Usability Testing
Usability testing: “Is this product or service easy to learn
and use?”
Participants are users.
Discovers the goals, motivations, and behaviors of
users.
@UXDiane
6

7. Interviewing Users by Portigal
•Leave your worldview at the
door.
•The user is the expert.
•Build Rapport.
@UXDiane
7

8. Nielsen Norman Group
The best results come
from testing no more
than 5 users and
running as many
small tests as you
can afford.
@UXDiane
8

9. My First Study
• Interviewed Participants
• Wrote up observations
• Scheduled a meeting
• Made recommendations
@UXDiane
9

10. My First Study
• Interviewed Participants
• Wrote up observations
• Scheduled a meeting
• Made recommendations
This wasn’t very effective
@UXDiane
10

11. A Better Way
Ask questions
• Who wants this research?
• Why are we testing this?
• Why is this a business goal?
• What answers would inform the development team?
@UXDiane
11

12. A Better Way
Ask questions
• What are the key parts to test?
• What does the perfect participant look like?
• When does the Product Owner need results?
@UXDiane
12

13. A Better Way
• Enlist participants
• Write jargon-free and non-leading interview questions
• Minimize interview questions that have high
cognitive load
@UXDiane
13

14. Observing the User Experience by Kuniavsky
Six phases of an interview:
• Introduction
• Warm-up
• General Issues
• Deep Focus
• Retrospective
• Wrap-up
@UXDiane
14

15. A Better Way
• Pilot test before holding interviews.
• Schedule interviews and invite the Product Team.
• Schedule debriefing meetings directly following the
interview.
• Email the findings to the Product Team.
@UXDiane
15

16. A Better Way
Webster’s definition of Synthesis:
“The composition or combination of parts or elements to
as to form a whole.”
Synthesis is insight from analysis
@UXDiane
16

17. A Recent Example
@UXDiane
• Take notes during the
interview
• Debrief with the team
immediately after the
interview
• Transcribe notes, one
thought per sticky note.
17

18. Usability and Synthesis Workshop
@UXDiane
• Place the sticky notes
into a task analysis.
• Identify trouble spots.
18

19. Sources
• Portigal, Steve. Interviewing Users. Brooklyn. Rosenfeld Media. 2013. Print
• Nielsen, Jakob. “Why You Only Need to Test With Five Users.” Nielsen Norman
Group. http://www.nngroup.com/articles/why-you-only-need-to-test-with-5-
users/, Accessed 8-26-2015
• Kuniavsky, Mike. Observing the User Experience: A practitioner’s guide to user
research. San Francisco. Elsevier. 2003. Print.
@UXDiane
19

20. Uncover User Insights and Deliver Value
@UXDiane
• Talk with users.
• Invite the Product Team to observe
the interviews.
• Debrief with the Team. Include
everyone’s thoughts.
• Send a synthesized findings doc to
the team to remind them of
their discoveries.
Diane Bowen
UX Practitioner, Covenant Eyes
@UXDiane 20