How many times have you seen a website or application stumble or crash once it gets to production, despite meeting the requirements perfectly? Have you felt like there was something missing in the development process? In this presentation, Norm Sun will talk about what usability testing is, why you should be doing it, and how you can start incorporating it into your development process.

1. Usability Testing
Adding A New Level To Your Toolbox

2. About Me
• Working in the field of User Experience since 2010
• Secretary for the User Experience Professionals
Association (UXPA), DC Chapter
• Certified Usability Analyst (CUA)
• Technical Business Analyst background

3. Agenda
•
•
•
•
•
•
•
What Is Usability Testing?
Why Usability Testing Is Important to a Business Analyst
Components of Usability Testing
Types of Usability Tests
When to Conduct a Usability Test
Usability Test Exercises
Questions

4. What Is Usability?
People being able to successfully use a product, system, or
service and how they expect it to work.
Great usability is invisible.
Bad usability…you know it when you see it.

6. Image Credit: http://www.flickr.com/photos/paulhagon/

7. Identifying Usability Issues
•
•
•
•
•
High number of errors
Significant time-on-task
Low customer satisfaction scores
Lots of help desk calls / tickets
High abandonment rates

10. What Is Good Usability?
•
•
•
•
•
Simple
Easy to understand
Recognizable
Clear purpose
Learnable

13. The 5 E’s
•
•
•
•
•
Effective
Efficient
Engaging
Error Tolerant
Easy to Learn
http://www.wqusability.com/articles/getting-started.html
Image Credit: http://www.flickr.com/photos/whatwhat/

14. Why Do Usability Testing?
• Aligns the customer’s goals with the
business goals
• Ignoring small issues can turn into big
problems
• If the user is not happy, everyone is
unhappy

15. Usability Testing vs. UAT
• User Acceptance
Testing:
o Process of verifying that the
deliverable meets the
requirements
o Conducted at the end of the
development cycle
o Quality control
• Usability Testing:
o Verifies whether an
implementation approach
meets the user needs
o Conducted at any time
during the development
cycle
o Does not serve as quality
control
o Increases the quality of
UAT outputs

16. Buying A Suit Online

17. Buying A Suit Online
UAT verifies that I can select a suit
style, size, and complete my purchase
online.

18. Buying A Suit Online
UAT verifies that I can select a suit style,
size, and complete my purchase online.
Usability Testing can verify:
• How quickly I can purchase a suit
online
• How satisfied I am with the online
purchase process
• The intuitiveness of the purchase
process is for me
• How much the online purchase
process will help me recover from an
error
• What unforeseen issues I will
encounter in the purchase process

19. What is Success?
• Reduction in hidden downstream maintenance costs
(developer time)
• Reduction in training time
• Reduction in help desk support
• Higher task completion rates

20. Most Importantly…
More satisfied users!

21. When Should You Test?
Research
Brainstorm
Ideas
Prototype
Test
Usability Testing Process
Implement

22. Usability Testing Cycle
Develop
Tasks
Create a
More Usable
Product
Watch Your
Users
Report
Findings

23. Components
• What will you test?
• Where will you test?
• Who will you test with?
• How will you test?

24. Testing Materials
•
•
•
•
•
•
•
Wireframes
Data Entry Form
Screenshots
HTML prototype
Live or staging site
Mobile Apps
Competitor’s site

25. Where Will You Test?
• What is the product?
• What is the context of use?
Image Credit: http://www.flickr.com/photos/nestle/

26. How Will You Test?
Single Usability Test
o Laboratory Test
o Remote Test
o Hallway Test

28. Laboratory Test
• Traditional type of usability test
• Scheduling participants to meet
you at a predetermined location
• Takes course over several days
• Average time: 60 minutes per
user

30. Remote Test
• Virtual meeting held over the
phone and a computer
• Scheduling participants to call
you on a conference line
• Takes course over several days
• Average time: 60 minutes per
user

32. Image Credit: http://www.flickr.com/photos/oreocookies/

33. Hallway Test
• Attract people in a high
pedestrian traffic area
• Can be completed in one day
• Least Expensive
• Average time: Up to 20
minutes per user
Image Credit: http://www.flickr.com/photos/oreocookies/

34. Key Players
• Facilitator – Person who conducts a usability test
• Observer – Person who records notes during a usability
test
• Participant – Person who is testing the website

35. Designing a Usability Test
•
•
•
•
Open with easy intro questions
Ask open ended questions when possible
Questions can ask a user to accomplish a task
Questions can ask a user to provide their perspective

36. Qualitative Testing
• Looking for common themes and specific insights from
your users
• Testing five users per user type
o
o
o
o
Who are your users?
Doctors vs Patients
Buyers vs Sellers
Federal vs. Non-Federal

37. How Many Participants?
“The answer is five…except when it’s not.”
Nielsen Norman Group, June 4, 2012
http://www.nngroup.com/articles/how-many-test-users/

38. Discuss Your Findings

39. Document Your Findings!
Image Credit: http://www.flickr.com/photos/juhansonin

40. Usability
Test
Exercise

41. Put Your Participant At Ease
Image Credit: http://www.flickr.com/photos/lukeydargons/

42. Be An Active
Listener
Image Credit: http://www.flickr.com/photos/brizzlebornandbred/

43. Be Unbiased!
Image Credit: http://seniorhousingforum.net/blog/2014/01/09/you-be-the-judge-cool-or-scary/

44. Don’t Lead Your Participant
Image Credit: http://www.flickr.com/photos/johnragai/

45. Let the
Participant Fail

46. Thank Your Participant

47. Facilitator Best Practices
1.
2.
3.
4.
5.
6.
Put your participant at ease
Be an active listener
Be unbiased
Avoid leading your participant
Let the participant fail
Thank your participant
Prompts:
• Can you tell me what you are thinking?
• Do you have any additional thoughts?

48. Wrap Up