Presented by Jonathan Rubin, MS, on September 7, 2012 at the third annual Center for Health Literacy Conference: Plain Talk in Complex Times.

1. The benefits of “Do It
Yourself” Usability Testing
By Jonathan Rubin
First Fridays Program Manager
General Services Administration

2. Let me tell you a story
2

3. Hanging chads
3

4. inspections
4

5. butteflu
5

6. Why did problems occur?
They didn’t listen to customer’s
wants, needs and expectations.
TAKEWAY MESSAGE: Testing
with customers gets you
valuable feedback (duh)!

7. In this hour…
1. What is usability
2. Why you need usability
3. Idea: Do your own testing!
7

8. 1. What is usability?

9. Usability =
How people interact with a
product, system or service,
and how they expect it to
work.

10. Usability ≠ Quality Control
10

11. Good products are usable
11

12. What is it? How do you use it?
12

13. What is it? How do you use it?
13

14. Don’t feel stupid!
14

15. 4 Steps to Working with Users
1. Observe users (customers) using your
product
2. (Re)design with users in mind
3. Get a usable product
4. Repeat

16. • Education
• Demonstration Tests
• Awareness

17. We’ve helped these folks
• DOT • State
• Interior • FCC
• IRS • USAJobs
• U.S. Army • Census
• NASA • NSF
• Labor • NIH
• OMB • And 35
• GSA others
17

18. Free @ Howto.gov/firstfridays
• Usability Testing Scripts
• Email Templates
• Best Practices
• Final Report Examples
• Project Management
• Before and After Screenshots
• Videos
• Etc.

19. 2. Why you need usability

20. Top 5 Government Usability
Problems
• Too many words

21. Top 5 Government Usability
Problems
• Too many words
• Gov speak / jargon

22. Top 5 Government Usability
Problems
• Too many words
• Gov speak / jargon
• Top tasks hard to find

23. Top 5 Government Usability
Problems
• Too many words
• Gov speak / jargon
• Top tasks hard to find
• Ineffective navigation

24. Top 5 Government Usability
Problems
• Too many words
• Gov speak / jargon
• Top tasks hard to find
• Ineffective navigation
• Ineffective search results

25. Example 2: GSA Intranet

26. ?

27. Before After
27
9/11/2012

31. Usability’s Return on Investment (ROI)
• Reduced developer time

32. Usability’s Return on Investment (ROI)
• Reduced developer time
• Reduced training time

33. Usability’s Return on Investment (ROI)
• Reduced developer time
• Reduced training time
• Reduced help desk calls / emails

34. Usability’s Return on Investment (ROI)
• Reduced developer time
• Reduced training time
• Reduced help desk calls / emails
• Higher task completion rate

35. Usability’s Return on Investment (ROI)
• Reduced developer time
• Reduced training time
• Reduced help desk calls / emails
• Higher task completion rate
• PR boost

36. Know your users,
for you are not them
- Usability Yoda

37. 3. Idea: Do your own testing!

38. Must read: ‘Rocket Surgery Made Easy’
• Commit to monthly
product testing
• Same time each
month
• More observers +
more testing = more
buy-in
38

39. Needed for “discount” usability testing
• Executive champion!

40. Needed for “discount” usability testing
• Executive champion!
• One morning a month
Needed for UT

41. Needed for “discount” usability testing
• Executive champion!
• One morning a month
• 1-2 rooms

42. Needed for “discount” usability testing
• Executive champion!
• One morning a month
• 1-2 rooms
• Laptops

43. Needed for “discount” usability testing
• Executive champion!
• One morning a month
• 1-2 rooms
• Laptops
• Screen sharing
software ($1000/yr)

44. Needed for “discount” usability testing
• Executive champion!
• One morning a month
• 1-2 rooms
• Laptops
• Screen sharing
software ($1000/yr)
• Volunteers

45. Needed for “discount” usability testing
• Executive champion!
• One morning a month
• 1-2 rooms
• Laptops
• Screen sharing
software ($1000/yr)
• Volunteers
• Food ($10/person)

46. Needed for “discount” usability testing
• Executive champion!
• One morning a month
• 1-2 rooms
• Laptops
• Screen sharing
software ($1000/yr)
• Volunteers
• Food ($10/person)
• Incentives ($0- $80)

47. Needed for “discount” usability testing
• Executive champion!
• One morning a month
• 1-2 rooms
• Laptops
• Screen sharing
software ($1000/yr)
• Volunteers
• Food ($10/person)
• Incentives ($0- $80)
• Training (FF)

48. How many people do I need?
48

49. Spending on usability
When usability is present, 10% of
development budget spent on
usability
- Usability Return on Investment (2008, Nielsen
Norman Group Report)
49

50. Staffing options (all are good)
1. Usability Contractor ($17,000 per test)
2. Dedicated Usability Staff ($60,000+)
3. Get training and do it yourself (Free)
50

51. Matrixed Approach to Staffing
• Part-time volunteers!
• Borrow from other
departments
• You are “providing skill
building opportunities”
• Flexible needs: 1 -20
hours a month
51

52. Types of First Fridays Tests
• 1. Formal, talk-aloud tests

53. Stakeholder
Meeting
Follow-up Develop
Meeting Scenarios
Formal Testing
Process
Recruit
Write
Participants
Report
+ Observers
Conduct
Test

54. It takes two (rooms) to test
Observation Room Testing Room

55. Formal Testing Schedule
• 8:20 – 8:30 Welcome
• 8:30 - 9:30 Tester #1
• 9:30 – 10:30 Tester #2
• 10:30 – 11:30 Tester #3
• 11:30 – Noon Lunch
• Noon – 1:30 Debrief / Critique
55

56. Absolute Minimum T = Tester,
Usability Staffing F = Facilitator
T
F
Testing Room

57. Minimum First O = Observer
F = Facilitator
Fridays Staffing M = Room Manager
O O O
T
F
O O M
Observation Room Testing Room

58. Better O = Observer G = Greeter
F = Facilitator DG = Data Gatherer
Staffing M = Room Manager BU = Back up
O O F
BU
BU T
BU F
DG O M
Observation Room G Testing Room

59. Types of First Fridays Tests
1. Formal, talk-aloud tests
2. Hallway Tests

60. 2. Hallway Test

61. Types of First Fridays Tests
1. Formal, talk-aloud tests
2. Hallway Tests
3. Navigation Improvement / Card sort

62. 3. Navigation Improvement

63. Types of First Fridays Tests
1. Formal, talk-aloud tests
2. Hallway Tests
3. Navigation Improvement / Card sort
4. Quick Win Evaluation (Heuristics Analysis)

64. 4. Quick Win Evaluation (Heuristics)
• Not for newbies
• Group or single person feedback
– Checklist
– Task run-through
• PRO: Fast! <1 week
• CON: No users
64

65. Teach ‘em to fish
Teach em to fish

66. Review: What you need to fish
• Executive champion!
• One morning a month
• 1-2 rooms
• Laptops
• Screen sharing
software
• Volunteers
• Food
• Incentives for testers
• Training (First Fridays!)

67. Free usability resources!
Howto.gov/firstfridays

68. More info on First Fridays
• Observe a test

69. More info on First Fridays
• Observe a test
• Recommend a website

70. More info on First Fridays
• Observe a test
• Recommend a website
• Volunteer as a tester
(remote or in person)

71. More info on First Fridays
• Observe a test
• Recommend a website
• Volunteer as a tester
(remote or in person)
• Learn to fish (Facilitator
in Training program)

72. Once a month

73. Thanks!
Jonathan.rubin@gsa.gov
firstfridays@gsa.gov
Howto.gov/firstfridays