At it’s most basic, usability is about insuring something, such as a website, works well. Without usability testing results, design and functionality decisions are based on opinion. Despite the understood importance, many organizations believe usability testing is too expensive and time consuming to fit their budget and schedule. We will show how usability tests can be performed both quickly and inexpensively using popular DIY usability techniques. We will cover both analog and online tools for user surveys, card sorting, tree testing, first click testing, and user testing. You don’t need a lot of participants either—many valuable tests may be performed with as few as five subjects.

1. Get It Right the FirstTime
Through Cheap and Easy
DIY Usability Testing
Wednesday, July 22th
Drupal GovCon 2015
David Minton @DH_David
@DH_Stephen
@DesignHammer
1

2. Overview
• Introduction
• What is Usability
• Who,What,Where, and When
• Free and/or Inexpensive Tools
2

3. Introduction
3

4. Disclaimer
Let’s get this out of the way first!
4

5. Disclaimer
We are not Usability experts
• We’ll share easy DIY techniques & tools
We won’t tell you how to fix your website
• But we will point you in the right direction
5

6. Scope
What we’ll cover and what we won’t.
6

7. What we will cover
• What is usability?
• Why should I bother?
• How can I test my site?
• How do I keep my site usable?
7

8. What we won’t cover
• Usability best practices
• Statistical analysis of results
• Every type of usability test
8

9. What is Usability?
9

10. What is usability?
“Usability testing refers to evaluating a
product or service by testing it with
representative users.The goal is to identify
any usability problems, collect qualitative and
quantitative data and determine the
participant's satisfaction with the product.”
usability.gov
10

11. What is usability?
“If there's one thing you learn by working on
a lot of different Web sites, it's that almost any
design idea—no matter how appallingly bad—
can be made usable in the right circumstances,
with enough effort.”
Steve Krug, author
Don't Make MeThink:
A Common Sense Approach toWeb Usability
11

12. Keep in mind
• Popular≠Good
• Attractive≠Usable
• Ugly≠Unusable
12

13. What, when,
and who to test?
13

14. What to test?
Anything or everything?
14

15. What should you test?
• Information Architecture
• Navigation
• User Interface elements
• New Functionality
• Critical User Interactions
• Everything!
15

16. When to test?
Is it too soon, or too late?
16

17. When to test
• The sooner the better
• Tests should be part of the change process
• Test early and test often!
17

18. Who to test?
Your opinion probably isn’t the one that matters.
18

19. Potential test subjects
• Actual site users
• Representative site users
• Crowdsourced
• Artificial intelligence
19

20. Actual site users
Pros:
• Most relevant to site
Cons:
• May have limited availability
• May require compensation
20

21. Representative users
Pros:
• Relevant to site
• More relevant than crowdsourced and AI
Cons:
• Only as relevant as your recruiting
• May require compensation
21

22. Crowdsourced
Pros:
• Readily available
• Likely more relevant than and AI
Cons:
• May not closely model actual site users
• May require compensation
22

23. Artificial intelligence
Pros:
• Readily available
• Complete tests in real time
Cons:
• May not closely model actual human users
23

24. Which is right for you?
• What kind of site are you testing?
• What is your site’s demographic?
• What access do you have to site users?
24

25. How to test?
25

26. Types of tests
One size does not fit all!
26

27. Formative Tests
• Used throughout the change process.
• Support tool for decision making.
• Different tools for different stages.
27

28. Summative Tests
• Generally QA testing.
• Determine if finished feature/site is usable.
• Confirm improved usability.
• Used prior to making changes to determine
and document any usability problems.
28

29. Types of testing
• Analytics
• User Testing
• Usability Testing
29

30. Usability testing
• Based on task completion.
• Usually measurable.
30

31. User Testing
• Based on user opinion.
• Not tied to specific task completion.
31

32. Analytics
• Measure of actual user behavior.
• May suggest usability problems.
• Not a substitute for usability testing.
• May provide baseline to measure success of
plans derived from usability testing.
32

33. Analytics
It takes time to collect data; start today!
33

34. Web Analytics
Formative
• Records actual site visitor interaction.
• Can suggest usability problems.
34

35. Web Analytics
Possible tools
• Google Analytics (Google)
• WebTrends Analytics (WebTrends)
• AWStats (AWStats)
35

36. Heat Maps
Formative
• Records specific actual site visitor
interaction: mouse click and scroll depth.
• Can suggest usability problems.
36

37. Heat Maps
Possible tools
• Crazy Egg (Crazy Egg)
• Hotjar (Hotjar)
• Lucky Orange (Lucky Orange)
• Google Analytics* (Google)
*sort of
37

38. User testing
Opinions are like noses, everyone’s got one.
38

39. User Surveys
Formative
• Ask users what they like or don’t like.
• Good starting point for refining or adding
new features.
39

40. User Surveys
Possible tools
• Paper
• SurveyMonkey (SurveyMonkey)
• SurveyGizmo (Widgix)
• Qualtrics (Qualtrics)
40

41. A/B & Multivariate Tests
Formative
• Users are randomly given an
implementation.
• Determine which implementation is most
effective based on conversion rates.
41

42. A/B & Multivariate Tests
Possible tools
• GoogleWebsite Optimizer (Google)
• CMS Specific tools (e.g. Drupal Multivariate)
• Optimizely (Optimizely)
42

43. Usability testing
Choose the test based on your question.
43

44. Open Card Sort
Formative
• Participants organize a list of items into
categories they provide.
44

45. Open Card Sort
Possible tools
• Index cards
• Cardsort (Optimal Workshop)
• UserZoom (UserZoom)
45

46. Closed Card Sort
Formative
• Participants organize a list of items into
categories you provide.
46

47. Closed Card Sort
Possible tools
• Index cards
• Cardsort (Optimal Workshop)
• UserZoom (UserZoom)
47

48. Tree Test
Formative
• Asks test participants where they would go
to find specific content.
• Tests the organization of content if
navigation labels make sense to users
• Test website’s pre-defined navigation.
48

49. Tree Test
Possible tools
• Index cards
• Treejack (Optimal Workshop)
49

50. Expectancy Test
Formative
• Tests a user’s impressions of an early
prototype of the interface.
• Typically paper prototype, wireframes, or
design concepts.
50

51. Expectancy Test
Possible tools
• Paper Prototype
• Wireframes
• Design Concepts
51

52. 5-Second Usability Test
Formative
• Tests individual pages (not homepage) for a
specific scenario and task.
• Not suitable for testing complex systems
or user interaction.
52

53. 5-Second Usability Test
Possible tools
• Paper Prototype
• Wireframes
• Design Concepts
• FiveSecondTest (UsabilityHub)
53

54. Visual Affordance Test
Formative
• Tests the “Clickability/Tappability.”
• Show subject web page or design concept.
• Subject indicate every item believed
clickable, and items believed not clickable.
54

55. Visual Affordance Test
Possible tools
• Paper Prototype
• Wireframes
• Design Concepts
55

56. Click Test
Summative
• Provide scenario and task.
• User clicks on the design element to
complete task.
56

57. Click Test
Possible tools
• Paper Prototype
• Wireframes
• Design Concepts
• Chalkmark (Optimal Workshop)
• Flippa (UsabilityHub)
57

58. Sample-of-One Test
Summative
• Document fundamental errors in interface
layout, labels, content organization,
affordance, and functionality
• Use single subject who fits main persona.
58

59. Sample-of-One Test
Possible tools
• Silverback (Clearleft)
• OpenHallway (OpenHallway)
• TryMyUI (TryMyUI)
59

60. Performance Test
Formative or Summative
• Can be to look for problems in existing site
or confirm function in a new site.
• Provide participants a task with a scenario
• Observe user interact with website.
60

61. Performance Test
Possible tools
• Silverback (Clearleft)
• OpenHallway (OpenHallway)
• TryMyUI (TryMyUI)
61

62. Free Exploration Test
Summative
• Observe users interaction with website for
set period of time (5 minutes?).
• Use when navigation is no longer an issue.
62

63. Free Exploration Test
Possible tools
• Silverback (Clearleft)
• OpenHallway (OpenHallway)
• TryMyUI (TryMyUI)
63

64. Free and Inexpensive
TestingTools
64

65. Survey Monkey
Likes:
• Robust question functionality & reporting.
• Filter response data by response or URL.
• Data export to allow manipulation in excel.
Be Aware:
• Some features require more pricey plans.
65

66. Survey Monkey
66

67. Survey Monkey
67

68. Survey Monkey
68

69. Survey Monkey
69

70. Survey Monkey
70

71. Survey Monkey
71

72. Survey Monkey
72

73. CrazyEgg
Likes:
• Variety of ways to filter/visual data
• Can test mobile and desktop
Be Aware:
• Not real time.
• Not cheap to monitor entire sites.
73

74. Crazy Egg
74

75. Crazy Egg
75

76. Crazy Egg
76

77. Crazy Egg
77

78. Crazy Egg
78

79. Crazy Egg
79

80. Crazy Egg
80

81. Optimal Sort
Likes:
• Easy to set up.
• Can use for remote testing.
Be Aware:
• Normalizing data requires assumptions.
• Not cheap if you need the full version.
81

82. Optimal Sort
82

83. Optimal Sort
83

84. Optimal Sort
84

85. Optimal Sort
85

86. Optimal Sort
86

87. Optimal Sort
87

88. Treejack
Likes:
• Easy to set up.
• Can use for remote testing.
Be Aware:
• Not cheap if you need the full version.
88

89. Treejack
89

90. Treejack
90

91. Treejack
91

92. Treejack
92

93. Treejack
93

94. Chalkmark
Likes:
• Easy to set up.
• Can use for remote testing.
Be Aware:
• Only single response per question.
• Not cheap if you need the full version.
94

95. Chalkmark
95

96. Chalkmark
96

97. Chalkmark
97

98. Chalkmark
98

99. Silverback
Likes:
• Portable, works well on a standard laptop.
• Does not require a dedicated usability lab.
Be Aware:
• Mac only.
• For on location testing.
99

100. Silverback
100

101. Conclusion
101

102. Conclusion
• No silver bullet to make sites usable.
• Usability is a never-ending process.
• Making sites more usable is fun for all!
102

103. Special thanks
103

104. Follow us!
@DesignHammer
facebook.com/DesignHammer
www.designhammer.com
104