Have you ever wanted to test your website or web app with users, but haven’t been sure which tools and techniques to use to ensure you get the most out of it? I'll walk you through the core concepts of guerrilla usability testing in agile environments and how you can ensure that your team is getting the insights you need to keep your project user centered throughout the development process. Topics include: - An introduction to guerrilla usability testing - Why it matters and some common findings - Planning a usability testing session - Making your users comfortable and setting up a room for testing - Simple approaches to testing on laptops and devices - Some tips and tricks for capturing your notes that can save you hours of analysis time

2. Getting the most from
guerrilla usability testing
Chris Bush
Head of Experience Design
@wearesigma
@suthen#agilecam

3.  15 years in user centred design roles
 Heads up Sigma’s talented UX team
(Manchester & Cambridge)
 Help out @NUXUK,
Previously: WaSP ILG and WaSP Interact

6.  What is usability testing and why it matters
 Some common usability findings
 How to run a guerrilla usability testing
session

7. Raise your hands 

8. When did you usability test?
Project start 1st release

9. When do we usability test?
Project start 1st release

10. What is usability testing?

11. Usability means making sure something works well, and
that a person of average ability or experience can use it
for its intended purpose without getting hopelessly
frustrated.
- Steve Krug

12. Why does it matter?

13. The procedure is actually quite simple. First, you arrange the items into different groups. Of course one pile may
be sufficient depending on how much there is to do. If you have to go somewhere else due to lack of facilities that
is the next step; otherwise you are pretty well set. It is important not to overdo things. That is, it is better to do too
few things at once than too many. In the short run this may not seem important but complications can easily
arise. A mistake can be expensive as well. At first, the whole procedure will seem complicated. Soon, however, it
will become just another facet of life. It is difficult to foresee any end to the necessity for this task in the
immediate future, but then, one can never tell. After the procedure is completed one arranges the materials into
different groups again. Then they can be put into their appropriate places. Eventually they will be used once
more and the whole cycle will then have to be repeated. However, that is part of life.

14. Washing clothes
The procedure is actually quite simple. First, you arrange the items into different groups. Of course one pile may
be sufficient depending on how much there is to do. If you have to go somewhere else due to lack of facilities that
is the next step; otherwise you are pretty well set. It is important not to overdo things. That is, it is better to do too
few things at once than too many. In the short run this may not seem important but complications can easily
arise. A mistake can be expensive as well. At first, the whole procedure will seem complicated. Soon, however, it
will become just another facet of life. It is difficult to foresee any end to the necessity for this task in the
immediate future, but then, one can never tell. After the procedure is completed one arranges the materials into
different groups again. Then they can be put into their appropriate places. Eventually they will be used once
more and the whole cycle will then have to be repeated. However, that is part of life.
Context aids
understanding

16. Common findings

17. Navigation

18. Navigation
isn’t
predictable

19. ..or
obvious

20. Good sites
have clear
sign posting

21. Page flow

22. Poor
page
flow

24. What next??
Scroll back up!

25. And fill in this...

26. Right side
content
is often
overlooked

27. Much
better

28. Calls to action

29. Primary action Secondary action Tertiary action
CTAs
are often
formulaic

30. Primary action Secondary action Tertiary action

31. Primary action Secondary action Tertiary action

33. Simplicity

34. Which form do visitors use most?

35. Which form do visitors use most?
14
%
86
%

36. Which form do visitors use most?
This one requires
you to think about
what you need...
...You probably
don’t really know
yet

37. Which form do visitors use most?
By asking users
relevant questions.
You limit the
amount of
thinking they need
to do to achieve
their overall goal

38. Which form do visitors use most?
Which in turn
provides users
with a much
better and more
positive
experience.

39. How well do they convert?
17
%
55
%

40. Running your own
usability test
- it’s easier than you think

41. Pros:
• Requires no specialist equipment
• Very portable (on location, café, office, home)
• Great for early stage testing (Prototypes, etc.)
• Great for AGILE and Rapid Iterative Testing and Evaluation (RITE)
• Minimal overhead (Time, People, Cost)
Cons:
• Moderator or reviewer bias
• The team takes on responsibility for ensuring
everything is organised
(Users, Room, Tasks, Write up)
• Finding a good testing room is sometimes tricky
Go Guerrilla

42. Getting the most
from your research

43. In general
• Any users are better than none (5 to 10 is ideal)
• Aim for a good cross-section of representative users
• Max 30-45 minutes to complete
Task design *
• Create a list of research objectives
• Design your tasks
• and then word tasks as scenarios
On the day
• Ask users to think aloud to help you gain insights
• Rotate tasks
• Use a discussion guide for consistency
• Use a scribe
Where to start

44. Research objective
• Do users understand the secondary navigation system?
• Do users have any challenges with forms errors?
Task
• Can users find and complete the returns form?
Scenario
“After receiving your new camera you have
noticed the lens is cracked. Using the site can you
request a replacement?“
• Be pragmatic about leading questions
• Do a control test (you’re testing your scenarios at this point)
Designing your tasks

45. Software
• Open broadcast software – FREE!
(https://obsproject.com)
• Reflector ($15)
Hardware
• PC / Mac
• Webcam / Document camera
Alternative
• Silverback v2 (free – Mac only)
• Camtasia - PC
Technical kit

48. Choosing users

49. Don’t forget that
editors are your
users too!

50. On the day

52. Don’t do this!

53. • Find the most comfortable room you can find
• Sit back from the user
- Sit outside of the users field of view
• Use reflective questioning
- Participant: What does this do?
- You: What do you think it does?
• Don’t be a monster!
If the user is struggling, it’s ok to help them.
Just wait a bit, then wait a bit longer before you do.
• Use your research questions to keep you focused
• Document both the positive and negative findings
• Take 30mins to review notes after each session
On the day

54. Insights
& reporting

55. Creating a
matrix of issues
can help your
team keep track
of your finding

56. Keep your
reporting
Simple
&
Actionable

57. In summary
 Aim to use 5-10 users
 Pilot your scenarios before engaging real users - Tasks nearly always need rewording
 Make the test room as comfortable as possible
 Sit back from the user, so they don’t feel like they are being watched
 Use reflective questioning
 Capture good and bad findings
 Wrap up after each session
 Make your write-up actionable

58. Thank You
Any questions?
@wearesigma - @suthen