The document discusses guerrilla usability testing as an affordable and informal method for evaluating products. Key points include: - Guerrilla testing involves getting people off the street to complete tasks on a product while being observed to uncover unexpected issues. - Only a small number of testers (5-12) are needed to find most usability problems. - The method was used by the author to test an online booking engine, finding issues with map controls and elements that were unclear. - While informal, guerrilla testing can effectively inform the design process and highlight problems early to improve the user experience.

1. Guerrilla Usability Testing

2. Usability
"Good design is problem solving"
Jeffrey Veen
(Analytics,Typekit, author...)
Book: Psychology of everyday things
by Donald Norman

3. What’s Usability
• Usable, "easy to use". It's a very intuitive and simple
concept, yet many companies don't do it (officially)
• Like CT, but ABE one of the simplest Booking Engines.
• User Centered design. It has to be self-explained.
• Technology shouldn’t make us feel stupid, neither force
us to learn complicated stuff. “Don’t Make me Think”,
book by Steve Krug
• Jakob Nielsen : Focus on monetizing

4. Example 1 : Useit.com

5. Example 2 : Craiglists, another dull yet successful (usable) site

6. Example 3 : Amazon

7. Example 6 : Apple
and more; Google,Youtube...

8. Example 7 : ABE step 1. 3 simple steps (not just good for marketing)

9. ABE Step 2 grid layout. Not the most common layout, but it works.

10. ABE Step 3

12. Why things are unusable?
• Development focus on the machine
• It's hard to achieve
• Teams don't work integrated
• Design vs implementation
• Developer’s lack of perspective.
• like...

13. Unusable: Irishrail.ie : Complex, uses its own language...

14. Why things are usable?
• Early focus on users and task. Interactive Design
• Prototyping and wireframing
• Evaluation and measurement
• Research, Surveys, Observation
• like...

15. Usable : Dropbox

16. Development process stages
Analysis
Design
Evaluation
Implementation

17. Usability Testing
Usability testing is a technique
used to evaluate a product by
testing it on users...
Wikipedia
Usability testing generally involves
setting a series of tasks for people
to complete and noting any
problems they encounter
Andy Budd

18. Usability Testing Lab

19. Benefits on usability testing
For designers For site owners
• Help inform the design process • Improve customer satisfaction
and retention
• Uncover unknown or
unexpected issues • Increase conversion rates
• Fix problems early (easier) • Reduce maintenance, trainning
and support costs
• Test assumptions
• Reduce project risk
• Highlight unnecessary features
• Highlight unnecessary features
• Provide Objectivity solve
opinion battles • It’s so cheap you’d be stupid
not to
• Set baselines and measure
improvement
Andy Budd

20. Effective ways of testing
• Eye tracking. Heat maps
• A/B Testing
• Formal testing (made by a company, lab, focus groups...)
• Guerrilla style

21. Eye tracking: Heat map

22. A/B Testing : Used by Argus to implement Lightbox

23. A/B Testing : Used by Argus to implement Lightbox
“The results were encouraging: after 3 weeks of testing, the average landing page conversion rate
had gone from 2.70% to 3.33% (an average rise of 20% in the conversion rate)
Following the positive impact of the test, the light box functionality was rolled out across all PPC
landing pages, leading to marked increase in conversions (+19%)
This was then rolled out on to the rest of the site leading to an improvement of 5.99% in the
conversion rate for all traffic mediums”
(Thanks Dave)

24. Guerrilla Usability Testing
Guerrilla warfare is the
unconventional warfare and
combat in which a small group
of combatants use mobile
tactics in the form of ambushes
and raids to combat a larger and
less mobile formal army.

25. Guerrilla usability testing
• Get somebody from the street, and ask them to do simple task,
observe and record (using Camtasia or Silverback)
• Affordable (small rewards), informal. Everybody can help or bring
somebody
• Small group of people (5 get 85% issues, 12 get 99%)
• Moderator and “Think Aloud” protocol
• Precise task definition
• Using tools for metrics like analytics, surveys and feedbacks...

26. Pros and Cons of Guerrilla style
PROS CONS
• Help inform the design process • Improve customer satisfaction
and retention
• Uncover unknown or
unexpected issues • Increase conversion rates
• Fix problems early (easier) • Reduce maintenance, trainning
and support costs
• Test assumptions
• Reduce project risk
• Highlight unnecessary features
• Highlight unnecessary features
• Provide Objectivity solve
opinion battles • It’s so cheap you’d be stupid
not to
• Set baselines and measure
improvement

27. Silverback

28. How does it work

29. Gerrilla Testing
• We did it for the first time in September 2010 before push
live the new Step 3
• We found just a few issues; actually step 1: Magic locations
• For a moment we though, Is this really working?

30. Results

31. Gerrilla Testing
• Then we did it again, in the process of developing the map, this time
we did it earlier stage.
• Very different results, tough but positive feedback. They liked the idea.
• Things to work on and improve. But at least we knew what to focus
on.

32. 2nd Testing: The Map

33. Issues we found
• Pick up locations, most didn’t know they could put the address
• Zooms and controls
• Some didn’t see “I’m here”, confused the best price icon...
• Missing elements like scales
• Ignoring funcionalities, (Legend, Sort by, Refres)
• Didn’t fully understand which car was which flag in the map

34. Gerrilla Testing
Video

35. Work in progress...
Thanks
questions?