Guerilla
Usability Testing
by Andy Budd
So what is usability
        testing anyway?

Usability testing generally involves setting a series
of tasks for people to...
Example Usability Test
Think aloud protocol
As the name suggests, you set a series of tasks
and ask the subject to verbalise their thoughts
and f...
Rock, paper, scissors

You can test on everything from a low fidelity
paper prototype, through a HTML prototypes,
to a live...
Testing Shadowgrounds
Benefits of usability
        testing for designers
•   Help inform the design process
•   Uncover unknown or unexpected i...
Benefits of usability
       testing for site owners
•   Fix problems early in the process when they are cheaper to fix
•  ...
Back to Shadowgrounds
Formative vs. Summative
Behavior vs. Opinion
Performance Metrics
Completion Rates – Are users able to complete the task?

Time on Task – How long does it take users to...
Usability Testing Halo3
Data
Results
Formal testing
Pros
•   Will recruit test subjects for you
•   Facilitators less likely to bias results
•   Comfortable facilities
•   Th...
Cons
•   Time consuming to set up, run and analyse results
•   Expensive so often only run once at the end of the project
...
Informal (guerilla) testing
Pros
•   Very quick and easy to perform
•   Relatively inexpensive so can afford multiple tests
•   Qualitative nature can...
Cons
• Have to recruit subjects, arrange location and write tests
  yourself.
• Familiarity with project may introduce per...
RITE Methodology
How to get corporate/
          client buy-in
•   Don’t ask permission. Just do it anyway.
•   Show, don’t explain
•   Exp...
Select your tasks
 (5-6 tasks, 45 min max)
Good and Bad Tasks
Bad: Search for a bookcase

Good: You have 200+ books in your fiction
collection, currently in boxes str...
Testing plan
•   Introduce the project
•   State the goals of the test
•   Profile your desired test subjects
•   Create a ...
Select your subjects
Recruiting test subjects
•   Client contacts
•   Via the client website (ehtnio.com?)
•   Recruitment agency
•   Market re...
Remuneration
•   Cash
•   Vouchers
•   Wine and Chocolates
•   Beer and pizza
•   Movie tickets
•   Nothing
How many test subjects
    are enough?
Scouting for Locations
The basic set-up
Lab in a box
Morae
Silverback
Selecting a moderator
On the day
• Don’t keep people waiting around
• Have everything already set up and waiting
• Welcome people and thank them...
Think aloud exercise
Be a good moderator
• Be friendly and put the person at ease
• Stay objective and detached. Try not to bond
• Start with a...
Moderation Techniques
•   “What are you currently thinking?”
•   “Describe the steps you’re going through here”
•   “What ...
Usability Testing Mistakes
•   Not doing a proper run through of the test in advance
•   Incomplete or buggy prototypes
• ...
Wrapping up
•   Answer any questions they may have
•   Administer the post test survey
•   Thank them for their help
•   S...
Review and implement
        ASAP
Reports and Presentations
Guerilla Usability Testing
Guerilla Usability Testing
Guerilla Usability Testing
Upcoming SlideShare
Loading in...5
×

Guerilla Usability Testing

43,497

Published on

A cut down version on the Guerilla Usability Testing workshop I gave at Web Directions UX.

Published in: Business, Technology
9 Comments
209 Likes
Statistics
Notes
No Downloads
Views
Total Views
43,497
On Slideshare
0
From Embeds
0
Number of Embeds
22
Actions
Shares
0
Downloads
1,825
Comments
9
Likes
209
Embeds 0
No embeds

No notes for slide

Guerilla Usability Testing

  1. 1. Guerilla Usability Testing by Andy Budd
  2. 2. So what is usability testing anyway? Usability testing generally involves setting a series of tasks for people to complete and noting any problems they encounter – It’s as simple as that!
  3. 3. Example Usability Test
  4. 4. Think aloud protocol As the name suggests, you set a series of tasks and ask the subject to verbalise their thoughts and feelings by ‘thinking aloud’. The object of this is to gain valuable insight into the thought processes behind the users actions.
  5. 5. Rock, paper, scissors You can test on everything from a low fidelity paper prototype, through a HTML prototypes, to a live site. You can test your existing site, your competitors sites or a brand new solutuion.
  6. 6. Testing Shadowgrounds
  7. 7. Benefits of usability testing for designers • Help inform the design process • Uncover unknown or unexpected issues • Fix problems early in the process when they are easy to fix • Test assumptions • Highlight unnecessary features • Provide objectivity help and solve opinion battles • Set baselines and measure improvement • Get stakeholder buy-in
  8. 8. Benefits of usability testing for site owners • Fix problems early in the process when they are cheaper to fix • Improve customer satisfaction and retention • Increase conversion rates • Reduce maintenance, training and support costs • Get stakeholder buy-in • Reduce project risk • Set baselines and measure improvement • It’s so cheap you’d be stupid not to
  9. 9. Back to Shadowgrounds
  10. 10. Formative vs. Summative
  11. 11. Behavior vs. Opinion
  12. 12. Performance Metrics Completion Rates – Are users able to complete the task? Time on Task – How long does it take users to complete? Page Views – How many pages does it take to complete? Errors – The number and severity of errors per task? Satisfaction – How does the user rate the system?
  13. 13. Usability Testing Halo3
  14. 14. Data
  15. 15. Results
  16. 16. Formal testing
  17. 17. Pros • Will recruit test subjects for you • Facilitators less likely to bias results • Comfortable facilities • The whole team can be present • Quantitative data likely to have more statistical validity • Professionally written report or presentation
  18. 18. Cons • Time consuming to set up, run and analyse results • Expensive so often only run once at the end of the project • Facilitators only have shallow knowledge of the project • Focus on quantitative over qualitative reduces insight • Smaller test samples can invalidate quantitative results • Most projects don’t require the level of statistical rigour • Easy for the results to be ‘thrown over the fence’ and forgotton
  19. 19. Informal (guerilla) testing
  20. 20. Pros • Very quick and easy to perform • Relatively inexpensive so can afford multiple tests • Qualitative nature can provide improved design insight • Results can be fed back into the design process almost immediately with no loss of signal
  21. 21. Cons • Have to recruit subjects, arrange location and write tests yourself. • Familiarity with project may introduce personal bias • Less statistical rigour on quantitative results
  22. 22. RITE Methodology
  23. 23. How to get corporate/ client buy-in • Don’t ask permission. Just do it anyway. • Show, don’t explain • Explain how it will reduce the risk that the project will fail. • Demonstrate the return on investment. • Highlight the benefits for your team. • Explain how it will provide hard data you can measure success against.
  24. 24. Select your tasks (5-6 tasks, 45 min max)
  25. 25. Good and Bad Tasks Bad: Search for a bookcase Good: You have 200+ books in your fiction collection, currently in boxes strewn around your living room. Find a way to organize them.
  26. 26. Testing plan • Introduce the project • State the goals of the test • Profile your desired test subjects • Create a test subject screener • Outline your testing methodology • Define your tasks • Write a testing script • Design a post test survey
  27. 27. Select your subjects
  28. 28. Recruiting test subjects • Client contacts • Via the client website (ehtnio.com?) • Recruitment agency • Market research agency • Twitter • Local notice boards and mailing lists • Other offices/departments • Classified ads e.g. gumtree
  29. 29. Remuneration • Cash • Vouchers • Wine and Chocolates • Beer and pizza • Movie tickets • Nothing
  30. 30. How many test subjects are enough?
  31. 31. Scouting for Locations
  32. 32. The basic set-up
  33. 33. Lab in a box
  34. 34. Morae
  35. 35. Silverback
  36. 36. Selecting a moderator
  37. 37. On the day • Don’t keep people waiting around • Have everything already set up and waiting • Welcome people and thank them for their time • Explain what you’ll be doing and how the test will work • Remunerate them in advance so they understand it’s not contingent on their performance • Ask them to sign a waiver • Remind them that you’re testing the system, not them • Encourage them to think aloud
  38. 38. Think aloud exercise
  39. 39. Be a good moderator • Be friendly and put the person at ease • Stay objective and detached. Try not to bond • Start with a simple task • Keep instructions simple • Don’t give hints or ask leading questions • Give encouraging but non-committal feedback • Don’t ask opinion-based questions • Avoid asking what the user would do in a certain sittuation • If you don’t understand an action feel free to ask why they did something • Know when to stop a task but don’t rescue people too soon • Summarise key findings
  40. 40. Moderation Techniques • “What are you currently thinking?” • “Describe the steps you’re going through here” • “What do you think will happen next?” • “Is that what you expected to happen?” • “That was confusing?” • “And you were expecting ...?” • “Would you mind repeating that again?”
  41. 41. Usability Testing Mistakes • Not doing a proper run through of the test in advance • Incomplete or buggy prototypes • Not scheduling enough time between sessions • Bad task design • Accidental Revelation • Getting frustrated or losing detachment
  42. 42. Wrapping up • Answer any questions they may have • Administer the post test survey • Thank them for their help • Show them out • Capture any important notes immediatly • Prepare for the next test
  43. 43. Review and implement ASAP
  44. 44. Reports and Presentations
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×