How To Run Your Own Guerrilla Usability Testing

  • 3,963 views
Uploaded on

A workshop we conducted at the University of British Columbia. A brief how-to guide on how to plan, recruit, set up, facilitate and analyze a usability testing campaign.

A workshop we conducted at the University of British Columbia. A brief how-to guide on how to plan, recruit, set up, facilitate and analyze a usability testing campaign.

More in: Technology , Business
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
No Downloads

Views

Total Views
3,963
On Slideshare
0
From Embeds
0
Number of Embeds
6

Actions

Shares
Downloads
4
Comments
4
Likes
30

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. How to Run Your Own Guerrilla Usability Testing A Guide By John Chan & Jonathan Kift April 18, 2012Friday, 20 April, 12
  • 2. So What Is Usability Testing Anyway? @jtcchan & @daftkiftyFriday, 20 April, 12
  • 3. So what is USABILITY TESTING anyway? “ Usability really just means making sure that something works well — whether it’s a Web site, a fighter jet or a revolving door. Steve Krug Author of “Don’t Make Me Think” @jtcchan & @daftkiftyFriday, 20 April, 12
  • 4. So what is USABILITY TESTING anyway? Usability testing usually involves 1. Making a series of tasks for people to complete and 2. Noting any problems they come across @jtcchan & @daftkiftyFriday, 20 April, 12
  • 5. Here’s an example: @jtcchan & @daftkiftyFriday, 20 April, 12
  • 6. Why Should We Test? @jtcchan & @daftkiftyFriday, 20 April, 12
  • 7. Why should we test? Cultivate Surprises It is an environment where we can find out how we intend something will be used vs. how it actually is being used. @jtcchan & @daftkiftyFriday, 20 April, 12
  • 8. Why should we test? Testing Works 99.99%* Of The Time Even a ‘bad’ test with the wrong users can show you important things you can do to improve your site. * Can someone really make a design worse from testing? Possibly but I honestly believe you can’t do wrong. @jtcchan & @daftkiftyFriday, 20 April, 12
  • 9. Why should we test? Buy-In From Team & Stakeholders User testing puts stakeholder and teams on common goals with data. Also, great for informing decisions with greater confidence. (e.g. picking “a” vs “b”) @jtcchan & @daftkiftyFriday, 20 April, 12
  • 10. What Should We Test and When? @jtcchan & @daftkiftyFriday, 20 April, 12
  • 11. What should we test and when? When Research What Current site Competitors site New Ideas @jtcchan & @daftkiftyFriday, 20 April, 12
  • 12. What should we test and when? When Research Design What Current site Paper Prototypes Competitors site Wireframes New Ideas Mockups @jtcchan & @daftkiftyFriday, 20 April, 12
  • 13. What should we test and when? When Research Design Develop What Current site Paper Prototypes HTML Prototypes Competitors site Wireframes Navigation New Ideas Mockups Interactions @jtcchan & @daftkiftyFriday, 20 April, 12
  • 14. What should we test and when? When Research Design Develop Evaluate What Current site Paper Prototypes HTML Prototypes Live Site Competitors site Wireframes Navigation A/B Testing New Ideas Mockups Interactions Multivariate Testing @jtcchan & @daftkiftyFriday, 20 April, 12
  • 15. How to Run Your Own Test @jtcchan & @daftkiftyFriday, 20 April, 12
  • 16. How to Run Your Own Test What you’ll need: 1 x Facilitator : someone that explains and goes through the tasks @jtcchan & @daftkiftyFriday, 20 April, 12
  • 17. How to Run Your Own Test What you’ll need: 1 x Facilitator : someone that explains and goes through the tasks 1 x Observer/Recorder : someone that notes key observations @jtcchan & @daftkiftyFriday, 20 April, 12
  • 18. How to Run Your Own Test What you’ll need: 1 x Facilitator : someone that explains and goes through the tasks 1 x Observer/Recorder : someone that notes key observations 1 x Laptop : for the participant to perform their test @jtcchan & @daftkiftyFriday, 20 April, 12
  • 19. How to Run Your Own Test Nice to haves: Some Money (for recruiting!) @jtcchan & @daftkiftyFriday, 20 April, 12
  • 20. How to Run Your Own Test Nice to haves: Some Money Screen Recorder (for recruiting!) (or a video camera) @jtcchan & @daftkiftyFriday, 20 April, 12
  • 21. How to Run Your Own Test Planning Before During After @jtcchan & @daftkiftyFriday, 20 April, 12
  • 22. How to Run Your Own Test Planning Facilitating Before During After @jtcchan & @daftkiftyFriday, 20 April, 12
  • 23. How to Run Your Own Test Planning Facilitating Analyzing Before During After @jtcchan & @daftkiftyFriday, 20 April, 12
  • 24. How to Run Your Own Test Planning Facilitating Analyzing Before During After @jtcchan & @daftkiftyFriday, 20 April, 12
  • 25. Planning Your Test @jtcchan & @daftkiftyFriday, 20 April, 12
  • 26. Planning Your Test @jtcchan & @daftkiftyFriday, 20 April, 12
  • 27. Planning Your Test 1 Preparing Tasks @jtcchan & @daftkiftyFriday, 20 April, 12
  • 28. Planning Your Test 1 Preparing Tasks 2 Recruiting Participants @jtcchan & @daftkiftyFriday, 20 April, 12
  • 29. Planning Your Test 1 Preparing Tasks 2 Recruiting Participants 3 Finding a Location @jtcchan & @daftkiftyFriday, 20 April, 12
  • 30. 1 On Preparing Tasks Pick Tasks Based On Your Site/User Goals What is your site trying to accomplish? What are your users trying to accomplish? @jtcchan & @daftkiftyFriday, 20 April, 12
  • 31. 1 On Preparing Tasks Keep It Under 15 Tasks This takes about 45 minutes to complete, which is perfect if you require participants an hour of their time. It also reduces the amount of data analysis you’ll have to do, which increases significantly with each new task you introduce. @jtcchan & @daftkiftyFriday, 20 April, 12
  • 32. 1 On Preparing Tasks Create Tasks That Are Measurable Pick tasks that are easy to determine success or completion. This helps when you evaluate the data. @jtcchan & @daftkiftyFriday, 20 April, 12
  • 33. For Example Bad Tasks Good Tasks Search for a book. Please show me where would you look to see if the library has a copy of the book titled The Tipping Point by Malcolm Gladwell. @jtcchan & @daftkiftyFriday, 20 April, 12
  • 34. 2 Recruiting Participants A few places that subjects can be found: Coffee Shops, Bars, Malls, Libraries Client or Colleague Contacts Facebook/Twitter Bulletin Boards Classified Ads (e.g. Craigslist) Mailing Lists Amazon’s Mechanical Turk @jtcchan & @daftkiftyFriday, 20 April, 12
  • 35. 2 Recruiting Participants Don’t Worry About Finding The Right Users It’s good to test with the right people but it’s much more important to test often and test early. @jtcchan & @daftkiftyFriday, 20 April, 12
  • 36. 2 Recruiting Participants Use Incentives If You Need Their Time It’s important to properly incentivize your volunteers and participants. This makes recruitment much easier. Anything under about 3 minutes can be free. @jtcchan & @daftkiftyFriday, 20 April, 12
  • 37. 2 Recruiting Participants Some incentives you can use: Cash Gift Cards Food and Drinks (Coffee, Pizza, Beer, Chocolate) Movie Tickets @jtcchan & @daftkiftyFriday, 20 April, 12
  • 38. 2 Recruiting Participants It’s Better to Have More Participants and Fewer Tasks Per Session In our experience, we’ve been much more effective in recruiting for shorter tests and smaller incentives because there’s less commitment involved. @jtcchan & @daftkiftyFriday, 20 April, 12
  • 39. 2 Recruiting Participants You’ll Need 3-5 Participants Per Task Most of the major findings will be revealed with the first 3 participants and the value of each new participant significantly decreases over time. @jtcchan & @daftkiftyFriday, 20 April, 12
  • 40. 3 On Finding a Location Pick An Open Area This works better than a closed room as it is a more comfortable environment to reduce anxiety. @jtcchan & @daftkiftyFriday, 20 April, 12
  • 41. 3 On Finding a Location Ask beforehand If you’re using a space like a mall, a school or coffee shop, remember to reach out to the appropriate contacts to let them know what you’re doing in advance. And remember to buy coffee! @jtcchan & @daftkiftyFriday, 20 April, 12
  • 42. 3 On Finding a Location Their Office is Too Far? Visiting in person would be great but consider doing a remote user testing by doing a video call (e.g. Skype) to conduct your test. @jtcchan & @daftkiftyFriday, 20 April, 12
  • 43. How to Run Your Own Test Planning Facilitating Analyzing Before During After @jtcchan & @daftkiftyFriday, 20 April, 12
  • 44. Facilitating A Test @jtcchan & @daftkiftyFriday, 20 April, 12
  • 45. Facilitating a Test @jtcchan & @daftkiftyFriday, 20 April, 12
  • 46. Facilitating a Test 1 Preparation and Setup @jtcchan & @daftkiftyFriday, 20 April, 12
  • 47. Facilitating a Test 1 Preparation and Setup 2 Moderating the Test @jtcchan & @daftkiftyFriday, 20 April, 12
  • 48. Facilitating a Test 1 Preparation and Setup 2 Moderating the Test 3 Finishing Up a Session @jtcchan & @daftkiftyFriday, 20 April, 12
  • 49. Preparation and Setup Here’s an example of our set up. @jtcchan & @daftkiftyFriday, 20 April, 12
  • 50. 1 Preparation and Setup Once Your Participant Arrives @jtcchan & @daftkiftyFriday, 20 April, 12
  • 51. 1 Preparation and Setup Once Your Participant Arrives • Welcome them and thank them for joining you @jtcchan & @daftkiftyFriday, 20 April, 12
  • 52. 1 Preparation and Setup Once Your Participant Arrives • Welcome them and thank them for joining you • Explain what you’re doing and how the test works @jtcchan & @daftkiftyFriday, 20 April, 12
  • 53. 1 Preparation and Setup Once Your Participant Arrives • Welcome them and thank them for joining you • Explain what you’re doing and how the test works • Let them know you’re recording for research purposes and ask them to sign a waiver @jtcchan & @daftkiftyFriday, 20 April, 12
  • 54. 1 Preparation and Setup Once Your Participant Arrives • Welcome them and thank them for joining you • Explain what you’re doing and how the test works • Let them know you’re recording for research purposes and ask them to sign a waiver • Remind them you’re testing the site, not them - no pressure! @jtcchan & @daftkiftyFriday, 20 April, 12
  • 55. 1 Preparation and Setup Once Your Participant Arrives • Welcome them and thank them for joining you • Explain what you’re doing and how the test works • Let them know you’re recording for research purposes and ask them to sign a waiver • Remind them you’re testing the site, not them - no pressure! • Ask them to pretend they’re using the site “by themselves” but encourage them to think aloud @jtcchan & @daftkiftyFriday, 20 April, 12
  • 56. 1 Preparation and Setup Encourage Them To Think Aloud This bears repeating. This is HOW you find out what they’re thinking as they go through your site. Important! @jtcchan & @daftkiftyFriday, 20 April, 12
  • 57. 2 Moderating the Test Start With a Simple Task This helps them build their confidence and puts them at ease. You’ll want them to think “hey, this is easy - I can do this!” @jtcchan & @daftkiftyFriday, 20 April, 12
  • 58. 2 Moderating the Test Resist Giving Hints Or Ask Leading Questions Watch out for using “keywords” from your tasks as that may not be a sign that your site is easy to use. If appropriate, try to use synonyms for the answers they’re looking for. @jtcchan & @daftkiftyFriday, 20 April, 12
  • 59. 2 Moderating the Test Know When To Stop If your subject is clearly having troubles, it’s okay to ask “would you keep looking or give up”? @jtcchan & @daftkiftyFriday, 20 April, 12
  • 60. 2 Moderating the Test Good Questions for Moderating “What are you currently thinking?” “What would are you looking for? “Is that what you were expecting?” “What do you think would happen next?” @jtcchan & @daftkiftyFriday, 20 April, 12
  • 61. 3 Finishing Up Your Test Once You’re Done With Tasks @jtcchan & @daftkiftyFriday, 20 April, 12
  • 62. 3 Finishing Up Your Test Once You’re Done With Tasks • Answer any questions they may have @jtcchan & @daftkiftyFriday, 20 April, 12
  • 63. 3 Finishing Up Your Test Once You’re Done With Tasks • Answer any questions they may have • Start the post-test survey @jtcchan & @daftkiftyFriday, 20 April, 12
  • 64. 3 Finishing Up Your Test Once You’re Done With Tasks • Answer any questions they may have • Start the post-test survey • Thank them for their help and give them their incentive @jtcchan & @daftkiftyFriday, 20 April, 12
  • 65. 3 Finishing Up Your Test Once You’re Done With Tasks • Answer any questions they may have • Start the post-test survey • Thank them for their help and give them their incentive • Record any key takeaways immediately @jtcchan & @daftkiftyFriday, 20 April, 12
  • 66. 3 Finishing Up Your Test Once You’re Done With Tasks • Answer any questions they may have • Start the post-test survey • Thank them for their help and give them their incentive • Record any key takeaways immediately • Prepare for the next test @jtcchan & @daftkiftyFriday, 20 April, 12
  • 67. How to Run Your Own Test Planning Facilitating Analyzing Before During After @jtcchan & @daftkiftyFriday, 20 April, 12
  • 68. Analyzing Test Results @jtcchan & @daftkiftyFriday, 20 April, 12
  • 69. Analyzing Test Results Debrief With Your Team Immediately @jtcchan & @daftkiftyFriday, 20 April, 12
  • 70. Analyzing Test Results Debrief With Your Team Immediately • Discuss key takeaways and note problems @jtcchan & @daftkiftyFriday, 20 April, 12
  • 71. Analyzing Test Results Debrief With Your Team Immediately • Discuss key takeaways and note problems • Determine what is easy “easy to fix” @jtcchan & @daftkiftyFriday, 20 April, 12
  • 72. Analyzing Test Results Debrief With Your Team Immediately • Discuss key takeaways and note problems • Determine what is easy “easy to fix” • Schedule the changes in for the next iteration @jtcchan & @daftkiftyFriday, 20 April, 12
  • 73. Analyzing Test Results Debrief With Your Team Immediately • Discuss key takeaways and note problems • Determine what is easy “easy to fix” • Schedule the changes in for the next iteration • Prepare for the next test @jtcchan & @daftkiftyFriday, 20 April, 12
  • 74. Analyzing Test Results Ignore “Kayak” Problems @jtcchan & @daftkiftyFriday, 20 April, 12
  • 75. Analyzing Test Results Ignore “Kayak” Problems Unless you have none, look for “low hanging fruits”. If you find your participants are getting lost but end up finding the correct answer with the “wrong path”, the problem is probably safe to ignore. @jtcchan & @daftkiftyFriday, 20 April, 12
  • 76. How We Do Our Testing @jtcchan & @daftkiftyFriday, 20 April, 12
  • 77. “5-Second Test” @jtcchan & @daftkiftyFriday, 20 April, 12
  • 78. “5-Second Test” Goals Helps us understand if our users “get it” (e.g. if this page is trying to help them find and research stuff, can they figure this out on their own?). @jtcchan & @daftkiftyFriday, 20 April, 12
  • 79. “5-Second Test” Requirements A sketch, wireframe, mock-up or complete design you can show people. No interactivity is required. @jtcchan & @daftkiftyFriday, 20 April, 12
  • 80. “5-Second Test” Method Done in person. Approach strangers that “seem idle” and ask them if they have 2 minutes? No incentives required. If they’re on Facebook, they usually have to say yes! @jtcchan & @daftkiftyFriday, 20 April, 12
  • 81. “5-Second Test” Procedure Let them know you’re designing a site and you need their opinion. Show them the design for 5 seconds and ask them: 1. What they saw 2. What they they think the site is trying to do 3. If they were to do something on the site, what would they click? @jtcchan & @daftkiftyFriday, 20 April, 12
  • 82. Task Analysis @jtcchan & @daftkiftyFriday, 20 April, 12
  • 83. Task Analysis Goals Finds out if our visitors can use our site and elicit key design problems. @jtcchan & @daftkiftyFriday, 20 April, 12
  • 84. Task Analysis Requirements A prototype or complete design you can have people play with. You’ll need to go through the standard preparations (create tasks, find locations and a small budget for incentives). @jtcchan & @daftkiftyFriday, 20 April, 12
  • 85. Task Analysis Setup A table, 2 computers and 3 team members (one for facilitating, one for observing and one for dealing with sign- ups). And a BIG sign that says “FREE CHOCOLATE BARS” @jtcchan & @daftkiftyFriday, 20 April, 12
  • 86. Task Analysis Record And Review Our Findings We enter all our data into a spreadsheet to look for trends, key findings, and patterns. @jtcchan & @daftkiftyFriday, 20 April, 12
  • 87. Thank you! Questions? Email us at john.chan@ubc.ca / jonathan.kift@ubc.ca or on Twitter @jtcchan / @daftkiftyFriday, 20 April, 12