Crowdsourced Remote Unmoderated Usability Testing

3,254 views

Published on

Slides for my presentation at the IASummit 2012 on crowdsourced remote unmoderated usability testing.

Published in: Technology, Design
0 Comments
3 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
3,254
On SlideShare
0
From Embeds
0
Number of Embeds
1,526
Actions
Shares
0
Downloads
34
Comments
0
Likes
3
Embeds 0
No embeds

No notes for slide

Crowdsourced Remote Unmoderated Usability Testing

  1. 1. CrowdsourcedRemoteUnmoderatedUsability Testing And how to make it work for youInge De Bleecker@ingedebleeckerIASummit 2012
  2. 2. Overviewo  What is ito  How to do ito  Get started today (or Monday)
  3. 3. What is it RemoteUnmoderated Crowdsourced
  4. 4. Crowdsourcing“The act of taking a job traditionallyperformed by a designated agent (usuallyan employee) and outsourcing it to anundefined, generally large group ofpeople in the form of an open call.” — JEFF HOWE
  5. 5. Crowdsourcing“a model for enabling convenient, on-demand access to a (shared) pool ofhuman workers with different skills thatcan be rapidly provisioned and releasedwith minimal management effort” — PANOS IPEIROTIS
  6. 6. Different Flavors includingo  Screenshot click testo  Screenshot timed testo  Task-based usability study with online survey D  I  Y  
  7. 7. Good foro  Use of personal deviceso  In own environmento  Fast turnaroundo  Cheap(er)
  8. 8. Not good foro  Can’t observe participantso  No additional questionso  Subject video recording
  9. 9. ProcessStill the same process1  Recruit participants Need committed participants!2  Write task plan Not easy to write a good task plan3  Run pilot4  Run test5  Analyze results6  Write report
  10. 10. Committed participants Compensa/on   Loyalty   Commitment   Higher   Longer   Higher      comple/on     sessions   quality   rate  
  11. 11. Tips for writing a task plano  Participants can’t get blockedo  You only get one shoto  Guide without influencingo  Make all questions requiredo  Encourage think aloud/write downo  Task and question types depend on tool
  12. 12. Tasks & Questionso  Screenshot click test (@usabilityhub)
  13. 13. Tasks & Questionso  Screenshot timed test (@usabilityhub)
  14. 14. Tasks & Questionso  Online survey testing (Loop11)
  15. 15. Tasks & Questionso  Online survey testing (DIY)
  16. 16. Tasks & Questionso  Online survey testing (DIY)
  17. 17. ResultsSelf-reported data“I find grid views easier to scan because they showenough detail of several phones on one screenwithout forcing me to scroll”“I would select the "Compare this phone" optionbelow the phones I wanted. But then I am at a lossas to what to hit next to actually carry out thecomparison”
  18. 18. Getting startedGot money!o  Engage crowdsourcing companyo  Use specialized toolBroke?o  Make your own crowdo  Use generic survey tool (e.g. Surveymonkey, Google docs)
  19. 19. Q&A Thank you

×