Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

NEOUPA Remote Usability Studies


Published on

Published in: Technology, Design
  • Be the first to comment

NEOUPA Remote Usability Studies

  1. 1. Remote Usability Studies Get Great Results While Saving Time and Money Jason Holmes Aaron Rosenberg American Greetings Interactive
  2. 2. So…you need to do a user study… by tomorrow <ul><li>Your boss comes by and says, “We need to test this on users. I need answers by the end of the week.” </li></ul><ul><li>Options </li></ul><ul><ul><li>Call a marketing/UX consulting firm </li></ul></ul><ul><ul><li>Do it yourself at your desk </li></ul></ul><ul><ul><li>Do it yourself…remotely! </li></ul></ul>
  3. 3. What is Remote Usability? <ul><li>Research where the facilitator and participant are separated by time and/or space </li></ul><ul><li>Can involve unmoderated or moderated methods </li></ul><ul><li>Usually mediated by technology </li></ul>
  4. 4. Unmoderated tools <ul><li>Optimal Workshop </li></ul><ul><ul><li>OptimalSort </li></ul></ul><ul><ul><li>Treejack </li></ul></ul><ul><ul><li>Chalkmark </li></ul></ul><ul><li> </li></ul><ul><ul><li>Task-based testing </li></ul></ul>
  5. 5. OptimalSort - Open
  6. 6. OptimalSort - Closed
  7. 7. Treejack – Optimal Workshop
  8. 8. Where would you find a baby book? Where would you find a baby book? Treejack – Results
  9. 9. Chalkmark – Optimal Workshop
  10. 10.
  11. 11. What do you get with remote testing? <ul><li>More contextual, more ethnographic </li></ul><ul><li>Stakeholder participation </li></ul><ul><li>Remote logging </li></ul><ul><li>Instant messaging between </li></ul><ul><ul><li>Facilitator <> Observers </li></ul></ul><ul><ul><li>Facilitator <> Participant </li></ul></ul><ul><li>Import into Morae </li></ul><ul><ul><li>Compile in-person Morae studies with UserVue studies </li></ul></ul>
  12. 12. What do you give up with remote testing? <ul><li>Non-verbal cues </li></ul><ul><ul><li>Facial expressions </li></ul></ul><ul><ul><li>Head movement </li></ul></ul><ul><li>Moments of confusion and hesitation are more difficult to detect </li></ul><ul><li>Less control </li></ul><ul><li>More distractions </li></ul>
  13. 13. Are the results as good? <ul><li>According to Bolt|Peters, yes! </li></ul><ul><li>In a comparative study between in-person and remote methods, they found remote was better </li></ul><ul><ul><li>Remote yielded more key findings </li></ul></ul><ul><ul><li>Shorter sessions </li></ul></ul><ul><ul><li>1 day vs. 12 days </li></ul></ul><ul><ul><li>$17,000 vs. $26,000 </li></ul></ul>
  14. 14. Getting Remote Participants <ul><li>Tap existing customer base </li></ul><ul><ul><li>Limited to current customers…not prospective customers </li></ul></ul><ul><li>Ethnio (Bolt|Peters) </li></ul><ul><ul><li>Quick screener </li></ul></ul><ul><ul><li>DHTML popup using JavaScript </li></ul></ul><ul><ul><li>Geographically neutral </li></ul></ul><ul><li>By combining Ethnio and Uservue, you can live recruit and record what real users actually do! </li></ul>
  15. 18. UserVue <ul><li>Setting up study </li></ul><ul><li>Running study </li></ul><ul><ul><li>Install plug-in </li></ul></ul><ul><ul><li>Initiate call </li></ul></ul><ul><ul><li>Initiate screen connection </li></ul></ul><ul><ul><li>Logging </li></ul></ul><ul><li>Analyzing results with Morae </li></ul>
  16. 25. Analyzing Results
  17. 26. Remote Moderating Tips <ul><li>Explain that you can see everything on their screen </li></ul><ul><ul><li>“ If you have anything of a personal or sensitive nature on your screen, please close it now.” </li></ul></ul><ul><li>Use their name often </li></ul><ul><li>Explain the technology </li></ul><ul><ul><li>“ There’s a slight delay between when you do something and when I see it, so I might ask you what you just did.” </li></ul></ul><ul><li>Blame technology </li></ul><ul><ul><li>“ You just cut out.” </li></ul></ul>
  18. 27. Not everything is great <ul><li>Participant commitment not as great </li></ul><ul><ul><li>Prepare for “no-shows” </li></ul></ul><ul><li>Unforeseen logistical problems </li></ul><ul><ul><li>Give you their cell number but don’t want to use it for an hour </li></ul></ul><ul><ul><li>Computer is in basement, but their phone signal won’t reach </li></ul></ul><ul><li>“Can you hold on a second…” </li></ul>