Remote Usability Studies Get Great Results While Saving Time and Money Jason Holmes Aaron Rosenberg American Greetings Int...
So…you need to do a user study… by tomorrow <ul><li>Your boss comes by and says, “We need to test this on users.  I need a...
What is Remote Usability? <ul><li>Research where the facilitator and participant are separated by time and/or space </li><...
Unmoderated tools <ul><li>Optimal Workshop </li></ul><ul><ul><li>OptimalSort </li></ul></ul><ul><ul><li>Treejack </li></ul...
OptimalSort - Open
OptimalSort - Closed
Treejack – Optimal Workshop
Where would you find a baby book? Where would you find a baby book? Treejack – Results
Chalkmark – Optimal Workshop
Loop11.com
What do you get with remote testing? <ul><li>More contextual, more ethnographic  </li></ul><ul><li>Stakeholder participati...
What do you give up with remote testing? <ul><li>Non-verbal cues </li></ul><ul><ul><li>Facial expressions </li></ul></ul><...
Are the results as good? <ul><li>According to Bolt|Peters, yes! </li></ul><ul><li>In a comparative study between in-person...
Getting Remote Participants <ul><li>Tap existing customer base </li></ul><ul><ul><li>Limited to current customers…not pros...
 
 
 
UserVue <ul><li>Setting up study </li></ul><ul><li>Running study </li></ul><ul><ul><li>Install plug-in </li></ul></ul><ul>...
 
 
 
 
 
 
Analyzing Results
Remote Moderating Tips <ul><li>Explain that you can see everything on their screen </li></ul><ul><ul><li>“ If you have any...
Not everything is great <ul><li>Participant commitment not as great </li></ul><ul><ul><li>Prepare for “no-shows” </li></ul...
Upcoming SlideShare
Loading in...5
×

NEOUPA Remote Usability Studies

881

Published on

Published in: Technology, Design
0 Comments
3 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
881
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
13
Comments
0
Likes
3
Embeds 0
No embeds

No notes for slide

NEOUPA Remote Usability Studies

  1. 1. Remote Usability Studies Get Great Results While Saving Time and Money Jason Holmes Aaron Rosenberg American Greetings Interactive
  2. 2. So…you need to do a user study… by tomorrow <ul><li>Your boss comes by and says, “We need to test this on users. I need answers by the end of the week.” </li></ul><ul><li>Options </li></ul><ul><ul><li>Call a marketing/UX consulting firm </li></ul></ul><ul><ul><li>Do it yourself at your desk </li></ul></ul><ul><ul><li>Do it yourself…remotely! </li></ul></ul>
  3. 3. What is Remote Usability? <ul><li>Research where the facilitator and participant are separated by time and/or space </li></ul><ul><li>Can involve unmoderated or moderated methods </li></ul><ul><li>Usually mediated by technology </li></ul>
  4. 4. Unmoderated tools <ul><li>Optimal Workshop </li></ul><ul><ul><li>OptimalSort </li></ul></ul><ul><ul><li>Treejack </li></ul></ul><ul><ul><li>Chalkmark </li></ul></ul><ul><li>Loop11.com </li></ul><ul><ul><li>Task-based testing </li></ul></ul>
  5. 5. OptimalSort - Open
  6. 6. OptimalSort - Closed
  7. 7. Treejack – Optimal Workshop
  8. 8. Where would you find a baby book? Where would you find a baby book? Treejack – Results
  9. 9. Chalkmark – Optimal Workshop
  10. 10. Loop11.com
  11. 11. What do you get with remote testing? <ul><li>More contextual, more ethnographic </li></ul><ul><li>Stakeholder participation </li></ul><ul><li>Remote logging </li></ul><ul><li>Instant messaging between </li></ul><ul><ul><li>Facilitator <> Observers </li></ul></ul><ul><ul><li>Facilitator <> Participant </li></ul></ul><ul><li>Import into Morae </li></ul><ul><ul><li>Compile in-person Morae studies with UserVue studies </li></ul></ul>
  12. 12. What do you give up with remote testing? <ul><li>Non-verbal cues </li></ul><ul><ul><li>Facial expressions </li></ul></ul><ul><ul><li>Head movement </li></ul></ul><ul><li>Moments of confusion and hesitation are more difficult to detect </li></ul><ul><li>Less control </li></ul><ul><li>More distractions </li></ul>
  13. 13. Are the results as good? <ul><li>According to Bolt|Peters, yes! </li></ul><ul><li>In a comparative study between in-person and remote methods, they found remote was better </li></ul><ul><ul><li>Remote yielded more key findings </li></ul></ul><ul><ul><li>Shorter sessions </li></ul></ul><ul><ul><li>1 day vs. 12 days </li></ul></ul><ul><ul><li>$17,000 vs. $26,000 </li></ul></ul>
  14. 14. Getting Remote Participants <ul><li>Tap existing customer base </li></ul><ul><ul><li>Limited to current customers…not prospective customers </li></ul></ul><ul><li>Ethnio (Bolt|Peters) </li></ul><ul><ul><li>Quick screener </li></ul></ul><ul><ul><li>DHTML popup using JavaScript </li></ul></ul><ul><ul><li>Geographically neutral </li></ul></ul><ul><li>By combining Ethnio and Uservue, you can live recruit and record what real users actually do! </li></ul>
  15. 18. UserVue <ul><li>Setting up study </li></ul><ul><li>Running study </li></ul><ul><ul><li>Install plug-in </li></ul></ul><ul><ul><li>Initiate call </li></ul></ul><ul><ul><li>Initiate screen connection </li></ul></ul><ul><ul><li>Logging </li></ul></ul><ul><li>Analyzing results with Morae </li></ul>
  16. 25. Analyzing Results
  17. 26. Remote Moderating Tips <ul><li>Explain that you can see everything on their screen </li></ul><ul><ul><li>“ If you have anything of a personal or sensitive nature on your screen, please close it now.” </li></ul></ul><ul><li>Use their name often </li></ul><ul><li>Explain the technology </li></ul><ul><ul><li>“ There’s a slight delay between when you do something and when I see it, so I might ask you what you just did.” </li></ul></ul><ul><li>Blame technology </li></ul><ul><ul><li>“ You just cut out.” </li></ul></ul>
  18. 27. Not everything is great <ul><li>Participant commitment not as great </li></ul><ul><ul><li>Prepare for “no-shows” </li></ul></ul><ul><li>Unforeseen logistical problems </li></ul><ul><ul><li>Give you their cell number but don’t want to use it for an hour </li></ul></ul><ul><ul><li>Computer is in basement, but their phone signal won’t reach </li></ul></ul><ul><li>“Can you hold on a second…” </li></ul>
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×