Remote Usability Studies Get Great Results While Saving Time and Money Jason Holmes Aaron Rosenberg American Greetings Interactive

So…you need to do a user study… by tomorrow Your boss comes by and says, “We need to test this on users. I need answers by the end of the week.” Options Call a marketing/UX consulting firm Do it yourself at your desk Do it yourself…remotely!

Your boss comes by and says, “We need to test this on users. I need answers by the end of the week.”

Options

Call a marketing/UX consulting firm

Do it yourself at your desk

Do it yourself…remotely!

What is Remote Usability? Research where the facilitator and participant are separated by time and/or space Can involve unmoderated or moderated methods Usually mediated by technology

Research where the facilitator and participant are separated by time and/or space

Can involve unmoderated or moderated methods

Usually mediated by technology

Unmoderated tools Optimal Workshop OptimalSort Treejack Chalkmark Loop11.com Task-based testing

Optimal Workshop

OptimalSort

Treejack

Chalkmark

Loop11.com

Task-based testing

OptimalSort - Open

OptimalSort - Closed

Treejack – Optimal Workshop

Where would you find a baby book? Where would you find a baby book? Treejack – Results

Chalkmark – Optimal Workshop

Loop11.com

What do you get with remote testing? More contextual, more ethnographic Stakeholder participation Remote logging Instant messaging between Facilitator <> Observers Facilitator <> Participant Import into Morae Compile in-person Morae studies with UserVue studies

More contextual, more ethnographic

Stakeholder participation

Remote logging

Instant messaging between

Facilitator <> Observers

Facilitator <> Participant

Import into Morae

Compile in-person Morae studies with UserVue studies

What do you give up with remote testing? Non-verbal cues Facial expressions Head movement Moments of confusion and hesitation are more difficult to detect Less control More distractions

Non-verbal cues

Facial expressions

Head movement

Moments of confusion and hesitation are more difficult to detect

Less control

More distractions

Are the results as good? According to Bolt|Peters, yes! In a comparative study between in-person and remote methods, they found remote was better Remote yielded more key findings Shorter sessions 1 day vs. 12 days $17,000 vs. $26,000

According to Bolt|Peters, yes!

In a comparative study between in-person and remote methods, they found remote was better

Remote yielded more key findings

Shorter sessions

1 day vs. 12 days

$17,000 vs. $26,000

Getting Remote Participants Tap existing customer base Limited to current customers…not prospective customers Ethnio (Bolt|Peters) Quick screener DHTML popup using JavaScript Geographically neutral By combining Ethnio and Uservue, you can live recruit and record what real users actually do!

Tap existing customer base

Limited to current customers…not prospective customers

Ethnio (Bolt|Peters)

Quick screener

DHTML popup using JavaScript

Geographically neutral

By combining Ethnio and Uservue, you can live recruit and record what real users actually do!

UserVue Setting up study Running study Install plug-in Initiate call Initiate screen connection Logging Analyzing results with Morae

Setting up study

Running study

Install plug-in

Initiate call

Initiate screen connection

Logging

Analyzing results with Morae

Analyzing Results

Remote Moderating Tips Explain that you can see everything on their screen “ If you have anything of a personal or sensitive nature on your screen, please close it now.” Use their name often Explain the technology “ There’s a slight delay between when you do something and when I see it, so I might ask you what you just did.” Blame technology “ You just cut out.”

Explain that you can see everything on their screen

“ If you have anything of a personal or sensitive nature on your screen, please close it now.”

Use their name often

Explain the technology

“ There’s a slight delay between when you do something and when I see it, so I might ask you what you just did.”

Blame technology

“ You just cut out.”

Not everything is great Participant commitment not as great Prepare for “no-shows” Unforeseen logistical problems Give you their cell number but don’t want to use it for an hour Computer is in basement, but their phone signal won’t reach “Can you hold on a second…”

Participant commitment not as great

Prepare for “no-shows”

Unforeseen logistical problems

Give you their cell number but don’t want to use it for an hour

Computer is in basement, but their phone signal won’t reach

“Can you hold on a second…”