Escape The Lab Tri Upa Slides

ESCAPE THE LAB
LET’S GET STARTED MMKAY?

Thursday, October 1, 2009

I’m Nate BOLT


10 YEARS
234 STUDIES
2,615 PARTICIPANTS (MODERATED)
89 ORGANIZATIONS
19,120 HOURS


#ETLAB

remote
USABILITY

also

time

travel

lasers

HOW TO BE
graceful
When All Your
TECHNOLOGY
BREAKS

PLEASE DON’T TRIP
ON THE CABLES


SO ANYWAY WHAT’s UP WITH YOU?

Have you conducted remote testing before?
Most important thing to get out of today?
Biggest concern about remote research?


IT’S TIME TO BREAK OUT YOUR

LAPTOP
AND GET ON THE WIFI


WIFI: CED CLASSROOM
(NO PASSWORD)


GO TO escapethelab.coM


Remote Research

In person
Remote
Deﬁnition:

Research where the
participant is not
sharing physical space
with the researcher.
% of qualitative research
done remotely*

*Source: BP’s best guess


a demonstration


What we’ll cover today

When to go remote
Study Design
Recruiting
Toolkit
Moderating
Pitfalls


When does remote win?


Native Environment
Cheap ethnography, not cheap lab.


Real Time
Allows self-motivated tasks.


Involved Observers
Easier to collaborate in real time.


Geography


When does remote fail?


Bandwidth
Yes, it’s an issue.


Translation
It’s expensive and diﬃcult to do live.


Minors
Consent is a bear.
Especially if they’re under 13.


Faces
Most users don’t have webcams.
Webcams are a little invasive.


making the most


TIME-AWARE Tasks
Told to ﬁnd purple pumps. Has to wear this for sister’s wedding this
weekend.
Doesn’t care.
Whole diﬀerent interview.


Real Time OBservers


Change on the fly

“No one seems engaged in the laptop customizer”
“Let’s add questions about who they get recommendations from”


EXERCISE
JUMP IN THE POOL

Groups of 5
Pick one moderator
and one participant
10 minutes of
moderating


MODERATOR Instructions

Your partner has the same puzzle
Goal: ﬁnd out what’s hardest about it
Hint: ask them to give you a play by play of what
they are doing
Follow along with your puzzle (remote puzzle
sharing?)

NO helping, even if you could.


A PrizE

First person to finish the puzzle wins a
hand-imported organic Rococo chocolate
bar with four flavors to choose, including
bacon, if you’re into weird stuff.


Surprises?


Designing Remote Studies


AUTOMATED Tools


plan THIS


plan THIS

Method
Recruiting
Logistics
Incentives (optional)


MODERATED must-haves



Consent



Consent
Tech set-up



Consent
Tech set-up
The beef



Consent
Tech set-up
The beef
De-installation



Consent
Tech set-up
The beef
De-installation
Thanks & incentive


Re-script for remote


Re-script for remote

“Ok, I’d like you to start from the homepage,
pretend you are shopping for groceries for
a party, and buy some paper plates.”


Recruiting


Recruiting

Same: Good talkers
Same: Demographic ﬁts


Recruiting

Same: Good talkers
Same: Demographic fits
Different: Relevant current tasks
Different: Compatible tech


LogisticS: What & How

Provide test
target
Access
Credentials


Logistics: Who & When

How to connect with the user
(we’ll discuss in Tools)
Managing time zones
Session length


Paying Incentives

Collect minimal
personal info
Deliver quickly
Appropriate for
location


Advanced planning:
hybrid studies

Numbers actually do lie sometimes
And they don’t tell you the why
3-5 remote interviews is all you need


Portable Research


BREAK


Recruiting for Remote


We all need people

Relevant tasks
Compatible tech
Good talkers
Demographic ﬁts


Gold Standard
Live recruiting
Get as close as you can


Recruiting Options

Ethnio
Links or house ads to online forms
Recruiting agencies
Customer email lists
Panels
Craigslist/online ads


use Ethnio when

Website access
Some budget
Quotas are relatively simple


use forms when

You can place a link but not code
Low budget
Quotas are relatively complex
Options:
Google Docs
Wufoo


Google docs


use Agencies when

No website access
People whine about ﬁrst-time visitors
Higher budget
Online recruiting failed


Customer lists

Sub-segments
Ownership requirements
No website access


PANELS

Craigslist

When other methods don’t work
No dough
Takes a while
Embarrassing to admit


the Gold Standard again

Live recruiting
Get as close as you can


Live Recruiting Math

Target: 6 recruits/hour
Allow for: 1% response (low average)
You Need: 8-10,000 uniques/day


Screener constraints

Keep it short
Diﬀerent from surveys
Who gets paid?
Combining is clever


Remember the Point

Tasks that align with research goals
Technical compatibility
Control demographics


EXERCISE

Create a live recruiting screener
Ethnio or Wufoo
10 questions max
15 minutes
Technical compatibility
Control demographics
ethnio.com escape / escape


Your best questions


SwitchING TO ethnio...


Remote Research
Toolkit


5 easy pieces

Talk to the user
See what they’re doing
Take notes (optional)
Involve observers (optional)
Record audio & video (optional)


Communication

phone Skype / VOIP

Reliable Cheap or free
Can be expensive Harder for users
for international Can combine with
Conference calls screensharing to
needed for stress user’s
observers connection
The Past The Future


Screen-sharing

Standalone Integrated
Adobe Connect UserVue
LiveLook
WebEx
GoToMeeting


Adobe Connect


Livelook


Webex


Uservue


GoToMeeting


Taking Notes

Second monitor if possible
Excel Hack (for time codes)


Observer Involvement

IM
Web Chat Rooms
Integrated Chat
Passing Notes


Recording Tools

Hardware Software
Phone Patch Camtasia (Windows)
Cables iShowU HD (Mac)


LUNCH.
ASK THE INTERNET
OR ANYONE BESIDES

ME

djeat - A stupid way of saying "Did Ya
Eat?" or a nickname for Derek Jeter.


let’s get even more
technical


Phone patch set up


Simple Set-up


Intermediate Set-up


Advanced Set-up


AGAIN THE Tools


exercise

Roll your own
What will be your base set-up?
Write it down


Your version?


Moderating for Remote


Is it really any
different?
(Short answer: some.)


Chances are they’ve
never done this before


What they worry about


Who are you and can I trust you?


If you can see my screen,
can you see my face?


Will you be able to take
control of my computer?


Will you help me get the plug-in
oﬀ my computer at the end?


Will this work if I have a
ﬁrewall/proxy/strict IT department?


Is this going to end up on
YouTube?


Will I really get the incentive
if I do this?


How long will it take?


Is it OK if I have to take a quick call
in the middle?


Is it OK if my kids/dog/homies are
making noise in the background?


OXYMORON?
Remote empathy


Phone Interviewing
Specifics


Things sound leading

Pitfalls


Body language is huge
Actual language is important too
Make sure you’re comfortable


EXERCISE


How did it go?


Advanced Techniques
High Emotions


Advanced Techniques
One to many moderation

Multi-threading

Future Frontiers


Observer input
One more stream on your screen
(Possibly 2 or 3)


So, what could go
wrong?


3 points of failure

Technology
yours
participant’s
observer’s
Recruiting
Scripting


Control what you can
control

Which is, your set up.
You wouldn’t go backpacking
without checking your gear.
Do it 24 hours in advance. Or else.


Prepare for what
you can’t control
User’s dog trips over power
cord
Internet aﬀected by local
weather
User can’t install
Doesn’t know how to install


Mid-Session
Tech
Problems


Live Recruiting
Problems
What if they gave a study and nobody
came?


Increase exposure

When you’re not getting
enough views
When the wrong type of users
are responding


Increase incentives

When it’s cheaper
When you’re almost getting enough
When segments are tough


Switch methods
When base traﬃc level isn’t high enough


Script Problems
Cut the BS.


The bottom line
If they’re not engaged,
something’s wrong.


Burn the script
Find out what matters.


SPONSORS
Rosenfeldmedia.com - 20% oﬀ everything at with
discount code: TRIUPA

Loop11.com - use the referral code “Escape the
Lab”

UserZoom.com - 20% discount on any of the four
licensed plan. Discount Code: Bolt&Peter

quick feedback!
GO TO escapethelab.coM


QUESTIONS


@boltpeters
@BOLTRON
NATE@BOLTPETERS.COM
#ETLAB


kthxbai.


Escape The Lab Tri Upa Slides

Recommended

Recommended

More Related Content

Similar to Escape The Lab Tri Upa Slides

Similar to Escape The Lab Tri Upa Slides (16)

More from bolt peters

More from bolt peters (20)

Recently uploaded

Recently uploaded (20)

Escape The Lab Tri Upa Slides