ESCAPE THE LAB
                            LET’S GET STARTED MMKAY?




Thursday, October 1, 2009
Thursday, October 1, 2009
I’m Nate BOLT




Thursday, October 1, 2009
10 YEARS
                234 STUDIES
                2,615 PARTICIPANTS (MODERATED)
                89 ORGANIZATIONS
     ...
#ETLAB
Thursday, October 1, 2009
remote
                  USABILITY
Thursday, October 1, 2009
oh
Thursday, October 1, 2009
and
Thursday, October 1, 2009
also
Thursday, October 1, 2009
NO
Thursday, October 1, 2009
time
Thursday, October 1, 2009
travel
Thursday, October 1, 2009
OR
Thursday, October 1, 2009
lasers
Thursday, October 1, 2009
HOW TO BE
                            graceful
                             When All Your
                            TECH...
PLEASE DON’T TRIP
                            ON THE CABLES


Thursday, October 1, 2009
SO ANYWAY WHAT’s UP WITH YOU?


               Have you conducted remote testing before?
               Most important thi...
IT’S TIME TO BREAK OUT YOUR


                     LAPTOP
                     AND GET ON THE WIFI


Thursday, October 1, ...
WIFI: CED CLASSROOM
                    (NO PASSWORD)



Thursday, October 1, 2009
GO TO escapethelab.coM




Thursday, October 1, 2009
Remote Research

                                                       In person
                                        ...
a demonstration




Thursday, October 1, 2009
Thursday, October 1, 2009
Thursday, October 1, 2009
What we’ll cover today

                            When to go remote
                            Study Design
           ...
When does remote win?




Thursday, October 1, 2009
Native Environment
                            Cheap ethnography, not cheap lab.




Thursday, October 1, 2009
Real Time
                            Allows self-motivated tasks.




Thursday, October 1, 2009
Involved Observers
                            Easier to collaborate in real time.




Thursday, October 1, 2009
Geography




Thursday, October 1, 2009
When does remote fail?




Thursday, October 1, 2009
Bandwidth
                            Yes, it’s an issue.




Thursday, October 1, 2009
Translation
                            It’s expensive and difficult to do live.




Thursday, October 1, 2009
Minors
                                 Consent is a bear.
                            Especially if they’re under 13.



...
Faces
                            Most users don’t have webcams.
                             Webcams are a little invasiv...
making the most




Thursday, October 1, 2009
TIME-AWARE Tasks
                       Told to find purple pumps.   Has to wear this for sister’s wedding this
           ...
Real Time OBservers




Thursday, October 1, 2009
Change on the fly




                     “No one seems engaged in the laptop customizer”
  “Let’s add questions about wh...
EXERCISE
                            JUMP IN THE POOL


                               Groups of 5
                       ...
MODERATOR Instructions

                   Your partner has the same puzzle
                   Goal: find out what’s hardes...
A PrizE


                   First person to finish the puzzle wins a
                   hand-imported organic Rococo choco...
Thursday, October 1, 2009
Surprises?




Thursday, October 1, 2009
Designing Remote Studies




Thursday, October 1, 2009
Thursday, October 1, 2009
Thursday, October 1, 2009
AUTOMATED Tools




Thursday, October 1, 2009
plan THIS




Thursday, October 1, 2009
plan THIS


                            Method
                            Recruiting
                            Logistic...
MODERATED must-haves




Thursday, October 1, 2009
MODERATED must-haves

                            Consent




Thursday, October 1, 2009
MODERATED must-haves

                            Consent
                            Tech set-up




Thursday, October 1,...
MODERATED must-haves

                            Consent
                            Tech set-up
                        ...
MODERATED must-haves

                            Consent
                            Tech set-up
                        ...
MODERATED must-haves

                            Consent
                            Tech set-up
                        ...
Re-script for remote




Thursday, October 1, 2009
Re-script for remote



           “Ok, I’d like you to start from the homepage,
            pretend you are shopping for ...
Recruiting




Thursday, October 1, 2009
Recruiting


                            Same: Good talkers
                            Same: Demographic fits




Thursday...
Recruiting


                            Same: Good talkers
                            Same: Demographic fits
            ...
LogisticS: What & How


                            Provide test
                            target
                      ...
Logistics: Who & When


                            How to connect with the user
                            (we’ll discus...
Paying Incentives

                       Collect minimal
                       personal info
                       Deli...
Advanced planning:
                              hybrid studies


                   Numbers actually do lie sometimes
   ...
Portable Research




Thursday, October 1, 2009
BREAK




Thursday, October 1, 2009
Recruiting for Remote




Thursday, October 1, 2009
We all need people


                                Relevant tasks
                                Compatible tech
      ...
Gold Standard
                             Live recruiting
                             Get as close as you can




Thursd...
Recruiting Options

                            Ethnio
                            Links or house ads to online forms
    ...
use Ethnio when

                            Website access
                            Some budget
                      ...
use forms when

                            You can place a link but not code
                            Low budget
     ...
Google docs




Thursday, October 1, 2009
use Agencies when


                            No website access
                            People whine about first-time...
Customer lists


                             Sub-segments
                             Ownership requirements
           ...
PANELS
Thursday, October 1, 2009
Craigslist


                   When other methods don’t work
                   No dough
                   Takes a while...
the Gold Standard again



                            Live recruiting
                            Get as close as you can...
Live Recruiting Math


                             Target: 6 recruits/hour
                             Allow for: 1% res...
Thursday, October 1, 2009
Thursday, October 1, 2009
Screener constraints


                            Keep it short
                            Different from surveys
       ...
Remember the Point


                            Tasks that align with research goals
                            Technica...
EXERCISE

                            Create a live recruiting screener
                               Ethnio or Wufoo
   ...
Thursday, October 1, 2009
Your best questions




Thursday, October 1, 2009
SwitchING TO ethnio...




Thursday, October 1, 2009
Remote Research
                                Toolkit




Thursday, October 1, 2009
5 easy pieces

                            Talk to the user
                            See what they’re doing
           ...
Communication

                            phone               Skype / VOIP

                            Reliable         ...
Screen-sharing

                            Standalone       Integrated
                             Adobe Connect    User...
Adobe Connect




Thursday, October 1, 2009
Livelook




Thursday, October 1, 2009
Webex




Thursday, October 1, 2009
Uservue




Thursday, October 1, 2009
GoToMeeting




Thursday, October 1, 2009
Taking Notes



                            Second monitor if possible
                            Excel Hack (for time co...
Observer Involvement


                            IM
                            Web Chat Rooms
                         ...
Recording Tools

                            Hardware       Software
                             Phone Patch   Camtasia (...
LUNCH.
                            ASK THE INTERNET
                            OR ANYONE BESIDES

                       ...
djeat - A stupid way of saying "Did Ya
               Eat?" or a nickname for Derek Jeter.




Thursday, October 1, 2009
let’s get even more
                                 technical




Thursday, October 1, 2009
Phone patch set up




Thursday, October 1, 2009
Simple Set-up




Thursday, October 1, 2009
Intermediate Set-up




Thursday, October 1, 2009
Advanced Set-up




Thursday, October 1, 2009
AGAIN THE Tools




Thursday, October 1, 2009
exercise



                            Roll your own
                            What will be your base set-up?
         ...
Your version?




Thursday, October 1, 2009
Moderating for Remote




Thursday, October 1, 2009
Is it really any
                               different?
                             (Short answer: some.)




Thursday...
Chances are they’ve
                 never done this before




Thursday, October 1, 2009
What they worry about




Thursday, October 1, 2009
Who are you and can I trust you?




Thursday, October 1, 2009
If you can see my screen,
                               can you see my face?




Thursday, October 1, 2009
Will you be able to take
                            control of my computer?




Thursday, October 1, 2009
Will you help me get the plug-in
                             off my computer at the end?




Thursday, October 1, 2009
Will this work if I have a
                       firewall/proxy/strict IT department?




Thursday, October 1, 2009
Is this going to end up on
                                     YouTube?




Thursday, October 1, 2009
Will I really get the incentive
                                     if I do this?




Thursday, October 1, 2009
How long will it take?




Thursday, October 1, 2009
Is it OK if I have to take a quick call
                                    in the middle?




Thursday, October 1, 2009
Is it OK if my kids/dog/homies are
                             making noise in the background?




Thursday, October 1, 2...
OXYMORON?
                            Remote empathy




Thursday, October 1, 2009
Phone Interviewing
                                Specifics




Thursday, October 1, 2009
Things sound leading




                              Pitfalls




Thursday, October 1, 2009
Body language is huge
                            Actual language is important too
                             Make sure ...
BREAK




Thursday, October 1, 2009
EXERCISE




Thursday, October 1, 2009
Thursday, October 1, 2009
How did it go?




Thursday, October 1, 2009
Advanced Techniques
                             High Emotions




Thursday, October 1, 2009
Advanced Techniques
                            One to many moderation




                            Multi-threading
Thu...
Future Frontiers




Thursday, October 1, 2009
Observer input
                            One more stream on your screen
                                   (Possibly 2 o...
So, what could go
                                 wrong?




Thursday, October 1, 2009
3 points of failure

                                  Technology
                                     yours
             ...
Control what you can
                        control

                            Which is, your set up.
                 ...
Prepare for what
                            you can’t control
                              User’s dog trips over power
 ...
Mid-Session
                     Tech
                   Problems




Thursday, October 1, 2009
Live Recruiting
                               Problems
                      What if they gave a study and nobody
       ...
Increase exposure


                             When you’re not getting
                             enough views
       ...
Increase incentives


                            When it’s cheaper
                            When you’re almost getting...
Switch methods
                 When base traffic level isn’t high enough




Thursday, October 1, 2009
Script Problems
                                Cut the BS.




Thursday, October 1, 2009
The bottom line
                             If they’re not engaged,
                              something’s wrong.




...
Burn the script
                            Find out what matters.




Thursday, October 1, 2009
SPONSORS
      Rosenfeldmedia.com - 20% off everything at with
      discount code: TRIUPA

      Loop11.com - use the refe...
quick feedback!
                   GO TO escapethelab.coM



Thursday, October 1, 2009
QUESTIONS




Thursday, October 1, 2009
@boltpeters
                                 @BOLTRON
                            NATE@BOLTPETERS.COM
                    ...
kthxbai.




Thursday, October 1, 2009
Upcoming SlideShare
Loading in …5
×

Escape The Lab Tri Upa Slides

4,038 views
3,184 views

Published on

Those slides...

Published in: Technology

Escape The Lab Tri Upa Slides

  1. ESCAPE THE LAB LET’S GET STARTED MMKAY? Thursday, October 1, 2009
  2. Thursday, October 1, 2009
  3. I’m Nate BOLT Thursday, October 1, 2009
  4. 10 YEARS 234 STUDIES 2,615 PARTICIPANTS (MODERATED) 89 ORGANIZATIONS 19,120 HOURS Thursday, October 1, 2009
  5. #ETLAB Thursday, October 1, 2009
  6. remote USABILITY Thursday, October 1, 2009
  7. oh Thursday, October 1, 2009
  8. and Thursday, October 1, 2009
  9. also Thursday, October 1, 2009
  10. NO Thursday, October 1, 2009
  11. time Thursday, October 1, 2009
  12. travel Thursday, October 1, 2009
  13. OR Thursday, October 1, 2009
  14. lasers Thursday, October 1, 2009
  15. HOW TO BE graceful When All Your TECHNOLOGY BREAKS Thursday, October 1, 2009
  16. PLEASE DON’T TRIP ON THE CABLES Thursday, October 1, 2009
  17. SO ANYWAY WHAT’s UP WITH YOU? Have you conducted remote testing before? Most important thing to get out of today? Biggest concern about remote research? Thursday, October 1, 2009
  18. IT’S TIME TO BREAK OUT YOUR LAPTOP AND GET ON THE WIFI Thursday, October 1, 2009
  19. WIFI: CED CLASSROOM (NO PASSWORD) Thursday, October 1, 2009
  20. GO TO escapethelab.coM Thursday, October 1, 2009
  21. Remote Research In person Remote Definition: Research where the participant is not sharing physical space with the researcher. % of qualitative research done remotely* *Source: BP’s best guess Thursday, October 1, 2009
  22. a demonstration Thursday, October 1, 2009
  23. Thursday, October 1, 2009
  24. Thursday, October 1, 2009
  25. What we’ll cover today When to go remote Study Design Recruiting Toolkit Moderating Pitfalls Thursday, October 1, 2009
  26. When does remote win? Thursday, October 1, 2009
  27. Native Environment Cheap ethnography, not cheap lab. Thursday, October 1, 2009
  28. Real Time Allows self-motivated tasks. Thursday, October 1, 2009
  29. Involved Observers Easier to collaborate in real time. Thursday, October 1, 2009
  30. Geography Thursday, October 1, 2009
  31. When does remote fail? Thursday, October 1, 2009
  32. Bandwidth Yes, it’s an issue. Thursday, October 1, 2009
  33. Translation It’s expensive and difficult to do live. Thursday, October 1, 2009
  34. Minors Consent is a bear. Especially if they’re under 13. Thursday, October 1, 2009
  35. Faces Most users don’t have webcams. Webcams are a little invasive. Thursday, October 1, 2009
  36. making the most Thursday, October 1, 2009
  37. TIME-AWARE Tasks Told to find purple pumps. Has to wear this for sister’s wedding this weekend. Doesn’t care. Whole different interview. Thursday, October 1, 2009
  38. Real Time OBservers Thursday, October 1, 2009
  39. Change on the fly “No one seems engaged in the laptop customizer” “Let’s add questions about who they get recommendations from” Thursday, October 1, 2009
  40. EXERCISE JUMP IN THE POOL Groups of 5 Pick one moderator and one participant 10 minutes of moderating Thursday, October 1, 2009
  41. MODERATOR Instructions Your partner has the same puzzle Goal: find out what’s hardest about it Hint: ask them to give you a play by play of what they are doing Follow along with your puzzle (remote puzzle sharing?) NO helping, even if you could. Thursday, October 1, 2009
  42. A PrizE First person to finish the puzzle wins a hand-imported organic Rococo chocolate bar with four flavors to choose, including bacon, if you’re into weird stuff. Thursday, October 1, 2009
  43. Thursday, October 1, 2009
  44. Surprises? Thursday, October 1, 2009
  45. Designing Remote Studies Thursday, October 1, 2009
  46. Thursday, October 1, 2009
  47. Thursday, October 1, 2009
  48. AUTOMATED Tools Thursday, October 1, 2009
  49. plan THIS Thursday, October 1, 2009
  50. plan THIS Method Recruiting Logistics Incentives (optional) Thursday, October 1, 2009
  51. MODERATED must-haves Thursday, October 1, 2009
  52. MODERATED must-haves Consent Thursday, October 1, 2009
  53. MODERATED must-haves Consent Tech set-up Thursday, October 1, 2009
  54. MODERATED must-haves Consent Tech set-up The beef Thursday, October 1, 2009
  55. MODERATED must-haves Consent Tech set-up The beef De-installation Thursday, October 1, 2009
  56. MODERATED must-haves Consent Tech set-up The beef De-installation Thanks & incentive Thursday, October 1, 2009
  57. Re-script for remote Thursday, October 1, 2009
  58. Re-script for remote “Ok, I’d like you to start from the homepage, pretend you are shopping for groceries for a party, and buy some paper plates.” Thursday, October 1, 2009
  59. Recruiting Thursday, October 1, 2009
  60. Recruiting Same: Good talkers Same: Demographic fits Thursday, October 1, 2009
  61. Recruiting Same: Good talkers Same: Demographic fits Different: Relevant current tasks Different: Compatible tech Thursday, October 1, 2009
  62. LogisticS: What & How Provide test target Access Credentials Thursday, October 1, 2009
  63. Logistics: Who & When How to connect with the user (we’ll discuss in Tools) Managing time zones Session length Thursday, October 1, 2009
  64. Paying Incentives Collect minimal personal info Deliver quickly Appropriate for location Thursday, October 1, 2009
  65. Advanced planning: hybrid studies Numbers actually do lie sometimes And they don’t tell you the why 3-5 remote interviews is all you need Thursday, October 1, 2009
  66. Portable Research Thursday, October 1, 2009
  67. BREAK Thursday, October 1, 2009
  68. Recruiting for Remote Thursday, October 1, 2009
  69. We all need people Relevant tasks Compatible tech Good talkers Demographic fits Thursday, October 1, 2009
  70. Gold Standard Live recruiting Get as close as you can Thursday, October 1, 2009
  71. Recruiting Options Ethnio Links or house ads to online forms Recruiting agencies Customer email lists Panels Craigslist/online ads Thursday, October 1, 2009
  72. use Ethnio when Website access Some budget Quotas are relatively simple Thursday, October 1, 2009
  73. use forms when You can place a link but not code Low budget Quotas are relatively complex Options: Google Docs Wufoo Thursday, October 1, 2009
  74. Google docs Thursday, October 1, 2009
  75. use Agencies when No website access People whine about first-time visitors Higher budget Online recruiting failed Thursday, October 1, 2009
  76. Customer lists Sub-segments Ownership requirements No website access Thursday, October 1, 2009
  77. PANELS Thursday, October 1, 2009
  78. Craigslist When other methods don’t work No dough Takes a while Embarrassing to admit Thursday, October 1, 2009
  79. the Gold Standard again Live recruiting Get as close as you can Thursday, October 1, 2009
  80. Live Recruiting Math Target: 6 recruits/hour Allow for: 1% response (low average) You Need: 8-10,000 uniques/day Thursday, October 1, 2009
  81. Thursday, October 1, 2009
  82. Thursday, October 1, 2009
  83. Screener constraints Keep it short Different from surveys Who gets paid? Combining is clever Thursday, October 1, 2009
  84. Remember the Point Tasks that align with research goals Technical compatibility Control demographics Thursday, October 1, 2009
  85. EXERCISE Create a live recruiting screener Ethnio or Wufoo 10 questions max 15 minutes Technical compatibility Control demographics ethnio.com escape / escape Thursday, October 1, 2009
  86. Thursday, October 1, 2009
  87. Your best questions Thursday, October 1, 2009
  88. SwitchING TO ethnio... Thursday, October 1, 2009
  89. Remote Research Toolkit Thursday, October 1, 2009
  90. 5 easy pieces Talk to the user See what they’re doing Take notes (optional) Involve observers (optional) Record audio & video (optional) Thursday, October 1, 2009
  91. Communication phone Skype / VOIP Reliable Cheap or free Can be expensive Harder for users for international Can combine with Conference calls screensharing to needed for stress user’s observers connection The Past The Future Thursday, October 1, 2009
  92. Screen-sharing Standalone Integrated Adobe Connect UserVue LiveLook WebEx GoToMeeting Thursday, October 1, 2009
  93. Adobe Connect Thursday, October 1, 2009
  94. Livelook Thursday, October 1, 2009
  95. Webex Thursday, October 1, 2009
  96. Uservue Thursday, October 1, 2009
  97. GoToMeeting Thursday, October 1, 2009
  98. Taking Notes Second monitor if possible Excel Hack (for time codes) Thursday, October 1, 2009
  99. Observer Involvement IM Web Chat Rooms Integrated Chat Passing Notes Thursday, October 1, 2009
  100. Recording Tools Hardware Software Phone Patch Camtasia (Windows) Cables iShowU HD (Mac) Thursday, October 1, 2009
  101. LUNCH. ASK THE INTERNET OR ANYONE BESIDES ME Thursday, October 1, 2009
  102. djeat - A stupid way of saying "Did Ya Eat?" or a nickname for Derek Jeter. Thursday, October 1, 2009
  103. let’s get even more technical Thursday, October 1, 2009
  104. Phone patch set up Thursday, October 1, 2009
  105. Simple Set-up Thursday, October 1, 2009
  106. Intermediate Set-up Thursday, October 1, 2009
  107. Advanced Set-up Thursday, October 1, 2009
  108. AGAIN THE Tools Thursday, October 1, 2009
  109. exercise Roll your own What will be your base set-up? Write it down Thursday, October 1, 2009
  110. Your version? Thursday, October 1, 2009
  111. Moderating for Remote Thursday, October 1, 2009
  112. Is it really any different? (Short answer: some.) Thursday, October 1, 2009
  113. Chances are they’ve never done this before Thursday, October 1, 2009
  114. What they worry about Thursday, October 1, 2009
  115. Who are you and can I trust you? Thursday, October 1, 2009
  116. If you can see my screen, can you see my face? Thursday, October 1, 2009
  117. Will you be able to take control of my computer? Thursday, October 1, 2009
  118. Will you help me get the plug-in off my computer at the end? Thursday, October 1, 2009
  119. Will this work if I have a firewall/proxy/strict IT department? Thursday, October 1, 2009
  120. Is this going to end up on YouTube? Thursday, October 1, 2009
  121. Will I really get the incentive if I do this? Thursday, October 1, 2009
  122. How long will it take? Thursday, October 1, 2009
  123. Is it OK if I have to take a quick call in the middle? Thursday, October 1, 2009
  124. Is it OK if my kids/dog/homies are making noise in the background? Thursday, October 1, 2009
  125. OXYMORON? Remote empathy Thursday, October 1, 2009
  126. Phone Interviewing Specifics Thursday, October 1, 2009
  127. Things sound leading Pitfalls Thursday, October 1, 2009
  128. Body language is huge Actual language is important too Make sure you’re comfortable Thursday, October 1, 2009
  129. BREAK Thursday, October 1, 2009
  130. EXERCISE Thursday, October 1, 2009
  131. Thursday, October 1, 2009
  132. How did it go? Thursday, October 1, 2009
  133. Advanced Techniques High Emotions Thursday, October 1, 2009
  134. Advanced Techniques One to many moderation Multi-threading Thursday, October 1, 2009
  135. Future Frontiers Thursday, October 1, 2009
  136. Observer input One more stream on your screen (Possibly 2 or 3) Thursday, October 1, 2009
  137. So, what could go wrong? Thursday, October 1, 2009
  138. 3 points of failure Technology yours participant’s observer’s Recruiting Scripting Thursday, October 1, 2009
  139. Control what you can control Which is, your set up. You wouldn’t go backpacking without checking your gear. Do it 24 hours in advance. Or else. Thursday, October 1, 2009
  140. Prepare for what you can’t control User’s dog trips over power cord Internet affected by local weather User can’t install Doesn’t know how to install Thursday, October 1, 2009
  141. Mid-Session Tech Problems Thursday, October 1, 2009
  142. Live Recruiting Problems What if they gave a study and nobody came? Thursday, October 1, 2009
  143. Increase exposure When you’re not getting enough views When the wrong type of users are responding Thursday, October 1, 2009
  144. Increase incentives When it’s cheaper When you’re almost getting enough When segments are tough Thursday, October 1, 2009
  145. Switch methods When base traffic level isn’t high enough Thursday, October 1, 2009
  146. Script Problems Cut the BS. Thursday, October 1, 2009
  147. The bottom line If they’re not engaged, something’s wrong. Thursday, October 1, 2009
  148. Burn the script Find out what matters. Thursday, October 1, 2009
  149. SPONSORS Rosenfeldmedia.com - 20% off everything at with discount code: TRIUPA Loop11.com - use the referral code “Escape the Lab” UserZoom.com - 20% discount on any of the four licensed plan. Discount Code: Bolt&Peter Thursday, October 1, 2009
  150. quick feedback! GO TO escapethelab.coM Thursday, October 1, 2009
  151. QUESTIONS Thursday, October 1, 2009
  152. @boltpeters @BOLTRON NATE@BOLTPETERS.COM #ETLAB Thursday, October 1, 2009
  153. kthxbai. Thursday, October 1, 2009

×