Please
                        stand by




                      ESCAPE THE LAB
                        WILL BEGIN AT 9:0...
ESCAPE THE LAB
                      LET’S GET STARTED MMKAY?




Friday, May 7, 2010
Friday, May 7, 2010
I’m Cyd
                      He’s Nate




Friday, May 7, 2010
#ETLAB
Friday, May 7, 2010
remote
                      USABILITY
Friday, May 7, 2010
oh
Friday, May 7, 2010
and
Friday, May 7, 2010
also
Friday, May 7, 2010
NO
Friday, May 7, 2010
time
Friday, May 7, 2010
travel
Friday, May 7, 2010
OR
Friday, May 7, 2010
lasers
Friday, May 7, 2010
HOW TO BE
                      graceful
                       When All Your
                      TECHNOLOGY
           ...
PLEASE DON’T TRIP
                      ON THE CABLES


Friday, May 7, 2010
Follow the buddy
                           system




Friday, May 7, 2010
How about you all?

                      Best remote research story
                      Most important thing to get out...
Go TO escapethelab.com NOW




Friday, May 7, 2010
Friday, May 7, 2010
Remote Research

                                                        In person
                                       ...
What we’ll cover today

                      Recruiting
                      Moderated
                      Unmoderated...
We shall begin...
              with a little experiment




Friday, May 7, 2010
EXERCISE
                          JUMP IN THE POOL


                      Table groups
                      Pick one mo...
MODERATOR Instructions

                      Your participant has a sheet of paper and a link
                      Goal:...
It’s A RACE



                      First participant to make an origami
                      giraffe is the winner




F...
Friday, May 7, 2010
Surprises?




Friday, May 7, 2010
When does remote win?


                              Page
                              10-15

Friday, May 7, 2010
Native Environment
                      Cheap ethnography, not cheap lab.




Friday, May 7, 2010
Real Time
                      Allows self-motivated tasks.




Friday, May 7, 2010
Involved Observers
                      Easier to collaborate in real time.




Friday, May 7, 2010
Geography




Friday, May 7, 2010
When does remote fail?




Friday, May 7, 2010
Bandwidth
                      Yes, it’s still an issue.




Friday, May 7, 2010
Translation
                      It’s expensive and difficult to do live.




Friday, May 7, 2010
Minors
                           Consent is a bear.
                      Especially if they’re under 13.




Friday, May...
Faces
                      We’ve all got webcams today.
                           Most users don’t.
                    ...
CASE STUDY: Epic FAIL




Friday, May 7, 2010
What went wrong

                      Coins were way more valuable than we
                      expected
               ...
making the most




Friday, May 7, 2010
TIME-AWARE Tasks
                      Told to find purple pumps.   Has to wear this for sister’s wedding this
            ...
Real Time OBservers




                        More
                      on page
                       109-111


Friday...
Change on the fly


                  “No one seems engaged in the   “Let’s add questions about who they
                 ...
Designing Remote Studies




Friday, May 7, 2010
Friday, May 7, 2010
Friday, May 7, 2010
plan these things




Friday, May 7, 2010
plan these things


                          Method
                          Recruiting
                          Logist...
MODERATED must-haves




Friday, May 7, 2010
MODERATED must-haves

                           Consent




Friday, May 7, 2010
MODERATED must-haves

                           Consent
                           Tech set-up




Friday, May 7, 2010
MODERATED must-haves

                           Consent
                           Tech set-up
                          ...
MODERATED must-haves

                           Consent
                           Tech set-up
                          ...
MODERATED must-haves

                           Consent
                           Tech set-up
                          ...
UNMODERATED must-haves




Friday, May 7, 2010
UNMODERATED must-haves


                      Expectation setting




Friday, May 7, 2010
UNMODERATED must-haves


                      Expectation setting
                      Instructions




Friday, May 7, 2...
UNMODERATED must-haves


                      Expectation setting
                      Instructions
                    ...
UNMODERATED must-haves


                      Expectation setting
                      Instructions
                    ...
Recruiting




Friday, May 7, 2010
Recruiting


                      Same: Good “talkers”
                      Same: Demographic fits




Friday, May 7, 2010
Recruiting


                      Same: Good “talkers”
                      Same: Demographic fits
                      ...
Logistics: What & How


                       Provide test
                       target
                       Access
  ...
Logistics: Who & When


                        How to connect with the user
                        (we’ll discuss in Too...
Paying Incentives

                      Collect minimal
                      personal info
                      Deliver...
Advanced planning:
                          hybrid studies


                      Why use only one method?
             ...
Portable Research




Friday, May 7, 2010
BREAK




Friday, May 7, 2010
Recruiting for Remote




Friday, May 7, 2010
We all need people

                         Relevant tasks
                         Compatible tech
                     ...
Gold Standard              Page
                                                  52


                       Live recruit...
Recruiting Options

                      Ethnio
                      Links or house ads to online forms
                ...
use Ethnio when

                      Website access
                      Some budget
                      Quotas are r...
use forms when

                      You can place a link but not code
                      Low budget
                 ...
Google docs




Friday, May 7, 2010
use Agencies when


                        No website access
                        Higher budget
                      ...
Customer lists


                       Sub-segments
                       Ownership requirements
                       ...
PANELS
Friday, May 7, 2010
Craigslist & forums


                      When other methods don’t work
                      No dough
                 ...
the Gold Standard again



                      Live recruiting
                      Get as close as you can




Friday,...
Live Recruiting Math
                         (for interviews)


                       Target: 6 recruits/hour
          ...
Page
                       56




Friday, May 7, 2010
Screener constraints
                                              Page
                                              60-6...
Remember the Point


                      Tasks that align with research goals
                      Technical compatibil...
EXERCISE




Friday, May 7, 2010
Friday, May 7, 2010
Your best questions




Friday, May 7, 2010
GO EAT LUNCH

                             Famished




Friday, May 7, 2010
exercise

                      We recommend:
                      Split Pea Seduction, 6th & Minna
                     ...
exercise

                      We recommend:
                      Split Pea Seduction, 6th & Minna
                     ...
Please
                        stand by




                      ESCAPE THE LAB
                        WILL RESUME AT 1:...
welcome back




Friday, May 7, 2010
Time
                         for a
                       contest!


                      Tweet why you hate remote
    ...
Remote Research
                          Toolkit
                         MODERATED
                                     ...
5 easy pieces

                      Talk to the user
                      See what they’re doing
                      T...
Communication

                      phone                Skype
                       Reliable             Cheap or free
...
Screensharing
                       Standalone        Integrated
                         Adobe Connect    UserVue (RIP)
...
Adobe Connect




Friday, May 7, 2010
FuzeMeeting(New)




Friday, May 7, 2010
Livelook




Friday, May 7, 2010
Webex




Friday, May 7, 2010
GoToMeeting




Friday, May 7, 2010
Taking Notes



                      Second monitor if possible
                      Excel Hack (for time codes)




Fri...
How to rig your notes

                               What the formulas look like

                                       ...
Observer Involvement


                           IM
                           Web Chat Rooms
                           ...
Recording Tools

                      Hardware       Software
                       Phone Patch   Camtasia (Windows)
   ...
Phone patch set up




Friday, May 7, 2010
Friday, May 7, 2010
Simple Set-up




Friday, May 7, 2010
Intermediate Set-up




Friday, May 7, 2010
Advanced Set-up




Friday, May 7, 2010
Friday, May 7, 2010
exercise



                      Roll your own
                      What will be your base set-up?
                     ...
Friday, May 7, 2010
Your version?




Friday, May 7, 2010
Winners!!




Friday, May 7, 2010
Moderating for Remote




Friday, May 7, 2010
Is it really any
                         different?
                       (Short answer: some.)



                     ...
Question from the BP
                        inbox yesterday
            It would have actually helped to know how many us...
Making passionate tasks
                      work




Friday, May 7, 2010
Making passionate tasks
                      work


                      Recruit for relevant current tasks
            ...
Re-script for remote




Friday, May 7, 2010
Re-script for remote



            “Ok, I’d like you to start from the homepage,
             pretend you are shopping fo...
MODERATED must-haves


                                  Also see
                                   37-43




Friday, May...
MODERATED must-haves

                           Consent
                                     Also see
                   ...
MODERATED must-haves

                           Consent
                                         Also see
               ...
MODERATED must-haves

                           Consent
                                         Also see
               ...
MODERATED must-haves

                           Consent
                                         Also see
               ...
MODERATED must-haves

                           Consent
                                         Also see
               ...
Chances are they’ve
                 never done this before




Friday, May 7, 2010
What they worry about




Friday, May 7, 2010
Who are you and can I trust you?




Friday, May 7, 2010
If you can see my screen,
                         can you see my face?




Friday, May 7, 2010
Will you be able to take
                      control of my computer?




Friday, May 7, 2010
Will you help me get the plug-in
                       off my computer at the end?




Friday, May 7, 2010
Will this work if I have a
                      firewall/proxy/strict IT department?




Friday, May 7, 2010
Is this going to end up on
                               YouTube?




Friday, May 7, 2010
Will I really get the incentive
                               if I do this?




Friday, May 7, 2010
How long will it take?




Friday, May 7, 2010
Is it OK if I have to take a quick call
                                  in the middle?




Friday, May 7, 2010
Is it OK if my kids/dog/fratmates are
                         making noise in the background?




Friday, May 7, 2010
OXYMORON?
                      Remote empathy




Friday, May 7, 2010
Phone Interviewing
                          Specifics




Friday, May 7, 2010
Things sound leading




                        Pitfalls




Friday, May 7, 2010
Body language is huge
                         Actual language is important too
                          Make sure you’re...
EXERCISE
                      Practice screensharing
                      Moderator, Participant, at least one
         ...
Friday, May 7, 2010
How did it go?




Friday, May 7, 2010
Advanced Techniques
                            High Emotions




Friday, May 7, 2010
Advanced Techniques
                          One to many moderation




                          Multi-threading
Friday,...
Future Frontiers




Friday, May 7, 2010
BREAK




Friday, May 7, 2010
Remote Research
                          Toolkit
                        UNMODERATED



Friday, May 7, 2010
3 parts


                      Provide test target
                      Record user behavior
                      Captu...
AUTOMATED Tools




Friday, May 7, 2010
What do you want to
                              do?

                       Get freeform data cheaply
                  ...
This stuff is pretty
                                 cheap

                      •   Usabilla           $0-950/yr
      ...
UserTesting.com




Friday, May 7, 2010
UserTesting.com




Friday, May 7, 2010
UserTesting.com




Friday, May 7, 2010
UserTesting.com




Friday, May 7, 2010
Using your own recruits
                      Set up an account
                      Email support@usertesting.com with:
...
Loop11




Friday, May 7, 2010
Loop11




Friday, May 7, 2010
Loop11




Friday, May 7, 2010
Loop11




Friday, May 7, 2010
Loop11




Friday, May 7, 2010
Usabilla




Friday, May 7, 2010
Optimal Workshop




                      Treejack     Chalkmark



Friday, May 7, 2010
EXERCISE


                      Choose one of these tools
                      Sign up for a test account
              ...
Friday, May 7, 2010
RESULTS




Friday, May 7, 2010
Session Videos
                           Heatmaps
                              Notes
                       Completion R...
Telling a story
                      If you have time-coded notes, you can make
                      highlight videos
  ...
Case Study: Data
                         Mismatch



Friday, May 7, 2010
Qualitative Results

            16 remote interviews
            Worst findability BP has ever observed
            Recomm...
Quantitative Data
                                                          EFFECTIVENESS RATIOS                          ...
What happened?




Friday, May 7, 2010
32 page report, distilled
                                     Navigation     Search ease              Hospital
          ...
What’s different IRL?




Friday, May 7, 2010
Recording
                      1 more button to press




Friday, May 7, 2010
Observer input
                      One more stream on your screen
                             (Possibly 2 or 3)




Fri...
Notes
                      Tap, tap, tapping away




Friday, May 7, 2010
So, what could go
                           wrong?




Friday, May 7, 2010
3 points of failure

                            Technology
                               yours
                         ...
Control what you can
                        control
                      Which is, your set up.
                      Yo...
Prepare for what
                      you can’t control
                        User’s dog trips over power
             ...
Mid-Session
                         Tech
                       Problems




Friday, May 7, 2010
Live Recruiting
                              Problems
                      What if they gave a study and nobody
        ...
Increase exposure


                       When you’re not getting
                       enough views
                   ...
Increase incentives


                      When it’s cheaper
                      When you’re almost getting enough
    ...
Switch methods
                  When base traffic level isn’t high enough




Friday, May 7, 2010
Script Problems
                          Cut the BS.




Friday, May 7, 2010
The bottom line
                       If they’re not engaged,
                        something’s wrong.




Friday, May ...
Burn the script
                      Find out what matters.




Friday, May 7, 2010
QUESTIONS




Friday, May 7, 2010
@boltpeters
                      CYD@BOLTPETERS.COM
                      NATE@BOLTPETERS.COM




Friday, May 7, 2010
kthxbai.




Friday, May 7, 2010
Upcoming SlideShare
Loading in...5
×

ETL May 2010

1,610

Published on

The official slides from our day-long workshop, Escape The Lab

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
1,610
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
32
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

ETL May 2010

  1. 1. Please stand by ESCAPE THE LAB WILL BEGIN AT 9:00 Friday, May 7, 2010
  2. 2. ESCAPE THE LAB LET’S GET STARTED MMKAY? Friday, May 7, 2010
  3. 3. Friday, May 7, 2010
  4. 4. I’m Cyd He’s Nate Friday, May 7, 2010
  5. 5. #ETLAB Friday, May 7, 2010
  6. 6. remote USABILITY Friday, May 7, 2010
  7. 7. oh Friday, May 7, 2010
  8. 8. and Friday, May 7, 2010
  9. 9. also Friday, May 7, 2010
  10. 10. NO Friday, May 7, 2010
  11. 11. time Friday, May 7, 2010
  12. 12. travel Friday, May 7, 2010
  13. 13. OR Friday, May 7, 2010
  14. 14. lasers Friday, May 7, 2010
  15. 15. HOW TO BE graceful When All Your TECHNOLOGY BREAKS Friday, May 7, 2010
  16. 16. PLEASE DON’T TRIP ON THE CABLES Friday, May 7, 2010
  17. 17. Follow the buddy system Friday, May 7, 2010
  18. 18. How about you all? Best remote research story Most important thing to get out of today? Biggest fear or skepticism about remote research? Friday, May 7, 2010
  19. 19. Go TO escapethelab.com NOW Friday, May 7, 2010
  20. 20. Friday, May 7, 2010
  21. 21. Remote Research In person Remote Definition: Research where the participant is not sharing physical space with the researcher. % of qualitative research done remotely* *Source: BP’s best guess Friday, May 7, 2010
  22. 22. What we’ll cover today Recruiting Moderated Unmoderated Results Pitfalls Friday, May 7, 2010
  23. 23. We shall begin... with a little experiment Friday, May 7, 2010
  24. 24. EXERCISE JUMP IN THE POOL Table groups Pick one moderator and one participant (they need to be in different locations) 10 minutes of moderating Friday, May 7, 2010
  25. 25. MODERATOR Instructions Your participant has a sheet of paper and a link Goal: find out what’s hardest about presenting physical task instructions online Hint: ask them to give you a play by play of what they are doing Follow along with your own paper NO helping, even if you could. Friday, May 7, 2010
  26. 26. It’s A RACE First participant to make an origami giraffe is the winner Friday, May 7, 2010
  27. 27. Friday, May 7, 2010
  28. 28. Surprises? Friday, May 7, 2010
  29. 29. When does remote win? Page 10-15 Friday, May 7, 2010
  30. 30. Native Environment Cheap ethnography, not cheap lab. Friday, May 7, 2010
  31. 31. Real Time Allows self-motivated tasks. Friday, May 7, 2010
  32. 32. Involved Observers Easier to collaborate in real time. Friday, May 7, 2010
  33. 33. Geography Friday, May 7, 2010
  34. 34. When does remote fail? Friday, May 7, 2010
  35. 35. Bandwidth Yes, it’s still an issue. Friday, May 7, 2010
  36. 36. Translation It’s expensive and difficult to do live. Friday, May 7, 2010
  37. 37. Minors Consent is a bear. Especially if they’re under 13. Friday, May 7, 2010
  38. 38. Faces We’ve all got webcams today. Most users don’t. Webcams are invasive. Friday, May 7, 2010
  39. 39. CASE STUDY: Epic FAIL Friday, May 7, 2010
  40. 40. What went wrong Coins were way more valuable than we expected People mis-assessed their English levels Bandwidth was really a problem Nobody plays Habbo when their parents are home to consent Friday, May 7, 2010
  41. 41. making the most Friday, May 7, 2010
  42. 42. TIME-AWARE Tasks Told to find purple pumps. Has to wear this for sister’s wedding this weekend. Doesn’t care. Whole different interaction. Friday, May 7, 2010
  43. 43. Real Time OBservers More on page 109-111 Friday, May 7, 2010
  44. 44. Change on the fly “No one seems engaged in the “Let’s add questions about who they laptop customizer” get recommendations from” Friday, May 7, 2010
  45. 45. Designing Remote Studies Friday, May 7, 2010
  46. 46. Friday, May 7, 2010
  47. 47. Friday, May 7, 2010
  48. 48. plan these things Friday, May 7, 2010
  49. 49. plan these things Method Recruiting Logistics Friday, May 7, 2010
  50. 50. MODERATED must-haves Friday, May 7, 2010
  51. 51. MODERATED must-haves Consent Friday, May 7, 2010
  52. 52. MODERATED must-haves Consent Tech set-up Friday, May 7, 2010
  53. 53. MODERATED must-haves Consent Tech set-up The beef Friday, May 7, 2010
  54. 54. MODERATED must-haves Consent Tech set-up The beef De-installation Friday, May 7, 2010
  55. 55. MODERATED must-haves Consent Tech set-up The beef De-installation Thanks & incentive Friday, May 7, 2010
  56. 56. UNMODERATED must-haves Friday, May 7, 2010
  57. 57. UNMODERATED must-haves Expectation setting Friday, May 7, 2010
  58. 58. UNMODERATED must-haves Expectation setting Instructions Friday, May 7, 2010
  59. 59. UNMODERATED must-haves Expectation setting Instructions The beef Friday, May 7, 2010
  60. 60. UNMODERATED must-haves Expectation setting Instructions The beef Thanks & incentive Friday, May 7, 2010
  61. 61. Recruiting Friday, May 7, 2010
  62. 62. Recruiting Same: Good “talkers” Same: Demographic fits Friday, May 7, 2010
  63. 63. Recruiting Same: Good “talkers” Same: Demographic fits Different: Relevant current tasks Different: Compatible tech Friday, May 7, 2010
  64. 64. Logistics: What & How Provide test target Access Credentials Friday, May 7, 2010
  65. 65. Logistics: Who & When How to connect with the user (we’ll discuss in Tools) Managing time zones Session length Friday, May 7, 2010
  66. 66. Paying Incentives Collect minimal personal info Deliver quickly Appropriate for location Friday, May 7, 2010
  67. 67. Advanced planning: hybrid studies Why use only one method? Some methods don’t tell you the why 3-5 remote interviews is all you need Friday, May 7, 2010
  68. 68. Portable Research Friday, May 7, 2010
  69. 69. BREAK Friday, May 7, 2010
  70. 70. Recruiting for Remote Friday, May 7, 2010
  71. 71. We all need people Relevant tasks Compatible tech Good talkers Demographic fits In the moment Friday, May 7, 2010
  72. 72. Gold Standard Page 52 Live recruiting Get as close as you can Friday, May 7, 2010
  73. 73. Recruiting Options Ethnio Links or house ads to online forms Recruiting agencies Customer email lists Panels Craigslist/online ads Friday, May 7, 2010
  74. 74. use Ethnio when Website access Some budget Quotas are relatively simple Friday, May 7, 2010
  75. 75. use forms when You can place a link but not code Low budget Quotas are relatively complex Options: Google Docs Wufoo Friday, May 7, 2010
  76. 76. Google docs Friday, May 7, 2010
  77. 77. use Agencies when No website access Higher budget Online recruiting failed Friday, May 7, 2010
  78. 78. Customer lists Sub-segments Ownership requirements No website access Friday, May 7, 2010
  79. 79. PANELS Friday, May 7, 2010
  80. 80. Craigslist & forums When other methods don’t work No dough Takes a while Embarrassing to admit Friday, May 7, 2010
  81. 81. the Gold Standard again Live recruiting Get as close as you can Friday, May 7, 2010
  82. 82. Live Recruiting Math (for interviews) Target: 6 recruits/hour Allow for: 1% response (low average) You Need: 8-10,000 uniques/day Friday, May 7, 2010
  83. 83. Page 56 Friday, May 7, 2010
  84. 84. Screener constraints Page 60-63 Keep it short Different from surveys Who gets paid? Friday, May 7, 2010
  85. 85. Remember the Point Tasks that align with research goals Technical compatibility Control demographics Friday, May 7, 2010
  86. 86. EXERCISE Friday, May 7, 2010
  87. 87. Friday, May 7, 2010
  88. 88. Your best questions Friday, May 7, 2010
  89. 89. GO EAT LUNCH Famished Friday, May 7, 2010
  90. 90. exercise We recommend: Split Pea Seduction, 6th & Minna Custom Burger, 7th & Natoma Tony Baloney’s, 7th & Howard Tin Vietnamese, Howard & 6th Harvest Market, Natoma & 8th Sightglass Coffee, 7th & Folsom Friday, May 7, 2010
  91. 91. exercise We recommend: Split Pea Seduction, 6th & Minna Custom Burger, 7th & Natoma Tony Baloney’s, 7th & Howard Tin Vietnamese, Howard & 6th Harvest Market, Natoma & 8th Sightglass Coffee, 7th & Folsom Friday, May 7, 2010
  92. 92. Please stand by ESCAPE THE LAB WILL RESUME AT 1:15 Friday, May 7, 2010
  93. 93. welcome back Friday, May 7, 2010
  94. 94. Time for a contest! Tweet why you hate remote research #ETLAB Win a copy of Silverback Friday, May 7, 2010
  95. 95. Remote Research Toolkit MODERATED most of Chapter 8 Friday, May 7, 2010
  96. 96. 5 easy pieces Talk to the user See what they’re doing Take notes (optional) Involve observers (optional) Record audio & video (optional) Friday, May 7, 2010
  97. 97. Communication phone Skype Reliable Cheap or free Can be expensive Harder for users for international Can combine with Conference calls screensharing to needed for stress user’s observers connection Friday, May 7, 2010
  98. 98. Screensharing Standalone Integrated Adobe Connect UserVue (RIP) LiveLook FuzeMeeting (new) WebEx* GoToMeeting *WebEx also has mobile versions Friday, May 7, 2010
  99. 99. Adobe Connect Friday, May 7, 2010
  100. 100. FuzeMeeting(New) Friday, May 7, 2010
  101. 101. Livelook Friday, May 7, 2010
  102. 102. Webex Friday, May 7, 2010
  103. 103. GoToMeeting Friday, May 7, 2010
  104. 104. Taking Notes Second monitor if possible Excel Hack (for time codes) Friday, May 7, 2010
  105. 105. How to rig your notes What the formulas look like Auto timestamp! Leave one column (A) empty for notes. Set the cells of the next column (B) to record the clock time of each entry in A. =IF(A2=“”,“”,IF(B2=“”,NOW(),B2)) Set the cells of the next column (C) to record the difference between the first entry and the current entry =IF(ISERROR(B2-$B$2),“”,B2-$B$2) Fill down (Ctrl-D). Expect to use up to 120 rows for a session. Don’t Hide column B. You’ll never need to see it write this down, it’s on page 104 Friday, May 7, 2010
  106. 106. Observer Involvement IM Web Chat Rooms Integrated Chat Passing Notes Friday, May 7, 2010
  107. 107. Recording Tools Hardware Software Phone Patch Camtasia (Windows) Cables iShowU HD (Mac) Friday, May 7, 2010
  108. 108. Phone patch set up Friday, May 7, 2010
  109. 109. Friday, May 7, 2010
  110. 110. Simple Set-up Friday, May 7, 2010
  111. 111. Intermediate Set-up Friday, May 7, 2010
  112. 112. Advanced Set-up Friday, May 7, 2010
  113. 113. Friday, May 7, 2010
  114. 114. exercise Roll your own What will be your base set-up? Write it down Friday, May 7, 2010
  115. 115. Friday, May 7, 2010
  116. 116. Your version? Friday, May 7, 2010
  117. 117. Winners!! Friday, May 7, 2010
  118. 118. Moderating for Remote Friday, May 7, 2010
  119. 119. Is it really any different? (Short answer: some.) Page 94-99 Friday, May 7, 2010
  120. 120. Question from the BP inbox yesterday It would have actually helped to know how many users the authors recommend for a tar* type of test. In a regular user test , many of us have seen how findings repeat a er 5 -8 users of each user type are observed. However, here we don't always know which task a user will be doing when we interrupt them. So in cases where we really are following users along as they complete what they came to do, how many useres should we target ? Given that the authors have been doing this for a while a recommendation from them on how many users to target and their notes on the topic would have been really interesting.  *Time-Aware Research Friday, May 7, 2010
  121. 121. Making passionate tasks work Friday, May 7, 2010
  122. 122. Making passionate tasks work Recruit for relevant current tasks Script for customizable tasks Friday, May 7, 2010
  123. 123. Re-script for remote Friday, May 7, 2010
  124. 124. Re-script for remote “Ok, I’d like you to start from the homepage, pretend you are shopping for groceries for a party, and buy some paper plates.” Friday, May 7, 2010
  125. 125. MODERATED must-haves Also see 37-43 Friday, May 7, 2010
  126. 126. MODERATED must-haves Consent Also see 37-43 Friday, May 7, 2010
  127. 127. MODERATED must-haves Consent Also see Tech set-up 37-43 Friday, May 7, 2010
  128. 128. MODERATED must-haves Consent Also see Tech set-up 37-43 The beef Friday, May 7, 2010
  129. 129. MODERATED must-haves Consent Also see Tech set-up 37-43 The beef De-installation Friday, May 7, 2010
  130. 130. MODERATED must-haves Consent Also see Tech set-up 37-43 The beef De-installation Thanks & incentive Friday, May 7, 2010
  131. 131. Chances are they’ve never done this before Friday, May 7, 2010
  132. 132. What they worry about Friday, May 7, 2010
  133. 133. Who are you and can I trust you? Friday, May 7, 2010
  134. 134. If you can see my screen, can you see my face? Friday, May 7, 2010
  135. 135. Will you be able to take control of my computer? Friday, May 7, 2010
  136. 136. Will you help me get the plug-in off my computer at the end? Friday, May 7, 2010
  137. 137. Will this work if I have a firewall/proxy/strict IT department? Friday, May 7, 2010
  138. 138. Is this going to end up on YouTube? Friday, May 7, 2010
  139. 139. Will I really get the incentive if I do this? Friday, May 7, 2010
  140. 140. How long will it take? Friday, May 7, 2010
  141. 141. Is it OK if I have to take a quick call in the middle? Friday, May 7, 2010
  142. 142. Is it OK if my kids/dog/fratmates are making noise in the background? Friday, May 7, 2010
  143. 143. OXYMORON? Remote empathy Friday, May 7, 2010
  144. 144. Phone Interviewing Specifics Friday, May 7, 2010
  145. 145. Things sound leading Pitfalls Friday, May 7, 2010
  146. 146. Body language is huge Actual language is important too Make sure you’re comfortable Friday, May 7, 2010
  147. 147. EXERCISE Practice screensharing Moderator, Participant, at least one observer Use the script in your packet as a starting point Get your participant all set up and ready to start an interview Friday, May 7, 2010
  148. 148. Friday, May 7, 2010
  149. 149. How did it go? Friday, May 7, 2010
  150. 150. Advanced Techniques High Emotions Friday, May 7, 2010
  151. 151. Advanced Techniques One to many moderation Multi-threading Friday, May 7, 2010
  152. 152. Future Frontiers Friday, May 7, 2010
  153. 153. BREAK Friday, May 7, 2010
  154. 154. Remote Research Toolkit UNMODERATED Friday, May 7, 2010
  155. 155. 3 parts Provide test target Record user behavior Capture comments (optional) Friday, May 7, 2010
  156. 156. AUTOMATED Tools Friday, May 7, 2010
  157. 157. What do you want to do? Get freeform data cheaply Quantify user behaviors Conceptual & IA testing Friday, May 7, 2010
  158. 158. This stuff is pretty cheap • Usabilla $0-950/yr • Optimal Workshop $559-1350/yr • Loop11 $350/study • OpenHallway $19-199/mo • UserTesting.com $39/user • Feedback Army $10/10 users Friday, May 7, 2010
  159. 159. UserTesting.com Friday, May 7, 2010
  160. 160. UserTesting.com Friday, May 7, 2010
  161. 161. UserTesting.com Friday, May 7, 2010
  162. 162. UserTesting.com Friday, May 7, 2010
  163. 163. Using your own recruits Set up an account Email support@usertesting.com with: the URL where your target lives the scenario (500 char limit) tasks (2000 char limit) up to 4 questions to be answered in writing at the end Request a UserTesting URL for your study Friday, May 7, 2010
  164. 164. Loop11 Friday, May 7, 2010
  165. 165. Loop11 Friday, May 7, 2010
  166. 166. Loop11 Friday, May 7, 2010
  167. 167. Loop11 Friday, May 7, 2010
  168. 168. Loop11 Friday, May 7, 2010
  169. 169. Usabilla Friday, May 7, 2010
  170. 170. Optimal Workshop Treejack Chalkmark Friday, May 7, 2010
  171. 171. EXERCISE Choose one of these tools Sign up for a test account Create a 15-minute test for any website you like Friday, May 7, 2010
  172. 172. Friday, May 7, 2010
  173. 173. RESULTS Friday, May 7, 2010
  174. 174. Session Videos Heatmaps Notes Completion Rates Card Sort Groups Lions, Tigers, Bears... Friday, May 7, 2010
  175. 175. Telling a story If you have time-coded notes, you can make highlight videos If you have user comments, you can do tag analysis or make word clouds If you have numbers, you can compare to benchmarks If you have card groups you can do cluster analysis Friday, May 7, 2010
  176. 176. Case Study: Data Mismatch Friday, May 7, 2010
  177. 177. Qualitative Results 16 remote interviews Worst findability BP has ever observed Recommendation to burn it down and start over Friday, May 7, 2010
  178. 178. Quantitative Data EFFECTIVENESS RATIOS EFFICIENCY Your task is to make an Success users Only appointment with a pediatric Time (mm:ss) arrhythmia Mean time per task 01:36 specialist at Standard deviation 00:47 Children’s Hospital. Once you have found it, please remember Clicks or write down the 6 phone number you Mean clicks per task (exact value: 6) would call to make 4 an appointment by Standard deviation (exact value: 4.4) phone, as you will be asked for that information later on. Answers distribution ??? ! The phone number search has been... *[1=Extremely difficult / 7=Extremely easy] 1 2% (1) 2 2% (1) 3 10% (5) 4 18% (9) 5 18% (9) 6 24% (12) 7 26% (13) N 50 Mean* 5.2 Friday, May 7, 2010
  179. 179. What happened? Friday, May 7, 2010
  180. 180. 32 page report, distilled Navigation Search ease Hospital ease of use of use reputation 1 2 3 4 5 6 7 terrible neutral UserZoom’s red excellent flag minimum Friday, May 7, 2010
  181. 181. What’s different IRL? Friday, May 7, 2010
  182. 182. Recording 1 more button to press Friday, May 7, 2010
  183. 183. Observer input One more stream on your screen (Possibly 2 or 3) Friday, May 7, 2010
  184. 184. Notes Tap, tap, tapping away Friday, May 7, 2010
  185. 185. So, what could go wrong? Friday, May 7, 2010
  186. 186. 3 points of failure Technology yours participant’s observer’s Recruiting Test Design Friday, May 7, 2010
  187. 187. Control what you can control Which is, your set up. You wouldn’t go backpacking without checking your gear. Do it 24 hours in advance. Or else. Page 240-241 Friday, May 7, 2010
  188. 188. Prepare for what you can’t control User’s dog trips over power cord Internet affected by local weather User can’t install Doesn’t know how to install Friday, May 7, 2010
  189. 189. Mid-Session Tech Problems Friday, May 7, 2010
  190. 190. Live Recruiting Problems What if they gave a study and nobody came? Friday, May 7, 2010
  191. 191. Increase exposure When you’re not getting enough views When the wrong type of users are responding Friday, May 7, 2010
  192. 192. Increase incentives When it’s cheaper When you’re almost getting enough When segments are tough Friday, May 7, 2010
  193. 193. Switch methods When base traffic level isn’t high enough Friday, May 7, 2010
  194. 194. Script Problems Cut the BS. Friday, May 7, 2010
  195. 195. The bottom line If they’re not engaged, something’s wrong. Friday, May 7, 2010
  196. 196. Burn the script Find out what matters. Friday, May 7, 2010
  197. 197. QUESTIONS Friday, May 7, 2010
  198. 198. @boltpeters CYD@BOLTPETERS.COM NATE@BOLTPETERS.COM Friday, May 7, 2010
  199. 199. kthxbai. Friday, May 7, 2010
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×