Thursday, July 1, 2010
Thursday, July 1, 2010
flickr/thetruthabout

Thursday, July 1, 2010
REMOTE RESEARCH

                                             In person (95%)
                     Definition:             ...
#UXREMOTE


Thursday, July 1, 2010
Thursday, July 1, 2010
WHEN DOES REMOTE WIN?




Thursday, July 1, 2010
Marv.
         Creative director UX designer product manager
         developer IA.




Thursday, July 1, 2010
Marv’s Boss.
         He’s always on a plane and loves birds.




Thursday, July 1, 2010
Hey Marv, I want to save
                some money on research.




Thursday, July 1, 2010
NATIVE ENVIRONMENT
                     CHEAP ETHNOGRAPHY, NOT CHEAP LAB.




Thursday, July 1, 2010
TIME-AWARE
                         RESEARCH IS THE BEST
Thursday, July 1, 2010
Thursday, July 1, 2010
I’LL TRADE YOU TIME
              FOR PHYSICAL PRESENCE




Thursday, July 1, 2010
INVOLVED OBSERVERS
            Easier to collaborate in real time.




Thursday, July 1, 2010
Geography




                         TEXT


Thursday, July 1, 2010
WHEN DOES REMOTE LOSE?




Thursday, July 1, 2010
Hey Marv, I want feedback
                on our MMO video game
                from 13yo English speaking
               ...
I guess, Boss.
                 Might be hard
                 to do with
                 remote testing.




Thursday, J...
BANDWIDTH
            Yes, it’s an issue.




Thursday, July 1, 2010
TRANSLATION
            It’s expensive and di cult to do live.




Thursday, July 1, 2010
MINORS
            Consent is a bear.
            Especially if they’re under 13.




Thursday, July 1, 2010
FACES
            Most users don’t have webcams.
            Webcams are a little invasive.




Thursday, July 1, 2010
SUPER DUPER SECURITY




Thursday, July 1, 2010
AN EPIC FAIL.




Thursday, July 1, 2010
Thursday, July 1, 2010
How To Find Actual
                          English Speakers:

                         What Happens When
               ...
WHAT WENT WRONG
                   Coins were way more valuable
                   than we expected
                   Peo...
DESIGNING
                         REMOTE STUDIES




Thursday, July 1, 2010
Yo Marv, get me some
                insight on WHY people are
                bailing out of checkout.




Thursday, July...
Sure, Boss.
                 Let’s intercept
                 & observe 10
                 users live with
              ...
TIME-AWARE TASKS
                         Told to find purple pumps.   Has to wear this for sister’s wedding this
         ...
REAL TIME OBSERVERS




Thursday, July 1, 2010
Thursday, July 1, 2010
TEXT


Thursday, July 1, 2010
PROTOTYPES, WIREFRAMES,
                     SKETCHES?




Thursday, July 1, 2010
HECK YES.




Thursday, July 1, 2010
Thursday, July 1, 2010
10 YEARS
                 244 STUDIES
                 2,915 PARTICIPANTS (MODERATED)
                 95 ORGANIZATIONS
  ...
60% PROTOTYPE RESEARCH




Thursday, July 1, 2010
Thursday, July 1, 2010
A DINOSAUR FAMILY
        EXPLAINS INFORMATION
        ARCHITECTURE




Thursday, July 1, 2010
Marv-o: are their any UX
                web apps we can use for
                online research?




Thursday, July 1, 20...
Sure, boss.
                 There is an
                 AVALANCHE
                 of UX TOOLS.




Thursday, July 1, 20...
“At least a little
                 behavioral”
                 Surveys.
                 Web Analytics.




Thursday, Ju...
FIVE CATEGORIES

                 •       Moderated (screen-sharing + audio)
                 •       Self-moderated (user...
Moderated



     Self-Moderated

                         For A Long Time



     Automated Live

                       ...
Talk To People



     People Talk To A Machine

                         For A Long Time



     Task Bar On A Live Site
...
Insight



     Ammunition

                         For Ad Agencies



     More Ammunition

                         I.A...
Moderated



     Self-Moderated

                         For A Long Time



     Automated Live

                       ...
Moderated
                                           Opinions

     Self-Moderated

                         For A Long Ti...
Moderated                             Ad Agencies
                                             Love It
     Self-Moderated...
Moderated                              “Online
                                           Qualitative
     Self-Moderated
...
Moderated                             Jared Spool
                                             Hates It
     Self-Moderate...
Thursday, July 1, 2010
THAT WAS 293 WORDS




Thursday, July 1, 2010
USABILLA

                         Clicks & Questions
                         Heatmaps
                         You do yo...
Thursday, July 1, 2010
Thursday, July 1, 2010
LOOP11 (UN-MODERATED)




                               TEXT


Thursday, July 1, 2010
TEXT


Thursday, July 1, 2010
* $49/mo
                         * Integrated VOIP
                         * iPad Viewing


Thursday, July 1, 2010
Thursday, July 1, 2010
TEXT


Thursday, July 1, 2010
TEXT


Thursday, July 1, 2010
Thursday, July 1, 2010
Thursday, July 1, 2010
Thursday, July 1, 2010
Thursday, July 1, 2010
Thursday, July 1, 2010
Thursday, July 1, 2010
Moderated



     Self-Moderated




     Automated Live

                         Conceptual



     Automated Static



...
WHAT WE USE.

                                     A LOT.

                         • Ethnio
                         • Go...
RECRUITING
                         FOR REMOTE



Thursday, July 1, 2010
So Marv, how do we find
                people for these studies?




Thursday, July 1, 2010
Stop
                 harassing me
                 and get o
                 that weird
                 plane.




Thur...
RECRUITING OPTIONS
                                                                Realness
                          Grab...
GOLD STANDARD
                          Live recruiting
                          Get as close as you
                    ...
RECRUITING OPTIONS

                          Ethnio
                          Links or house ads to online
              ...
USE ETHNIO WHEN
                          Website access
                          Some budget
                          Q...
USE FORMS WHEN

                         You can place a link but not code
                         Low budget
           ...
USE AGENCIES WHEN

                          No website access
                          People whine about first-time
    ...
CUSTOMER LISTS


                          Sub-segments
                          Ownership
                          requ...
PANELS
Thursday, July 1, 2010
SORT OF
Thursday, July 1, 2010
Thursday, July 1, 2010
CRAIGSLIST

                    When other methods don’t work
                    No dough
                    Takes a whi...
LIVE RECRUITING MATH


                         Target: 6 recruits/hour
                         Allow for: 1% response (l...
TEXT


Thursday, July 1, 2010
TEXT


Thursday, July 1, 2010
TEXT


Thursday, July 1, 2010
Paying Incentives



                         Collect minimal
                         personal info
                     ...
HOW TO BE
                         graceful
                          When All Your
                         TECHNOLOGY
  ...
WHAT THEY WORRY ABOUT




Thursday, July 1, 2010
WHO ARE YOU AND CAN I
                 TRUST YOU?




Thursday, July 1, 2010
IF YOU CAN SEE MY
                         SCREEN,
                  CAN YOU SEE MY FACE?



Thursday, July 1, 2010
WILL YOU BE ABLE TO
                                TAKE
                           CONTROL OF MY
                        ...
WILL YOU HELP ME GET
                              THE PLUG-IN
                         OFF MY COMPUTER AT
               ...
WILL THIS WORK IF I HAVE A
     FIREWALL/PROXY/STRICT IT
           DEPARTMENT?



Thursday, July 1, 2010
IS THIS GOING TO END
                                 UP ON
                               YOUTUBE?



Thursday, July 1, 2...
OXYMORON?
                         REMOTE EMPATHY




Thursday, July 1, 2010
EMPATHY TIP



                   Get someone on the phone
                   Give them an impossible puzzle




Thursday,...
HYBRID STUDIES

                   Numbers actually do lie
                   sometimes
                   And they don’t ...
MOBILE REMOTE
                          RESEARCH?




Thursday, July 1, 2010
?

Thursday, July 1, 2010
ADVANCED TECHNIQUES
                         One to many moderation




                         Multi-threading
Thursday,...
PORTABLE RESEARCH




Thursday, July 1, 2010
Thursday, July 1, 2010
BURN THE SCRIPT
                         FIND OUT WHAT MATTERS.




Thursday, July 1, 2010
ADVANCED TECHNIQUES
                         High Emotions




Thursday, July 1, 2010
TEXT


Thursday, July 1, 2010
TEXT


Thursday, July 1, 2010
THE FUTURE




Thursday, July 1, 2010
remoteusability.com




Thursday, July 1, 2010
@boltron



Thursday, July 1, 2010
Upcoming SlideShare
Loading in …5
×

Remote Research, The Talk.

8,918 views

Published on

Slides from Nate's IxDA & UPA & SVA talks in Los Angeles and New York in May and June of 2010. Most of these won't make any damn sense but more information is in the actual book here: http://bit.ly/ZlDoQ and on our site http://remoteusability.com.

Published in: Technology, Design, Business
0 Comments
35 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
8,918
On SlideShare
0
From Embeds
0
Number of Embeds
4,584
Actions
Shares
0
Downloads
167
Comments
0
Likes
35
Embeds 0
No embeds

No notes for slide

Remote Research, The Talk.

  1. 1. Thursday, July 1, 2010
  2. 2. Thursday, July 1, 2010
  3. 3. flickr/thetruthabout Thursday, July 1, 2010
  4. 4. REMOTE RESEARCH In person (95%) Definition: Remote (5%) Research where the participant is not sharing physical space with the researcher. % of qualitative research done remotely* *Source: BP’s best guess Thursday, July 1, 2010
  5. 5. #UXREMOTE Thursday, July 1, 2010
  6. 6. Thursday, July 1, 2010
  7. 7. WHEN DOES REMOTE WIN? Thursday, July 1, 2010
  8. 8. Marv. Creative director UX designer product manager developer IA. Thursday, July 1, 2010
  9. 9. Marv’s Boss. He’s always on a plane and loves birds. Thursday, July 1, 2010
  10. 10. Hey Marv, I want to save some money on research. Thursday, July 1, 2010
  11. 11. NATIVE ENVIRONMENT CHEAP ETHNOGRAPHY, NOT CHEAP LAB. Thursday, July 1, 2010
  12. 12. TIME-AWARE RESEARCH IS THE BEST Thursday, July 1, 2010
  13. 13. Thursday, July 1, 2010
  14. 14. I’LL TRADE YOU TIME FOR PHYSICAL PRESENCE Thursday, July 1, 2010
  15. 15. INVOLVED OBSERVERS Easier to collaborate in real time. Thursday, July 1, 2010
  16. 16. Geography TEXT Thursday, July 1, 2010
  17. 17. WHEN DOES REMOTE LOSE? Thursday, July 1, 2010
  18. 18. Hey Marv, I want feedback on our MMO video game from 13yo English speaking kids in France & Latin America with webcams. Thursday, July 1, 2010
  19. 19. I guess, Boss. Might be hard to do with remote testing. Thursday, July 1, 2010
  20. 20. BANDWIDTH Yes, it’s an issue. Thursday, July 1, 2010
  21. 21. TRANSLATION It’s expensive and di cult to do live. Thursday, July 1, 2010
  22. 22. MINORS Consent is a bear. Especially if they’re under 13. Thursday, July 1, 2010
  23. 23. FACES Most users don’t have webcams. Webcams are a little invasive. Thursday, July 1, 2010
  24. 24. SUPER DUPER SECURITY Thursday, July 1, 2010
  25. 25. AN EPIC FAIL. Thursday, July 1, 2010
  26. 26. Thursday, July 1, 2010
  27. 27. How To Find Actual English Speakers: What Happens When You Let Go Of A Balloon? Thursday, July 1, 2010
  28. 28. WHAT WENT WRONG Coins were way more valuable than we expected People mis-assessed their English levels Bandwidth was really a problem Nobody plays Habbo when their parents are home to consent Thursday, July 1, 2010
  29. 29. DESIGNING REMOTE STUDIES Thursday, July 1, 2010
  30. 30. Yo Marv, get me some insight on WHY people are bailing out of checkout. Thursday, July 1, 2010
  31. 31. Sure, Boss. Let’s intercept & observe 10 users live with screen-sharing, and do 10 self- moderated with usertesting.com Thursday, July 1, 2010
  32. 32. TIME-AWARE TASKS Told to find purple pumps. Has to wear this for sister’s wedding this weekend. Doesn’t care. Whole different interview. Thursday, July 1, 2010
  33. 33. REAL TIME OBSERVERS Thursday, July 1, 2010
  34. 34. Thursday, July 1, 2010
  35. 35. TEXT Thursday, July 1, 2010
  36. 36. PROTOTYPES, WIREFRAMES, SKETCHES? Thursday, July 1, 2010
  37. 37. HECK YES. Thursday, July 1, 2010
  38. 38. Thursday, July 1, 2010
  39. 39. 10 YEARS 244 STUDIES 2,915 PARTICIPANTS (MODERATED) 95 ORGANIZATIONS 19,610 HOURS Thursday, July 1, 2010
  40. 40. 60% PROTOTYPE RESEARCH Thursday, July 1, 2010
  41. 41. Thursday, July 1, 2010
  42. 42. A DINOSAUR FAMILY EXPLAINS INFORMATION ARCHITECTURE Thursday, July 1, 2010
  43. 43. Marv-o: are their any UX web apps we can use for online research? Thursday, July 1, 2010
  44. 44. Sure, boss. There is an AVALANCHE of UX TOOLS. Thursday, July 1, 2010
  45. 45. “At least a little behavioral” Surveys. Web Analytics. Thursday, July 1, 2010
  46. 46. FIVE CATEGORIES • Moderated (screen-sharing + audio) • Self-moderated (usertesting.com) • Automated (loop11, usabilla) • Fancy Analytics on Steroids (crazyegg) • Diary (revelation) Thursday, July 1, 2010
  47. 47. Moderated Self-Moderated For A Long Time Automated Live Conceptual Automated Static Web Analytics Thursday, July 1, 2010
  48. 48. Talk To People People Talk To A Machine For A Long Time Task Bar On A Live Site Tasks On Concepts Task Bar On Images Fancy-Ass Analytics Thursday, July 1, 2010
  49. 49. Insight Ammunition For Ad Agencies More Ammunition I.A. & Navigation Still More Ammunition Just Better Analytics Thursday, July 1, 2010
  50. 50. Moderated Self-Moderated For A Long Time Automated Live Conceptual Automated Static Deep Behavioral UX Web Analytics Insight Thursday, July 1, 2010
  51. 51. Moderated Opinions Self-Moderated For A Long Time Automated Live Conceptual Automated Static Web Analytics Thursday, July 1, 2010
  52. 52. Moderated Ad Agencies Love It Self-Moderated For A Long Time Automated Live Conceptual Automated Static Web Analytics Thursday, July 1, 2010
  53. 53. Moderated “Online Qualitative Self-Moderated Research” For A Long Time Automated Live Conceptual Automated Static Web Analytics Thursday, July 1, 2010
  54. 54. Moderated Jared Spool Hates It Self-Moderated For A Long Time Automated Live Conceptual Automated Static Web Analytics Thursday, July 1, 2010
  55. 55. Thursday, July 1, 2010
  56. 56. THAT WAS 293 WORDS Thursday, July 1, 2010
  57. 57. USABILLA Clicks & Questions Heatmaps You do your own recruiting $49 - $199 Thursday, July 1, 2010
  58. 58. Thursday, July 1, 2010
  59. 59. Thursday, July 1, 2010
  60. 60. LOOP11 (UN-MODERATED) TEXT Thursday, July 1, 2010
  61. 61. TEXT Thursday, July 1, 2010
  62. 62. * $49/mo * Integrated VOIP * iPad Viewing Thursday, July 1, 2010
  63. 63. Thursday, July 1, 2010
  64. 64. TEXT Thursday, July 1, 2010
  65. 65. TEXT Thursday, July 1, 2010
  66. 66. Thursday, July 1, 2010
  67. 67. Thursday, July 1, 2010
  68. 68. Thursday, July 1, 2010
  69. 69. Thursday, July 1, 2010
  70. 70. Thursday, July 1, 2010
  71. 71. Thursday, July 1, 2010
  72. 72. Moderated Self-Moderated Automated Live Conceptual Automated Static Web Analytics Thursday, July 1, 2010
  73. 73. WHAT WE USE. A LOT. • Ethnio • GoToMeeting + iShowYouHD • Usabilla + Usertesting.com Thursday, July 1, 2010
  74. 74. RECRUITING FOR REMOTE Thursday, July 1, 2010
  75. 75. So Marv, how do we find people for these studies? Thursday, July 1, 2010
  76. 76. Stop harassing me and get o that weird plane. Thursday, July 1, 2010
  77. 77. RECRUITING OPTIONS Realness Grab someone in the hallway Grab someone who isn’t directly involved in your project IM your friends for a quick look Panels and recruiting agencies Real customers from marketing lists Site intercepts Thursday, July 1, 2010
  78. 78. GOLD STANDARD Live recruiting Get as close as you can Thursday, July 1, 2010
  79. 79. RECRUITING OPTIONS Ethnio Links or house ads to online forms Recruiting agencies Customer email lists Panels Craigslist/online ads Thursday, July 1, 2010
  80. 80. USE ETHNIO WHEN Website access Some budget Quotas are relatively simple Thursday, July 1, 2010
  81. 81. USE FORMS WHEN You can place a link but not code Low budget Quotas are relatively complex Options: Google Docs Wufoo Thursday, July 1, 2010
  82. 82. USE AGENCIES WHEN No website access People whine about first-time visitors Higher budget Online recruiting failed Thursday, July 1, 2010
  83. 83. CUSTOMER LISTS Sub-segments Ownership requirements No website access Thursday, July 1, 2010
  84. 84. PANELS Thursday, July 1, 2010
  85. 85. SORT OF Thursday, July 1, 2010
  86. 86. Thursday, July 1, 2010
  87. 87. CRAIGSLIST When other methods don’t work No dough Takes a while Embarrassing to admit to your friends Thursday, July 1, 2010
  88. 88. LIVE RECRUITING MATH Target: 6 recruits/hour Allow for: 1% response (low average You Need: 10,000 uniques/day Thursday, July 1, 2010
  89. 89. TEXT Thursday, July 1, 2010
  90. 90. TEXT Thursday, July 1, 2010
  91. 91. TEXT Thursday, July 1, 2010
  92. 92. Paying Incentives Collect minimal personal info Deliver quickly Appropriate for location Thursday, July 1, 2010
  93. 93. HOW TO BE graceful When All Your TECHNOLOGY BREAKS Thursday, July 1, 2010
  94. 94. WHAT THEY WORRY ABOUT Thursday, July 1, 2010
  95. 95. WHO ARE YOU AND CAN I TRUST YOU? Thursday, July 1, 2010
  96. 96. IF YOU CAN SEE MY SCREEN, CAN YOU SEE MY FACE? Thursday, July 1, 2010
  97. 97. WILL YOU BE ABLE TO TAKE CONTROL OF MY COMPUTER? Thursday, July 1, 2010
  98. 98. WILL YOU HELP ME GET THE PLUG-IN OFF MY COMPUTER AT THE END? Thursday, July 1, 2010
  99. 99. WILL THIS WORK IF I HAVE A FIREWALL/PROXY/STRICT IT DEPARTMENT? Thursday, July 1, 2010
  100. 100. IS THIS GOING TO END UP ON YOUTUBE? Thursday, July 1, 2010
  101. 101. OXYMORON? REMOTE EMPATHY Thursday, July 1, 2010
  102. 102. EMPATHY TIP Get someone on the phone Give them an impossible puzzle Thursday, July 1, 2010
  103. 103. HYBRID STUDIES Numbers actually do lie sometimes And they don’t tell you the why 3-5 remote interviews is all you need Thursday, July 1, 2010
  104. 104. MOBILE REMOTE RESEARCH? Thursday, July 1, 2010
  105. 105. ? Thursday, July 1, 2010
  106. 106. ADVANCED TECHNIQUES One to many moderation Multi-threading Thursday, July 1, 2010
  107. 107. PORTABLE RESEARCH Thursday, July 1, 2010
  108. 108. Thursday, July 1, 2010
  109. 109. BURN THE SCRIPT FIND OUT WHAT MATTERS. Thursday, July 1, 2010
  110. 110. ADVANCED TECHNIQUES High Emotions Thursday, July 1, 2010
  111. 111. TEXT Thursday, July 1, 2010
  112. 112. TEXT Thursday, July 1, 2010
  113. 113. THE FUTURE Thursday, July 1, 2010
  114. 114. remoteusability.com Thursday, July 1, 2010
  115. 115. @boltron Thursday, July 1, 2010

×