Your SlideShare is downloading. ×
  • Like
UX Lx 2012 - Nate Bolt
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Now you can save presentations on your phone or tablet

Available for both IPhone and Android

Text the download link to your phone

Standard text messaging rates apply

UX Lx 2012 - Nate Bolt

  • 10,282 views
Published

 

Published in Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
10,282
On SlideShare
0
From Embeds
0
Number of Embeds
10

Actions

Shares
Downloads
106
Comments
0
Likes
14

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Remote ResearchWorkshop
  • 2. Photo Credit: recados-orkut.net
  • 3. Remote Design Research In person RemoteDefinition:Research where theparticipant is not sharingphysical space with theresearcher. % of qualitative research done remotely* * Source: My wild guess
  • 4. #UXLX #RR
  • 5. Bayesian methods Adaptive1995Infer when users needed help Decision Theory &Animated pedagogical Systems Groupagency
  • 6. “ Misunderstood animated pedagogical agent or spawn of Satan? ”
  • 7. Bad Research
  • 8. 1) Academic Focus2) Lab-based3) False Premise
  • 9. THAT IS WHY WE WENT REMOTE
  • 10. Btw I’m Nate Bolt
  • 11. 12 YEARS293 STUDIES3,108 PARTICIPANTS (MODERATED)118 ORGANIZATIONS21,610 HOURS
  • 12. “SOCIAL IMPACT OF DIGITAL TECHNOLOGY”
  • 13. A DINOSAUR FAMILYEXPLAINS INFORMATION ARCHITECTURE
  • 14. SF to PARISIN TWO MINUTES
  • 15. jumboltron.com
  • 16. OMG PLEASEBUY THISBOOK
  • 17. Let’s jump right in
  • 18. What’s New with UX Research Tools?
  • 19. BUT FIRST
  • 20. Moderated versus Automated
  • 21. Moderated versus Automated10% 90%
  • 22. Moderated versus Automated Photo Credit: singularityhub.com
  • 23. ModeratedGoToMeeting 5.2Integrated RecordingThat’s really awesome
  • 24. ModeratedHuggingAt Mozillaand MailChimp Photo Credit: Diane Loviglio, Mozilla
  • 25. AutomatedUsabillaMobile
  • 26. HOW TO BEgraceful When All YourTECHNOLOGYBREAKS
  • 27. Follow the buddy system
  • 28. It’s laptop time
  • 29. remoteresear.ch
  • 30. remoteresear.ch/lisbon
  • 31. (BROWSER TIME)
  • 32. IN CASE THE INTERNET FAILSResearch experienceMost important thing to get out of today?Biggest fear or skepticism about remote research?
  • 33. Thanks, Volunteer!
  • 34. Four Takeaways1. See moderated method & recording with GoToMeeting2. Automated with Usabilla3.Self-moderated tablet hugging4.Quick incentive payment
  • 35. Eff you sales guy
  • 36. RecruitingWHAT WE’LL COVER TODAY Moderated Contextual Inquiry Automated Results Pitfalls
  • 37. SOME THOUGHTS ON WEB ANALYTICSWhat do helicopters do in action movies? Explode
  • 38. “ How much practical research experience do I need to have for this workshop? ” - Many People
  • 39. “ Will this be a long lecture or an actual workshop? ” - Some Guy on Twitter
  • 40. “ Can we do contextual inquiries remotely? ” - Nick Dunlavey | @ndunlavey
  • 41. The physical and computing ecosystem
  • 42. Now we do a little experiment
  • 43. EXERCISE #1 “Giraffe”Binder PDF Page 4Table groupsPick one moderator and one participant(they need to be in different locations) 10 minutes of moderating
  • 44. remoteresear.ch/lisbon“The Workshop Binder”
  • 45. MODERATOR InstructionsYour participant has a sheet of paper and that’s itGoal: find out what kind of physical taskinstructions onlineHint: ask them to give you a play by play of whatthey are doingTry and follow along with your own paper NO helping, even if you could.
  • 46. It’s A RACEYou only have 10 minutes to do thismax.
  • 47. Surprises?
  • 48. Was there emotion?
  • 49. WHEN DOES REMOTE WIN?
  • 50. Marv.Creative director UX designer product managerdeveloper IA co-founder.
  • 51. Marv’s Boss.He’s always on a plane and loves birds.
  • 52. Hey Marv, I want to savesome money on research.
  • 53. NATIVE ENVIRONMENTCHEAP ETHNOGRAPHY, NOT CHEAP LAB.
  • 54. TIME-AWARERESEARCH IS THE BEST
  • 55. I’LL TRADE YOU TIMEFOR PHYSICAL PRESENCE
  • 56. INVOLVED OBSERVERSEasier to collaborate in real time.
  • 57. GEOGRAPHYTITLE TEXT
  • 58. WHEN DOES REMOTE LOSE?
  • 59. Hey Marv, I want feedbackon our MMO video gamefrom 13yo English speakingkids in France & LatinAmerica with webcams.
  • 60. I guess, Boss.Might be hardto do withremote testing.
  • 61. BANDWIDTHYes, it’s an issue.
  • 62. TRANSLATIONIt’s expensive and difficult to do live.
  • 63. MINORS Consent is a bear.Especially if they’re under 13.
  • 64. FACESMost users don’t have webcams.Webcams are a little invasive.
  • 65. YOUR BOSS
  • 66. SUPER DUPER SECURITY
  • 67. AN EPIC FAIL.
  • 68. We basically didBad Research
  • 69. Kids will lie for coinsPeople mis-assessed their English levelsBandwidth was really a problemKids don’t play when their parents are home
  • 70. HOW TO FIND ACTUAL SPEAKERS OF YOUR LANGUAGE IN ANY COUNTRY:“ What happens when you let go of a balloon? ”“ Quest-ce qui se passe quand vous lâchez unballon? ”
  • 71. DESIGNINGREMOTE STUDIES
  • 72. Yo Marv, get me someinsight on WHY people arebailing out of checkout.
  • 73. Sure, Boss.Let’s combinemethods.Intercept &observe 10users live, anddo 10 self-moderated withusertesting.com
  • 74. TIME-AWARE TASKSTold to find purple pumps. Has to wear this for sister’s wedding this weekend. Doesn’t care. Whole different interview.
  • 75. HEART & SOUL OF REMOTE RESEARCHNot scheduling participants.
  • 76. REAL TIME OBSERVERS
  • 77. TITLE TEXT
  • 78. PROTOTYPES, WIREFRAMES, SKETCHES?
  • 79. HECK YES.
  • 80. RECRUITINGFOR REMOTE
  • 81. RECRUITING OPTIONSEthnioLinks or house ads to online formsRecruiting agenciesCustomer email listsPanelsCraigslist/online ads
  • 82. RECRUITING OPTIONS realnessGrab someone in the hallwayGrab someone who isn’t directly involvedin your projectIM your friends for a quick lookPanels and recruiting agenciesReal customers from marketing listsSite intercepts
  • 83. SITE ISN’T AN OPTIONGEO IPMEMBERS ONLY
  • 84. GOLD STANDARDLive recruitingGet as close as youcan
  • 85. USE ETHNIO WHENWebsite accessSome budgetQuotas are relatively simple
  • 86. USE FORMS WHENYou can place a link but not codeLow budgetQuotas are relatively complexOptions: Google Docs Wufoo
  • 87. USE AGENCIES WHENNo website accessPeople whine about first-timevisitorsHigher budgetOnline recruiting failed
  • 88. CUSTOMER LISTSSub-segmentsOwnershiprequirementsNo website access
  • 89. PANEL
  • 90. SORT OF
  • 91. CRAIGSLISTWhen other methods don’t workNo doughTakes a whileEmbarrassing to admit to your friends
  • 92. LIVE RECRUITING MATHTarget: 6 recruits/hourAllow for: 1% response (low averageYou Need: 10,000 uniques/day
  • 93. TITLE TEXT
  • 94. remoteresear.ch/samples “Online Consent”
  • 95. Paying IncentivesCollect minimalpersonal infoDeliver quicklyAppropriate forlocation
  • 96. Screener constraintsKeep it shortDifferent from surveysWho gets paid?
  • 97. MODERATED must-haves Consent Tech set-up The beef De-installation Thanks & incentive
  • 98. AUTOMATED must-haves Expectation setting Instructions The beef Thanks & incentive
  • 99. RecruitingSame: Good “talkers”Same: Demographic fitsDifferent: Relevant currenttasksDifferent: Compatible tech
  • 100. Logistics: What & HowProvide testtargetAccessCredentials
  • 101. Logistics: Who & WhenHow to connect with the userManaging time zonesSession length
  • 102. Paying IncentivesCollect minimalpersonal infoDeliver quicklyAppropriate forlocation
  • 103. STAGED IncentivesFor Contextual InquiryStart with $75, then at the end offer$25 more for environment photosSelf-made diary studies
  • 104. Advanced planning: hybrid studiesWhy use only one method?Some methods don’t tell you the why3-5 remote interviews is all you need
  • 105. EXERCISE #2okay, draft a screener Binder PDF Page 6 Use ethn.io or dev.ethn.io (beta) or wufoo Under 8 Questions Advanced Bonus: create an automated and moderated screener
  • 106. remoteresear.ch/lisbon“The Workshop Binder”
  • 107. Marv-o - what in the worlddoes automated/unmoderated/asynchronous researchmean?
  • 108. Sure, boss.There is anAVALANCHEof UX TOOLS.
  • 109. “At least a littlebehavioral”No surveys.No standard webanalytics.
  • 110. Since 2007
  • 111. Can tools make us better researchers?
  • 112. 5CATEGORIES
  • 113. ModeratedSelf-Moderated For A Long TimeAutomated Live ConceptualAutomated StaticWeb Analytics
  • 114. Talk To PeoplePeople Talk To A Machine For A Long TimeTasks On A Live Site Tasks On ConceptsTasks On ImagesFancy-Ass Analytics
  • 115. InsightAmmunition For Ad AgenciesMore Ammunition I.A. & NavigationStill More AmmunitionJust Better Analytics
  • 116. ModeratedSelf-Moderated For A Long TimeAutomated Live ConceptualAutomated Static Deep Behavioral UXWeb Analytics Insight
  • 117. Moderated OpinionsSelf-Moderated For A Long TimeAutomated Live ConceptualAutomated StaticWeb Analytics
  • 118. Moderated Ad Agencies Love ItSelf-Moderated For A Long TimeAutomated Live ConceptualAutomated StaticWeb Analytics
  • 119. Moderated Jared Spool Hates ItSelf-Moderated For A Long TimeAutomated Live ConceptualAutomated StaticWeb Analytics
  • 120. Moderated }Self-Moderated For A Long Time MobileAutomated Live ConceptualAutomated Static
  • 121. Remote Research ToolkitUNMODERATED (AKA AUTOMATED)
  • 122. 3 partsProvide test targetRecord user behaviorCapture comments (optional)
  • 123. USABILLAClicks & QuestionsHeatmapsYou do your own recruiting$49 - $199
  • 124. LOOP11 (UN-MODERATED) TITLE TEXT
  • 125. TITLE TEXT
  • 126. GOTOMEETING - $39/MO
  • 127. TITLE TEXT
  • 128. WHAT WE USE. A LOT.• Ethnio + GTM• Usabilla + Usertesting.com• Guerrilla mobile
  • 129. EXERCISE #3Either “unmoderated” or “practice interviewing” usertesting.com - 1 free user CODE: uxlisbon2012 usabilla - 2 months free (Sign up for an account, but cancel on the payment page. Send an email to support@usabilla.com and mention UXLX) ethnio - 1 month free or trial. CODE: WORKSHOPLOVE
  • 130. RESULTS
  • 131. REMEMBER, BEgraceful When All YourTECHNOLOGYBREAKS
  • 132. WHAT THEY WORRY ABOUT
  • 133. WHO ARE YOU AND CAN I TRUST YOU?
  • 134. IF YOU CAN SEE MY SCREEN, CAN YOU SEE MY FACE?
  • 135. WILL YOU BE ABLE TO TAKE CONTROL OF MY COMPUTER?
  • 136. WILL YOU HELP ME GET THE PLUG-INOFF MY COMPUTER AT THE END?
  • 137. WILL THIS WORK IF I HAVE AFIREWALL/PROXY/STRICT IT DEPARTMENT?
  • 138. IS THIS GOING TO END UP ON YOUTUBE?
  • 139. OXYMORON?REMOTE EMPATHY
  • 140. Taking NotesSecond monitor ifpossibleExcel Hack(remoteresear.ch/samples)
  • 141. Observer InvolvementIMCampfire (!)Integrated ChatPassing Notes
  • 142. HYBRID STUDIESNumbers actually do lie sometimesAnd they don’t tell you the why3-5 remote interviews is all you need
  • 143. So, what could go wrong?
  • 144. 3 points of failure Technology yours participant’s observer’s Recruiting Test Design
  • 145. Control what you can controlWhich is, your set up.You wouldn’t go backpackingwithout checking your gear.Do it 24 hours in advance. Orelse.
  • 146. Prepare for what you can’t controlUser’s dog trips over powercordInternet affected by localweatherUser can’t installDoesn’t know how to install
  • 147. Mid-SessionTech Problems
  • 148. Live Recruiting ProblemsWhat if they gave a study and nobody came?
  • 149. Increase exposureWhen you’re not gettingenough viewsWhen the wrong type ofusers are responding
  • 150. Increase incentivesWhen it’s cheaperWhen you’re almost gettingenoughWhen segments are tough
  • 151. Switch methodsWhen base traffic level isn’t high enough
  • 152. Script Problems Cut the BS.
  • 153. ADVANCED TECHNIQUES One to many moderation Multi-threading
  • 154. PORTABLE RESEARCH
  • 155. BURN THE SCRIPTFIND OUT WHAT MATTERS.
  • 156. ADVANCED TECHNIQUES High Emotions
  • 157. TITLE TEXT
  • 158. MOBILE MOBILE THE FUTUREMOBILE MOBILEMOBILE MOBILEMOBILE MOBILE
  • 159. DUAL LENSSTREAMINGFIELD RESEARCH• Livestream• Flip Cam• It’s the view finder, silly• Actual duct tape
  • 160. DISCOUNTS!!Usertesting.com 1 free user. uxlisbon2012Usabilla 2 months free of any plan. Sign upfor an account, but cancel on the paymentpage. Send an email tosupport@usabilla.com and mention UXLXEthnio 1 month free any plan.WORKSHOPLOVE
  • 161. DISCOUNTS!!remoteresear.ch/lisbon(look in the slides)(for, well, that last slide)
  • 162. nate@boltpeters.com @boltron