Your SlideShare is downloading. ×

UX Lx 2012 - Nate Bolt

10,672
views

Published on

Published in: Technology

0 Comments
14 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
10,672
On Slideshare
0
From Embeds
0
Number of Embeds
11
Actions
Shares
0
Downloads
108
Comments
0
Likes
14
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Remote ResearchWorkshop
  • 2. Photo Credit: recados-orkut.net
  • 3. Remote Design Research In person RemoteDefinition:Research where theparticipant is not sharingphysical space with theresearcher. % of qualitative research done remotely* * Source: My wild guess
  • 4. #UXLX #RR
  • 5. Bayesian methods Adaptive1995Infer when users needed help Decision Theory &Animated pedagogical Systems Groupagency
  • 6. “ Misunderstood animated pedagogical agent or spawn of Satan? ”
  • 7. Bad Research
  • 8. 1) Academic Focus2) Lab-based3) False Premise
  • 9. THAT IS WHY WE WENT REMOTE
  • 10. Btw I’m Nate Bolt
  • 11. 12 YEARS293 STUDIES3,108 PARTICIPANTS (MODERATED)118 ORGANIZATIONS21,610 HOURS
  • 12. “SOCIAL IMPACT OF DIGITAL TECHNOLOGY”
  • 13. A DINOSAUR FAMILYEXPLAINS INFORMATION ARCHITECTURE
  • 14. SF to PARISIN TWO MINUTES
  • 15. jumboltron.com
  • 16. OMG PLEASEBUY THISBOOK
  • 17. Let’s jump right in
  • 18. What’s New with UX Research Tools?
  • 19. BUT FIRST
  • 20. Moderated versus Automated
  • 21. Moderated versus Automated10% 90%
  • 22. Moderated versus Automated Photo Credit: singularityhub.com
  • 23. ModeratedGoToMeeting 5.2Integrated RecordingThat’s really awesome
  • 24. ModeratedHuggingAt Mozillaand MailChimp Photo Credit: Diane Loviglio, Mozilla
  • 25. AutomatedUsabillaMobile
  • 26. HOW TO BEgraceful When All YourTECHNOLOGYBREAKS
  • 27. Follow the buddy system
  • 28. It’s laptop time
  • 29. remoteresear.ch
  • 30. remoteresear.ch/lisbon
  • 31. (BROWSER TIME)
  • 32. IN CASE THE INTERNET FAILSResearch experienceMost important thing to get out of today?Biggest fear or skepticism about remote research?
  • 33. Thanks, Volunteer!
  • 34. Four Takeaways1. See moderated method & recording with GoToMeeting2. Automated with Usabilla3.Self-moderated tablet hugging4.Quick incentive payment
  • 35. Eff you sales guy
  • 36. RecruitingWHAT WE’LL COVER TODAY Moderated Contextual Inquiry Automated Results Pitfalls
  • 37. SOME THOUGHTS ON WEB ANALYTICSWhat do helicopters do in action movies? Explode
  • 38. “ How much practical research experience do I need to have for this workshop? ” - Many People
  • 39. “ Will this be a long lecture or an actual workshop? ” - Some Guy on Twitter
  • 40. “ Can we do contextual inquiries remotely? ” - Nick Dunlavey | @ndunlavey
  • 41. The physical and computing ecosystem
  • 42. Now we do a little experiment
  • 43. EXERCISE #1 “Giraffe”Binder PDF Page 4Table groupsPick one moderator and one participant(they need to be in different locations) 10 minutes of moderating
  • 44. remoteresear.ch/lisbon“The Workshop Binder”
  • 45. MODERATOR InstructionsYour participant has a sheet of paper and that’s itGoal: find out what kind of physical taskinstructions onlineHint: ask them to give you a play by play of whatthey are doingTry and follow along with your own paper NO helping, even if you could.
  • 46. It’s A RACEYou only have 10 minutes to do thismax.
  • 47. Surprises?
  • 48. Was there emotion?
  • 49. WHEN DOES REMOTE WIN?
  • 50. Marv.Creative director UX designer product managerdeveloper IA co-founder.
  • 51. Marv’s Boss.He’s always on a plane and loves birds.
  • 52. Hey Marv, I want to savesome money on research.
  • 53. NATIVE ENVIRONMENTCHEAP ETHNOGRAPHY, NOT CHEAP LAB.
  • 54. TIME-AWARERESEARCH IS THE BEST
  • 55. I’LL TRADE YOU TIMEFOR PHYSICAL PRESENCE
  • 56. INVOLVED OBSERVERSEasier to collaborate in real time.
  • 57. GEOGRAPHYTITLE TEXT
  • 58. WHEN DOES REMOTE LOSE?
  • 59. Hey Marv, I want feedbackon our MMO video gamefrom 13yo English speakingkids in France & LatinAmerica with webcams.
  • 60. I guess, Boss.Might be hardto do withremote testing.
  • 61. BANDWIDTHYes, it’s an issue.
  • 62. TRANSLATIONIt’s expensive and difficult to do live.
  • 63. MINORS Consent is a bear.Especially if they’re under 13.
  • 64. FACESMost users don’t have webcams.Webcams are a little invasive.
  • 65. YOUR BOSS
  • 66. SUPER DUPER SECURITY
  • 67. AN EPIC FAIL.
  • 68. We basically didBad Research
  • 69. Kids will lie for coinsPeople mis-assessed their English levelsBandwidth was really a problemKids don’t play when their parents are home
  • 70. HOW TO FIND ACTUAL SPEAKERS OF YOUR LANGUAGE IN ANY COUNTRY:“ What happens when you let go of a balloon? ”“ Quest-ce qui se passe quand vous lâchez unballon? ”
  • 71. DESIGNINGREMOTE STUDIES
  • 72. Yo Marv, get me someinsight on WHY people arebailing out of checkout.
  • 73. Sure, Boss.Let’s combinemethods.Intercept &observe 10users live, anddo 10 self-moderated withusertesting.com
  • 74. TIME-AWARE TASKSTold to find purple pumps. Has to wear this for sister’s wedding this weekend. Doesn’t care. Whole different interview.
  • 75. HEART & SOUL OF REMOTE RESEARCHNot scheduling participants.
  • 76. REAL TIME OBSERVERS
  • 77. TITLE TEXT
  • 78. PROTOTYPES, WIREFRAMES, SKETCHES?
  • 79. HECK YES.
  • 80. RECRUITINGFOR REMOTE
  • 81. RECRUITING OPTIONSEthnioLinks or house ads to online formsRecruiting agenciesCustomer email listsPanelsCraigslist/online ads
  • 82. RECRUITING OPTIONS realnessGrab someone in the hallwayGrab someone who isn’t directly involvedin your projectIM your friends for a quick lookPanels and recruiting agenciesReal customers from marketing listsSite intercepts
  • 83. SITE ISN’T AN OPTIONGEO IPMEMBERS ONLY
  • 84. GOLD STANDARDLive recruitingGet as close as youcan
  • 85. USE ETHNIO WHENWebsite accessSome budgetQuotas are relatively simple
  • 86. USE FORMS WHENYou can place a link but not codeLow budgetQuotas are relatively complexOptions: Google Docs Wufoo
  • 87. USE AGENCIES WHENNo website accessPeople whine about first-timevisitorsHigher budgetOnline recruiting failed
  • 88. CUSTOMER LISTSSub-segmentsOwnershiprequirementsNo website access
  • 89. PANEL
  • 90. SORT OF
  • 91. CRAIGSLISTWhen other methods don’t workNo doughTakes a whileEmbarrassing to admit to your friends
  • 92. LIVE RECRUITING MATHTarget: 6 recruits/hourAllow for: 1% response (low averageYou Need: 10,000 uniques/day
  • 93. TITLE TEXT
  • 94. remoteresear.ch/samples “Online Consent”
  • 95. Paying IncentivesCollect minimalpersonal infoDeliver quicklyAppropriate forlocation
  • 96. Screener constraintsKeep it shortDifferent from surveysWho gets paid?
  • 97. MODERATED must-haves Consent Tech set-up The beef De-installation Thanks & incentive
  • 98. AUTOMATED must-haves Expectation setting Instructions The beef Thanks & incentive
  • 99. RecruitingSame: Good “talkers”Same: Demographic fitsDifferent: Relevant currenttasksDifferent: Compatible tech
  • 100. Logistics: What & HowProvide testtargetAccessCredentials
  • 101. Logistics: Who & WhenHow to connect with the userManaging time zonesSession length
  • 102. Paying IncentivesCollect minimalpersonal infoDeliver quicklyAppropriate forlocation
  • 103. STAGED IncentivesFor Contextual InquiryStart with $75, then at the end offer$25 more for environment photosSelf-made diary studies
  • 104. Advanced planning: hybrid studiesWhy use only one method?Some methods don’t tell you the why3-5 remote interviews is all you need
  • 105. EXERCISE #2okay, draft a screener Binder PDF Page 6 Use ethn.io or dev.ethn.io (beta) or wufoo Under 8 Questions Advanced Bonus: create an automated and moderated screener
  • 106. remoteresear.ch/lisbon“The Workshop Binder”
  • 107. Marv-o - what in the worlddoes automated/unmoderated/asynchronous researchmean?
  • 108. Sure, boss.There is anAVALANCHEof UX TOOLS.
  • 109. “At least a littlebehavioral”No surveys.No standard webanalytics.
  • 110. Since 2007
  • 111. Can tools make us better researchers?
  • 112. 5CATEGORIES
  • 113. ModeratedSelf-Moderated For A Long TimeAutomated Live ConceptualAutomated StaticWeb Analytics
  • 114. Talk To PeoplePeople Talk To A Machine For A Long TimeTasks On A Live Site Tasks On ConceptsTasks On ImagesFancy-Ass Analytics
  • 115. InsightAmmunition For Ad AgenciesMore Ammunition I.A. & NavigationStill More AmmunitionJust Better Analytics
  • 116. ModeratedSelf-Moderated For A Long TimeAutomated Live ConceptualAutomated Static Deep Behavioral UXWeb Analytics Insight
  • 117. Moderated OpinionsSelf-Moderated For A Long TimeAutomated Live ConceptualAutomated StaticWeb Analytics
  • 118. Moderated Ad Agencies Love ItSelf-Moderated For A Long TimeAutomated Live ConceptualAutomated StaticWeb Analytics
  • 119. Moderated Jared Spool Hates ItSelf-Moderated For A Long TimeAutomated Live ConceptualAutomated StaticWeb Analytics
  • 120. Moderated }Self-Moderated For A Long Time MobileAutomated Live ConceptualAutomated Static
  • 121. Remote Research ToolkitUNMODERATED (AKA AUTOMATED)
  • 122. 3 partsProvide test targetRecord user behaviorCapture comments (optional)
  • 123. USABILLAClicks & QuestionsHeatmapsYou do your own recruiting$49 - $199
  • 124. LOOP11 (UN-MODERATED) TITLE TEXT
  • 125. TITLE TEXT
  • 126. GOTOMEETING - $39/MO
  • 127. TITLE TEXT
  • 128. WHAT WE USE. A LOT.• Ethnio + GTM• Usabilla + Usertesting.com• Guerrilla mobile
  • 129. EXERCISE #3Either “unmoderated” or “practice interviewing” usertesting.com - 1 free user CODE: uxlisbon2012 usabilla - 2 months free (Sign up for an account, but cancel on the payment page. Send an email to support@usabilla.com and mention UXLX) ethnio - 1 month free or trial. CODE: WORKSHOPLOVE
  • 130. RESULTS
  • 131. REMEMBER, BEgraceful When All YourTECHNOLOGYBREAKS
  • 132. WHAT THEY WORRY ABOUT
  • 133. WHO ARE YOU AND CAN I TRUST YOU?
  • 134. IF YOU CAN SEE MY SCREEN, CAN YOU SEE MY FACE?
  • 135. WILL YOU BE ABLE TO TAKE CONTROL OF MY COMPUTER?
  • 136. WILL YOU HELP ME GET THE PLUG-INOFF MY COMPUTER AT THE END?
  • 137. WILL THIS WORK IF I HAVE AFIREWALL/PROXY/STRICT IT DEPARTMENT?
  • 138. IS THIS GOING TO END UP ON YOUTUBE?
  • 139. OXYMORON?REMOTE EMPATHY
  • 140. Taking NotesSecond monitor ifpossibleExcel Hack(remoteresear.ch/samples)
  • 141. Observer InvolvementIMCampfire (!)Integrated ChatPassing Notes
  • 142. HYBRID STUDIESNumbers actually do lie sometimesAnd they don’t tell you the why3-5 remote interviews is all you need
  • 143. So, what could go wrong?
  • 144. 3 points of failure Technology yours participant’s observer’s Recruiting Test Design
  • 145. Control what you can controlWhich is, your set up.You wouldn’t go backpackingwithout checking your gear.Do it 24 hours in advance. Orelse.
  • 146. Prepare for what you can’t controlUser’s dog trips over powercordInternet affected by localweatherUser can’t installDoesn’t know how to install
  • 147. Mid-SessionTech Problems
  • 148. Live Recruiting ProblemsWhat if they gave a study and nobody came?
  • 149. Increase exposureWhen you’re not gettingenough viewsWhen the wrong type ofusers are responding
  • 150. Increase incentivesWhen it’s cheaperWhen you’re almost gettingenoughWhen segments are tough
  • 151. Switch methodsWhen base traffic level isn’t high enough
  • 152. Script Problems Cut the BS.
  • 153. ADVANCED TECHNIQUES One to many moderation Multi-threading
  • 154. PORTABLE RESEARCH
  • 155. BURN THE SCRIPTFIND OUT WHAT MATTERS.
  • 156. ADVANCED TECHNIQUES High Emotions
  • 157. TITLE TEXT
  • 158. MOBILE MOBILE THE FUTUREMOBILE MOBILEMOBILE MOBILEMOBILE MOBILE
  • 159. DUAL LENSSTREAMINGFIELD RESEARCH• Livestream• Flip Cam• It’s the view finder, silly• Actual duct tape
  • 160. DISCOUNTS!!Usertesting.com 1 free user. uxlisbon2012Usabilla 2 months free of any plan. Sign upfor an account, but cancel on the paymentpage. Send an email tosupport@usabilla.com and mention UXLXEthnio 1 month free any plan.WORKSHOPLOVE
  • 161. DISCOUNTS!!remoteresear.ch/lisbon(look in the slides)(for, well, that last slide)
  • 162. nate@boltpeters.com @boltron

×