UX Lx 2012 - Nate Bolt
Upcoming SlideShare
Loading in...5
×
 

UX Lx 2012 - Nate Bolt

on

  • 9,887 views

 

Statistics

Views

Total Views
9,887
Views on SlideShare
2,764
Embed Views
7,123

Actions

Likes
14
Downloads
98
Comments
0

18 Embeds 7,123

http://econsultancy.com 5202
https://econsultancy.com 1231
http://www.frankwatching.com 635
http://lanyrd.com 22
https://twitter.com 13
http://www.365online.nu 3
https://www.google.co.in 2
http://www.newsblur.com 2
https://www.google.ca 2
http://newsblur.com 2
http://bundlr.com 2
http://www.hanrss.com 1
http://dev.newsblur.com 1
http://www.pinterest.com 1
https://www.google.com 1
https://www.google.fr 1
http://search.daum.net 1
http://feeds2read.net 1
More...

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

UX Lx 2012 - Nate Bolt UX Lx 2012 - Nate Bolt Presentation Transcript

  • Remote ResearchWorkshop
  • Photo Credit: recados-orkut.net
  • Remote Design Research In person RemoteDefinition:Research where theparticipant is not sharingphysical space with theresearcher. % of qualitative research done remotely* * Source: My wild guess
  • #UXLX #RR
  • Bayesian methods Adaptive1995Infer when users needed help Decision Theory &Animated pedagogical Systems Groupagency
  • “ Misunderstood animated pedagogical agent or spawn of Satan? ”
  • Bad Research
  • 1) Academic Focus2) Lab-based3) False Premise
  • THAT IS WHY WE WENT REMOTE
  • Btw I’m Nate Bolt
  • 12 YEARS293 STUDIES3,108 PARTICIPANTS (MODERATED)118 ORGANIZATIONS21,610 HOURS
  • “SOCIAL IMPACT OF DIGITAL TECHNOLOGY”
  • A DINOSAUR FAMILYEXPLAINS INFORMATION ARCHITECTURE
  • SF to PARISIN TWO MINUTES
  • jumboltron.com
  • OMG PLEASEBUY THISBOOK
  • Let’s jump right in
  • What’s New with UX Research Tools?
  • BUT FIRST
  • Moderated versus Automated
  • Moderated versus Automated10% 90%
  • Moderated versus Automated Photo Credit: singularityhub.com
  • ModeratedGoToMeeting 5.2Integrated RecordingThat’s really awesome
  • ModeratedHuggingAt Mozillaand MailChimp Photo Credit: Diane Loviglio, Mozilla
  • AutomatedUsabillaMobile
  • HOW TO BEgraceful When All YourTECHNOLOGYBREAKS
  • Follow the buddy system
  • It’s laptop time
  • remoteresear.ch
  • remoteresear.ch/lisbon
  • (BROWSER TIME)
  • IN CASE THE INTERNET FAILSResearch experienceMost important thing to get out of today?Biggest fear or skepticism about remote research?
  • Thanks, Volunteer!
  • Four Takeaways1. See moderated method & recording with GoToMeeting2. Automated with Usabilla3.Self-moderated tablet hugging4.Quick incentive payment
  • Eff you sales guy
  • RecruitingWHAT WE’LL COVER TODAY Moderated Contextual Inquiry Automated Results Pitfalls
  • SOME THOUGHTS ON WEB ANALYTICSWhat do helicopters do in action movies? Explode
  • “ How much practical research experience do I need to have for this workshop? ” - Many People
  • “ Will this be a long lecture or an actual workshop? ” - Some Guy on Twitter
  • “ Can we do contextual inquiries remotely? ” - Nick Dunlavey | @ndunlavey
  • The physical and computing ecosystem
  • Now we do a little experiment
  • EXERCISE #1 “Giraffe”Binder PDF Page 4Table groupsPick one moderator and one participant(they need to be in different locations) 10 minutes of moderating
  • remoteresear.ch/lisbon“The Workshop Binder”
  • MODERATOR InstructionsYour participant has a sheet of paper and that’s itGoal: find out what kind of physical taskinstructions onlineHint: ask them to give you a play by play of whatthey are doingTry and follow along with your own paper NO helping, even if you could.
  • It’s A RACEYou only have 10 minutes to do thismax.
  • Surprises?
  • Was there emotion?
  • WHEN DOES REMOTE WIN?
  • Marv.Creative director UX designer product managerdeveloper IA co-founder.
  • Marv’s Boss.He’s always on a plane and loves birds.
  • Hey Marv, I want to savesome money on research.
  • NATIVE ENVIRONMENTCHEAP ETHNOGRAPHY, NOT CHEAP LAB.
  • TIME-AWARERESEARCH IS THE BEST
  • I’LL TRADE YOU TIMEFOR PHYSICAL PRESENCE
  • INVOLVED OBSERVERSEasier to collaborate in real time.
  • GEOGRAPHYTITLE TEXT
  • WHEN DOES REMOTE LOSE?
  • Hey Marv, I want feedbackon our MMO video gamefrom 13yo English speakingkids in France & LatinAmerica with webcams.
  • I guess, Boss.Might be hardto do withremote testing.
  • BANDWIDTHYes, it’s an issue.
  • TRANSLATIONIt’s expensive and difficult to do live.
  • MINORS Consent is a bear.Especially if they’re under 13.
  • FACESMost users don’t have webcams.Webcams are a little invasive.
  • YOUR BOSS
  • SUPER DUPER SECURITY
  • AN EPIC FAIL.
  • We basically didBad Research
  • Kids will lie for coinsPeople mis-assessed their English levelsBandwidth was really a problemKids don’t play when their parents are home
  • HOW TO FIND ACTUAL SPEAKERS OF YOUR LANGUAGE IN ANY COUNTRY:“ What happens when you let go of a balloon? ”“ Quest-ce qui se passe quand vous lâchez unballon? ”
  • DESIGNINGREMOTE STUDIES
  • Yo Marv, get me someinsight on WHY people arebailing out of checkout.
  • Sure, Boss.Let’s combinemethods.Intercept &observe 10users live, anddo 10 self-moderated withusertesting.com
  • TIME-AWARE TASKSTold to find purple pumps. Has to wear this for sister’s wedding this weekend. Doesn’t care. Whole different interview.
  • HEART & SOUL OF REMOTE RESEARCHNot scheduling participants.
  • REAL TIME OBSERVERS
  • TITLE TEXT
  • PROTOTYPES, WIREFRAMES, SKETCHES?
  • HECK YES.
  • RECRUITINGFOR REMOTE
  • RECRUITING OPTIONSEthnioLinks or house ads to online formsRecruiting agenciesCustomer email listsPanelsCraigslist/online ads
  • RECRUITING OPTIONS realnessGrab someone in the hallwayGrab someone who isn’t directly involvedin your projectIM your friends for a quick lookPanels and recruiting agenciesReal customers from marketing listsSite intercepts
  • SITE ISN’T AN OPTIONGEO IPMEMBERS ONLY
  • GOLD STANDARDLive recruitingGet as close as youcan
  • USE ETHNIO WHENWebsite accessSome budgetQuotas are relatively simple
  • USE FORMS WHENYou can place a link but not codeLow budgetQuotas are relatively complexOptions: Google Docs Wufoo
  • USE AGENCIES WHENNo website accessPeople whine about first-timevisitorsHigher budgetOnline recruiting failed
  • CUSTOMER LISTSSub-segmentsOwnershiprequirementsNo website access
  • PANEL
  • SORT OF
  • CRAIGSLISTWhen other methods don’t workNo doughTakes a whileEmbarrassing to admit to your friends
  • LIVE RECRUITING MATHTarget: 6 recruits/hourAllow for: 1% response (low averageYou Need: 10,000 uniques/day
  • TITLE TEXT
  • remoteresear.ch/samples “Online Consent”
  • Paying IncentivesCollect minimalpersonal infoDeliver quicklyAppropriate forlocation
  • Screener constraintsKeep it shortDifferent from surveysWho gets paid?
  • MODERATED must-haves Consent Tech set-up The beef De-installation Thanks & incentive
  • AUTOMATED must-haves Expectation setting Instructions The beef Thanks & incentive
  • RecruitingSame: Good “talkers”Same: Demographic fitsDifferent: Relevant currenttasksDifferent: Compatible tech
  • Logistics: What & HowProvide testtargetAccessCredentials
  • Logistics: Who & WhenHow to connect with the userManaging time zonesSession length
  • Paying IncentivesCollect minimalpersonal infoDeliver quicklyAppropriate forlocation
  • STAGED IncentivesFor Contextual InquiryStart with $75, then at the end offer$25 more for environment photosSelf-made diary studies
  • Advanced planning: hybrid studiesWhy use only one method?Some methods don’t tell you the why3-5 remote interviews is all you need
  • EXERCISE #2okay, draft a screener Binder PDF Page 6 Use ethn.io or dev.ethn.io (beta) or wufoo Under 8 Questions Advanced Bonus: create an automated and moderated screener
  • remoteresear.ch/lisbon“The Workshop Binder”
  • Marv-o - what in the worlddoes automated/unmoderated/asynchronous researchmean?
  • Sure, boss.There is anAVALANCHEof UX TOOLS.
  • “At least a littlebehavioral”No surveys.No standard webanalytics.
  • Since 2007
  • Can tools make us better researchers?
  • 5CATEGORIES
  • ModeratedSelf-Moderated For A Long TimeAutomated Live ConceptualAutomated StaticWeb Analytics
  • Talk To PeoplePeople Talk To A Machine For A Long TimeTasks On A Live Site Tasks On ConceptsTasks On ImagesFancy-Ass Analytics
  • InsightAmmunition For Ad AgenciesMore Ammunition I.A. & NavigationStill More AmmunitionJust Better Analytics
  • ModeratedSelf-Moderated For A Long TimeAutomated Live ConceptualAutomated Static Deep Behavioral UXWeb Analytics Insight
  • Moderated OpinionsSelf-Moderated For A Long TimeAutomated Live ConceptualAutomated StaticWeb Analytics
  • Moderated Ad Agencies Love ItSelf-Moderated For A Long TimeAutomated Live ConceptualAutomated StaticWeb Analytics
  • Moderated Jared Spool Hates ItSelf-Moderated For A Long TimeAutomated Live ConceptualAutomated StaticWeb Analytics
  • Moderated }Self-Moderated For A Long Time MobileAutomated Live ConceptualAutomated Static
  • Remote Research ToolkitUNMODERATED (AKA AUTOMATED)
  • 3 partsProvide test targetRecord user behaviorCapture comments (optional)
  • USABILLAClicks & QuestionsHeatmapsYou do your own recruiting$49 - $199
  • LOOP11 (UN-MODERATED) TITLE TEXT
  • TITLE TEXT
  • GOTOMEETING - $39/MO
  • TITLE TEXT
  • WHAT WE USE. A LOT.• Ethnio + GTM• Usabilla + Usertesting.com• Guerrilla mobile
  • EXERCISE #3Either “unmoderated” or “practice interviewing” usertesting.com - 1 free user CODE: uxlisbon2012 usabilla - 2 months free (Sign up for an account, but cancel on the payment page. Send an email to support@usabilla.com and mention UXLX) ethnio - 1 month free or trial. CODE: WORKSHOPLOVE
  • RESULTS
  • REMEMBER, BEgraceful When All YourTECHNOLOGYBREAKS
  • WHAT THEY WORRY ABOUT
  • WHO ARE YOU AND CAN I TRUST YOU?
  • IF YOU CAN SEE MY SCREEN, CAN YOU SEE MY FACE?
  • WILL YOU BE ABLE TO TAKE CONTROL OF MY COMPUTER?
  • WILL YOU HELP ME GET THE PLUG-INOFF MY COMPUTER AT THE END?
  • WILL THIS WORK IF I HAVE AFIREWALL/PROXY/STRICT IT DEPARTMENT?
  • IS THIS GOING TO END UP ON YOUTUBE?
  • OXYMORON?REMOTE EMPATHY
  • Taking NotesSecond monitor ifpossibleExcel Hack(remoteresear.ch/samples)
  • Observer InvolvementIMCampfire (!)Integrated ChatPassing Notes
  • HYBRID STUDIESNumbers actually do lie sometimesAnd they don’t tell you the why3-5 remote interviews is all you need
  • So, what could go wrong?
  • 3 points of failure Technology yours participant’s observer’s Recruiting Test Design
  • Control what you can controlWhich is, your set up.You wouldn’t go backpackingwithout checking your gear.Do it 24 hours in advance. Orelse.
  • Prepare for what you can’t controlUser’s dog trips over powercordInternet affected by localweatherUser can’t installDoesn’t know how to install
  • Mid-SessionTech Problems
  • Live Recruiting ProblemsWhat if they gave a study and nobody came?
  • Increase exposureWhen you’re not gettingenough viewsWhen the wrong type ofusers are responding
  • Increase incentivesWhen it’s cheaperWhen you’re almost gettingenoughWhen segments are tough
  • Switch methodsWhen base traffic level isn’t high enough
  • Script Problems Cut the BS.
  • ADVANCED TECHNIQUES One to many moderation Multi-threading
  • PORTABLE RESEARCH
  • BURN THE SCRIPTFIND OUT WHAT MATTERS.
  • ADVANCED TECHNIQUES High Emotions
  • TITLE TEXT
  • MOBILE MOBILE THE FUTUREMOBILE MOBILEMOBILE MOBILEMOBILE MOBILE
  • DUAL LENSSTREAMINGFIELD RESEARCH• Livestream• Flip Cam• It’s the view finder, silly• Actual duct tape
  • DISCOUNTS!!Usertesting.com 1 free user. uxlisbon2012Usabilla 2 months free of any plan. Sign upfor an account, but cancel on the paymentpage. Send an email tosupport@usabilla.com and mention UXLXEthnio 1 month free any plan.WORKSHOPLOVE
  • DISCOUNTS!!remoteresear.ch/lisbon(look in the slides)(for, well, that last slide)
  • nate@boltpeters.com @boltron