Download as PDF, PPTX
![Quantitative Data
EFFECTIVENESS RATIOS EFFICIENCY
Your task is to
make an
Success users Only
appointment with
a pediatric Time (mm:ss)
arrhythmia Mean time per task 01:36
specialist at
Standard deviation 00:47
Children’s
Hospital. Once you
have found it,
please remember Clicks
or write down the
6
phone number you Mean clicks per task
(exact value: 6)
would call to make
4
an appointment by Standard deviation
(exact value: 4.4)
phone, as you will
be asked for that
information later
on.
Answers distribution
??? ! The phone number search has been...
*[1=Extremely difficult / 7=Extremely easy]
1
2%
(1)
2
2%
(1)
3
10%
(5)
4
18%
(9)
5
18%
(9)
6
24%
(12)
7
26%
(13)
N
50
Mean*
5.2
Friday, May 7, 2010](https://image.slidesharecdn.com/2010slidesv4-100507174309-phpapp01/75/ETL-May-2010-178-2048.jpg)
Uploaded bybolt peters
ETL May 2010
The document provides instructions and information for a remote research event called "Escape the Lab". It begins at 9:00 AM on Friday, May 7, 2010. Various hashtags and instructions are provided throughout for participating in the remote event. Information is also shared about conducting remote user research, including recruiting participants, moderating studies remotely, tools that can be used, and best practices.
![Coded Agents – with UiPath SDK + LangGraph [Virtual Hands-on Workshop]](https://cdn.slidesharecdn.com/ss_thumbnails/codedagentsdeck-251215155422-5497c599-thumbnail.jpg?width=640&height=640&fit=bounds)
ETL May 2010
- 1. Please stand by ESCAPE THE LAB WILL BEGIN AT 9:00 Friday, May 7, 2010
- 2. ESCAPE THE LAB LET’S GET STARTED MMKAY? Friday, May 7, 2010
- 3. Friday, May 7,2010
- 4. I’m Cyd He’s Nate Friday, May 7, 2010
- 5.
- 6. remote USABILITY Friday, May 7, 2010
- 7.
- 8.
- 9.
- 10.
- 11.
- 12.
- 13.
- 14.
- 15. HOW TO BE graceful When All Your TECHNOLOGY BREAKS Friday, May 7, 2010
- 16. PLEASE DON’T TRIP ON THE CABLES Friday, May 7, 2010
- 17. Follow the buddy system Friday, May 7, 2010
- 18. How about youall? Best remote research story Most important thing to get out of today? Biggest fear or skepticism about remote research? Friday, May 7, 2010
- 19. Go TO escapethelab.comNOW Friday, May 7, 2010
- 20. Friday, May 7,2010
- 21. Remote Research In person Remote Definition: Research where the participant is not sharing physical space with the researcher. % of qualitative research done remotely* *Source: BP’s best guess Friday, May 7, 2010
- 22. What we’ll covertoday Recruiting Moderated Unmoderated Results Pitfalls Friday, May 7, 2010
- 23. We shall begin... with a little experiment Friday, May 7, 2010
- 24. EXERCISE JUMP IN THE POOL Table groups Pick one moderator and one participant (they need to be in different locations) 10 minutes of moderating Friday, May 7, 2010
- 25. MODERATOR Instructions Your participant has a sheet of paper and a link Goal: find out what’s hardest about presenting physical task instructions online Hint: ask them to give you a play by play of what they are doing Follow along with your own paper NO helping, even if you could. Friday, May 7, 2010
- 26. It’s A RACE First participant to make an origami giraffe is the winner Friday, May 7, 2010
- 27. Friday, May 7,2010
- 28.
- 29. When does remotewin? Page 10-15 Friday, May 7, 2010
- 30. Native Environment Cheap ethnography, not cheap lab. Friday, May 7, 2010
- 31. Real Time Allows self-motivated tasks. Friday, May 7, 2010
- 32. Involved Observers Easier to collaborate in real time. Friday, May 7, 2010
- 33.
- 34. When does remotefail? Friday, May 7, 2010
- 35. Bandwidth Yes, it’s still an issue. Friday, May 7, 2010
- 36. Translation It’s expensive and difficult to do live. Friday, May 7, 2010
- 37. Minors Consent is a bear. Especially if they’re under 13. Friday, May 7, 2010
- 38. Faces We’ve all got webcams today. Most users don’t. Webcams are invasive. Friday, May 7, 2010
- 39. CASE STUDY: EpicFAIL Friday, May 7, 2010
- 40. What went wrong Coins were way more valuable than we expected People mis-assessed their English levels Bandwidth was really a problem Nobody plays Habbo when their parents are home to consent Friday, May 7, 2010
- 41. making the most Friday,May 7, 2010
- 42. TIME-AWARE Tasks Told to find purple pumps. Has to wear this for sister’s wedding this weekend. Doesn’t care. Whole different interaction. Friday, May 7, 2010
- 43. Real Time OBservers More on page 109-111 Friday, May 7, 2010
- 44. Change on thefly “No one seems engaged in the “Let’s add questions about who they laptop customizer” get recommendations from” Friday, May 7, 2010
- 45. Designing Remote Studies Friday,May 7, 2010
- 46. Friday, May 7,2010
- 47. Friday, May 7,2010
- 48. plan these things Friday,May 7, 2010
- 49. plan these things Method Recruiting Logistics Friday, May 7, 2010
- 50.
- 51. MODERATED must-haves Consent Friday, May 7, 2010
- 52. MODERATED must-haves Consent Tech set-up Friday, May 7, 2010
- 53. MODERATED must-haves Consent Tech set-up The beef Friday, May 7, 2010
- 54. MODERATED must-haves Consent Tech set-up The beef De-installation Friday, May 7, 2010
- 55. MODERATED must-haves Consent Tech set-up The beef De-installation Thanks & incentive Friday, May 7, 2010
- 56.
- 57. UNMODERATED must-haves Expectation setting Friday, May 7, 2010
- 58. UNMODERATED must-haves Expectation setting Instructions Friday, May 7, 2010
- 59. UNMODERATED must-haves Expectation setting Instructions The beef Friday, May 7, 2010
- 60. UNMODERATED must-haves Expectation setting Instructions The beef Thanks & incentive Friday, May 7, 2010
- 61.
- 62. Recruiting Same: Good “talkers” Same: Demographic fits Friday, May 7, 2010
- 63. Recruiting Same: Good “talkers” Same: Demographic fits Different: Relevant current tasks Different: Compatible tech Friday, May 7, 2010
- 64. Logistics: What &How Provide test target Access Credentials Friday, May 7, 2010
- 65. Logistics: Who &When How to connect with the user (we’ll discuss in Tools) Managing time zones Session length Friday, May 7, 2010
- 66. Paying Incentives Collect minimal personal info Deliver quickly Appropriate for location Friday, May 7, 2010
- 67. Advanced planning: hybrid studies Why use only one method? Some methods don’t tell you the why 3-5 remote interviews is all you need Friday, May 7, 2010
- 68.
- 69.
- 70. Recruiting for Remote Friday,May 7, 2010
- 71. We all needpeople Relevant tasks Compatible tech Good talkers Demographic fits In the moment Friday, May 7, 2010
- 72. Gold Standard Page 52 Live recruiting Get as close as you can Friday, May 7, 2010
- 73. Recruiting Options Ethnio Links or house ads to online forms Recruiting agencies Customer email lists Panels Craigslist/online ads Friday, May 7, 2010
- 74. use Ethnio when Website access Some budget Quotas are relatively simple Friday, May 7, 2010
- 75. use forms when You can place a link but not code Low budget Quotas are relatively complex Options: Google Docs Wufoo Friday, May 7, 2010
- 76. Google docs Friday, May7, 2010
- 77. use Agencies when No website access Higher budget Online recruiting failed Friday, May 7, 2010
- 78. Customer lists Sub-segments Ownership requirements No website access Friday, May 7, 2010
- 79.
- 80. Craigslist & forums When other methods don’t work No dough Takes a while Embarrassing to admit Friday, May 7, 2010
- 81. the Gold Standardagain Live recruiting Get as close as you can Friday, May 7, 2010
- 82. Live Recruiting Math (for interviews) Target: 6 recruits/hour Allow for: 1% response (low average) You Need: 8-10,000 uniques/day Friday, May 7, 2010
- 83. Page 56 Friday, May 7, 2010
- 84. Screener constraints Page 60-63 Keep it short Different from surveys Who gets paid? Friday, May 7, 2010
- 85. Remember the Point Tasks that align with research goals Technical compatibility Control demographics Friday, May 7, 2010
- 86.
- 87. Friday, May 7,2010
- 88. Your best questions Friday,May 7, 2010
- 89. GO EAT LUNCH Famished Friday, May 7, 2010
- 90. exercise We recommend: Split Pea Seduction, 6th & Minna Custom Burger, 7th & Natoma Tony Baloney’s, 7th & Howard Tin Vietnamese, Howard & 6th Harvest Market, Natoma & 8th Sightglass Coffee, 7th & Folsom Friday, May 7, 2010
- 91. exercise We recommend: Split Pea Seduction, 6th & Minna Custom Burger, 7th & Natoma Tony Baloney’s, 7th & Howard Tin Vietnamese, Howard & 6th Harvest Market, Natoma & 8th Sightglass Coffee, 7th & Folsom Friday, May 7, 2010
- 92. Please stand by ESCAPE THE LAB WILL RESUME AT 1:15 Friday, May 7, 2010
- 93. welcome back Friday, May7, 2010
- 94. Time for a contest! Tweet why you hate remote research #ETLAB Win a copy of Silverback Friday, May 7, 2010
- 95. Remote Research Toolkit MODERATED most of Chapter 8 Friday, May 7, 2010
- 96. 5 easy pieces Talk to the user See what they’re doing Take notes (optional) Involve observers (optional) Record audio & video (optional) Friday, May 7, 2010
- 97. Communication phone Skype Reliable Cheap or free Can be expensive Harder for users for international Can combine with Conference calls screensharing to needed for stress user’s observers connection Friday, May 7, 2010
- 98. Screensharing Standalone Integrated Adobe Connect UserVue (RIP) LiveLook FuzeMeeting (new) WebEx* GoToMeeting *WebEx also has mobile versions Friday, May 7, 2010
- 99. Adobe Connect Friday, May7, 2010
- 100.
- 101.
- 102.
- 103.
- 104. Taking Notes Second monitor if possible Excel Hack (for time codes) Friday, May 7, 2010
- 105. How to rigyour notes What the formulas look like Auto timestamp! Leave one column (A) empty for notes. Set the cells of the next column (B) to record the clock time of each entry in A. =IF(A2=“”,“”,IF(B2=“”,NOW(),B2)) Set the cells of the next column (C) to record the difference between the first entry and the current entry =IF(ISERROR(B2-$B$2),“”,B2-$B$2) Fill down (Ctrl-D). Expect to use up to 120 rows for a session. Don’t Hide column B. You’ll never need to see it write this down, it’s on page 104 Friday, May 7, 2010
- 106. Observer Involvement IM Web Chat Rooms Integrated Chat Passing Notes Friday, May 7, 2010
- 107. Recording Tools Hardware Software Phone Patch Camtasia (Windows) Cables iShowU HD (Mac) Friday, May 7, 2010
- 108. Phone patch setup Friday, May 7, 2010
- 109. Friday, May 7,2010
- 110. Simple Set-up Friday, May7, 2010
- 111.
- 112. Advanced Set-up Friday, May7, 2010
- 113. Friday, May 7,2010
- 114. exercise Roll your own What will be your base set-up? Write it down Friday, May 7, 2010
- 115. Friday, May 7,2010
- 116. Your version? Friday, May7, 2010
- 117.
- 118. Moderating for Remote Friday,May 7, 2010
- 119. Is it reallyany different? (Short answer: some.) Page 94-99 Friday, May 7, 2010
- 120. Question from theBP inbox yesterday It would have actually helped to know how many users the authors recommend for a tar* type of test. In a regular user test , many of us have seen how findings repeat a er 5 -8 users of each user type are observed. However, here we don't always know which task a user will be doing when we interrupt them. So in cases where we really are following users along as they complete what they came to do, how many useres should we target ? Given that the authors have been doing this for a while a recommendation from them on how many users to target and their notes on the topic would have been really interesting. *Time-Aware Research Friday, May 7, 2010
- 121. Making passionate tasks work Friday, May 7, 2010
- 122. Making passionate tasks work Recruit for relevant current tasks Script for customizable tasks Friday, May 7, 2010
- 123. Re-script for remote Friday,May 7, 2010
- 124. Re-script for remote “Ok, I’d like you to start from the homepage, pretend you are shopping for groceries for a party, and buy some paper plates.” Friday, May 7, 2010
- 125. MODERATED must-haves Also see 37-43 Friday, May 7, 2010
- 126. MODERATED must-haves Consent Also see 37-43 Friday, May 7, 2010
- 127. MODERATED must-haves Consent Also see Tech set-up 37-43 Friday, May 7, 2010
- 128. MODERATED must-haves Consent Also see Tech set-up 37-43 The beef Friday, May 7, 2010
- 129. MODERATED must-haves Consent Also see Tech set-up 37-43 The beef De-installation Friday, May 7, 2010
- 130. MODERATED must-haves Consent Also see Tech set-up 37-43 The beef De-installation Thanks & incentive Friday, May 7, 2010
- 131. Chances are they’ve never done this before Friday, May 7, 2010
- 132. What they worryabout Friday, May 7, 2010
- 133. Who are youand can I trust you? Friday, May 7, 2010
- 134. If you cansee my screen, can you see my face? Friday, May 7, 2010
- 135. Will you beable to take control of my computer? Friday, May 7, 2010
- 136. Will you helpme get the plug-in off my computer at the end? Friday, May 7, 2010
- 137. Will this workif I have a firewall/proxy/strict IT department? Friday, May 7, 2010
- 138. Is this goingto end up on YouTube? Friday, May 7, 2010
- 139. Will I reallyget the incentive if I do this? Friday, May 7, 2010
- 140. How long willit take? Friday, May 7, 2010
- 141. Is it OKif I have to take a quick call in the middle? Friday, May 7, 2010
- 142. Is it OKif my kids/dog/fratmates are making noise in the background? Friday, May 7, 2010
- 143. OXYMORON? Remote empathy Friday, May 7, 2010
- 144. Phone Interviewing Specifics Friday, May 7, 2010
- 145. Things sound leading Pitfalls Friday, May 7, 2010
- 146. Body language ishuge Actual language is important too Make sure you’re comfortable Friday, May 7, 2010
- 147. EXERCISE Practice screensharing Moderator, Participant, at least one observer Use the script in your packet as a starting point Get your participant all set up and ready to start an interview Friday, May 7, 2010
- 148. Friday, May 7,2010
- 149. How did itgo? Friday, May 7, 2010
- 150. Advanced Techniques High Emotions Friday, May 7, 2010
- 151. Advanced Techniques One to many moderation Multi-threading Friday, May 7, 2010
- 152. Future Frontiers Friday, May7, 2010
- 153.
- 154. Remote Research Toolkit UNMODERATED Friday, May 7, 2010
- 155. 3 parts Provide test target Record user behavior Capture comments (optional) Friday, May 7, 2010
- 156. AUTOMATED Tools Friday, May7, 2010
- 157. What do youwant to do? Get freeform data cheaply Quantify user behaviors Conceptual & IA testing Friday, May 7, 2010
- 158. This stuff ispretty cheap • Usabilla $0-950/yr • Optimal Workshop $559-1350/yr • Loop11 $350/study • OpenHallway $19-199/mo • UserTesting.com $39/user • Feedback Army $10/10 users Friday, May 7, 2010
- 159.
- 160.
- 161.
- 162.
- 163. Using your ownrecruits Set up an account Email support@usertesting.com with: the URL where your target lives the scenario (500 char limit) tasks (2000 char limit) up to 4 questions to be answered in writing at the end Request a UserTesting URL for your study Friday, May 7, 2010
- 164.
- 165.
- 166.
- 167.
- 168.
- 169.
- 170. Optimal Workshop Treejack Chalkmark Friday, May 7, 2010
- 171. EXERCISE Choose one of these tools Sign up for a test account Create a 15-minute test for any website you like Friday, May 7, 2010
- 172. Friday, May 7,2010
- 173.
- 174. Session Videos Heatmaps Notes Completion Rates Card Sort Groups Lions, Tigers, Bears... Friday, May 7, 2010
- 175. Telling a story If you have time-coded notes, you can make highlight videos If you have user comments, you can do tag analysis or make word clouds If you have numbers, you can compare to benchmarks If you have card groups you can do cluster analysis Friday, May 7, 2010
- 176. Case Study: Data Mismatch Friday, May 7, 2010
- 177. Qualitative Results 16 remote interviews Worst findability BP has ever observed Recommendation to burn it down and start over Friday, May 7, 2010
- 178. Quantitative Data EFFECTIVENESS RATIOS EFFICIENCY Your task is to make an Success users Only appointment with a pediatric Time (mm:ss) arrhythmia Mean time per task 01:36 specialist at Standard deviation 00:47 Children’s Hospital. Once you have found it, please remember Clicks or write down the 6 phone number you Mean clicks per task (exact value: 6) would call to make 4 an appointment by Standard deviation (exact value: 4.4) phone, as you will be asked for that information later on. Answers distribution ??? ! The phone number search has been... *[1=Extremely difficult / 7=Extremely easy] 1 2% (1) 2 2% (1) 3 10% (5) 4 18% (9) 5 18% (9) 6 24% (12) 7 26% (13) N 50 Mean* 5.2 Friday, May 7, 2010
- 179. What happened? Friday, May7, 2010
- 180. 32 page report,distilled Navigation Search ease Hospital ease of use of use reputation 1 2 3 4 5 6 7 terrible neutral UserZoom’s red excellent flag minimum Friday, May 7, 2010
- 181. What’s different IRL? Friday,May 7, 2010
- 182. Recording 1 more button to press Friday, May 7, 2010
- 183. Observer input One more stream on your screen (Possibly 2 or 3) Friday, May 7, 2010
- 184. Notes Tap, tap, tapping away Friday, May 7, 2010
- 185. So, what couldgo wrong? Friday, May 7, 2010
- 186. 3 points offailure Technology yours participant’s observer’s Recruiting Test Design Friday, May 7, 2010
- 187. Control what youcan control Which is, your set up. You wouldn’t go backpacking without checking your gear. Do it 24 hours in advance. Or else. Page 240-241 Friday, May 7, 2010
- 188. Prepare for what you can’t control User’s dog trips over power cord Internet affected by local weather User can’t install Doesn’t know how to install Friday, May 7, 2010
- 189. Mid-Session Tech Problems Friday, May 7, 2010
- 190. Live Recruiting Problems What if they gave a study and nobody came? Friday, May 7, 2010
- 191. Increase exposure When you’re not getting enough views When the wrong type of users are responding Friday, May 7, 2010
- 192. Increase incentives When it’s cheaper When you’re almost getting enough When segments are tough Friday, May 7, 2010
- 193. Switch methods When base traffic level isn’t high enough Friday, May 7, 2010
- 194. Script Problems Cut the BS. Friday, May 7, 2010
- 195. The bottom line If they’re not engaged, something’s wrong. Friday, May 7, 2010
- 196. Burn the script Find out what matters. Friday, May 7, 2010
- 197.
- 198. @boltpeters CYD@BOLTPETERS.COM NATE@BOLTPETERS.COM Friday, May 7, 2010
- 199.