Your SlideShare is downloading. ×
Open Plans User Testing Workshop
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Saving this for later?

Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime - even offline.

Text the download link to your phone

Standard text messaging rates apply

Open Plans User Testing Workshop

594
views

Published on

Introductory user testing & user research workshop for government staff & civic hackers, delivered at Thoughtworks in NYC on 4/23/14

Introductory user testing & user research workshop for government staff & civic hackers, delivered at Thoughtworks in NYC on 4/23/14

Published in: Design, Technology

0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
594
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
20
Comments
0
Likes
2
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Testing & User Research Workshop
  • 2. Photo Credit: kaboodle.com
  • 3. “The best tool for resolving disputes within a design team, for making design decisions based on data rather than opinion, is sitting next to someone who is a real person who wants to accomplish something as they use your design to do it.” --Dana Chisnell, August 2, 2010
  • 4. Btw I’m Cyd Harrell I’m UX Evangelist at Code for America My favorite tools: empathy and duct tape
  • 5. Today’s Agenda 10:00-10:15: intros 10:15-11:30: core user research technique 11:45-12:30: your research set-up 1:30 - 2:45: recruiting participants 3:00 - 4:00: going beyond usability
  • 6. #1 Find out about the people who will use your thing #2 Align your core team around what you’re doing to do #3 Find out what your core team is up against in their work Research
  • 7. The hardest two things about research: ‣ providing the prompt ‣ collecting the response OversimplificationAlert
  • 8. The hardest two three things about research: ‣ providing the prompt ‣ collecting the response ‣ getting everyone to buy in OversimplificationAlert
  • 9. The hardest three four things: ‣ getting the right participants ‣ providing the prompt ‣ collecting the response ‣ getting everyone to buy in OversimplificationAlert
  • 10. o.k.soafewthingsarehard
  • 11. Let’s jump right in
  • 12. What’s the latest with UX Research?
  • 13. WHAT’S DIFFERENT IN 2014 Source: Pew Internet and American Life Project, January 2014
  • 14. WHAT’S DIFFERENT IN 2014 Source: Knight Foundation Digital Access Update, February 2014
  • 15. TOOLKIT
  • 16. TOOLKIT
  • 17. REMEMBER BE graceful When All Your TECHNOLOGY BREAKS
  • 18. uximresearch.tumblr.com
  • 19. Now we do a little experiment
  • 20. EXERCISE #1 “Giraffe” ‣ 2 groups per table ‣ Pick one moderator and one participant ‣ 10 minutes of observation
  • 21. MODERATOR Instructions Your participant has a sheet of paper and that’s it Goal: find out what kind of physical task instructions work Hint: ask them to give you a play by play of what they are doing Try and follow along with your own paper NO helping, even if you could.
  • 22. It’s A RACE You only have 10 minutes to do this max.
  • 23. Surprises?
  • 24. Useful phrases 1 the approach “Hi, would you like to participate in a study today?” “I’m working to improve XXXX for citizens and I’m collecting feedback.”
  • 25. Useful phrases 2 reassurance “This isn’t a test of you, it’s a test of the system” “There are no right or wrong answers” “Positive and negative feedback is equally valuable” “Nothing you say is going to hurt my feelings”
  • 26. Useful phrases 3 instructions “I just need you to be yourself and act as you naturally would” “One little thing I’d like you to do differently is talk aloud as you use this today”
  • 27. Useful phrases 4,5,6,7 keeping the conversation going “mm-hm” “oh interesting” “sure, I hear you” “right” “ok” “uh-huh” (lather, rinse, repeat)
  • 28. Useful phrases 8 probes “Can you tell me a little more about that?” “When you did X a minute ago, can you tell me how you made that choice?” “Talk me through what happened there.”
  • 29. Useful phrases 9 ratings “If you had to give this a letter grade, you know A, B, C, D, F like we used to get in school, what would you give it?”
  • 30. Useful phrases 10 self-explanatory “Thank you so much, I really appreciate your taking the time to talk to me.”
  • 31. DESIGNINGUSER STUDIES
  • 32. FIND THE RIGHT PARTICIPANTS uximresearch.tumblr.com Photo Credit: Ed Yourdon
  • 33. Photo from blog post by Dustin Larimer http://www.dustinlarimer.com/design-management-summer-camp/ OFFER A TASK
  • 34. SCRIPT FOR CUSTOMIZATION “Please look for a pair of pink pumps in your size.” “Please look for a pair of shoes you’d like to buy”
  • 35. MODERATED must-haves Consent Tech set-up The beef De-installation Thanks & incentive
  • 36. AUTOMATED must-haves Expectation setting Instructions The beef Thanks & incentive
  • 37. Photo from blog post by Dustin Larimer http://www.dustinlarimer.com/design-management-summer-camp/ STREAM & RECORD WHAT HAPPENS.
  • 38. BREAK Photo Credit: Sorin Petcu
  • 39. Testing & User Research Part 2: your set-up
  • 40. What’s New with ResearchTools?
  • 41. Remote Design Research Definition: Research where the participant is not sharing physical space with the researcher. % of qualitative research done remotely* * Source: My wild guess In person Remote (2008) Remote (2012)
  • 42. Screen Easier Sharing
  • 43. Awesome compared to this!
  • 44. Prototype So Many Tools Flinto
  • 45. Experience Sampling Task-driven iOS & Android
  • 46. Mobile Client Simultaneous Website Testing
  • 47. WHOSE DEVICE?
  • 48. Photo Credit: groundsel, inju Pro: ‣ better context ‣ more comfortable users Con: ‣ extra installation time ‣ less control
  • 49. Pro tips for working with users’ own phones: ‣ Have a backup iPhone and backup Android device on hand. ‣ Make a power strip part of your testing station ‣ Have chargers for all the phones you’re expecting ‣ Instruct participants to install necessary apps in advance, but leave time in case they don’t ‣ Adjust the screen brightness before you start each session
  • 50. Pro tips for offering lab devices: ‣ Have a backup iPhone and backup Android device on hand. ‣ Charge devices between every session. ‣ Remove passwords, lock screens, etc. ‣ Dry-run every aspect of the test, since you have the opportunity. uximresearch.tumblr.com
  • 51. Physical Setup for MobileTests
  • 52. Photo Credit: Diane Loviglio, Mozilla
  • 53. RealTalk About Holding Phones Source: Steve Hoober in UX Matters, February 2013
  • 54. TestTargets
  • 55. PROTOTYPES,WIREFRAMES, SKETCHES?
  • 56. HECKYES.
  • 57. Photo from blog post by Dustin Larimer http://www.dustinlarimer.com/design-management-summer-camp/ PAPER PROTOTYPES, POST-IT STYLE
  • 58. Photo from UX Magazine article by Tania Lang PAPER PROTOTYPES, WITH SCROLLING
  • 59. Lo-fi clickable prototypes
  • 60. Photo from UX Magazine article by Tania Lang Prototyping Apps for Higher Fidelity Proto.io Fluid Flinto PhoneGap
  • 61. Photo from UX Magazine article by Tania Lang Cool Hack: Screenshots in an album or photostream
  • 62. The hardest two things about research: ‣ providing the prompt ‣ collecting the response InTheLab
  • 63. Photo Credit: Hans Makkee high resolution portable streaming camera
  • 64. Mobile Screen Recording
  • 65. Source: OpenDeviceLab.com Device Labs
  • 66. uximresearch.tumblr.com
  • 67. EXERCISE #2 “RollYour Own” ‣ Share out
  • 68. Taking Notes Second monitor if possible Excel Hack (remoteresear.ch/samples)
  • 69. Observer Involvement IM Campfire (!) Integrated Chat Passing Notes
  • 70. Questions?
  • 71. Let’s do lunch
  • 72. Testing & User Research Part 3: recruiting participants
  • 73. RECRUITING OPTIONS ‣ Grab someone in the hallway ‣ Grab someone who isn’t involved in your project ‣ IM your outside friends for a quick look ‣ Panels and recruiting agencies ‣ Real citizens from email & consent lists ‣ Physical & web intercepts realness
  • 74. TWITTER RECRUITING
  • 75. ANATOMY OF A RECRUITING TWEET @person we’re interviewing [job or characteristic] about [topic] on [dates] (paid research) -interested? [LINK] I’m looking for [job or characteristic(s)] for a [research activity] on [dates] incentive $X [LINK] please RT Pro tip: edit your profile to mention “UX Researcher” and “Current Study: Martian Life”
  • 76. BIGGER THAN A TWEET ‣ Why are you doing this work? ‣ What exactly are you asking? (How much time, where, when) ‣ Who do you need? (tasks or demographics) ‣ Who is asking? ‣ Who vouches for you? (official sponsor or trusted leader) ‣ Will there be a reward or incentive? ‣ Can this be confidential or anonymous?
  • 77. Screener constraints Keep it short Always ask an open-ended question Different from surveys Who gets paid?
  • 78. MOBILE FORMS ARE GETTING WAY BETTER!
  • 79. PANELS
  • 80. “What happens when you let go of a balloon?” PRO TIP: HOW TO FIND ACTUAL SPEAKERS OF A LANGUAGE: “ Qu'est-ce qui se passe quand vous lâchez un ballon? ”
  • 81. WHAT PARTICIPANTS WORRY ABOUT
  • 82. Who are you and who sent you? (can I trust you?)
  • 83. Will participating affect my actual outcome?
  • 84. Is this going to end up on YouTube?
  • 85. remoteresear.ch/samples “Online Consent”
  • 86. Paying Incentives Collect minimal personal info Deliver quickly Appropriate for location
  • 87. Paying Incentives “Free cookies and lemonade for your city feedback”
  • 88. EXERCISE ‣ Consider format (poster, landing page, online form) ‣ Cover the questions draft a recruiting appeal
  • 89. ‣ Why are you doing this work? ‣ What exactly are you asking? (How much time, where, when) ‣ Who do you need? (tasks or demographics) ‣ Who is asking? ‣ Who vouches for you? (official sponsor or trusted leader) ‣ Will there be a reward or incentive? ‣ Can this be confidential or anonymous?
  • 90. share out
  • 91. BREAK Photo Credit: Mrs Magic
  • 92. User Testing & Research Part 4: going beyond usability
  • 93. WHAT IF IT’S NOT USABILITY?
  • 94. EXPERIENCE SAMPLING
  • 95. EXPERIENCE SAMPLING uximresearch.tumblr.com
  • 96. Cyd’sResults
  • 97. Cyd’sResults
  • 98. CONCEPT & NAVIGATION TESTING: EXERCISE
  • 99. UNMODERATED STUDIES
  • 100. 3 parts Provide test target Record user behavior Capture comments (optional)
  • 101. Prototype Unmoderated Testing
  • 102. Mobile Diary Studies iOS & Android
  • 103. Results
  • 104. Results
  • 105. Results
  • 106. CAPTURE WHAT HAPPENS.
  • 107. Surveys via SMS
  • 108. About getting buy-in AFewWords
  • 109. How “Testing” Is Perceived
  • 110. Numbers (& Numbers People) AreYour Friends Photo Credit: Search Influence on Flickr
  • 111. Video Is Another Friend Photo Credit: Search Influence on Flickr
  • 112. Try asking for One Day $200 PeopleToWatch Photo Credit: Search Influence on Flickr
  • 113. “Fix the machine, not the person” -- Aaron Swartz, 9.25.12 ActualAdvice
  • 114. There’s always a way
  • 115. Questions?
  • 116. cyd@codeforamerica.org @cydharrell
  • 117. Thanks for coming. Go run some awesome research. Photo Credit: Wade Rockett on Flickr