Rapid usability testing


Published on

Published in: Technology, Education
1 Like
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Rapid usability testing

  2. 2. Lisa Rex @lisarex#frontendunited
  3. 3. AND YOU ARE?
  4. 4. AGENDAThe basicsPreparation for a studyConducting a usability studyAnalysis & debrief
  5. 5. THE BASICS
  6. 6. WHY ARE USABILITY STUDIES IMPORTANT? http://www.flickr.com/photos/0olong/479270001/
  7. 7. WHAT DOES YOUR INTERFACE SAY?We hope you have excellent eyesight. We hope you have excellent eyesight. We hope you have excellent eyesight.
  8. 8. Yeah, hmm, we don’t know what that error message is either. http://www.flickr.com/photos/kk/206835799/
  9. 9. Sorry we couldnt be bothered to test our site actually makes sense to anyone outside our organization. http://www.flickr.com/photos/ballookey/222118221/
  10. 10. Ahttp://www.flickr.com/photos/chrisinplymouth/4456434875
  11. 11. Bhttp://www.flickr.com/photos/epsos/4444576795/
  12. 12. Chttp://www.flickr.com/photos/whistlepunch/5176579823/
  13. 13. “Testing with one user is 100% better than testing with none.” - Steve Krug
  14. 14. WHAT IS RAPID USABILITY Stakeholder buy-in. Observe real users doing real tasks. Quick turnaround. No. Reports. Ever.
  16. 16. DO NOT BE ALARMED Preparation will take longer than steps 2 & 3 combined
  17. 17. PREP: GOALS Talk to stakeholders:“What are the business goals of the study?” “What do we need to learn?” “When do we need it?” “What’s our budget?”
  18. 18. PREP: WHAT TO TESTIdeaInformation architecturePrototypeStaging siteProduction site You can test at any point during the project lifecycle!
  19. 19. PREP: METHODOLOGYModerated vs unmoderatedOnline tool vs own setupRemote with screenshare vs in personOne-on-one vs many-to-one
  20. 20. PREP: METHODOLOGYModerated vs unmoderatedOnline tool vs own setupRemote with screenshare vs in personOne-on-one vs many-to-one
  21. 21. PREP: METHODOLOGYModerated vs unmoderatedOnline tool vs own setupRemote with screenshare vs in person
  22. 22. PREP: WHO TO TEST Type of skillset Expertise New users vs. current users 5 people per study is ideal
  23. 23. PREP: TESTING ENVIRONMENTHow will they access the site?Content and user account setup is needed?Do you need to set up separate sites for each? Or re-setbetween sessions? (hint: use Drush)
  24. 24. PREP: STUDY OUTLINEOverviewParticipant briefingPre-session questionsScenarioLogin detailsTasks and questionsPost-session questions
  25. 25. PREP: WRITING TASKSDo not use words that are in the UIPhrase things in the way people think and in plain languageWrite in cues for moderator (e.g. “Begin recording” or“Observe how they ____”)Dependencies between the tasks & task orderWrite moderated assistsAsk them to rate their experience
  26. 26. CONTENT CREATION UX STUDY PLAN http://groups.drupal.org/node/223959
  27. 27. PREP: THE SCREENERDevelop a screener to weedout the ‘wrong’ participantFind out when they’reavailable (use Doodle?)Set expectations withparticipants (when, howlong, compensation)
  28. 28. PREP: RECRUITINGWhen: 0-2 days before you want to talk to them, ifmoderatedHow: social networks, online free ads, groups ororganizations, meetups, even family and friends“General” participant profile: can use a bar or cafeSend them to your screener, e.g. bit.ly/example
  29. 29. PREP: SCHEDULINGConfirm timeSend calendar invite(s)Remind them of expectationsThank them
  30. 30. PREP: OBSERVATION FORMPriority Participants who had the issue (frequency)Issue summary (quotes!) Other things: totals,Link to screenshot estimated fix date, commentsTask name
  31. 31. PREP: OBSERVERS 1 = catastrophic 2 = serious 3= important 4 = minor 5 = observation 6 = positive 7.x = resolved
  33. 33. MODERATED STUDY She’s working on a site that is highly focused on a specific type of user: the Drupal developer. She needs feedback from Drupal developers... anyone else won’t provide the knowledge that is needed to improve the site.15-60 minute session | 5 participants | 3-5 tasks | 3-5 pre- and post-session questions
  34. 34. CONDUCTING A MODERATED STUDYUse your study outline prepared scriptParticipant needs come above everything elseGet good information from them Gonna talk about this in detail at 13:45!
  35. 35. HAVE FUN WITH IT!
  36. 36. THE RESULTSAnalyze and add to your observation spreadsheet
  37. 37. TREEJACK STUDY He’s working on a Drupal distribution that is being redeveloped for Drupal 7. The previous distribution has some issues, so will users understand the new information architecture? He needs feedback from both new and current users.10 minute session | 40-400 (120 optimum) participants | 3-10 tasks | 3-5 pre- and post-session questions
  38. 38. TREEJACKWhen? Anytime...tree is isolated• Where would they go to do ___?• Can they find a specific piece of content?• Do navigation labels make sense?• Are my subsections well-organized?• How does Tree A compare to Tree B?
  39. 39. BUILDING THE TREEBuild the structure in a spreadsheet, one column for eachlevelInclude sample contentInclude controls within the site, if that’s what you want totest (e.g. “add comment”)Only the leaves of the tree are selectable as answers,branches are not.
  41. 41. TREEJACK SETUPThink about what you want to validate, then write the tasksDecide the right answers; used to calculate the success ofyour tasksCustomize pre-session and post-session questionmessage options
  42. 42. TREEJACK PROTIPSRun a few moderated trials with people unfamiliar with yoursite, to catch the WTFs ....Set your ‘right’ answers at the very end, as you will blowthem away if you rebuild the treeConsider hacky chunking of optionsDon’t allow skip tasks, but include [None of these]
  43. 43. THE RESULTSWhat you’ll get is several graphs & complex visuals torepresent the paths people took, the success of each task, etc Analyze and add to your observation spreadsheet
  44. 44. USERTESTING.COM STUDY He works at a university library. He isn’t a usability expert but wants to make sure that the students can perform their primary tasks easily and quickly (like finding a book, making the reservations for a meeting room). Unfortunately, he has little time and little budget. 15 minute session | 5-8 participants | 4-5 tasks | 3-4 post-session questions
  45. 45. USERTESTING.COM SETUPReally flexible: show them a URL to live site or prototypeGive them detailed instructions at each stepWrite hints, just in case they missed itPre-session and post-session question and messageoptions
  46. 46. Tip: Run a pilot | Ask for refund
  47. 47. THE RESULTSWhat you’ll get is recordings of the tester’s browser and voice, plus the answers they fill in at the end. Analyze and add to your observation spreadsheet
  48. 48. THE ANALYSIS & DEBRIEF Positive comments Have a scribe, if no other stakeholders are presentIssues/Problems/Bugs and the session isn’t recorded Observations If recording, revisit the Quotes tapes and make notesAnnotated screenshotsTake notes during the session
  49. 49. ANALYSISTeam (or just you) should review observations andcondense, if possibleCategorize the positive negative/observationsSeverity: Low/Medium/ High or 1-5Frequency: Rare/ Common / Very CommonScope: Local/ Global
  50. 50. EXECUTIVE SUMMARYThere were no Urgent problems uncovered. Overall, the response was positive and participants could find their way towards achieving the tasks quickly. Task scores Tasks are rated from 1 (low) to 5 (high) by the moderator for efficiency and effectiveness, based on what the participant does and says. Anything 3.5 or high is acceptable, but 4 or higher is the goal! Average overall score by task: [Task 1] = 3.85 [Task 2] = 3.01 [Task 3] = 3.55 [Task 4] = 3.66When asked what their overall experience was, given everything they’d seen in the session, the average score provided by participants was 3.6.
  51. 51. GOING OVER RESULTSGet all stakeholders togetherWithin 2 days of final studyInformal discussionMake recommendations
  52. 52. NEEDS MOAR RAPID• Test, debrief and present • Inventtasks and questions findings within 3 days based on participants thoughts = less prep, less• Recruit and test at an event analysis e.g. from the booth or nabbing people in the • Have feedback incorporated hallway into current or next sprint• Getdecision-makers to observe sessions
  53. 53. TOOLS AND ARTICLESClickHeat, ClickTale Usertesting.comCrazyEgg UserVueEthnio VerifyappFive Second TestFeedback Army Article: Extremely Rapid Usability TestingInvision App http://www.upassoc.org/ upa_publications/jus/2009may/Loop11 pawson1.htmlMechanical TurkMorae Presentation: Rapid turnaround usability testing: not just a pipe dream http://Open Hallway www.slideshare.net/usableinterface/rapid-OptimalWorkshop (OptimalSort, turnaround-usability-testing-not-just-a-Treejack, ChalkMark) pipe-dreamUsabilla
  54. 54. DRUPAL USABILITY DIRECTORYhttp://groups.drupal.org/usabilityUsability Group for Drupalhttp://groups.drupal.org/node/223959Content Creation UX Study Planhttp://drupal.org/community-initiatives/drupal-core/usabilityDrupal 8 User Experiencehttp://www.d7ux.org/Drupal 7 User Experience Project
  55. 55. Wanna volunteer in the workshop? Needed: 3 moderators 3 participants several observers for 3 short studies on groups.drupal.org drupal.org drupalshowcase.com
  56. 56. Q&A
  57. 57. Thank you! Lisa Rex @lisarex#frontendunited
  59. 59. Lisa Rex @lisarex#frontendunited
  60. 60. MODERATINGUse a study guideExplain the purpose of the study and what to expectGet consent to recordEncourage thinking out loudAssure them it’s a test of the software, not their skills orabilitiesExplain your role as an unbiased observer
  61. 61. MODERATINGEstablish rapport, but be professionalEnsure their physical and emotional comfortBe observantBe greedy for informationWatch the timeGive assists, but only if neededThank them
  62. 62. STUDY GUIDEdrupal.org:http://bit.ly/fu-usabilitygroups.drupal.org:http://bit.ly/fu-groups