13. “Testing with one user is 100% better than testing with none.” - Steve Krug
14. WHAT IS RAPID USABILITY Stakeholder buy-in. Observe real users doing real tasks. Quick turnaround. No. Reports. Ever.
15. THE PREPARATION
16. DO NOT BE ALARMED Preparation will take longer than steps 2 & 3 combined
17. PREP: GOALS Talk to stakeholders:“What are the business goals of the study?” “What do we need to learn?” “When do we need it?” “What’s our budget?”
18. PREP: WHAT TO TESTIdeaInformation architecturePrototypeStaging siteProduction site You can test at any point during the project lifecycle!
19. PREP: METHODOLOGYModerated vs unmoderatedOnline tool vs own setupRemote with screenshare vs in personOne-on-one vs many-to-one
20. PREP: METHODOLOGYModerated vs unmoderatedOnline tool vs own setupRemote with screenshare vs in personOne-on-one vs many-to-one
21. PREP: METHODOLOGYModerated vs unmoderatedOnline tool vs own setupRemote with screenshare vs in person
22. PREP: WHO TO TEST Type of skillset Expertise New users vs. current users 5 people per study is ideal
23. PREP: TESTING ENVIRONMENTHow will they access the site?Content and user account setup is needed?Do you need to set up separate sites for each? Or re-setbetween sessions? (hint: use Drush)
24. PREP: STUDY OUTLINEOverviewParticipant brieﬁngPre-session questionsScenarioLogin detailsTasks and questionsPost-session questions
25. PREP: WRITING TASKSDo not use words that are in the UIPhrase things in the way people think and in plain languageWrite in cues for moderator (e.g. “Begin recording” or“Observe how they ____”)Dependencies between the tasks & task orderWrite moderated assistsAsk them to rate their experience
26. CONTENT CREATION UX STUDY PLAN http://groups.drupal.org/node/223959
27. PREP: THE SCREENERDevelop a screener to weedout the ‘wrong’ participantFind out when they’reavailable (use Doodle?)Set expectations withparticipants (when, howlong, compensation)
28. PREP: RECRUITINGWhen: 0-2 days before you want to talk to them, ifmoderatedHow: social networks, online free ads, groups ororganizations, meetups, even family and friends“General” participant proﬁle: can use a bar or cafeSend them to your screener, e.g. bit.ly/example
29. PREP: SCHEDULINGConﬁrm timeSend calendar invite(s)Remind them of expectationsThank them
30. PREP: OBSERVATION FORMPriority Participants who had the issue (frequency)Issue summary (quotes!) Other things: totals,Link to screenshot estimated ﬁx date, commentsTask name
33. MODERATED STUDY She’s working on a site that is highly focused on a speciﬁc type of user: the Drupal developer. She needs feedback from Drupal developers... anyone else won’t provide the knowledge that is needed to improve the site.15-60 minute session | 5 participants | 3-5 tasks | 3-5 pre- and post-session questions
34. CONDUCTING A MODERATED STUDYUse your study outline prepared scriptParticipant needs come above everything elseGet good information from them Gonna talk about this in detail at 13:45!
35. HAVE FUN WITH IT!
36. THE RESULTSAnalyze and add to your observation spreadsheet
37. TREEJACK STUDY He’s working on a Drupal distribution that is being redeveloped for Drupal 7. The previous distribution has some issues, so will users understand the new information architecture? He needs feedback from both new and current users.10 minute session | 40-400 (120 optimum) participants | 3-10 tasks | 3-5 pre- and post-session questions
38. TREEJACKWhen? Anytime...tree is isolated• Where would they go to do ___?• Can they ﬁnd a speciﬁc piece of content?• Do navigation labels make sense?• Are my subsections well-organized?• How does Tree A compare to Tree B?
39. BUILDING THE TREEBuild the structure in a spreadsheet, one column for eachlevelInclude sample contentInclude controls within the site, if that’s what you want totest (e.g. “add comment”)Only the leaves of the tree are selectable as answers,branches are not.
41. TREEJACK SETUPThink about what you want to validate, then write the tasksDecide the right answers; used to calculate the success ofyour tasksCustomize pre-session and post-session questionmessage options
42. TREEJACK PROTIPSRun a few moderated trials with people unfamiliar with yoursite, to catch the WTFs ....Set your ‘right’ answers at the very end, as you will blowthem away if you rebuild the treeConsider hacky chunking of optionsDon’t allow skip tasks, but include [None of these]
43. THE RESULTSWhat you’ll get is several graphs & complex visuals torepresent the paths people took, the success of each task, etc Analyze and add to your observation spreadsheet
44. USERTESTING.COM STUDY He works at a university library. He isn’t a usability expert but wants to make sure that the students can perform their primary tasks easily and quickly (like ﬁnding a book, making the reservations for a meeting room). Unfortunately, he has little time and little budget. 15 minute session | 5-8 participants | 4-5 tasks | 3-4 post-session questions
45. USERTESTING.COM SETUPReally ﬂexible: show them a URL to live site or prototypeGive them detailed instructions at each stepWrite hints, just in case they missed itPre-session and post-session question and messageoptions
46. Tip: Run a pilot | Ask for refund
47. THE RESULTSWhat you’ll get is recordings of the tester’s browser and voice, plus the answers they ﬁll in at the end. Analyze and add to your observation spreadsheet
48. THE ANALYSIS & DEBRIEF Positive comments Have a scribe, if no other stakeholders are presentIssues/Problems/Bugs and the session isn’t recorded Observations If recording, revisit the Quotes tapes and make notesAnnotated screenshotsTake notes during the session
49. ANALYSISTeam (or just you) should review observations andcondense, if possibleCategorize the positive negative/observationsSeverity: Low/Medium/ High or 1-5Frequency: Rare/ Common / Very CommonScope: Local/ Global
50. EXECUTIVE SUMMARYThere were no Urgent problems uncovered. Overall, the response was positive and participants could ﬁnd their way towards achieving the tasks quickly. Task scores Tasks are rated from 1 (low) to 5 (high) by the moderator for efﬁciency and effectiveness, based on what the participant does and says. Anything 3.5 or high is acceptable, but 4 or higher is the goal! Average overall score by task: [Task 1] = 3.85 [Task 2] = 3.01 [Task 3] = 3.55 [Task 4] = 3.66When asked what their overall experience was, given everything they’d seen in the session, the average score provided by participants was 3.6.
51. GOING OVER RESULTSGet all stakeholders togetherWithin 2 days of ﬁnal studyInformal discussionMake recommendations
52. NEEDS MOAR RAPID• Test, debrief and present • Inventtasks and questions ﬁndings within 3 days based on participants thoughts = less prep, less• Recruit and test at an event analysis e.g. from the booth or nabbing people in the • Have feedback incorporated hallway into current or next sprint• Getdecision-makers to observe sessions
53. TOOLS AND ARTICLESClickHeat, ClickTale Usertesting.comCrazyEgg UserVueEthnio VerifyappFive Second TestFeedback Army Article: Extremely Rapid Usability TestingInvision App http://www.upassoc.org/ upa_publications/jus/2009may/Loop11 pawson1.htmlMechanical TurkMorae Presentation: Rapid turnaround usability testing: not just a pipe dream http://Open Hallway www.slideshare.net/usableinterface/rapid-OptimalWorkshop (OptimalSort, turnaround-usability-testing-not-just-a-Treejack, ChalkMark) pipe-dreamUsabilla
54. DRUPAL USABILITY DIRECTORYhttp://groups.drupal.org/usabilityUsability Group for Drupalhttp://groups.drupal.org/node/223959Content Creation UX Study Planhttp://drupal.org/community-initiatives/drupal-core/usabilityDrupal 8 User Experiencehttp://www.d7ux.org/Drupal 7 User Experience Project
55. Wanna volunteer in the workshop? Needed: 3 moderators 3 participants several observers for 3 short studies on groups.drupal.org drupal.org drupalshowcase.com
57. Thank you! Lisa Rex @lisarex#frontendunited
58. WORKSHOP: MODERATING A USABILITY SESSION
59. Lisa Rex @lisarex#frontendunited
60. MODERATINGUse a study guideExplain the purpose of the study and what to expectGet consent to recordEncourage thinking out loudAssure them it’s a test of the software, not their skills orabilitiesExplain your role as an unbiased observer
61. MODERATINGEstablish rapport, but be professionalEnsure their physical and emotional comfortBe observantBe greedy for informationWatch the timeGive assists, but only if neededThank them
62. STUDY GUIDEdrupal.org:http://bit.ly/fu-usabilitygroups.drupal.org:http://bit.ly/fu-groups