0
HOW TO DO RAPIDUSABILITY TESTING
Lisa Rex   @lisarex#frontendunited
AND YOU ARE?
AGENDAThe basicsPreparation for a studyConducting a usability studyAnalysis & debrief
THE BASICS
WHY ARE USABILITY STUDIES      IMPORTANT? http://www.flickr.com/photos/0olong/479270001/
WHAT DOES YOUR                INTERFACE SAY?We hope you have excellent eyesight.                                  We hope ...
Yeah, hmm, we don’t know what that error message                     is either.         http://www.flickr.com/photos/kk/206...
Sorry we couldnt be bothered to test our site actually makes sense to anyone outside our                organization. http...
Ahttp://www.flickr.com/photos/chrisinplymouth/4456434875
Bhttp://www.flickr.com/photos/epsos/4444576795/
Chttp://www.flickr.com/photos/whistlepunch/5176579823/
“Testing with one user is 100% better than testing with                         none.”                     - Steve Krug
WHAT IS RAPID USABILITY           Stakeholder buy-in.   Observe real users doing real tasks.           Quick turnaround.  ...
THE PREPARATION
DO NOT BE ALARMED Preparation will take longer than      steps 2 & 3 combined
PREP: GOALS           Talk to stakeholders:“What are the business goals of the study?”      “What do we need to learn?”   ...
PREP: WHAT TO TESTIdeaInformation architecturePrototypeStaging siteProduction site             You can test at            ...
PREP: METHODOLOGYModerated vs unmoderatedOnline tool vs own setupRemote with screenshare vs in personOne-on-one vs many-to...
PREP: METHODOLOGYModerated vs unmoderatedOnline tool vs own setupRemote with screenshare vs in personOne-on-one vs many-to...
PREP: METHODOLOGYModerated vs unmoderatedOnline tool vs own setupRemote with screenshare vs in person
PREP: WHO TO TEST         Type of skillset           Expertise   New users vs. current users   5 people per study is ideal
PREP: TESTING             ENVIRONMENTHow will they access the site?Content and user account setup is needed?Do you need to...
PREP: STUDY OUTLINEOverviewParticipant briefingPre-session questionsScenarioLogin detailsTasks and questionsPost-session qu...
PREP: WRITING TASKSDo not use words that are in the UIPhrase things in the way people think and in plain languageWrite in ...
CONTENT CREATION UX    STUDY PLAN http://groups.drupal.org/node/223959
PREP: THE SCREENERDevelop a screener to weedout the ‘wrong’ participantFind out when they’reavailable (use Doodle?)Set exp...
PREP: RECRUITINGWhen: 0-2 days before you want to talk to them, ifmoderatedHow: social networks, online free ads, groups o...
PREP: SCHEDULINGConfirm timeSend calendar invite(s)Remind them of expectationsThank them
PREP: OBSERVATION FORMPriority                  Participants who had the                          issue (frequency)Issue s...
PREP: OBSERVERS   1 = catastrophic   2 = serious   3= important   4 = minor   5 = observation   6 = positive   7.x = resol...
CONDUCTING THE    STUDY
MODERATED STUDY              She’s working on a site that is highly             focused on a specific type of user: the    ...
CONDUCTING A MODERATED STUDYUse your study outline prepared scriptParticipant needs come above everything elseGet good inf...
HAVE FUN WITH IT!
THE RESULTSAnalyze and add to your observation spreadsheet
TREEJACK STUDY                He’s working on a Drupal distribution                 that is being redeveloped for Drupal  ...
TREEJACKWhen? Anytime...tree is isolated• Where would they go to do ___?• Can they find a specific piece of content?• Do nav...
BUILDING THE TREEBuild the structure in a spreadsheet, one column for eachlevelInclude sample contentInclude controls with...
SPREADSHEET
TREEJACK SETUPThink about what you want to validate, then write the tasksDecide the right answers; used to calculate the s...
TREEJACK PROTIPSRun a few moderated trials with people unfamiliar with yoursite, to catch the WTFs ....Set your ‘right’ an...
THE RESULTSWhat you’ll get is several graphs & complex visuals torepresent the paths people took, the success of each     ...
USERTESTING.COM STUDY           He works at a university library. He           isn’t a usability expert but wants to      ...
USERTESTING.COM SETUPReally flexible: show them a URL to live site or prototypeGive them detailed instructions at each step...
Tip: Run a pilot | Ask for refund
THE RESULTSWhat you’ll get is recordings of the tester’s browser and    voice, plus the answers they fill in at the end.   ...
THE ANALYSIS & DEBRIEF  Positive comments     Have a scribe, if no other                        stakeholders are presentIs...
ANALYSISTeam (or just you) should review observations andcondense, if possibleCategorize the positive negative/observation...
EXECUTIVE SUMMARYThere were no Urgent problems uncovered. Overall, the response was positive and       participants could ...
GOING OVER RESULTSGet all stakeholders togetherWithin 2 days of final studyInformal discussionMake recommendations
NEEDS MOAR RAPID• Test, debrief             and present         • Inventtasks and questions findings within 3 days         ...
TOOLS AND ARTICLESClickHeat, ClickTale            Usertesting.comCrazyEgg                        UserVueEthnio            ...
DRUPAL USABILITY                   DIRECTORYhttp://groups.drupal.org/usabilityUsability Group for Drupalhttp://groups.drup...
Wanna volunteer in the workshop?           Needed:         3 moderators         3 participants       several observers    ...
Q&A
Thank you!   Lisa Rex   @lisarex#frontendunited
WORKSHOP: MODERATING A   USABILITY SESSION
Lisa Rex   @lisarex#frontendunited
MODERATINGUse a study guideExplain the purpose of the study and what to expectGet consent to recordEncourage thinking out ...
MODERATINGEstablish rapport, but be professionalEnsure their physical and emotional comfortBe observantBe greedy for infor...
STUDY GUIDEdrupal.org:http://bit.ly/fu-usabilitygroups.drupal.org:http://bit.ly/fu-groups
Rapid usability testing
Rapid usability testing
Rapid usability testing
Rapid usability testing
Rapid usability testing
Upcoming SlideShare
Loading in...5
×

Rapid usability testing

803

Published on

Published in: Technology, Education
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
803
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
19
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Transcript of "Rapid usability testing"

  1. 1. HOW TO DO RAPIDUSABILITY TESTING
  2. 2. Lisa Rex @lisarex#frontendunited
  3. 3. AND YOU ARE?
  4. 4. AGENDAThe basicsPreparation for a studyConducting a usability studyAnalysis & debrief
  5. 5. THE BASICS
  6. 6. WHY ARE USABILITY STUDIES IMPORTANT? http://www.flickr.com/photos/0olong/479270001/
  7. 7. WHAT DOES YOUR INTERFACE SAY?We hope you have excellent eyesight. We hope you have excellent eyesight. We hope you have excellent eyesight.
  8. 8. Yeah, hmm, we don’t know what that error message is either. http://www.flickr.com/photos/kk/206835799/
  9. 9. Sorry we couldnt be bothered to test our site actually makes sense to anyone outside our organization. http://www.flickr.com/photos/ballookey/222118221/
  10. 10. Ahttp://www.flickr.com/photos/chrisinplymouth/4456434875
  11. 11. Bhttp://www.flickr.com/photos/epsos/4444576795/
  12. 12. Chttp://www.flickr.com/photos/whistlepunch/5176579823/
  13. 13. “Testing with one user is 100% better than testing with none.” - Steve Krug
  14. 14. WHAT IS RAPID USABILITY Stakeholder buy-in. Observe real users doing real tasks. Quick turnaround. No. Reports. Ever.
  15. 15. THE PREPARATION
  16. 16. DO NOT BE ALARMED Preparation will take longer than steps 2 & 3 combined
  17. 17. PREP: GOALS Talk to stakeholders:“What are the business goals of the study?” “What do we need to learn?” “When do we need it?” “What’s our budget?”
  18. 18. PREP: WHAT TO TESTIdeaInformation architecturePrototypeStaging siteProduction site You can test at any point during the project lifecycle!
  19. 19. PREP: METHODOLOGYModerated vs unmoderatedOnline tool vs own setupRemote with screenshare vs in personOne-on-one vs many-to-one
  20. 20. PREP: METHODOLOGYModerated vs unmoderatedOnline tool vs own setupRemote with screenshare vs in personOne-on-one vs many-to-one
  21. 21. PREP: METHODOLOGYModerated vs unmoderatedOnline tool vs own setupRemote with screenshare vs in person
  22. 22. PREP: WHO TO TEST Type of skillset Expertise New users vs. current users 5 people per study is ideal
  23. 23. PREP: TESTING ENVIRONMENTHow will they access the site?Content and user account setup is needed?Do you need to set up separate sites for each? Or re-setbetween sessions? (hint: use Drush)
  24. 24. PREP: STUDY OUTLINEOverviewParticipant briefingPre-session questionsScenarioLogin detailsTasks and questionsPost-session questions
  25. 25. PREP: WRITING TASKSDo not use words that are in the UIPhrase things in the way people think and in plain languageWrite in cues for moderator (e.g. “Begin recording” or“Observe how they ____”)Dependencies between the tasks & task orderWrite moderated assistsAsk them to rate their experience
  26. 26. CONTENT CREATION UX STUDY PLAN http://groups.drupal.org/node/223959
  27. 27. PREP: THE SCREENERDevelop a screener to weedout the ‘wrong’ participantFind out when they’reavailable (use Doodle?)Set expectations withparticipants (when, howlong, compensation)
  28. 28. PREP: RECRUITINGWhen: 0-2 days before you want to talk to them, ifmoderatedHow: social networks, online free ads, groups ororganizations, meetups, even family and friends“General” participant profile: can use a bar or cafeSend them to your screener, e.g. bit.ly/example
  29. 29. PREP: SCHEDULINGConfirm timeSend calendar invite(s)Remind them of expectationsThank them
  30. 30. PREP: OBSERVATION FORMPriority Participants who had the issue (frequency)Issue summary (quotes!) Other things: totals,Link to screenshot estimated fix date, commentsTask name
  31. 31. PREP: OBSERVERS 1 = catastrophic 2 = serious 3= important 4 = minor 5 = observation 6 = positive 7.x = resolved
  32. 32. CONDUCTING THE STUDY
  33. 33. MODERATED STUDY She’s working on a site that is highly focused on a specific type of user: the Drupal developer. She needs feedback from Drupal developers... anyone else won’t provide the knowledge that is needed to improve the site.15-60 minute session | 5 participants | 3-5 tasks | 3-5 pre- and post-session questions
  34. 34. CONDUCTING A MODERATED STUDYUse your study outline prepared scriptParticipant needs come above everything elseGet good information from them Gonna talk about this in detail at 13:45!
  35. 35. HAVE FUN WITH IT!
  36. 36. THE RESULTSAnalyze and add to your observation spreadsheet
  37. 37. TREEJACK STUDY He’s working on a Drupal distribution that is being redeveloped for Drupal 7. The previous distribution has some issues, so will users understand the new information architecture? He needs feedback from both new and current users.10 minute session | 40-400 (120 optimum) participants | 3-10 tasks | 3-5 pre- and post-session questions
  38. 38. TREEJACKWhen? Anytime...tree is isolated• Where would they go to do ___?• Can they find a specific piece of content?• Do navigation labels make sense?• Are my subsections well-organized?• How does Tree A compare to Tree B?
  39. 39. BUILDING THE TREEBuild the structure in a spreadsheet, one column for eachlevelInclude sample contentInclude controls within the site, if that’s what you want totest (e.g. “add comment”)Only the leaves of the tree are selectable as answers,branches are not.
  40. 40. SPREADSHEET
  41. 41. TREEJACK SETUPThink about what you want to validate, then write the tasksDecide the right answers; used to calculate the success ofyour tasksCustomize pre-session and post-session questionmessage options
  42. 42. TREEJACK PROTIPSRun a few moderated trials with people unfamiliar with yoursite, to catch the WTFs ....Set your ‘right’ answers at the very end, as you will blowthem away if you rebuild the treeConsider hacky chunking of optionsDon’t allow skip tasks, but include [None of these]
  43. 43. THE RESULTSWhat you’ll get is several graphs & complex visuals torepresent the paths people took, the success of each task, etc Analyze and add to your observation spreadsheet
  44. 44. USERTESTING.COM STUDY He works at a university library. He isn’t a usability expert but wants to make sure that the students can perform their primary tasks easily and quickly (like finding a book, making the reservations for a meeting room). Unfortunately, he has little time and little budget. 15 minute session | 5-8 participants | 4-5 tasks | 3-4 post-session questions
  45. 45. USERTESTING.COM SETUPReally flexible: show them a URL to live site or prototypeGive them detailed instructions at each stepWrite hints, just in case they missed itPre-session and post-session question and messageoptions
  46. 46. Tip: Run a pilot | Ask for refund
  47. 47. THE RESULTSWhat you’ll get is recordings of the tester’s browser and voice, plus the answers they fill in at the end. Analyze and add to your observation spreadsheet
  48. 48. THE ANALYSIS & DEBRIEF Positive comments Have a scribe, if no other stakeholders are presentIssues/Problems/Bugs and the session isn’t recorded Observations If recording, revisit the Quotes tapes and make notesAnnotated screenshotsTake notes during the session
  49. 49. ANALYSISTeam (or just you) should review observations andcondense, if possibleCategorize the positive negative/observationsSeverity: Low/Medium/ High or 1-5Frequency: Rare/ Common / Very CommonScope: Local/ Global
  50. 50. EXECUTIVE SUMMARYThere were no Urgent problems uncovered. Overall, the response was positive and participants could find their way towards achieving the tasks quickly. Task scores Tasks are rated from 1 (low) to 5 (high) by the moderator for efficiency and effectiveness, based on what the participant does and says. Anything 3.5 or high is acceptable, but 4 or higher is the goal! Average overall score by task: [Task 1] = 3.85 [Task 2] = 3.01 [Task 3] = 3.55 [Task 4] = 3.66When asked what their overall experience was, given everything they’d seen in the session, the average score provided by participants was 3.6.
  51. 51. GOING OVER RESULTSGet all stakeholders togetherWithin 2 days of final studyInformal discussionMake recommendations
  52. 52. NEEDS MOAR RAPID• Test, debrief and present • Inventtasks and questions findings within 3 days based on participants thoughts = less prep, less• Recruit and test at an event analysis e.g. from the booth or nabbing people in the • Have feedback incorporated hallway into current or next sprint• Getdecision-makers to observe sessions
  53. 53. TOOLS AND ARTICLESClickHeat, ClickTale Usertesting.comCrazyEgg UserVueEthnio VerifyappFive Second TestFeedback Army Article: Extremely Rapid Usability TestingInvision App http://www.upassoc.org/ upa_publications/jus/2009may/Loop11 pawson1.htmlMechanical TurkMorae Presentation: Rapid turnaround usability testing: not just a pipe dream http://Open Hallway www.slideshare.net/usableinterface/rapid-OptimalWorkshop (OptimalSort, turnaround-usability-testing-not-just-a-Treejack, ChalkMark) pipe-dreamUsabilla
  54. 54. DRUPAL USABILITY DIRECTORYhttp://groups.drupal.org/usabilityUsability Group for Drupalhttp://groups.drupal.org/node/223959Content Creation UX Study Planhttp://drupal.org/community-initiatives/drupal-core/usabilityDrupal 8 User Experiencehttp://www.d7ux.org/Drupal 7 User Experience Project
  55. 55. Wanna volunteer in the workshop? Needed: 3 moderators 3 participants several observers for 3 short studies on groups.drupal.org drupal.org drupalshowcase.com
  56. 56. Q&A
  57. 57. Thank you! Lisa Rex @lisarex#frontendunited
  58. 58. WORKSHOP: MODERATING A USABILITY SESSION
  59. 59. Lisa Rex @lisarex#frontendunited
  60. 60. MODERATINGUse a study guideExplain the purpose of the study and what to expectGet consent to recordEncourage thinking out loudAssure them it’s a test of the software, not their skills orabilitiesExplain your role as an unbiased observer
  61. 61. MODERATINGEstablish rapport, but be professionalEnsure their physical and emotional comfortBe observantBe greedy for informationWatch the timeGive assists, but only if neededThank them
  62. 62. STUDY GUIDEdrupal.org:http://bit.ly/fu-usabilitygroups.drupal.org:http://bit.ly/fu-groups
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×