How to Easily Uncover Usability Issues


Published on

Published in: Technology
1 Like
  • Be the first to comment

No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

How to Easily Uncover Usability Issues

  1. 1. How to Easily Uncover UsabilityIssuesDharmesh MistryUX Researcher, AcquiaJune 20, id/ Twitter: @dcmistry
  2. 2. Welcome Dharmesh MistryUX Researcher, Acquia•  UX for Acquia, Drupal•  UXPA Boston, Board of Director•  Co-organizer: Boston Design Jam, Design 4Drupal, UX/UI Summit, UXPA Boston AnnualConference•  Speaker, Blogger, Trainer, Evangelist•  Bentley University•  Desserts and Photography
  3. 3. Agenda•  Why usability matters?•  A few basics•  Preparation for a study•  Unmoderated usability study•  Analyzing observations•  Reporting your findings•  Q & A
  4. 4. Why usability matters?
  5. 5. “Testing with one user is 100% betterthan testing with none.”- Steve Krug
  6. 6. What is usability testing?Design + Technology + PsychologyImprove the user experience; competitive edgeObserving real users doing real tasks
  7. 7. Evaluating usability“Usability testing is the second most used evaluationmethod and the method that has the most impact onmaking products better.”-Rosebaum, et al (2000)
  8. 8. Types of usability studies•  Formative vs. Summative•  Moderated vs. Unmoderated•  In Person vs. Remote
  9. 9. When can you uncover issues?•  Idea/ concept stage•  Design is in prototype stage•  Design is ready to go live•  Design is liveAt any point during the lifecycle of a product.
  10. 10. The preparation for anunmoderated usability study
  11. 11. General Construct1.  Goals2.  Prepare3.  Recruit4.  Set up5.  Analyze6.  Report
  12. 12. Step 1: Determining GoalsTalk to stakeholders to determine thebusiness and usability goals of the studyIt is necessary to collaborate withstakeholders to produce a high qualitystudy
  13. 13. Step 2: Answering the questions•  Where is the product in terms of development cycle•  What tasks will be covered?•  Duration of the session?•  Who should be your participants?•  What should be your methodology?•  How many participants should you have?•  How should you compensate your participants?
  14. 14. Step 2: Preparing, cont.Study Outline •  Scenario•  Login details (if applicable)•  Tasks •  Post-session questions
  15. 15. Step 2: Preparing, cont. The more severe the problem, the more likely it is to be uncovered withinthe first few sessions.“Approximately 80% of usability problems identified would have foundafter only five subjects.”Number of participants 3 – 65% / 5 – 80% / 9 - 95%“Virtually all of the problems judged high in severity were discovered afterrunning 5 subjects, whereas only 55% of the problems judged low inseverity were found.”Source: Virzi study
  16. 16. Conducting an 
unmoderated usability study
  17. 17. The unmoderated studyHe works for University of Minnesotaand is considering re-designing thelibrary website. He is somewhat awareabout usability but has not had the timeto dive into it. He wants to make surethat the students can perform the taskseasily and quickly (like finding a book,making the reservations for a meetingroom). Unfortunately, he has little timeand little budget. Tip: 15 minutes | 5 participants | 4 tasks | 4 post session questions
  18. 18. The unmoderated study
  19. 19. URL and Scenario
  20. 20. Tasks
  21. 21. Post Session
  22. 22. Writing scenarios and tasksHow to choose the right user and tasks :•  Tasks that are typical or frequent/ critical tooperation•  New or existing users
  23. 23. Writing scenarios and tasksChoose your words carefully!•  Do not use words that are in the product•  Write in plain language•  What order should tasks be done?•  How many tasks should you have?•  What are the dependencies between the tasks?•  Can I compare designs?•  How can I give moderated assists?
  24. 24. Writing unbiased scenarios andtasksFollowing are some of the biased questions that you should avoid: Rate the degree to which the software you just used has no performanceproblemAre you interested in obtaining another degree in the next 10 years?About how many times in the last year did you use online help?____Number of timesHow often did you use Product X during the last month?(Check one answer)_____ Never_____ Rarely_____ Occasionally_____ Regularly
  25. 25. Finding the right participantsTip: Running a pilot | Getting refund back
  26. 26. Analyzing your observations
  27. 27. Video recordings
  28. 28. Step 5: AnalysisWhat to note?•  Positive•  Issues•  Bugs•  Observations•  Quotations and annotated screenshotsWhen to note?•  Revisit the recordings and make notes •  During the session•  Have a scribe, if possible
  29. 29. Prioritize your findings Categorize the positive/ negative/ observationSeverity low/ medium/ highFrequency rare/ common/ very commonScope local/ global
  30. 30. Step 6: Reporting your findings •  Purpose•  Why did we test?•  How did we test?•  Whom did we test?•  Executive Summary•  Positives•  Major Issues•  Detailed findings•  Recommendations•  Observations•  Next Steps
  31. 31. The executive summaryThere were no Urgent problems uncovered. Overall, the response was positive and participants could find their waytowards achieving the tasks quickly.High severity findingsBulleted listMedium severity findingsBulleted listTask scoresTasks are rated from 1 (low) to 5 (high) by the moderator for efficiency and effectiveness, based on what the participantdoes and says. Anything 3.5 or high is acceptable, but 4 or higher is the goal!Average overall score by task:[Task 1] = 3.85[Task 2] = 3.01[Task 3] = 3.55[Task 4] = 3.66When asked what their overall experience was, given everything they’d seen in the session, the average score providedby participants was 3.6.
  32. 32. Other unmoderated usability studytools to consider•  Chalkmark•  Loop11•  Open Hallway•  Usabilla•  UserZoom"Handbook:
  33. 33. Q & A