Your SlideShare is downloading. ×
How to Easily Uncover Usability Issues
Upcoming SlideShare
Loading in...5

Thanks for flagging this SlideShare!

Oops! An error has occurred.

Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

How to Easily Uncover Usability Issues


Published on

Published in: Technology
1 Like
  • Be the first to comment

No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

No notes for slide


  • 1. How to Easily Uncover UsabilityIssuesDharmesh MistryUX Researcher, AcquiaJune 20, id/ Twitter: @dcmistry
  • 2. Welcome Dharmesh MistryUX Researcher, Acquia•  UX for Acquia, Drupal•  UXPA Boston, Board of Director•  Co-organizer: Boston Design Jam, Design 4Drupal, UX/UI Summit, UXPA Boston AnnualConference•  Speaker, Blogger, Trainer, Evangelist•  Bentley University•  Desserts and Photography
  • 3. Agenda•  Why usability matters?•  A few basics•  Preparation for a study•  Unmoderated usability study•  Analyzing observations•  Reporting your findings•  Q & A
  • 4. Why usability matters?
  • 5. “Testing with one user is 100% betterthan testing with none.”- Steve Krug
  • 6. What is usability testing?Design + Technology + PsychologyImprove the user experience; competitive edgeObserving real users doing real tasks
  • 7. Evaluating usability“Usability testing is the second most used evaluationmethod and the method that has the most impact onmaking products better.”-Rosebaum, et al (2000)
  • 8. Types of usability studies•  Formative vs. Summative•  Moderated vs. Unmoderated•  In Person vs. Remote
  • 9. When can you uncover issues?•  Idea/ concept stage•  Design is in prototype stage•  Design is ready to go live•  Design is liveAt any point during the lifecycle of a product.
  • 10. The preparation for anunmoderated usability study
  • 11. General Construct1.  Goals2.  Prepare3.  Recruit4.  Set up5.  Analyze6.  Report
  • 12. Step 1: Determining GoalsTalk to stakeholders to determine thebusiness and usability goals of the studyIt is necessary to collaborate withstakeholders to produce a high qualitystudy
  • 13. Step 2: Answering the questions•  Where is the product in terms of development cycle•  What tasks will be covered?•  Duration of the session?•  Who should be your participants?•  What should be your methodology?•  How many participants should you have?•  How should you compensate your participants?
  • 14. Step 2: Preparing, cont.Study Outline •  Scenario•  Login details (if applicable)•  Tasks •  Post-session questions
  • 15. Step 2: Preparing, cont. The more severe the problem, the more likely it is to be uncovered withinthe first few sessions.“Approximately 80% of usability problems identified would have foundafter only five subjects.”Number of participants 3 – 65% / 5 – 80% / 9 - 95%“Virtually all of the problems judged high in severity were discovered afterrunning 5 subjects, whereas only 55% of the problems judged low inseverity were found.”Source: Virzi study
  • 16. Conducting an 
unmoderated usability study
  • 17. The unmoderated studyHe works for University of Minnesotaand is considering re-designing thelibrary website. He is somewhat awareabout usability but has not had the timeto dive into it. He wants to make surethat the students can perform the taskseasily and quickly (like finding a book,making the reservations for a meetingroom). Unfortunately, he has little timeand little budget. Tip: 15 minutes | 5 participants | 4 tasks | 4 post session questions
  • 18. The unmoderated study
  • 19. URL and Scenario
  • 20. Tasks
  • 21. Post Session
  • 22. Writing scenarios and tasksHow to choose the right user and tasks :•  Tasks that are typical or frequent/ critical tooperation•  New or existing users
  • 23. Writing scenarios and tasksChoose your words carefully!•  Do not use words that are in the product•  Write in plain language•  What order should tasks be done?•  How many tasks should you have?•  What are the dependencies between the tasks?•  Can I compare designs?•  How can I give moderated assists?
  • 24. Writing unbiased scenarios andtasksFollowing are some of the biased questions that you should avoid: Rate the degree to which the software you just used has no performanceproblemAre you interested in obtaining another degree in the next 10 years?About how many times in the last year did you use online help?____Number of timesHow often did you use Product X during the last month?(Check one answer)_____ Never_____ Rarely_____ Occasionally_____ Regularly
  • 25. Finding the right participantsTip: Running a pilot | Getting refund back
  • 26. Analyzing your observations
  • 27. Video recordings
  • 28. Step 5: AnalysisWhat to note?•  Positive•  Issues•  Bugs•  Observations•  Quotations and annotated screenshotsWhen to note?•  Revisit the recordings and make notes •  During the session•  Have a scribe, if possible
  • 29. Prioritize your findings Categorize the positive/ negative/ observationSeverity low/ medium/ highFrequency rare/ common/ very commonScope local/ global
  • 30. Step 6: Reporting your findings •  Purpose•  Why did we test?•  How did we test?•  Whom did we test?•  Executive Summary•  Positives•  Major Issues•  Detailed findings•  Recommendations•  Observations•  Next Steps
  • 31. The executive summaryThere were no Urgent problems uncovered. Overall, the response was positive and participants could find their waytowards achieving the tasks quickly.High severity findingsBulleted listMedium severity findingsBulleted listTask scoresTasks are rated from 1 (low) to 5 (high) by the moderator for efficiency and effectiveness, based on what the participantdoes and says. Anything 3.5 or high is acceptable, but 4 or higher is the goal!Average overall score by task:[Task 1] = 3.85[Task 2] = 3.01[Task 3] = 3.55[Task 4] = 3.66When asked what their overall experience was, given everything they’d seen in the session, the average score providedby participants was 3.6.
  • 32. Other unmoderated usability studytools to consider•  Chalkmark•  Loop11•  Open Hallway•  Usabilla•  UserZoom"Handbook:
  • 33. Q & A