Module 10: Usability Testing


Published on

Module 10 in the one-week intensive for community college instructors, offered by

Published in: Design, Technology
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Module 10: Usability Testing

  1. 1. Usability Testing Usability testing is the bookend to user research. ! Both are needed to support what goes in between.
  2. 2. Usability Testing What is usability testing? ! — First point: We are not testing the user (you will hear the term "user testing", but that may make Jared Spool sad) ! — Second point: It’s not a science ! !
  3. 3. Usability Testing What is usability testing? ! "“It takes only five users to uncover 80 percent of high-level usability problems." ! "… debugging a design." ! — Jakob Nielsen !
  4. 4. Usability Testing What is usability testing? ! "Every time a person has a great experience with a website, a web app, a gadget, or a service, it’s because a design team made excellent decisions about both design and implementation — decisions based on data about how people use designs. And how can you get that data? Usability testing." — Dana Chisnell
  5. 5. Usability Testing What is usability testing? ! (Dana co-wrote this book)
  6. 6. Usability Testing What is usability testing? ! "Watching people try to use what you’re creating/designing/ building (or something you’ve already created/designed/built), with the intention of (a) making it easier for people to use or (b) proving that it is easy to use." — Steve Krug
  7. 7. Usability Testing What is usability testing? ! (Steve wrote this book)
  8. 8. Usability Testing What is usability testing? ! "In a usability test, individual users work through a series of fairly realistic tasks. Some tasks may be timed, but most often, each participant talks out loud to describe his thought process as he uses an actual product or prototype." — Kim Goodwin !
  9. 9. Usability Testing What is usability testing? ! — It can demonstrate a problem exists — It can help us understand how something we design should behave ! !
  10. 10. Usability Testing What is usability testing? ! —There are lots of online resources, even some you might not expect ! ! !
  11. 11. Usability Testing What is usability testing? ! — It can be time-consuming and not easy; you can outsource it to specialized companies at different scales ! — —AnswerLab — — others
  12. 12. Usability Testing What is usability testing? ! Please note: it is NOT a focus group ! People want to agree and show favor. If you basically ask them if they like the work you’ve done, they’ll try to please you (and give useless results).
  13. 13. Usability Testing What is usability testing? ! Please note: it is NOT a focus group ! The difference? ! "I’m showing you this. Do you like it?" ! "Show me how you do this."
  14. 14. Usability Testing What is usability testing? ! It is NOT
  15. 15. Usability Testing When to do what? ! ! by Hong Qi,Donna Lichaw; Adapted & taught by Christina Wodtke 11/13/13 What do we know? User Interviews Contextual Inquiry,Field Studies & Ethnography Diary & Journal Studies Benchmark Usability Tests Card Sorting Surveys How are we doing? Usability & Concept Tests Eye Tracking Diary & Journal Studies How did we do? Usability & Concept Tests Eye Tracking A/B & Multivariate Tests Data Analysis
  16. 16. Usability Testing This isn’t rocket science or deep thinking ! But this does require keeping many, things in mind, and self-control, as well as practice.
  17. 17. Usability Testing The "classic model" ! — Develop a test plan — Choose a testing environment — Find and select participants — Prepare test materials — Conduct the sessions — Debrief with participants and observers —Analyze data and observations — Create findings and recommendations
  18. 18. Usability Testing When to go classic ! —Your team/company does it ! —You need to deliver lots of data/stats ! —Your team/company works with extensive requirements documents !
  19. 19. Usability Testing Plan ! — Discuss with the whole team ! —Agree on a test objective (something besides “determine whether users can use it”) ! —The methods and measures you’ll use to learn the answers to your research questions ! —This can take < 1hr
  20. 20. Usability Testing Plan ! One template: ! tools/resources/templates/usability- test-plan-template.html
  21. 21. Usability Testing Environment ! —You can do it in a formal or dedicated lab ! !
  22. 22. Usability Testing Environment ! —You can do it in a more casual setting ! !
  23. 23. Usability Testing Environment ! — It helps if there is recording equipment (even just software recording the screen and user on the laptop or desktop being used), but if you only have a person to record observations and take notes, that can work, too. ! !
  24. 24. Usability Testing Environment ! —You can even do it remotely (screen sharing via Skype, JoinMe, GoToMeeting; these can record both the user’s screen and the user via webcam) ! —And there’s always the ambush method: "Hi. I’m ______________. Do you have a minute?"
  25. 25. Usability Testing
  26. 26. Usability Testing Recruiting ! — Can be hard and time-consuming ! — Focusing on the behavior that you’re interested in observing is easier than trying to select for market segmentation or demographics ! — If you can, offload. Many services will charge $75-150 per participant and take care of everything
  27. 27. Usability Testing Recruiting ! — mention the gift certificate — friends and family and friends of friends — post to user groups of similar apps/sites — professional mailing lists — did we mention the gift certificate? — Facebook fan pages/Twitter hashtags/LinkedIn groups — probably not people at the same company (see: Google Buzz) — NOT people supplied by the marketing department
  28. 28. Usability Testing Three may be a magic number
  29. 29. Usability Testing Screening ! —"If you want to test a kiosk for checking people into and out of education programs, you want people who are attending those programs. Make sense?  Don’t make recruiting harder than it has to be."— Dana Chisnell ! — make sure they’re available ! — make sure they meet your qualifications ! — make sure you’ve explained the process to them
  30. 30. Usability Testing Prep for tests ! — Guide or checklist for the moderator ! — Includes the research questions ! — I any specific interview questions you might want to ask, prompts for follow-up questions, as well as closing, debriefing questions that you want to ask each participant
  31. 31. Usability Testing Prep for tests ! See Hdotcom_test_plan.pdf
  32. 32. Usability Testing Choosing tasks for the test ! —Translate the research questions into task scenarios that represent realistic user goals ! —Tasks = user goals, not app/site features — e.g.,"How would you book a hotel room?", not "How do you navigate our hotel interface?" ! — Give a task that realistically represents a user goal
  33. 33. Usability Testing Conducting the test
  34. 34. Usability Testing How to structure the test ! (See "session_script.pdf") ! Prep reminders for moderator — Rehearsal and preparation needs — Before each session ("Set up computer", etc.) —After each session ("Turn off recording", etc.) !
  35. 35. Usability Testing How to structure the test ! (See "session_script.pdf") ! Session introduction — Introduce yourself nicely —Think-out-loud protocol —"You can’t do anything wrong" —"You’re helping us find where we confuse you." !
  36. 36. Usability Testing How to structure the test ! (See "session_script.pdf") ! Tasks !
  37. 37. Usability Testing How to structure the test ! (See "session_script.pdf") ! Close the session ! This often involves overall closing questions. ! Remember: Thank the participant. !
  38. 38. Usability Testing How to conduct yourself ! — Be nice, empathetic ! —Ask open-ended questions: "What would you expect to happen? Where would you look for that?" ! — Be reassuring: "We’re trying to make this better, so you’re helping us by pointing out where it fails you."
  39. 39. Usability Testing How to conduct yourself ! — Do NOT say,"Click this/go there" ! — Remind participant to think out loud ! — Mirror their statements, allow them to clarify ! — Don’t interrupt. LISTEN.
  40. 40. Usability Testing Don’t lead the witness (phrasing and idea: Hong Qi, Donna Lichaw, Christina Wodtke)
  41. 41. Usability Testing How to take notes ! —A dedicated observer/notetaker helps ! — Note participants’ facial or non-verbal expressions, body language ! —Add time labels to notes ("This at 9:42") ! NOTES OR IT DIDN’T HAPPEN
  42. 42. Usability Testing How to take notes !
  43. 43. Usability Testing Conducting the test ! —The moderator as MC: sees to the safety and comfort of the participants, manages the team members observing, and handles the data collected. !
  44. 44. Usability Testing Debrief ! —Take a step back with the participant and ask,“How’d that go?” Invite the trained observers to pass follow-up questions to the moderator. ! —Thank the participant, compensate, and say good-bye. !
  45. 45. Usability Testing Debrief ! — Now, the team observing should talk briefly about what they saw and what they heard. (This discussion is not about solving design problems, yet.)
  46. 46. Usability Testing The RITE way ! Rapid Iteration Testing Environment — integrates well into Lean UX and agile development in general — Boxes and Arrows podcast "Rapid-Turnaround Usability Testing" by Kyle Soucy and Holly Phillips: https:// id275459507?i=82723199&mt=2 — Keys include: small testing sample, rapid design iterations, debriefs of all stakeholders time-limited and tied to physical observations
  47. 47. Usability Testing When to test
  48. 48. Usability Testing The Lean UX"3-12-1" ! ! !
  49. 49. Usability Testing The Lean UX"3-12-1" ! ! !
  50. 50. Usability Testing Analyze ! — Can be stories (qualitative), can be stats (quantitative) !
  51. 51. Usability Testing Findings and recs ! — Concentrate on the most significant and serious findings, not all the details ! — It’s a good idea to include a summary of the main points !
  52. 52. Usability Testing Findings and recs ! —Think visual. You ideally want lots of screen shots and perhaps even video clips to showcase issues and findings ! — Be ready to discuss in person
  53. 53. Usability Testing Krug says: You can do this.
  54. 54. Usability Testing Any testing helps: ! Why? ! —The "second set of eyes" principle ! —The "it’s always been right in front of me" principle ! — Everything is broken, even if everything is awesome ! (Can still be a pain.)
  55. 55. Usability Testing Hey, ho, let’s go ! A: Create a script ! B: Pair up ! C: Each person interviews the other.Then switch. ! D: Report back to the group what you learned
  56. 56. Review Do = !! ! Why > Do
  57. 57. Review Know the Why ! Then Do
  58. 58. Review And thanks twitter web