Successfully reported this slideshow.
Your SlideShare is downloading. ×

Usability testing / Nearly everything you need to know to get started

Ad

Usability testing
Nearly everything you need to know to get started
Rebecca Destello

Ad

Rebecca Destello
•  Restaurant manager
•  Front-end web developer
•  User researcher
•  Affiliate lecturer, UW HCDE
•  Chair...

Ad

Atlas Recall is a searchable photographic memory
for your digital life — if you see it, you can search
it and share with a...

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Loading in …3
×

Check these out next

1 of 60 Ad
1 of 60 Ad
Advertisement

More Related Content

Advertisement

Usability testing / Nearly everything you need to know to get started

  1. 1. Usability testing Nearly everything you need to know to get started Rebecca Destello
  2. 2. Rebecca Destello •  Restaurant manager •  Front-end web developer •  User researcher •  Affiliate lecturer, UW HCDE •  Chair of Puget Sound PSSIGCHI •  Manager, Research & Insights, Atlas Informatics R. Destello | Usability Testing
  3. 3. Atlas Recall is a searchable photographic memory for your digital life — if you see it, you can search it and share with anyone, from anywhere. atlas.co R. Destello | Usability Testing
  4. 4. Are you experienced? R. Destello | Usability Testing
  5. 5. R. Destello | Usability Testing Tonight’s lecture What you’ll need to know to do this as a professional
  6. 6. R. Destello | Usability Testing Why do we do usability testing?
  7. 7. Because there is a science to how people use things. R. Destello | Usability Testing
  8. 8. usability testing: the flow R. Destello | Usability Testing Identify problems in the system Create a study Recruit and schedule target users Test with participants Analyze + Synthesize Report out Evangelize findings! Repeat
  9. 9. The one book you need: Handbook of Usability Testing by Jeffery Rubin and Dana Chisnell R. Destello | Usability Testing
  10. 10. Planning R. Destello | Usability Testing
  11. 11. •  Identify key stakeholders •  Scope and priorities •  Identify & recruit target users •  Establish a timeline •  Identify usability concerns planning a usability study R. Destello | Usability Testing
  12. 12. It is critical to get key players involved up-front •  Project sponsors •  Product owners •  People who will have to act on the findings identify key stakeholders R. Destello | Usability Testing
  13. 13. R. Destello | Usability Testing How do we generate support for usability testing?
  14. 14. 1. Start running studies! 2. Get people to observations 3. Always be prepared with talking points; include findings from other studies that made a critical difference 4. Be a good team player - be willing to compromise and negotiate 5. Map usability planning to the company and product(s) 5 steps to getting buy-in R. Destello | Usability Testing
  15. 15. Think about: •  What is the political landscape? How much buy-in do you have from the stakeholders? •  Has usability been done before? •  What questions are you trying to answer? •  What is your budget? •  What time constraints are you working with? scope and priorities R. Destello | Usability Testing
  16. 16. Tips for identifying areas of concern: •  Ask your stakeholders •  What do you want to learn from this test? •  What are your greatest fears about this product? •  Are there different points of view in the group about this issue? •  What are you willing to change as a result of this test? •  Mine customer feedback •  Perform a task analysis •  Evaluate the product (heuristic evaluation or cog. walkthrough) identifying usability concerns R. Destello | Usability Testing
  17. 17. ! Bad recruiting is bad testing R. Destello | Usability Testing
  18. 18. •  Find your target user by asking your client to add a survey to their website •  Use Craigslist (with caution) •  Ping your social networks •  Go where your users might congregate And, write a good screener… tips to find your target user R. Destello | Usability Testing
  19. 19. The screening questionnaire needs to be •  Specific, inclusive of all necessary criteria •  Contain boundary conditions •  Contain exclusion criteria May need to stand alone •  May be passed off to others, firm, colleague Can be lengthy •  10 pages •  28 questions screeners R. Destello | Usability Testing
  20. 20. Tips for writing a good screener •  Ask the elimination questions first •  Eliminate conflicts of interest •  Screen for experience •  Recruit based on behavior and attitudes •  Screen out “professional” users •  Write a good screener (bit.ly/uxmattersrecruiting) screeners R. Destello | Usability Testing
  21. 21. usability test participant incentives Why should you give an incentive? •  Participants are helping you by giving up their time •  Participants will leave with a positive attitude •  Participants will tell their friends (larger pool of participants for future studies) •  Best way to get qualified, representative participants R. Destello | Usability Testing
  22. 22. usability test participant incentives What should the incentive be? •  Incentives are determined based on the client / budget •  Example: –  Microsoft gives out free software –  Boeing gives out movie tickets –  Chocolate (at the very least) How much should an incentive be? •  Based on budget and target audience R. Destello | Usability Testing
  23. 23. Your timeline will be determined by •  The complexity of the product •  The broadness of the scope of the test •  What set up will be needed (facilities, travel, special equip, etc.) •  Level of difficulty in recruiting the target users •  # of participants needed •  How formal the report out will need to be •  Related deadlines (design, dev, and even holidays!) establish a timeline R. Destello | Usability Testing
  24. 24. Conducting R. Destello | Usability Testing
  25. 25. what’s in the usability study? ü  Product or assets to test ü  Participants (5 works if doing iterative testing, if only 1 user group) ü  Moderator script with scenarios and tasks ü  Pre/Post task questionnaires ü  Checklists for study setup ü  Participant documents (Informed Consent, NDAs, Questionnaires) ü  Data collection tools ü  Participant incentives R. Destello | Usability Testing
  26. 26. •  Greet and make participant comfortable •  Explain study, setup, and complete forms •  Explain Thinking Aloud Protocol •  Remind them they are not being tested •  Begin tasks •  Wrap-up •  Questionnaire, if applicable •  Final Comments/Questions •  Thank them profusely •  Provide gratuity •  Parking validation, if applicable testing framework R. Destello | Usability Testing
  27. 27. what about scheduling? Allow time during the study: •  Time for test (30 minutes - 1 hour) •  Time in between (1 hour between participants) –  Setup for the next study, check in with observers, clean up notes, start examining data while it’s fresh R. Destello | Usability Testing E X A M P L E
  28. 28. stay organized with a checklist Why use a checklist? •  Checklists are good, reusable tools for any usability study •  Checklists outline all things that must be completed before, during, and after study •  Helps ensure study consistency and increases internal validity of the study from day 1! Key Takeaway: Even if you don’t think you’ll need it, create a checklist of all the things you need to do for study (you’ll be surprised what you forget!) R. Destello | Usability Testing
  29. 29. participant documents Before Testing •  Pre-Test Questionnaire (optional) •  Informed Consent/NDA (required) During Testing •  Post-task Questionnaires After Testing •  Post-Test Questionnaire R. Destello | Usability Testing
  30. 30. R. Destello | Usability Testing What’s the role of the moderator?
  31. 31. so what does the “moderator” do? ü  Interacts with the participant ü  Ensures the participant’s comfort ü  Sets the pace and tone of the study ü  Administers scenarios, tasks, and questionnaires ü  Handles prompting and working with the participant to understand the reasons behind problems. ü  Provides clarifications so note taker can capture data ü  Takes care of the participant! R. Destello | Usability Testing
  32. 32. why is moderating challenging? •  Goal is unbiased feedback •  Participants need to be comfortable, but the test situation is often unsettling •  You need to know what the participant is thinking, but you can’t read their minds R. Destello | Usability Testing
  33. 33. intervention: what are they doing? When and how to intervene during a test: •  In general, it’s preferable to let participants decide how much time to work on a task •  Some exceptions –  If they are getting very frustrated –  If you feel that you’ve got data on the task at hand and you want to keep going and get to other tasks –  If they’ve wandered so far down a bunny hole they are unable to proceed with the remaining tasks R. Destello | Usability Testing
  34. 34. prompting: guidelines When prompting, you want to… •  Get the user to elaborate on something that is unclear Without… •  Introducing bias •  Influencing the user’s behavior •  Taking over as the expert or primary speaker •  Unnerving the user through implication that she did something wrong R. Destello | Usability Testing
  35. 35. prompting: techniques Provide Acknowledgment Tokens at natural intervals •  Mm hmm, I see….. Focus on Questions •  How do you feel about that? •  Tell me what you are thinking… •  Tell me more about that… Repeat their word or phrase back to them as a question •  They say: That was confusing / You say: “confusing…?” Make the alternative explicit •  Was this task easy or difficult? R. Destello | Usability Testing
  36. 36. Keep in mind… •  People cannot reliably reflect on their own cognition •  Without video to playback questions like “what were you thinking when…” or “why do you think you…” you will find you often have missing / bad data •  Participants will struggle to come up with a believable rationale while being unaware of the true cause for their actions prompting: words of warning R. Destello | Usability Testing
  37. 37. participant questions Participants sometimes ask the moderator for help. Turn their question into a question. How would you respond to the following? •  Is that right? Did I do it right? •  Am I supposed to keep going? •  What should I do now? •  Should I click here? •  What is this? R. Destello | Usability Testing
  38. 38. The most important thing to remember about moderating is this: Being in a study is stressful for the participant – their comfort level and state of mind are vital. When people are stressed, their behavior/memory are impaired. R. Destello | Usability Testing
  39. 39. Things to keep in mind while moderating: •  Be patient •  Be excited about the process of discovery •  Be “small” in relationship with the participant •  Be as neutral as possible •  Be professional •  Be caring to the participant’s needs Important: The newer you are, the less you should talk. moderate with care R. Destello | Usability Testing
  40. 40. q  Does the moderator use relaxed body language? q  Does the moderator encourages/discourages responses? q  Does the moderator follow-up (when allowed) and dig into ambiguous statements or promising topics or ideas q  Does the moderator have any verbal or behavioral tics that would be distracting? q  Does the moderator provide excessive reinforcements? q  Is the moderator aware of expressions of support, sympathy, joy, anger, and other emotions? q  Does the moderator recover from miss-steps well. q  Does the moderator make the interviewee comfortable during the initial briefing and questions. q  Does the moderator clearly explain the limits of his/her ability to help the participant? q  Does the moderator follow protocols for prompting? q  Was the moderator sensitive to cultural/org. issues? CHECKYOURSELF beforeyouwreckyourself R. Destello | Usability Testing – Moderator Checklist credit: Chauncey Wilson
  41. 41. moderator script: the essentials Prepares participants for the session •  Describes what to expect (how long, what they’ll be doing, if they’ll be observed, if they’ll be videotaped) •  Gives think-aloud instructions •  Lets them know their rights Why is the script important? •  Ensures that the facilitator gives the same information to all the participants •  Stresses that the study is a test of the product, not of the participant R. Destello | Usability Testing
  42. 42. moderator script: the essentials How to write unbiased, non-leading usability tasks (at the 10,000 ft. level): First think: What is the user naturally trying to do at this step? Then – •  Make them short (hard to remember long tasks) •  Write tasks in the user’s natural language •  Avoid jargon •  Don’t use words found in the system R. Destello | Usability Testing
  43. 43. For a successful study… •  Make sure to know what data you are looking for (what questions are you trying to answer?) •  Define success criteria for each task •  Pilot the study at least once •  Do a quick “clean up” of notes after each session •  Have a plan for effective note-taking so you can spend less time analyzing data other considerations R. Destello | Usability Testing
  44. 44. Analyzing + Synthesizing R. Destello | Usability Testing
  45. 45. Data you can capture during the study: •  Pathways •  Verbal commentary •  Think Aloud Protocol (TAP) •  Time on task (only when not using TAP) •  Task completion •  And more data collection: available data Data can be gathered using: •  Handwriting •  Document •  Spreadsheet •  Usability software R. Destello | Usability Testing
  46. 46. data collection: DOs Do capture: •  Observations; what you see the participant do •  Quotes; what you hear the participant say •  Details; what specifically caused issues or prevented task completion R. Destello | Usability Testing
  47. 47. data collection: DON’Ts Don’t capture: •  Why you think they are doing what they are doing (if you aren’t sure, ask the participant) •  Broad generalizations that don’t contain a description - “She had a lot of trouble finding the link” won’t be helpful when you’re analyzing findings R. Destello | Usability Testing
  48. 48. Use codes to note instances of issues that you’re tracking Example: F = failure E = error message H = goes to help Keep an issues matrix and add to it after each participant. data collection: note-taking tools R. Destello | Usability Testing
  49. 49. data collection: note-taking tools R. Destello | Usability Testing
  50. 50. usability software: Morae R. Destello | Usability Testing
  51. 51. usability software: Silverback R. Destello | Usability Testing
  52. 52. Reporting + Evangelizing R. Destello | Usability Testing
  53. 53. •  Word of mouth and email •  Executive summary •  Formal report •  Presentation How you choose to communicate results can depend upon your audience, the culture, timeline, and the potential “Freak Out Factor” communicating results R. Destello | Usability Testing
  54. 54. •  Know your user (reader) – what’s important to them? •  Accentuate the positive, but deliver the news •  Report the facts (don’t overreach) •  Be thoughtful about word choice (opportunity vs. issue) •  When in doubt, recommend further testing •  Check grammar and spelling communicating results: tips R. Destello | Usability Testing
  55. 55. R. Destello | Usability Testing
  56. 56. In groups of 2, using a phone 1 person act as the participant, the other as the note taker and go to Nordstrom.com and complete the following tasks: 1.  You’d like to see the hot new items for this season. Where on the site would you browse what’s trending this fall for yourself? 2.  You want to buy a new shirt for your 5 year old niece. Where would you find a shirt that might work for her? Focus on problems in navigation and filter use. in-class exercise: Navigation R. Destello | Usability Testing
  57. 57. Don’t forget! •  Take notes (Where did the participant have trouble? What did they say? What worked?) •  Use prompts to keep them talking: “What are you thinking?” “Say more.” “Is that what you expected?” •  Don’t say much and listen closely in-class exercise: Navigation R. Destello | Usability Testing
  58. 58. in-class exercise: Navigation R. Destello | Usability Testing In groups of 2, using a phone 1 person act as the participant, the other as the note taker and go to Nordstrom.com and complete the following tasks: 1.  You’d like to see the hot new items for this season. Where on the site would you browse what’s trending this fall for yourself? SWITCH ROLES 2.  You want to buy a new shirt for your 5 year old niece. Where would you find a shirt that might work for her?
  59. 59. What was the experience like? in-class exercise: Navigation R. Destello | Usability Testing
  60. 60. Questions? RebeccaDestello@gmail.com

×