Better Results with User Testing - Why, When, What, and How


Published on

Published in: Internet
1 Like
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Better Results with User Testing - Why, When, What, and How

  1. 1. BETTER RESULTS WITH USER TESTING why, when, what and how User Experience Design | 6.3.14 Drew Gorton | Lynn Winter
  2. 2. How to be a usability expert ALL UX research and evaluation tools Instructions on how to run a specific usability test Designing for Touch, Lewis Nyman WE WILL NOT discuss
  3. 3. WHEN all signs point to testing HOW to set expectations WHAT tools you should use WHY care about your users WE WILL discuss 1 2 3 4
  4. 4. RESOURCES – books, articles, and more tools LESSONS we learned along the way WE WILL discuss 5 6
  5. 5. HELLO!
  6. 6. 1 WHY CARE
  7. 7. We all want to do something
  8. 8. I want to make the world a better place
  9. 9. but I need help
  10. 10. from clients who are doing good things
  11. 11. and with my magical web powers
  12. 12. I can make them more effective
  13. 13. together we can impact people
  14. 14. but we are only successful if those people
  15. 15. Listen | Care | Act | Change
  16. 16. how?
  17. 17. understand our users
  18. 18. act on what we learn
  19. 19. what do you want to do?
  20. 20. it will involve people
  21. 21. you will need to listen to your users
  22. 22. on board with user testing yet?
  23. 23. The Benefits of user testing
  24. 24. Fix People don’t like using things that confuse them — or make them angry. Find those things and fix them.
  25. 25. Outshine Test competitors. Learn where you’re better. Learn where you aren’t. Tell the marketing people-they’ll thank you.
  26. 26. Decide Do you really need that feature before launch? Is it good enough as is? (You might be surprised)
  27. 27. Humility Ever worked with someone who won’t let go of a bad idea? Even been that person?
  28. 28. Morale We make things. We want them to be useful. Making things that won’t be useful sucks. Stop it.
  29. 29. Trust User testing feels right. User testing feels professional. Your clients will see this and feel that you’re doing the right thing—because you are.
  30. 30. 2 WHEN TO TEST
  31. 31. Complex or high-risk functionality Unsure of who your users are Accessibility requirements are high priority Company or product is a start-up WHEN TO TEST
  32. 32. Conversion rate has big financial impact Use to backup expertise Task driven website Need user buy-in on decisions Maneuvering through political environment WHEN TO TEST
  34. 34. What is the goal of user testing? What is the budget for testing? How much time can be added to the project? Who will be working with your users? SET EXPECTATIONS
  35. 35. Where will the testing take place? Who will interpret the results? Who are your users? How will you find them? SET EXPECTATIONS
  36. 36. 4 TOOLS YOU CAN USE
  37. 37. Navigation Content Layout and Design Initial Discovery Working Prototype PHASES
  38. 38. Initial Discovery Usability Test | User Survey | User Interviews
  39. 39. Usability Test Observe users complete tasks with a specific tool
  40. 40. Want to pinpoint what exactly is working and what isn’t on the existing site To learn from your competitors At start of project-the earlier the feedback, the more likely you can address it. USE WHEN
  41. 41. Clients can join the observation Can reach a large, diverse group faster with remote options Only need a few users for good data PROS Lab setting is expensive Long timeline, pushes the start of build even later Testing & analysis is time consuming CONS
  42. 42. Create tasks, recruit users, secure facility, test, and analyze THE DETAILS Tasks | 3 – 8 weeksTimeline | Usability Lab-$15,000-$25,000Hard Costs | Recruiting & Screening-$2000 Online Services-$40-$70 / user
  43. 43. THE TOOLS
  44. 44. Bottom Line Observation is golden. It allows you to become familiar with the product and industry. Grab some users in a room and listen to them. No fancy tool required!
  45. 45. User Survey Ask questions to gather user feedback and ideas
  46. 46. Want to generate a list of ideas Unsure of your users demographics and needs Have a member-based audience that drives your organizational decisions USE WHEN
  47. 47. Able to gather user demographics Can recruit users for future testing Fast and easy to create, post, & promote PROS Analysis can take quite a bit of time Will not give you insight on specific user behavior Accuracy of data can be suspect CONS
  48. 48. Create survey, promote, post, and analyze THE DETAILS Tasks | 2 – 3 weeksTimeline | Online Services-$0 to $200+Hard Costs |
  49. 49. THE TOOLS
  50. 50. Bottom Line Why not? It is quick, easy, and affordable. The key is to ask the right questions and recruit the right users.
  51. 51. User Interviews Contextual discussions with internal and external stakeholders
  52. 52. Internal users are a high priority Know very little about the topic Have a task-driven website USE WHEN Can piggyback on an upcoming event
  53. 53. Few users make a big impact Alert you of workaround & cheat sheets Get out of the office & into their space PROS Limited perspective will come with few users Internal stakeholders can take over process Traveling to multiple locations takes time CONS
  54. 54. Create questions for unique users, recruit users, interview, analyze, create report THE DETAILS Tasks | 3 – 5 weeksTimeline | Mileage-$0.56 / mileHard Costs | Food-$30+
  55. 55. Bottom Line This technique can be time consuming-but don’t overthink it. Building a college website? Go sit in the student union and talk to people.
  56. 56. Navigation Card Sorting | Tree Testing
  57. 57. Card Sorting Observing how users group information together
  58. 58. Labels are a concern Content duplicated in multiple areas Overhauling the current grouping of content USE WHEN
  59. 59. Quick & affordable Weak points exposed Easy exercise for both internal & external users PROS Large data sets are overwhelming Specialized content difficult to group Only guideline and not a structure CONS
  60. 60. Create cards, recruit users, test, & analyze THE DETAILS Tasks | 1.5 – 3 weeksTimeline | Pens, paper, scissors-$15Hard Costs | Online Services-$0 to $109+
  61. 61. THE TOOLS
  62. 62. Bottom Line Observe your users on-site, don’t use too many cards & include internal stakeholders
  63. 63. Tree Testing Users search through a site skeleton to complete a task
  64. 64. Continue to address label concerns Help client visualize flow Validate the new structure is working – findability of items and groups USE WHEN
  65. 65. Able to iterate quickly to fine tune deeper levels Helps visual stakeholders find holes Quick & affordable PROS Can’t test cross- linking, search, & design impacts Iterations add to the project timeline Can only test large sites in pieces CONS
  66. 66. Create site map, create tasks, recruit users, test, analyze, revise, and repeat THE DETAILS Tasks | 2.5 – 4 weeksTimeline | Paper-$0Hard Costs | Online Services-$0 to $109+
  67. 67. THE TOOLS
  68. 68. Bottom Line If you have available users, give it a try! Just make sure to save time and budget to iterate
  69. 69. Content Readability Formulas | Cloze Testing
  70. 70. 21% of US adults read below a 5th grade level US Department of Education, National Institute of Literacy April 2013 ‘’ ‘’
  71. 71. Complex topics need to be communicated clearly Creating content, use as a guide to improve Critical to convince your user to take an action based on the content USE WHEN
  72. 72. Readability Formulas Measure quantifiable writing elements to estimate if the content is readable
  73. 73. Allows you to assess if you are writing to your users level Quick, easy, & free online tool PROS The tool won’t fix the content Can’t test layout & organization of content Test doesn’t assess meaning CONS
  74. 74. THE TOOLS
  75. 75. Readability measures the complexity of words and sentences Comprehension Measures whether a user can understand the meaning of sentences vs
  76. 76. Cloze Testing Remove every fifth word and ask users to fill in the blank
  77. 77. Way to test comprehension of select content Easy to create, facilitate, and score PROS Test doesn’t provide any qualitative feedback CONS Who doesn’t love Mad Libs? Usability testing is the best way to get comprehensive feedback
  78. 78. THE TOOLS
  79. 79. Bottom Line To gain useful feedback on the comprehension of content, you need to ask real users questions.
  80. 80. Layout and Design Click Test
  81. 81. Click Test Tracks the first place a user would click to complete a task
  82. 82. Research has shown that when users’ first click down the right path, 87% eventually succeed. ‘’ ‘’
  83. 83. When they click down an incorrect path, only 46% eventually succeed ‘’ ‘’Jeff Sauro Measuring Usability
  84. 84. Analytics show that certain critical pages are failing but you don’t know why Conversion rate is critical to the bottom line Completion of specific tasks are critical. The user must find the right buttons and links. USE WHEN
  85. 85. Large sample sizes possible No Observer Effect (live sites) Fast to create and execute PROS Click testing does not show eye movement When observing the live site, you don’t control motivations No direct feedback; you can’t ask why CONS
  86. 86. Create layout/design, create tasks, recruit users, test, analyze, revise, and repeat THE DETAILS Tasks | 2.5 – 4 weeksTimeline | Online Services-$0 to $200Hard Costs |
  87. 87. THE TOOLS
  88. 88. Bottom Line The first click is critical to your users’ success. If you understand where and how your current site is failing, this tool can be powerful.
  89. 89. Bottom Line Can tell you what users notice and don’t. Can help diagnose and help solve first- click navigation problems.
  90. 90. Working Prototype A/B Testing
  91. 91. A/B Testing Users compare two versions of the same page
  92. 92. Content layout is a concern Working on your conversion rate optimization (CRO) Need to ensure you have the right layout and/or look and feel USE WHEN
  93. 93. Great way to get a sense of users gut reaction Small changes can provide you with a big pay-off Quick process to test and iterate PROS No control over motivations Analysis has limits – missing the why of the decision CONS Have to build both A and B
  94. 94. Create two options, recruit users, test, analyze, and iterate as needed THE DETAILS Tasks | 1.5 – 4 weeksTimeline | Paper or your computer -$0Hard Costs | Online Services-$0 to $300+
  95. 95. THE TOOLS
  96. 96. Bottom Line Can be great for focused goals. Analysis and iterations can be endless.
  98. 98. There is no substitute for in-person observation The more time you put in, the more value you will receive Leave the analysis up to the experts Analytics are instrumental LESSONS LEARNED
  99. 99. Testing will not make your decisions You can’t test everything, focus on high priority items Balance the results with your expertise Don’t make it too complicated-any feedback is helpful LESSONS LEARNED
  100. 100. 6 RESOURCES
  101. 101.
  102. 102. Evaluate this session