Better Results with User Testing - Why, When, What, and How
Upcoming SlideShare
Loading in...5

Like this? Share it with your network


Better Results with User Testing - Why, When, What, and How






Total Views
Views on SlideShare
Embed Views



1 Embed 25 25



Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
Post Comment
Edit your comment

Better Results with User Testing - Why, When, What, and How Presentation Transcript

  • 1. BETTER RESULTS WITH USER TESTING why, when, what and how User Experience Design | 6.3.14 Drew Gorton | Lynn Winter
  • 2. How to be a usability expert ALL UX research and evaluation tools Instructions on how to run a specific usability test Designing for Touch, Lewis Nyman WE WILL NOT discuss
  • 3. WHEN all signs point to testing HOW to set expectations WHAT tools you should use WHY care about your users WE WILL discuss 1 2 3 4
  • 4. RESOURCES – books, articles, and more tools LESSONS we learned along the way WE WILL discuss 5 6
  • 5. HELLO!
  • 6. 1 WHY CARE
  • 7. We all want to do something
  • 8. I want to make the world a better place
  • 9. but I need help
  • 10. from clients who are doing good things
  • 11. and with my magical web powers
  • 12. I can make them more effective
  • 13. together we can impact people
  • 14. but we are only successful if those people
  • 15. Listen | Care | Act | Change
  • 16. how?
  • 17. understand our users
  • 18. act on what we learn
  • 19. what do you want to do?
  • 20. it will involve people
  • 21. you will need to listen to your users
  • 22. on board with user testing yet?
  • 23. The Benefits of user testing
  • 24. Fix People don’t like using things that confuse them — or make them angry. Find those things and fix them.
  • 25. Outshine Test competitors. Learn where you’re better. Learn where you aren’t. Tell the marketing people-they’ll thank you.
  • 26. Decide Do you really need that feature before launch? Is it good enough as is? (You might be surprised)
  • 27. Humility Ever worked with someone who won’t let go of a bad idea? Even been that person?
  • 28. Morale We make things. We want them to be useful. Making things that won’t be useful sucks. Stop it.
  • 29. Trust User testing feels right. User testing feels professional. Your clients will see this and feel that you’re doing the right thing—because you are.
  • 30. 2 WHEN TO TEST
  • 31. Complex or high-risk functionality Unsure of who your users are Accessibility requirements are high priority Company or product is a start-up WHEN TO TEST
  • 32. Conversion rate has big financial impact Use to backup expertise Task driven website Need user buy-in on decisions Maneuvering through political environment WHEN TO TEST
  • 34. What is the goal of user testing? What is the budget for testing? How much time can be added to the project? Who will be working with your users? SET EXPECTATIONS
  • 35. Where will the testing take place? Who will interpret the results? Who are your users? How will you find them? SET EXPECTATIONS
  • 37. Navigation Content Layout and Design Initial Discovery Working Prototype PHASES
  • 38. Initial Discovery Usability Test | User Survey | User Interviews
  • 39. Usability Test Observe users complete tasks with a specific tool
  • 40. Want to pinpoint what exactly is working and what isn’t on the existing site To learn from your competitors At start of project-the earlier the feedback, the more likely you can address it. USE WHEN
  • 41. Clients can join the observation Can reach a large, diverse group faster with remote options Only need a few users for good data PROS Lab setting is expensive Long timeline, pushes the start of build even later Testing & analysis is time consuming CONS
  • 42. Create tasks, recruit users, secure facility, test, and analyze THE DETAILS Tasks | 3 – 8 weeksTimeline | Usability Lab-$15,000-$25,000Hard Costs | Recruiting & Screening-$2000 Online Services-$40-$70 / user
  • 43. THE TOOLS
  • 44. Bottom Line Observation is golden. It allows you to become familiar with the product and industry. Grab some users in a room and listen to them. No fancy tool required!
  • 45. User Survey Ask questions to gather user feedback and ideas
  • 46. Want to generate a list of ideas Unsure of your users demographics and needs Have a member-based audience that drives your organizational decisions USE WHEN
  • 47. Able to gather user demographics Can recruit users for future testing Fast and easy to create, post, & promote PROS Analysis can take quite a bit of time Will not give you insight on specific user behavior Accuracy of data can be suspect CONS
  • 48. Create survey, promote, post, and analyze THE DETAILS Tasks | 2 – 3 weeksTimeline | Online Services-$0 to $200+Hard Costs |
  • 49. THE TOOLS
  • 50. Bottom Line Why not? It is quick, easy, and affordable. The key is to ask the right questions and recruit the right users.
  • 51. User Interviews Contextual discussions with internal and external stakeholders
  • 52. Internal users are a high priority Know very little about the topic Have a task-driven website USE WHEN Can piggyback on an upcoming event
  • 53. Few users make a big impact Alert you of workaround & cheat sheets Get out of the office & into their space PROS Limited perspective will come with few users Internal stakeholders can take over process Traveling to multiple locations takes time CONS
  • 54. Create questions for unique users, recruit users, interview, analyze, create report THE DETAILS Tasks | 3 – 5 weeksTimeline | Mileage-$0.56 / mileHard Costs | Food-$30+
  • 55. Bottom Line This technique can be time consuming-but don’t overthink it. Building a college website? Go sit in the student union and talk to people.
  • 56. Navigation Card Sorting | Tree Testing
  • 57. Card Sorting Observing how users group information together
  • 58. Labels are a concern Content duplicated in multiple areas Overhauling the current grouping of content USE WHEN
  • 59. Quick & affordable Weak points exposed Easy exercise for both internal & external users PROS Large data sets are overwhelming Specialized content difficult to group Only guideline and not a structure CONS
  • 60. Create cards, recruit users, test, & analyze THE DETAILS Tasks | 1.5 – 3 weeksTimeline | Pens, paper, scissors-$15Hard Costs | Online Services-$0 to $109+
  • 61. THE TOOLS
  • 62. Bottom Line Observe your users on-site, don’t use too many cards & include internal stakeholders
  • 63. Tree Testing Users search through a site skeleton to complete a task
  • 64. Continue to address label concerns Help client visualize flow Validate the new structure is working – findability of items and groups USE WHEN
  • 65. Able to iterate quickly to fine tune deeper levels Helps visual stakeholders find holes Quick & affordable PROS Can’t test cross- linking, search, & design impacts Iterations add to the project timeline Can only test large sites in pieces CONS
  • 66. Create site map, create tasks, recruit users, test, analyze, revise, and repeat THE DETAILS Tasks | 2.5 – 4 weeksTimeline | Paper-$0Hard Costs | Online Services-$0 to $109+
  • 67. THE TOOLS
  • 68. Bottom Line If you have available users, give it a try! Just make sure to save time and budget to iterate
  • 69. Content Readability Formulas | Cloze Testing
  • 70. 21% of US adults read below a 5th grade level US Department of Education, National Institute of Literacy April 2013 ‘’ ‘’
  • 71. Complex topics need to be communicated clearly Creating content, use as a guide to improve Critical to convince your user to take an action based on the content USE WHEN
  • 72. Readability Formulas Measure quantifiable writing elements to estimate if the content is readable
  • 73. Allows you to assess if you are writing to your users level Quick, easy, & free online tool PROS The tool won’t fix the content Can’t test layout & organization of content Test doesn’t assess meaning CONS
  • 74. THE TOOLS
  • 75. Readability measures the complexity of words and sentences Comprehension Measures whether a user can understand the meaning of sentences vs
  • 76. Cloze Testing Remove every fifth word and ask users to fill in the blank
  • 77. Way to test comprehension of select content Easy to create, facilitate, and score PROS Test doesn’t provide any qualitative feedback CONS Who doesn’t love Mad Libs? Usability testing is the best way to get comprehensive feedback
  • 78. THE TOOLS
  • 79. Bottom Line To gain useful feedback on the comprehension of content, you need to ask real users questions.
  • 80. Layout and Design Click Test
  • 81. Click Test Tracks the first place a user would click to complete a task
  • 82. Research has shown that when users’ first click down the right path, 87% eventually succeed. ‘’ ‘’
  • 83. When they click down an incorrect path, only 46% eventually succeed ‘’ ‘’Jeff Sauro Measuring Usability
  • 84. Analytics show that certain critical pages are failing but you don’t know why Conversion rate is critical to the bottom line Completion of specific tasks are critical. The user must find the right buttons and links. USE WHEN
  • 85. Large sample sizes possible No Observer Effect (live sites) Fast to create and execute PROS Click testing does not show eye movement When observing the live site, you don’t control motivations No direct feedback; you can’t ask why CONS
  • 86. Create layout/design, create tasks, recruit users, test, analyze, revise, and repeat THE DETAILS Tasks | 2.5 – 4 weeksTimeline | Online Services-$0 to $200Hard Costs |
  • 87. THE TOOLS
  • 88. Bottom Line The first click is critical to your users’ success. If you understand where and how your current site is failing, this tool can be powerful.
  • 89. Bottom Line Can tell you what users notice and don’t. Can help diagnose and help solve first- click navigation problems.
  • 90. Working Prototype A/B Testing
  • 91. A/B Testing Users compare two versions of the same page
  • 92. Content layout is a concern Working on your conversion rate optimization (CRO) Need to ensure you have the right layout and/or look and feel USE WHEN
  • 93. Great way to get a sense of users gut reaction Small changes can provide you with a big pay-off Quick process to test and iterate PROS No control over motivations Analysis has limits – missing the why of the decision CONS Have to build both A and B
  • 94. Create two options, recruit users, test, analyze, and iterate as needed THE DETAILS Tasks | 1.5 – 4 weeksTimeline | Paper or your computer -$0Hard Costs | Online Services-$0 to $300+
  • 95. THE TOOLS
  • 96. Bottom Line Can be great for focused goals. Analysis and iterations can be endless.
  • 98. There is no substitute for in-person observation The more time you put in, the more value you will receive Leave the analysis up to the experts Analytics are instrumental LESSONS LEARNED
  • 99. Testing will not make your decisions You can’t test everything, focus on high priority items Balance the results with your expertise Don’t make it too complicated-any feedback is helpful LESSONS LEARNED
  • 100. 6 RESOURCES
  • 101.
  • 102. Evaluate this session