Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Moderated user testing: do's and don'ts


Published on

User testing is a fantastic method to discover problems. But why is it such a great user research method? How to make sure you recruit the right participants? How to write the right questions and tasks for your usability test? And what is your job as a moderator? This slide deck answers all your questions on usability testing!

Published in: Marketing
  • Login to see the comments

Moderated user testing: do's and don'ts

  1. 1. Do's & don'ts of moderated user testing Karl Gilis @agconsult
  2. 2. > 3.500 user tests Since 2001
  3. 3. Some of our clients
  4. 4. What will we be talking about? 1. Moderated user testing: the basics 2. What can you test? And where? 3. 8 reasons why user testing is so valuable 4. Recruiting the right users & making sure they show up 5. Writing the scenario 6. Your job on the test day itself 7. What to do next?
  5. 5. 1. Moderated user testing: the basics
  6. 6. 3 key methods to make your website better AB-tests User testing Expert review
  7. 7. What are moderated user tests? • You arrange for a number of people to perform a series of typical tasks on your website, intranet, app, … • The test takes place on an individual basis. • A moderator asks the questions and leads the user test. • Observing the user while he's using the website is key to this method.
  8. 8. 2. What can you test? And where?
  9. 9. What to test? Your website, intranet, app, … Those of your competitors Early prototype to final design
  10. 10. On every screen!
  11. 11. In-person user testing Guerilla testing Remote user testing 3 types of moderated user testing
  12. 12. User testing myth You need a test lab
  13. 13. User testing myth #2: Don’t test in the client’s office
  14. 14. 3. 8 reasons why user testing is so valuable
  15. 15. 1. Experts don't know everything
  16. 16. "The more I learn, the more I realize how much I don't know." Albert Einstein
  17. 17. 2. Launch something that people will like to use
  18. 18. Old website New website 8% less sales Investment: 150 million GBP
  19. 19. 3. Discover problems that really matter (but are hard to find with other methods)
  20. 20. 4. Discover why something is a problem
  21. 21. 5. Generate ideas for AB-testing
  22. 22. 6. Facts trump opinions
  23. 23. 7. Ignoring your own clients is difficult
  24. 24. 8. Convincing and durable proof
  25. 25. 4. Recruiting the right users & making sure they show up
  26. 26. Don't test with your mother!
  27. 27. Profile of the test users • Most important: • Belongs to your target audience • Mix between customer/non-customer • You also want a good mix between • Gender • Age • Educational level
  28. 28. Relax. Don't become too fixated on a specific mix of test users or on a ridiculously precise profile.
  29. 29. Screen your candidates
  30. 30. Speak to them beforehand!
  31. 31. Avoid people who are in it for the money
  32. 32. Don't use the word 'test'
  33. 33. Other recruiting pitfalls • Avoid professional testers • Don't recruit people with an alcohol problem  • Call every participant the day before the test to reconfirm • Make sure you have a back-up participant
  34. 34. How many people do you need?
  35. 35. Iterate
  36. 36. How do you find those users? • Pop-up or slide-in on your site • Mail your newsletter subscribers • Mail your client database • Facebook, Twitter, LinkedIn, … • Your call centre • Notice board of your physical location • Recruitment agency
  37. 37. Let's talk about money • Cash • Gift vouchers • Coupon codes • Your own product, gadgets …
  38. 38. 5. Writing the scenario
  39. 39. 1. Set the scope of the test
  40. 40. 2. Focus on the important things
  41. 41. 3. Make a list of possible questions
  42. 42. Typical pitfalls when making a scenario • Using words that are on your website • Tasks that are too specific or aren’t relevant for the participant • Book a citytrip to Rome for Valentine for 2 people, half board? • Buy a pair of running shoes, size 38, in pink • Not using plain language • Too many tasks that are too difficult
  43. 43. 6. On the test day itself
  44. 44. Relax. You are prepared.
  45. 45. As a moderator you have to combine 4 'personalities'
  46. 46. 1. Steward
  47. 47. 2. Switzerland
  48. 48. 3. Scientist
  49. 49. 4. Reporter
  50. 50. Steward: make the participant feel at ease • Ask if it was easy to find the location, offer them a refreshment… • Explain what will happen: research of an existing site, app, wireframes (I call it a 'draft version' of a website), etc. • Never use the word test. It's not a test, you're doing research. • Make it clear you're researching the website or app, not the user. They can't do anything wrong. There are no wrong answers. • Assure the user you were not involved in building the site (even if you were). Nothing the test user says will therefore flatter or insult you.
  51. 51. Steward: start gently • Move from the introduction to an interview • Ask the test user a number of questions on his experience with the site, on similar sites, on surfing in general. • Refer to what you already know about him/her (during the recruitment phase) • Start the test with a familiar or easy task • If the user is familiar with the site or app (or related ones) • Do you remember why you used the website on the most recent occasion? • Did it work well then? • We will try and do the same now. • If not, start with an easy task
  52. 52. How to deal with the scenario? • The scenario is a guideline, not a straitjacket • Try to adapt your tasks/questions to the user • Leave room for the test user's input • Ask a simple task after a failure • Mix up the order of your questions
  53. 53. Your job once you've given a task? • The best thing you can do is to just shut up. • Give people the time to think and to express their thoughts • Do not interrupt people, even if they’re on the wrong track • Of course you can ask additional questions • When depends on the situation • Interact based on their behavior, facial expressions, what they say, how they say something, …
  54. 54. Of course there's so much more… • How to respond to typical situations? • How to deal with 'difficult' users? • In what situations can you help the user? • How useful is it to ask questions after each task? • How to make sure the observers understand what's happening? • How to take notes? • How to make sure your observations and conclusions aren't biased? • How to report on a user test? • How to find solutions for the problems you discovered?
  55. 55. What to do next? • Learn more during our deep dive workshop • London – October 11 2016 • Berlin – November 7 2016 • Mail if you want an in-house training • Do a user test! • Never forget to AB-test the ideas you have to tackle the problems you discovered.
  56. 56. Get my free ebook now  10 conversion tips that work on every website  15 cases from the real world  6 best practices from leading websites   Already more than 3.216 downloads
  57. 57. Questions?