Usability Testing For Business ROI


Published on

Even after over a decade of business growth online, it seems usability testing remains one of the best-kept secrets on the web, in terms of ways to separate you from your competition and place you well ahead of the curve. How can you make your site easier to navigate? What testing methods work best to identify and exploit key performance enhancements? Find this out and more from one of the CMS world\'s leaders, Merav Knafo.

As a business owner herself, Merav understands what you\'re looking for in this information-packed session. Her no-nonsense, (no-bull) approach is refreshing and to-the-point. But most of all, her information and insights will prove to be extremely valuable to you, the minute you choose to take her advice and apply it on your own sites.

What You Will Learn:

* What is usability testing
* Who should perform usability testing
* Different usability testing methods, pros and cons of each
* Watch a real test and use a tool to analyze it
* Usability testing cycle
* Tips and tricks to get more out of your testing

Usability Testing For Business ROI

  1. 1. The Best-kept-Secret for a Successful Website: Merav Knafo Founder Usability Testing
  2. 2. We will discuss: <ul><li>What is Usability Testing </li></ul><ul><li>Who should perform Usability Testing </li></ul><ul><li>Why you should perform Usability Testing </li></ul><ul><li>Usability Testing Methods </li></ul><ul><li>Make a plan for you to get started </li></ul><ul><li>Watch a Usability Testing Video and analyze it </li></ul>
  3. 3. What is… <ul><li>Usability? </li></ul><ul><li>Usability Testing? </li></ul><ul><li>ROI ? </li></ul>
  4. 4. Definitions: <ul><li>Usability: </li></ul><ul><li>Actions/changes you make on your site to make it more user-friendly </li></ul><ul><li>Usability Testing: </li></ul><ul><li>Observing people's attempts to use a website in order to test how user-friendly it is </li></ul><ul><li>ROI - Return on Investment </li></ul><ul><li>A measure of how much return you receive from each marketing dollar. ROI= (Profit – Investment)/Investment </li></ul>
  5. 5. The formula for website success is: <ul><li>Where </li></ul><ul><li>B = amount of business done by the site </li></ul><ul><li>V = unique visitors coming to the site </li></ul><ul><li>C = conversion rate </li></ul><ul><li>L = loyalty rate </li></ul>Usability Return on Investment (ROI) By Jakob Nielsen, J.M. Berger, and Shuli Gilutz B = V x C x L Jakob Nielsen
  6. 6. Fact: <ul><li>Doing Usability DOUBLES ROI </li></ul>Usability Return on Investment (ROI) By Jakob Nielsen, J.M. Berger, and Shuli Gilutz
  7. 7. The Problem: Most sites have a weak usability that effects their ROI 80% of all design projects have zero investment in usability Usability Return on Investment (ROI) By Jakob Nielsen, J.M. Berger, and Shuli Gilutz Don’t do any usability Do usability (at least some)
  8. 8. It’s safe to say that… <ul><li>Designing a website without Usability Testing is like using a computer blindfolded. </li></ul>
  9. 9. What to expect if you don’t do Usability Testing <ul><li>Customer dissatisfaction </li></ul><ul><li>Lose of potential income </li></ul><ul><li>Losing money on marketing </li></ul><ul><li>Bad reputation </li></ul><ul><li>Staying behind the competition </li></ul>No Soup For You!
  10. 10. The Solution: Usability Testing <ul><li>Definition: Testing the quality and assessing the problems of your website in order to </li></ul><ul><li>Measure the site's ease of use </li></ul><ul><li>The time it takes to complete required tasks </li></ul><ul><li>The user's perception of the experience </li></ul>
  11. 11. How it works <ul><li>Tester(s) will use your site and try to perform the tasks you’ve identified in advance </li></ul><ul><li>Their actions/words will be recorded </li></ul><ul><li>Deliverables: screen capture video & a written report </li></ul>
  12. 12. What is a Task? <ul><li>Knowledge </li></ul><ul><ul><li>Decide what the site is about. </li></ul></ul><ul><ul><li>Describe our products/services. </li></ul></ul><ul><ul><li>Download a PDF. </li></ul></ul><ul><ul><li>Post a question on the forum. </li></ul></ul><ul><li>Feedback </li></ul><ul><ul><li>Contact you </li></ul></ul><ul><ul><li>Find our phone number. </li></ul></ul><ul><ul><li>Send feedback. </li></ul></ul><ul><li>Action </li></ul><ul><ul><li>Subscribe to our newsletter. </li></ul></ul><ul><ul><li>Make a donation. </li></ul></ul><ul><ul><li>Buy from our store. </li></ul></ul>UT requires identification of the tasks you want people to accomplish on your site, it can be anything:
  13. 13. Who should perform Usability Testing? <ul><li>Website managers </li></ul><ul><li>Website owners </li></ul><ul><li>Web designers </li></ul><ul><li>Web developers </li></ul><ul><li>Anyone who owns/creates/manages/architects a website! </li></ul>
  14. 14. Why should you perform Usability Testing? <ul><li>You have blind spots </li></ul><ul><li>You have no idea how people use your site till you actually see them doing so </li></ul><ul><li>You can’t possibly predict/find/be aware of all the issues of your site, no matter how good you are </li></ul>
  15. 15. Excuses for not performing usability testing on your site <ul><li>I don’t have the time </li></ul><ul><li>I don’t have the money </li></ul><ul><li>I don’t have the expertise </li></ul><ul><li>I don’t have a usability lab/special equipment </li></ul><ul><li>I wouldn’t know what to do with the results </li></ul>Yada Yada Yada “ Don't Make Me Think” – Steve Krug
  16. 16. But… <ul><li>It can take no time at all </li></ul><ul><li>It can be free or almost free </li></ul><ul><li>You need no lab or special equipment </li></ul><ul><li>You don’t need to be an expert </li></ul><ul><li>I will give you a tool for analyzing the test </li></ul>
  17. 17. Usability Testing Cycle Test Analyze Fix
  18. 18. Usability Testing Methods <ul><li>Traditional </li></ul><ul><li>Do-it-yourself </li></ul><ul><li>Expert </li></ul><ul><li>User </li></ul>
  19. 19. Traditional Usability Testing
  20. 20. Traditional Usability Testing 20 Page report You get Once usually (it’s too expensive) How often Lab with one-way mirror Where Usability professional Who performs Selected carefully Testers Quality 8-10 users to justify the cost Testers Expensive ($5,000-15,000 or more) Cost:
  21. 21. Traditional Usability Testing: Pros <ul><li>Very targeted, you can specify the exact testers you need/want </li></ul><ul><li>More accurate </li></ul><ul><li>Done by experts </li></ul><ul><li>More comprehensive than other methods </li></ul>
  22. 22. Traditional Usability Testing: Cons <ul><li>Expensive! </li></ul><ul><li>A “big deal” </li></ul><ul><li>Takes time and resources </li></ul><ul><li>Not spontaneous </li></ul>
  23. 23. Do-it-yourself Usability Testing
  24. 24. Do-it-yourself Usability Testing Video capture of test, that can be viewed later You get every few weeks throughout the development process How often Any place with an internet Where You/Someone from your team Who performs Anybody who can use the web Testers Quality 3-4 Testers $50-100 per user Cost:
  25. 25. Do-it-yourself Usability Testing: Pros <ul><li>Affordable </li></ul><ul><li>Can be pretty targeted </li></ul><ul><li>Pretty easily done </li></ul><ul><li>Pretty accurate </li></ul><ul><li>Face-to-face interaction with tester </li></ul><ul><li>Ability to interview tester after the test </li></ul>
  26. 26. Do-it-yourself Usability Testing: Cons <ul><li>Requires some planning </li></ul><ul><li>Can take some time to perform </li></ul><ul><li>Requires some patience </li></ul>
  27. 27. You will need <ul><li>Camtasia studio – download at </li></ul><ul><li>A microphone </li></ul><ul><li>Testing Script – download at </li></ul>
  28. 28. <ul><li>Post an ad on or grab a friend/neighbor </li></ul><ul><li>Install a free trial of Camtasia </li></ul><ul><li>Make sure you have a microphone or at least a webcam to record the sounds </li></ul><ul><li>Test before you start </li></ul><ul><li>Read the script (provided) to the tester </li></ul><ul><li>Start recording (important!!!) </li></ul><ul><li>Ask the tester to perform the first task </li></ul><ul><li>Make sure throughout the test that the tester talks out loud about what’s going on in their mind. Don’t let them be quiet for too long </li></ul><ul><li>Guide them with questions such as “how did it make you feel?” </li></ul><ul><li>Don’t tell them how to do things! </li></ul>How to do it yourself
  29. 29. User Usability Testing
  30. 30. User Usability Testing Video capture & Short Written report You get every few weeks throughout the development process How often Tester’s home computer Where Testers available on Who performs Anyone Testers Quality 3-4 Testers $19 per user Cost:
  31. 31. User Usability Testing: Pros <ul><li>Affordable </li></ul><ul><li>Fast </li></ul><ul><li>Easy </li></ul><ul><li>Hassle-free </li></ul>Giddyup!
  32. 32. User Usability Testing: Cons <ul><li>Users will try harder than real users to accomplish the tasks </li></ul><ul><li>No direct interaction with the tester </li></ul><ul><li>You can’t ask them questions throughout the test </li></ul><ul><li>You can’t interview the tester </li></ul><ul><li>You can’t see their facial expressions </li></ul><ul><li>Harder to target </li></ul><ul><li>No testing of real payment at this time </li></ul>
  33. 33. How to do “User Usability Testing” <ul><li>Go to </li></ul><ul><li>Register (click the “try it” button on top) </li></ul><ul><li>Enter the test request and payment info </li></ul><ul><li>Wait for the email with the result </li></ul>Click “Try It”
  34. 34. Expert Usability Testing That's me!
  35. 35. Expert Usability Testing Video capture & Written report with suggestions You get every few weeks throughout the development process How often Tester’s office computer Where Usability Expert Who performs Must be a Usability Expert Testers Quality 1 (or more) Testers $299 or more per test Cost:
  36. 36. Expert Usability Testing: Pros <ul><li>Get an expert insight of your website </li></ul><ul><li>Enjoy his/hers years of experience </li></ul><ul><li>Get a solid advice of things to do/change/fix in order to get better results </li></ul>
  37. 37. Expert Usability Testing: Cons <ul><li>Not a “Regular user” </li></ul><ul><li>May find problems where others don’t </li></ul>
  38. 38. How to do “Expert Usability Testing” <ul><li>Go to </li></ul><ul><li>Click the “buy now” button </li></ul><ul><li>Choose the service </li></ul><ul><li>Register and order </li></ul>Click “Buy Now”
  39. 39. Usability Testing Cycle Test Analyze Fix
  40. 40. 1. Analyze <ul><li>Once you have the video capture & written report, it’s time to sit down and analyze it.. </li></ul><ul><li>Watch the video & Read the report </li></ul><ul><li>Use the “Usability Testing Analysis Tool” to: </li></ul><ul><ul><li>Write down the problems </li></ul></ul><ul><ul><li>Identify the “low hanging fruits” </li></ul></ul>
  41. 41. Watching the video can be <ul><li>Frustrating </li></ul><ul><li>Annoying </li></ul><ul><li>Painful </li></ul><ul><li>And even (sometimes) boring </li></ul>
  42. 42. You will discover <ul><li>Things that you thought were obvious, are not </li></ul><ul><li>Users can’t buy/do what you want them to do </li></ul><ul><li>Things don’t look/work the same in different browsers/operating systems </li></ul><ul><li>What you think is easy to find, is not </li></ul><ul><li>Bugs </li></ul><ul><li>Broken links </li></ul><ul><li>Other disasters…that make you lose money/prospects </li></ul>
  43. 43. Before My site is awesome! I am awesome! Watching the test video: YOU
  44. 44. After My site sucks! I suck! My life sucks! Watching the test video: YOU
  45. 45. Pull yourself together! <ul><li>Start by fixing the easy things to fix </li></ul><ul><li>Then fix the harder things to fix </li></ul><ul><li>You will feel better </li></ul><ul><li>Then do another test </li></ul><ul><li>The second, third, fourth test won’t be as bad as the first one! I guarantee it! </li></ul>
  46. 46. Remember: You ARE Awesome! <ul><li>Just the fact that you are doing Usability Testing puts you way ahead of the curve! </li></ul><ul><li>I bet you that your competition is NOT doing Usability Testing (yet); you win! </li></ul>You're Spongeworthy
  47. 47. Let’s get started! <ul><li>Tasks (what are your tasks?) </li></ul><ul><li>Budget (what is your budget?) </li></ul><ul><li>Plan (Got plan?) </li></ul>Goal: Have at least one Usability Testing done on your site by the end of today!
  48. 48. Tasks What are the tasks you want users to accomplish on your website? Fill out 1-5 tasks in the “Usability Testing Request Form” (Provided)
  49. 49. Tips for filling out the Usability Testing Request Form: <ul><li>Always start by asking to perform the task “What do we do?” To make sure it’s clear. </li></ul><ul><li>Take advantage of this platform. Ask questions you would ask a Market Research , for example “Would you order our service/product, if not, why?” </li></ul>
  50. 50. Budget “… we recommend spending 10% of a project’s budget on usability, but optimal ROI will probably require spending 20% or more…” Usability Return on Investment (ROI) By Jakob Nielsen, J.M. Berger, and Shuli Gilutz
  51. 51. However… You can do Usability Testing With Any Budget: <ul><li>$0 </li></ul><ul><li>~$20 </li></ul><ul><li>~$100 </li></ul><ul><li>~$350 </li></ul><ul><li>~$1000 </li></ul>Invite some friends/neighbors <ul><li>5 testers on or </li></ul><ul><li>2 rounds of 2-3 testers on </li></ul><ul><li>1 Expert testing + 2 user testing on or </li></ul><ul><li>4 Rounds of 4 testers on </li></ul><ul><li>3 rounds of </li></ul><ul><li>1 Expert testing AND </li></ul><ul><li>3 user testing on </li></ul>One tester on
  52. 52. Make a plan <ul><li>Use the “Usability Testing Request Form” to buy at least one user test on by the end of the day! </li></ul><ul><li>Ask the person who sits next to you if they would like to exchange a “Do-it-yourself usability test” with you. Do their site and then they will do yours. Be sure to record the tests! </li></ul>So by the end of today, you will have at least 2 tests: one “User Test”, and one “Do-it-yourself” test.
  53. 53. Tasks: <ul><li>Obtain information about a product. </li></ul><ul><li>Find and read the FAQ of the product. </li></ul><ul><li>Purchase the product. </li></ul><ul><li>Download the product. </li></ul>EXPERT USABILITY TEST -- REAL EXAMPLE!
  54. 54. Watch this video – <ul><li>Here’s the Expert Usability Testing video </li></ul><ul><li>Use the “Usability Testing Video Analysis Tool” and: </li></ul><ul><li>Write down all the problems </li></ul><ul><li>Rate the problems from 1 to 5 </li></ul><ul><li>Identify the “low hanging fruits </li></ul>
  55. 55. Find the full video in:
  56. 56. Be the “Master of Your domain” <ul><li>Usability testing is affordable , easy , and effective </li></ul><ul><li>A proven way to increase your ROI </li></ul><ul><li>Identify tasks, test, analyze and repeat </li></ul><ul><li>Can be tailored to your needs and budget </li></ul><ul><li>Get ahead of the curve and the competition </li></ul>Hoochie Mama!
  57. 57. Q & A <ul><li>Ask away… </li></ul>
  58. 58. Resources <ul><li> – for some tools and great information </li></ul><ul><li> – for whitepapers and articles </li></ul><ul><li> –for some great usability test videos and reports. </li></ul>cmsexpo2009 $50 OFF for Expert Usability Testing. Promo code: Visit and enter the promo code to get your discount.
  59. 59. Thank you! Merav Knafo [email_address]