Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Conducting Usability Testing with a Team of One


Published on

Here's the presentation I gave at the 2009 HighEdWeb Regional Conference at Cornell University

Published in: Technology, Design
  • Be the first to comment

  • Be the first to like this

Conducting Usability Testing with a Team of One

  1. 1. Conducting usability research with a team of one By Christine Kowalski
  2. 2. “ Redesign? Sure, but let’s do some usability testing first…”
  3. 3. Spring 2007 – Pre-Redesign Usability Research <ul><li>Had a TOTAL “ t h ank you for your time” budget of $150 dollars for one focus group, dozens of card sorts and usability tests, and an online web survey that was sent to over 27,000 people </li></ul><ul><li>Had 3 students help with the “ t e sting and analysis” phase of the project </li></ul>
  4. 4. “ Hooray we’re finished! Well, not quite…”
  5. 5. Fall 2008 – Post-Redesign Usability Research <ul><li>Had a TOTAL “t h ank you for your time” budget of $100 dollars for one focus group, 25 usability tests, and an online web survey that was sent to over 10,000 people </li></ul><ul><li>No students helping with the “t e sting and analysis” phase of the project, and only other “w e b team” co-worker resigned in April 2008 </li></ul>
  6. 6. Wait a minute…Total budgets of $100 and $150 - really? <ul><li>Well, no. The actual costs were… </li></ul><ul><ul><li>In 2007, $253.04 </li></ul></ul><ul><ul><ul><li>Catering = $96.24 </li></ul></ul></ul><ul><ul><ul><li>Focus group video taping = $106.80 </li></ul></ul></ul><ul><ul><ul><li>5 Starbucks gift cards = $50.00 </li></ul></ul></ul><ul><ul><li>In 2008, $277.90 </li></ul></ul><ul><ul><ul><li>Catering = $126.50 </li></ul></ul></ul><ul><ul><ul><li>Focus group video taping = $151.40 </li></ul></ul></ul>
  7. 7. So how do you do this research by yourself? <ul><li>Step back and take a look at what it’s going to take to accomplish your goals: </li></ul><ul><ul><li>Time </li></ul></ul><ul><ul><li>People </li></ul></ul><ul><ul><li>Money </li></ul></ul><ul><ul><li>Supervisor approval </li></ul></ul>
  8. 8. Or in my case, it was more like, “How do I overcome these four major hurdles?” <ul><li>“I don’t have time during the day” </li></ul><ul><li>“I don’t have enough manpower” </li></ul><ul><li>“I don’t have a budget” </li></ul><ul><li>“I don’t have buy-in from my supervisor” </li></ul>
  9. 9. Hurdle #1 - “I don’t have enough time during the day” <ul><li>Test end-users on your lunch break </li></ul><ul><li>Evaluate participants before or after work </li></ul><ul><li>Arrange to do the usability test on the weekends </li></ul><ul><li>Have your student intern perform the evaluations </li></ul><ul><li>Ask a user to take a “5-Second Test” </li></ul>How to overcome it
  10. 10. Helpful hint to overcome hurdle #1: <ul><li>More people tend to be available on campus for a focus group and usability testing at lunchtime. How many people would pass up a free lunch? </li></ul>
  11. 11. Hurdle #2 - “I don’t have enough manpower” <ul><li>Get web stat collection tools to do the work for you </li></ul><ul><li>Use a web cam to capture usability tests </li></ul><ul><li>Test fewer participants: “5 users is all you need” </li></ul><ul><li>Collect the data over a longer time-frame </li></ul>How to overcome it
  12. 12. Helpful hint to overcome hurdle #2: <ul><li>You can do the testing yourself -- just have the patience to spread the data collection out over a longer period of time. </li></ul>
  13. 13. Hurdle #3 - “I don’t have a budget” <ul><li>Use free tools to help you collect web site data </li></ul><ul><li>Ask your Admissions department, Alumni Association, and fundraising departments for freebies they might be trying to get rid of </li></ul><ul><li>Use pen/paper instead of renting video equipment </li></ul><ul><li>Ask people to volunteer their time </li></ul>How to overcome it
  14. 14. Helpful hint to overcome hurdle #3: <ul><li>No matter which web site you are testing, see what you can get for free from the fundraising departments to give away to the participants -- be resourceful! </li></ul>
  15. 15. Hurdle #4 - “I don’t have buy-in from my supervisor” <ul><li>Show how usability testing is a not big production </li></ul><ul><li>Illustrate the results/statistics of preliminary tests </li></ul><ul><li>Ask your design team to watch a usability test </li></ul><ul><li>Hold a post-focus group viewing party </li></ul>How to overcome it
  16. 16. Helpful hint to overcome hurdle #4: <ul><li>Identify which groups would benefit the most from your website getting a lot of hits. Have them champion your cause. </li></ul>
  17. 17. An overview of a few helpful tools I use… <ul><li>Userfly </li></ul><ul><li>Crazy Egg </li></ul><ul><li>Google Analytics </li></ul>
  18. 18. Userfly -- <ul><li>A web usability test capturing tool that records a screencast of your users' behavior: </li></ul><ul><li>Allows you to record 10 captures per month for free </li></ul><ul><li>It tracks everything from simple mouse movements to complex interactions with AJAX elements </li></ul>
  19. 19. Userfly - Pros and Cons <ul><li>Advantages: </li></ul><ul><ul><li>Provides info on the user's browser, each page visited, and time spent on each page </li></ul></ul><ul><ul><li>Clicking on a particular page plays back the interaction, showing where the user moved his or her mouse and where he or she clicked </li></ul></ul><ul><ul><li>After watching the capture you can decide to keep or discard it </li></ul></ul>
  20. 20. Userfly - Pros and Cons <ul><li>Disadvantages: </li></ul><ul><ul><li>The walkthrough fails to automatically pan as the user scrolls - you have to manually scroll to keep up with the mouse movements </li></ul></ul><ul><ul><li>No sound or video capture, so you can’t read the person’s facial expressions or hear what they say </li></ul></ul><ul><ul><li>You do not receive another “extra” capture if you delete one </li></ul></ul>
  21. 21. Crazy Egg -- http: //crazyegg .com/ <ul><li>A free web analytics tool that has four different ways to view your data: </li></ul><ul><li>Overlay - find out how many people clicked on each individual element [links, graphics, etc.] </li></ul><ul><li>Confetti - get user info on each click or “dot” </li></ul><ul><li>Lis t - view a full summary of raw click data </li></ul><ul><li>Heat Map - see what's hot and what's not </li></ul>
  22. 22. Crazy Egg - Overlay
  23. 23. Crazy Egg - Confetti
  24. 24. Crazy Egg - Heat Map
  25. 25. Google Analytics -- http://www. google .com/analytics/ <ul><li>A free web analytics tool that shows how visitors actually interact with your site: </li></ul><ul><li>Users can see a statistical output of graphs and charts of data, and make informed site design improvements </li></ul><ul><li>Unique feature - a clickable world map, where you can see what regions, countries, states and cities your users are from. </li></ul>
  26. 26. Google Analytics - Dashboard
  27. 27. Google Analytics - Map
  28. 28. Thank you for your time! <ul><li>Any questions? </li></ul><ul><li>Christine Kowalski </li></ul><ul><li>Web Design Specialist </li></ul><ul><li>University at Buffalo </li></ul><ul><li>[email_address] </li></ul>