Conducting Usability Testing with a Team of One


Published on

Here's the presentation I gave at the 2009 HighEdWeb Regional Conference at Cornell University

Published in: Technology, Design
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • Conducting Usability Testing with a Team of One

    1. 1. Conducting usability research with a team of one By Christine Kowalski
    2. 2. “ Redesign? Sure, but let’s do some usability testing first…”
    3. 3. Spring 2007 – Pre-Redesign Usability Research <ul><li>Had a TOTAL “ t h ank you for your time” budget of $150 dollars for one focus group, dozens of card sorts and usability tests, and an online web survey that was sent to over 27,000 people </li></ul><ul><li>Had 3 students help with the “ t e sting and analysis” phase of the project </li></ul>
    4. 4. “ Hooray we’re finished! Well, not quite…”
    5. 5. Fall 2008 – Post-Redesign Usability Research <ul><li>Had a TOTAL “t h ank you for your time” budget of $100 dollars for one focus group, 25 usability tests, and an online web survey that was sent to over 10,000 people </li></ul><ul><li>No students helping with the “t e sting and analysis” phase of the project, and only other “w e b team” co-worker resigned in April 2008 </li></ul>
    6. 6. Wait a minute…Total budgets of $100 and $150 - really? <ul><li>Well, no. The actual costs were… </li></ul><ul><ul><li>In 2007, $253.04 </li></ul></ul><ul><ul><ul><li>Catering = $96.24 </li></ul></ul></ul><ul><ul><ul><li>Focus group video taping = $106.80 </li></ul></ul></ul><ul><ul><ul><li>5 Starbucks gift cards = $50.00 </li></ul></ul></ul><ul><ul><li>In 2008, $277.90 </li></ul></ul><ul><ul><ul><li>Catering = $126.50 </li></ul></ul></ul><ul><ul><ul><li>Focus group video taping = $151.40 </li></ul></ul></ul>
    7. 7. So how do you do this research by yourself? <ul><li>Step back and take a look at what it’s going to take to accomplish your goals: </li></ul><ul><ul><li>Time </li></ul></ul><ul><ul><li>People </li></ul></ul><ul><ul><li>Money </li></ul></ul><ul><ul><li>Supervisor approval </li></ul></ul>
    8. 8. Or in my case, it was more like, “How do I overcome these four major hurdles?” <ul><li>“I don’t have time during the day” </li></ul><ul><li>“I don’t have enough manpower” </li></ul><ul><li>“I don’t have a budget” </li></ul><ul><li>“I don’t have buy-in from my supervisor” </li></ul>
    9. 9. Hurdle #1 - “I don’t have enough time during the day” <ul><li>Test end-users on your lunch break </li></ul><ul><li>Evaluate participants before or after work </li></ul><ul><li>Arrange to do the usability test on the weekends </li></ul><ul><li>Have your student intern perform the evaluations </li></ul><ul><li>Ask a user to take a “5-Second Test” </li></ul>How to overcome it
    10. 10. Helpful hint to overcome hurdle #1: <ul><li>More people tend to be available on campus for a focus group and usability testing at lunchtime. How many people would pass up a free lunch? </li></ul>
    11. 11. Hurdle #2 - “I don’t have enough manpower” <ul><li>Get web stat collection tools to do the work for you </li></ul><ul><li>Use a web cam to capture usability tests </li></ul><ul><li>Test fewer participants: “5 users is all you need” </li></ul><ul><li>Collect the data over a longer time-frame </li></ul>How to overcome it
    12. 12. Helpful hint to overcome hurdle #2: <ul><li>You can do the testing yourself -- just have the patience to spread the data collection out over a longer period of time. </li></ul>
    13. 13. Hurdle #3 - “I don’t have a budget” <ul><li>Use free tools to help you collect web site data </li></ul><ul><li>Ask your Admissions department, Alumni Association, and fundraising departments for freebies they might be trying to get rid of </li></ul><ul><li>Use pen/paper instead of renting video equipment </li></ul><ul><li>Ask people to volunteer their time </li></ul>How to overcome it
    14. 14. Helpful hint to overcome hurdle #3: <ul><li>No matter which web site you are testing, see what you can get for free from the fundraising departments to give away to the participants -- be resourceful! </li></ul>
    15. 15. Hurdle #4 - “I don’t have buy-in from my supervisor” <ul><li>Show how usability testing is a not big production </li></ul><ul><li>Illustrate the results/statistics of preliminary tests </li></ul><ul><li>Ask your design team to watch a usability test </li></ul><ul><li>Hold a post-focus group viewing party </li></ul>How to overcome it
    16. 16. Helpful hint to overcome hurdle #4: <ul><li>Identify which groups would benefit the most from your website getting a lot of hits. Have them champion your cause. </li></ul>
    17. 17. An overview of a few helpful tools I use… <ul><li>Userfly </li></ul><ul><li>Crazy Egg </li></ul><ul><li>Google Analytics </li></ul>
    18. 18. Userfly -- <ul><li>A web usability test capturing tool that records a screencast of your users' behavior: </li></ul><ul><li>Allows you to record 10 captures per month for free </li></ul><ul><li>It tracks everything from simple mouse movements to complex interactions with AJAX elements </li></ul>
    19. 19. Userfly - Pros and Cons <ul><li>Advantages: </li></ul><ul><ul><li>Provides info on the user's browser, each page visited, and time spent on each page </li></ul></ul><ul><ul><li>Clicking on a particular page plays back the interaction, showing where the user moved his or her mouse and where he or she clicked </li></ul></ul><ul><ul><li>After watching the capture you can decide to keep or discard it </li></ul></ul>
    20. 20. Userfly - Pros and Cons <ul><li>Disadvantages: </li></ul><ul><ul><li>The walkthrough fails to automatically pan as the user scrolls - you have to manually scroll to keep up with the mouse movements </li></ul></ul><ul><ul><li>No sound or video capture, so you can’t read the person’s facial expressions or hear what they say </li></ul></ul><ul><ul><li>You do not receive another “extra” capture if you delete one </li></ul></ul>
    21. 21. Crazy Egg -- http: //crazyegg .com/ <ul><li>A free web analytics tool that has four different ways to view your data: </li></ul><ul><li>Overlay - find out how many people clicked on each individual element [links, graphics, etc.] </li></ul><ul><li>Confetti - get user info on each click or “dot” </li></ul><ul><li>Lis t - view a full summary of raw click data </li></ul><ul><li>Heat Map - see what's hot and what's not </li></ul>
    22. 22. Crazy Egg - Overlay
    23. 23. Crazy Egg - Confetti
    24. 24. Crazy Egg - Heat Map
    25. 25. Google Analytics -- http://www. google .com/analytics/ <ul><li>A free web analytics tool that shows how visitors actually interact with your site: </li></ul><ul><li>Users can see a statistical output of graphs and charts of data, and make informed site design improvements </li></ul><ul><li>Unique feature - a clickable world map, where you can see what regions, countries, states and cities your users are from. </li></ul>
    26. 26. Google Analytics - Dashboard
    27. 27. Google Analytics - Map
    28. 28. Thank you for your time! <ul><li>Any questions? </li></ul><ul><li>Christine Kowalski </li></ul><ul><li>Web Design Specialist </li></ul><ul><li>University at Buffalo </li></ul><ul><li>[email_address] </li></ul>