Usability Testing Options


Published on

A review of usability testing options to improve your Web site performance. We cover expert reviews, task analysis and our blueprint package.

Published in: Technology, Design
No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • AGENDA Web analytics intro Rewards & Barriers Where to begin Presentation matters
  • Usability Testing Options

    1. 1. Usability Testing: Options to Fit Any Budget Kathy McShea 2008
    2. 2. Usability is: “the effectiveness and efficiency of a system to meet the user’s needs, resulting in user satisfaction and productivity .” Usability is an essential part of all great Web sites Usability Technology Content Usability Engineering is: “ tools , techniques and user centered processes applied to achieve usability.”
    3. 3. Satisfaction = Loyalty <ul><li>Rational Loyalty </li></ul><ul><li>Are you satisfied? </li></ul><ul><li>Would you recommend? </li></ul><ul><li>Would you come back? </li></ul><ul><li>Emotional Loyalty </li></ul><ul><li>Confidence </li></ul><ul><li>This {brand} delivers what they promise </li></ul><ul><li>This {brand} is a name I can always trust </li></ul><ul><li>Integrity </li></ul><ul><li>{brand} always treats me fairly </li></ul><ul><li>If a problem arises, I can count on {brand} to reach a fair and satisfactory resolution </li></ul><ul><li>Pride </li></ul><ul><li>I feel proud to be a {brand} customer </li></ul><ul><li>{brand} always treats me with respect </li></ul><ul><li>Passion </li></ul><ul><li>{brand} is the perfect company for people like me </li></ul><ul><li>I can’t imagine a world without {brand} </li></ul>
    4. 4. The elephant in the room... <ul><li>VISITORS </li></ul><ul><li>85% abandon a site due to poor design </li></ul><ul><li>83% leave a site if they have to make too many clicks to find what they are looking for </li></ul><ul><li>62% gave up looking for an item while shopping online </li></ul><ul><li>40% don’t return due to a negative experience like hard to use content </li></ul><ul><li>ORGANIZATIONS </li></ul><ul><li>50% of sales are lost because visitors can’t find content </li></ul><ul><li>51% of sites comply with simple usability principles </li></ul>
    5. 5. Why usability works... <ul><li>Ensures your Web site is: </li></ul><ul><li>Easy to learn and use </li></ul><ul><li>Useful to the intended audience </li></ul><ul><li>Satisfying to use </li></ul><ul><li>Users can: </li></ul><ul><li>Easily find information on your site </li></ul><ul><li>Perform tasks on your site with confidence </li></ul><ul><li>Understand information (jargon, acronyms, wall of words) </li></ul><ul><li>Use the information to accomplish their goal </li></ul><ul><li>Leave satisfied with the experience, recommend and decide to return </li></ul>
    6. 6. Top 3 Benefits <ul><li>Get actionable results to measure progress: are you succeeding or failing? </li></ul><ul><li>Align with industry best practices – don’t reinvent the wheel </li></ul><ul><li>Lead with facts not opinions; it is a research-based orientation </li></ul>
    7. 7. Who is your user? <ul><li>User characteristics: </li></ul><ul><li>Who is the site for? </li></ul><ul><li>What are the users like? </li></ul><ul><li>Environmental characteristics: </li></ul><ul><li>When/where will they access the site </li></ul><ul><li>Goal and Task Characteristics: </li></ul><ul><li>Why will they come to the site? </li></ul><ul><li>What will they do on the site? </li></ul>
    8. 8. What are the right tasks? <ul><li>Assess audience goals using... </li></ul><ul><ul><li>Surveys </li></ul></ul><ul><ul><li>Call record data </li></ul></ul><ul><ul><li>Interviews </li></ul></ul><ul><ul><li>Search log analysis </li></ul></ul><ul><ul><li>Field studies </li></ul></ul><ul><li>Assess business and organization goals... </li></ul><ul><ul><li>Collect data from leadership </li></ul></ul><ul><ul><ul><li>Surveys </li></ul></ul></ul><ul><ul><ul><li>Interviews </li></ul></ul></ul><ul><ul><li>Review relevant business documents </li></ul></ul><ul><ul><li>Conduct competitive analysis </li></ul></ul>
    9. 9. What’s the right test? <ul><li>Inspection evaluations </li></ul><ul><ul><li>Expert Review with scorecard and optional report (1 week) </li></ul></ul><ul><li>Human performance testing </li></ul><ul><ul><li>Top Task Tune-up (3 to 4 weeks) </li></ul></ul><ul><ul><li>Participant recruitment can be done by client </li></ul></ul><ul><li>Software inspection </li></ul><ul><ul><li>We can run the LIFT MACHINE test in conjunction with expert review to evaluate accessibility and usability issues </li></ul></ul><ul><li>Qualitative research </li></ul><ul><ul><li>Web blueprint package ; facilitating stakeholder and/or audience feedback with competitive benchmarking (4 to 6 weeks) </li></ul></ul>
    10. 10. Expert Review <ul><li>A scorecard maps the user experience to a “value” ladder </li></ul><ul><li>Prioritize: take it a step at a time , to rise above “discouraged” </li></ul>
    11. 11. Climb the ladder… <ul><li>Awareness : Access to content </li></ul><ul><li>Tactics : </li></ul><ul><ul><li>Speed and reliability of the framework must be high </li></ul></ul><ul><ul><li>Information must be accessible to qualified users </li></ul></ul><ul><ul><li>Adhere to guidelines for accessibility </li></ul></ul><ul><ul><li>Marketing your site </li></ul></ul><ul><li>Satisfaction : Deliver an error-free experience to users – Does it work? Can I recover quickly? </li></ul><ul><li>Tactics : </li></ul><ul><ul><li>Contact info for the Web team/Help should be visible in the expected location </li></ul></ul><ul><ul><li>Broken links are fixed quickly </li></ul></ul><ul><ul><li>Content is searchable </li></ul></ul><ul><ul><li>Indexes work as expected </li></ul></ul>
    12. 12. Higher steps… <ul><li>Confidence : Is the site intuitive and learnable? Can I find it? </li></ul><ul><li>Tactics : </li></ul><ul><ul><li>Navigation </li></ul></ul><ul><ul><li>Link behavior </li></ul></ul><ul><ul><li>Ease of use </li></ul></ul><ul><ul><li>Readability </li></ul></ul><ul><ul><li>Packaging </li></ul></ul><ul><ul><li>Forms </li></ul></ul><ul><li>Trust: Do you deliver what you promise? Are you transparent? </li></ul><ul><li>Tactics : </li></ul><ul><ul><li>Use a privacy policy </li></ul></ul><ul><ul><li>Populate the “about us” programs on their pages </li></ul></ul><ul><ul><li>Show content is current by adopting a review schedule </li></ul></ul><ul><ul><li>Be authoritative, avoid duplication, use related links </li></ul></ul><ul><ul><li>Graphic display guidelines </li></ul></ul>
    13. 13. We’ve completed over 200 Web Site evaluations using our custom scorecard to measure, usability, 508 compliance, content and policy rules.
    14. 14. Current ES Scorecard <ul><li>Custom Scorecard reviews 36 factors on usability against best practice standards </li></ul><ul><li>Some items weighted: total score of 100 </li></ul><ul><ul><li>Providing valuable content is worth 46 points </li></ul></ul><ul><ul><li>Helping people find stuff is worth 31 points </li></ul></ul><ul><ul><li>Leveraging feedback is worth 23 points </li></ul></ul>feedback finding content NOTE: The 36 factors could form the basis for future revisions to your editorial guidelines or publishing rules
    15. 15. Publishing Rules We’ve created documentation for standards and publishing rules to govern Web sites with the House Democratic Caucus and HouseNet at the U.S. House of Representatives and the AF Portal Intranet at the USAF.
    16. 16. Expert Review pros and cons <ul><li>Advantages </li></ul><ul><li>The yardstick you use is best practice standards and evidence-based research </li></ul><ul><li>Can pave the way for embracing standards across your organization </li></ul><ul><li>“ Grades” are familiar </li></ul><ul><li>Actionable report focused on problem spots </li></ul><ul><li>You can track improvement over time </li></ul><ul><li>You can compare score with competition </li></ul><ul><li>Disadvantages </li></ul><ul><li>Uniform understanding of terms and definitions needed </li></ul><ul><li>Expert reviewer track record is critical to success </li></ul><ul><li>You must avoid single point of failure: For two-sets of eyes use software inspection OR second reviewer </li></ul><ul><li>Binary report means no shades of success </li></ul><ul><li>Does not involve users </li></ul><ul><li>Not possible to review features, tasks, functionality </li></ul>
    17. 17. Why More Can be Less.... People judge their own satisfaction based on the peak moments and the end points... SOURCE: Barry Schwartz, author The Paradox of Choice, Why More is Less (2004) Too many choices = paralysis Imagined alternatives = regret Choice = escalation of expectations Today: no excuse for failure
    18. 18. Usability testing process <ul><li>Recruit participants (NLT 8) </li></ul><ul><li>Create scenarios </li></ul><ul><li>Conduct the test </li></ul><ul><ul><li>Bailey’s Usability Testing Environment </li></ul></ul><ul><ul><li>Remote testing with Go to Meeting </li></ul></ul><ul><li>Analyze the results </li></ul><ul><li>Prepare the test report </li></ul>
    19. 19. Our favorite tool: UTE
    20. 20. Usability testing process <ul><li>Create scenarios (user tasks) </li></ul><ul><ul><li>Usually each person attempts to complete several tasks (up to 7) </li></ul></ul><ul><ul><li>The scenarios are presented in random order </li></ul></ul><ul><li>Secure participants </li></ul><ul><ul><li>Representative of typical users </li></ul></ul><ul><ul><li>Large enough number to represent typical users </li></ul></ul>
    21. 21. First time <ul><li>At least 95% of typical users will complete a specific task (“purchase a camera”) in less than three minutes </li></ul><ul><li>At least 90% of users will find information on a specific topic (“risks related to taking aspirin”) within 30 seconds </li></ul><ul><li>At least 95% of users will be able to find a particular report (“cancer mortality in the U.S.”) within 45 seconds </li></ul>Objectives
    22. 22. Compared with a Baseline <ul><li>The average time to complete a particular task (“make an airline reservation”) will be 10% faster when using the revised Web site than when using the original Web site </li></ul><ul><li>At least 95% of typical users will complete a specific task (“purchase a camera”) in 10% less time than the previous test </li></ul><ul><li>At least 95% of users will be able to find a particular report (“cancer mortality in the U.S.”) in 10% less time than the previous test </li></ul>Objectives
    23. 23. Compared to the Competition <ul><li>The average time to complete a particular task (“make an airline reservation”) will be reliably faster when using our Web site than when using a competitors Web site </li></ul><ul><li>At least 95% of typical users of our website will complete a specific task (“purchase a camera”) reliably faster than a competitors Web site </li></ul><ul><li>At least 90% of users will find information on a specific topic (“risks related to taking aspirin”) in reliably faster time than a competitors Web site </li></ul>Objectives
    24. 24. Usability Test Participants Participants 40:05 10 years or more Advanced degree 60-75 Female P16 37:33 5 to 9 years High school 18-39 Male P15 21:25 10 years or more Advanced degree 40-59 Female P14 37:43 10 years or more Advanced degree 60-75 Male P13 54:29 10 years or more Some college 18-39 Male P12 18:55 10 years or more Bachelors degree 40-59 Male P11 45:27 10 years or more Some college 40-59 Female P10 40:26 5 to 9 years Bachelors degree 40-59 Female P9 36:28 10 years or more Bachelors degree 18-39 Male P8 49:51 10 years or more Bachelors degree 40-59 Female P7 38:18 5 to 9 years Bachelors degree 18-39 Male P6 36:47 5 to 9 years High school 40-59 Female P5 29:43 5 to 9 years Some college 40-59 Female P4 36:29 1 to 4 years Bachelors degree 60-75 Male P3 40:16 5 to 9 years Some college 40-59 Female P2 53:50 10 years or more Advanced degree 40-59 Male P1 Test Time Web Experience Education Age Gender Participant
    25. 25. Age and Education <ul><li>Gender </li></ul><ul><ul><li>Female: 50% </li></ul></ul><ul><ul><li>Male: 50% </li></ul></ul><ul><li>Age </li></ul><ul><ul><li>18 - 39: 25% </li></ul></ul><ul><ul><li>40 - 59: 56% </li></ul></ul><ul><ul><li>60 - 74: 19% </li></ul></ul><ul><li>Education </li></ul><ul><ul><li>High school: 13% </li></ul></ul><ul><ul><li>Some college: 25% </li></ul></ul><ul><ul><li>Bachelor’s degree: 37% </li></ul></ul><ul><ul><li>Advanced degree: 25% </li></ul></ul>Participants KEY QUESTION: Does the participant profile match your target audience?
    26. 26. We help you target common tasks users may find difficult and use task analysis studies to recommend improvements in usability.
    27. 27. Performance Results Between the old HHS site and the Redesign What Indicated Success 92% 41% Average success rate 88% 8% Your cousin is considering a career in medical research and asked you if HHS offers financial aid to undergraduate students. 94% 71% You want to know what the Fiscal Year 2001 budget for HHS was. 94% 13% You want to know what housing organizations are available to help assist the homeless in your area. 94% 73% You want to know what diabetes is and how you can prevent it. 88% 38% You want to find a nursing home for a relative. Final Prototype Baseline Test Success Rate Scenario Text
    28. 28. Sample Reporting Format <ul><ul><li>Scenario: You want to know what diabetes is and how you can prevent it? Where would you get this information? </li></ul></ul>____Successful ____Unsuccessful ____Time -Diseases and Conditions -CDC Health Topics Notes/observations Task Completion and Time Pathways
    29. 29. Task Analysis pros and cons <ul><li>Advantages </li></ul><ul><li>Software automates measures of time, clicks and success </li></ul><ul><li>Think-aloud method reveals cues on user preferences </li></ul><ul><li>Good results possible with as few as 8 testers </li></ul><ul><li>Results helps you focus on what matters most </li></ul><ul><li>Can be repeated over time </li></ul><ul><li>Disadvantages </li></ul><ul><li>Must set up software test correctly to collect data </li></ul><ul><li>Remote testing format hides visual cues </li></ul><ul><li>Selecting no less than 8 testers critical </li></ul><ul><li>Selection bias possible without good screening </li></ul><ul><li>Deeper review of content and navigation missing </li></ul><ul><li>One shot test limits impact </li></ul>
    30. 30. 3 Phases of Web Management
    31. 31. Draw the blueprint <ul><li>Endorsement </li></ul><ul><li>Get it in writing from the organization's top brass: the visionary executive </li></ul><ul><li>Set goals </li></ul><ul><li>Baseline current activities and define the &quot;problem&quot; your Web site wants to tackle. </li></ul><ul><li>Analyze audiences </li></ul><ul><li>Talk to internal and external audiences and define your market. </li></ul><ul><li>Benchmark </li></ul><ul><li>Examine the competition and surface best practices, new ideas. </li></ul><ul><li>Document requirements </li></ul><ul><li>Develop list of high-level needs; validate with stakeholder team. </li></ul><ul><li>Select navigation </li></ul><ul><li>Lead stakeholder team to decide on primary and secondary navigation with meaningful labels. </li></ul><ul><li>Brand it </li></ul><ul><li>Pick a tag line and theme to match your value proposition and fit your market and objectives. </li></ul>
    32. 32. Web blueprint package <ul><li>When? </li></ul><ul><ul><li>If you are at the start of a site redesign project </li></ul></ul><ul><ul><li>If you want to avoid an expensive redesign and simply improve navigation </li></ul></ul><ul><ul><li>If you need a competitive analysis </li></ul></ul><ul><li>What? </li></ul><ul><ul><li>Lead a card-sorting exercise to validate navigation </li></ul></ul><ul><ul><li>Facilitate a session for stakeholder feedback </li></ul></ul><ul><ul><li>Benchmark capabilities against top competitors </li></ul></ul>
    33. 33. Card-sorting definitions <ul><li>...a user-centered design method for increasing a system’s findability. The process involves sorting a series of cards, each labeled with a piece of content or functionality into groups that make sense to users or participants </li></ul><ul><ul><li>Source: Card-sorting: a Definitive Guide by Donna Maurer and Todd Warfel </li></ul></ul><ul><li>“ ...can provide insight into users’ mental models, illuminating the way that they often tacitly group sort and label tasks and content within their own heads” </li></ul><ul><ul><li>Source: Information Architecture for the World Wide Web </li></ul></ul>
    34. 34. Card-sorting <ul><li>What is it? </li></ul><ul><li>Method that generates an overall structure for your information, as well as suggestions for navigation, menus and possible taxonomies </li></ul><ul><li>Benefits? </li></ul><ul><li>Explore how users group items – low cost + low tech </li></ul><ul><li>Develop structures that are logical to users </li></ul><ul><li>Maximize probability of users finding information </li></ul>
    35. 35. How do you do it? <ul><li>Item names to be categorized printed on note cards, 1 item/ card </li></ul><ul><li>Participants are asked to group cards in a way that makes sense to them; Participants may be asked to... </li></ul><ul><ul><li>assign cards to groups and label groups by giving each category names (open sort) </li></ul></ul><ul><ul><li>sort cards into pre-defined categories (closed sort) </li></ul></ul><ul><li>Options: </li></ul><ul><li>Use larger cards and ask whether they understood the term (yes, no or unsure) or how frequently the user uses info. </li></ul><ul><li>Ask participants to rename cards or think of synonyms </li></ul><ul><ul><li>clarify taxonomy </li></ul></ul><ul><ul><li>unmask jargon </li></ul></ul>
    36. 36. <ul><li>Do users want to see information grouped by subject, process, business group or information type? </li></ul><ul><li>How many potential main categories are there? (typically relates to navigation) </li></ul><ul><li>What should these groups be called? </li></ul><ul><li>Which terms/labels are logical and intuitive to users? </li></ul><ul><li>How similar (or different) are the needs of user groups? </li></ul>Card-sorting answers questions
    37. 37. Card-sorting pros and cons <ul><li>Advantages </li></ul><ul><li>Easy and inexpensive </li></ul><ul><li>Reliable </li></ul><ul><li>User-centric </li></ul><ul><li>Helps to develop categories that are logical to users </li></ul><ul><li>Helps to identify items that need to be renamed </li></ul><ul><li>Helps with terminology </li></ul><ul><li>Disadvantages </li></ul><ul><li>Does not consider users’ tasks </li></ul><ul><li>Not an evaluation technique (like usability testing) </li></ul><ul><li>Can be time-consuming </li></ul><ul><li>Sometimes difficult to analyze – tools have limitations </li></ul><ul><li>Limited number of participants </li></ul>
    38. 38. Facilitated feedback <ul><li>Organization feedback </li></ul><ul><li>Where is the organization today? (Customers? Suppliers? Competitors? Investors and shareholders?) </li></ul><ul><li>What are our distinct competencies? </li></ul><ul><li>What are our vulnerabilities? Where could we easily be attacked? </li></ul><ul><li>What could we improve easily? </li></ul><ul><li>What do we feel proud and sorry (rant and rave) about? </li></ul><ul><li>What are the issues, problems and complaints that we have? </li></ul><ul><li>Stakeholder feedback </li></ul><ul><li>How is our world changing? </li></ul><ul><li>What are the new market conditions and forces? </li></ul><ul><li>What are the new ways to win? </li></ul><ul><li>What are others doing? </li></ul><ul><li>What is threatening us or what are the most urgent things for us to do? </li></ul><ul><li>Who do we want to be like? </li></ul>Sample
    39. 39. What are YOU ready for? <ul><li>Expert Review </li></ul><ul><ul><li>with scorecard </li></ul></ul><ul><ul><li>with scorecard and report </li></ul></ul><ul><li>Top Task Tune-up </li></ul><ul><ul><li>with client recruiting participants </li></ul></ul><ul><ul><li>with participant recruitment </li></ul></ul><ul><li>Web Blueprint Package </li></ul>
    40. 40. Contact us <ul><ul><li>Kathy McShea Owner & Founder Emerald Strategies, Inc. 202.543.2112 Web Management & Marketing ~ http:// Media Training ~ http:// Join my professional network ~ http:// </li></ul></ul>