Usability Testing Options

Slide 1: Usability Testing: Options to Fit Any Budget Kathy McShea 2008

Slide 2: Usability Usability is Usability is: “the effectiveness an essential and efficiency of a system part of all to meet the user’s needs, great Web resulting in user satisfaction and sites productivity.” Content Technology Usability Engineering is: “tools, techniques and user centered processes applied to achieve usability.” (c) 2008, Emerald Strategies, Inc. All Rights Reserved 2

Slide 3: Satisfaction = Loyalty Rational Loyalty Emotional Loyalty Are you satisfied? Confidence • This {brand} delivers what they promise Would you recommend? • This {brand} is a name I can always trust Integrity Would you come back? • {brand} always treats me fairly • If a problem arises, I can count on {brand} to reach a fair and satisfactory resolution Pride • I feel proud to be a {brand} customer • {brand} always treats me with respect Passion • {brand} is the perfect company for people like me • I can’t imagine a world without {brand} (c) 2008, Emerald Strategies, Inc. All Rights Reserved 3

Slide 4: The elephant in the room... VISITORS ORGANIZATIONS • 85% abandon a site due to • 50% of sales are lost because poor design visitors can’t find content • 83% leave a site if they have • 51% of sites comply with to make too many clicks to find simple usability principles what they are looking for • 62% gave up looking for an item while shopping online • 40% don’t return due to a negative experience like hard to use content (c) 2008, Emerald Strategies, Inc. All Rights Reserved 4

Slide 5: Why usability works... Ensures your Web site is: Users can: - Easily find information on your - Easy to learn and use site - Useful to the intended - Perform tasks on your site with audience confidence - Satisfying to use - Understand information (jargon, acronyms, wall of words) - Use the information to accomplish their goal - Leave satisfied with the experience, recommend and decide to return (c) 2008, Emerald Strategies, Inc. All Rights Reserved 5

Slide 6: Top 3 Benefits • Get actionable results to measure progress: are you succeeding or failing? • Align with industry best practices – don’t reinvent the wheel • Lead with facts not opinions; it is a research- based orientation (c) 2008, Emerald Strategies, Inc. All Rights Reserved 6

Slide 7: Who is your user? User characteristics: • Who is the site for? • What are the users like? Environmental characteristics: • When/where will they access the site Goal and Task Characteristics: • Why will they come to the site? • What will they do on the site? (c) 2008, Emerald Strategies, Inc. All Rights Reserved 7

Slide 8: What are the right tasks? • Assess audience goals • Assess business and using... organization goals... – Surveys – Collect data from – Call record data leadership – Interviews • Surveys – Search log analysis • Interviews – Field studies – Review relevant business documents – Conduct competitive analysis (c) 2008, Emerald Strategies, Inc. All Rights Reserved 8

Slide 9: What’s the right test? • Inspection evaluations – Expert Review with scorecard and optional report (1 week) • Human performance testing – Top Task Tune-up (3 to 4 weeks) – Participant recruitment can be done by client • Software inspection – We can run the LIFT MACHINE test in conjunction with expert review to evaluate accessibility and usability issues • Qualitative research – Web blueprint package; facilitating stakeholder and/or audience feedback with competitive benchmarking (4 to 6 weeks) (c) 2008, Emerald Strategies, Inc. All Rights Reserved 9

Slide 10: Expert Review • A scorecard maps the user experience to a “value” ladder • Prioritize: take it a step at a time, to rise above “discouraged” (c) 2008, Emerald Strategies, Inc. All Rights Reserved 10

Slide 11: Climb the ladder… • Awareness: • Satisfaction: Access Deliver an error-free to content experience to users – Does it work? Can I recover quickly? • Tactics: – Speed and reliability of the • Tactics: framework must be high – Contact info for the Web – Information must be team/Help should be visible accessible to qualified in the expected location users – Broken links are fixed – Adhere to guidelines for quickly accessibility – Content is searchable – Marketing your site – Indexes work as expected (c) 2008, Emerald Strategies, Inc. All Rights Reserved 11

Slide 12: Higher steps… • Trust: • Confidence: Do you deliver what you Is the site intuitive and promise? Are you learnable? Can I find it? transparent? • Tactics: • Tactics: – Use a privacy policy – Navigation – Populate the “about us” – Link behavior programs on their pages – Ease of use – Show content is current – Readability by adopting a review schedule – Packaging – Be authoritative, avoid – Forms duplication, use related links – Graphic display guidelines (c) 2008, Emerald Strategies, Inc. All Rights Reserved 12

Slide 13: We’ve completed over 200 Web Site evaluations using our custom scorecard to measure, usability, 508 compliance, content and policy rules. (c) 2008, Emerald Strategies, Inc. All Rights Reserved 13

Slide 14: Current ES Scorecard Usability Factors • Custom Scorecard content reviews 36 factors on feedback 23% 22% Confidence usability against best Efficiency Transparency practice standards 15% 11% Trust Navigation • Some items weighted: finding 16% 9% 4% Search Feedback total score of 100 – Providing valuable content is worth 46 points – Helping people find stuff NOTE: is worth 31 points The 36 factors could form the – Leveraging feedback basis for future revisions to is worth 23 points your editorial guidelines or publishing rules (c) 2008, Emerald Strategies, Inc. All Rights Reserved 14

Slide 15: Publishing Rules We’ve created documentation for standards and publishing rules to govern Web sites with the House Democratic Caucus and HouseNet at the U.S. House of Representatives and the AF Portal Intranet at the USAF. (c) 2008, Emerald Strategies, Inc. All Rights Reserved 15

Slide 16: Expert Review pros and cons Advantages Disadvantages - The yardstick you use is best - Uniform understanding of practice standards and terms and definitions needed evidence-based research - Expert reviewer track record is - Can pave the way for critical to success embracing standards across - You must avoid single point of your organization failure: For two-sets of eyes - “Grades” are familiar use software inspection OR - Actionable report focused on second reviewer problem spots - Binary report means no - You can track improvement shades of success over time - Does not involve users - You can compare score with - Not possible to review competition features, tasks, functionality (c) 2008, Emerald Strategies, Inc. All Rights Reserved 16

Slide 17: Why More Can be Less.... People judge their own satisfaction based on the peak moments and the end points... Too many choices = paralysis Imagined alternatives = regret Choice = escalation of expectations Today: no excuse for failure SOURCE: Barry Schwartz, author The Paradox of Choice, Why More is Less (2004) (c) 2008, Emerald Strategies, Inc. All Rights Reserved 17

Slide 18: Usability testing process • Recruit participants (NLT 8) • Create scenarios • Conduct the test – Bailey’s Usability Testing Environment – Remote testing with Go to Meeting • Analyze the results • Prepare the test report (c) 2008, Emerald Strategies, Inc. All Rights Reserved 18

Slide 20: Usability testing process • Create scenarios (user tasks) – Usually each person attempts to complete several tasks (up to 7) – The scenarios are presented in random order • Secure participants – Representative of typical users – Large enough number to represent typical users (c) 2008, Emerald Strategies, Inc. All Rights Reserved 20

Slide 21: First time • At least 95% of typical users will complete a specific task (“purchase a camera”) in less than three minutes • At least 90% of users will find information on a specific topic (“risks related to taking aspirin”) within 30 seconds • At least 95% of users will be able to find a particular report (“cancer mortality in the U.S.”) within 45 seconds (c) 2008, Emerald Strategies, Inc. All Rights Reserved 21

Slide 22: Compared with a Baseline • The average time to complete a particular task (“make an airline reservation”) will be 10% faster when using the revised Web site than when using the original Web site • At least 95% of typical users will complete a specific task (“purchase a camera”) in 10% less time than the previous test • At least 95% of users will be able to find a particular report (“cancer mortality in the U.S.”) in 10% less time than the previous test (c) 2008, Emerald Strategies, Inc. All Rights Reserved 22

Slide 23: Compared to the Competition • The average time to complete a particular task (“make an airline reservation”) will be reliably faster when using our Web site than when using a competitors Web site • At least 95% of typical users of our website will complete a specific task (“purchase a camera”) reliably faster than a competitors Web site • At least 90% of users will find information on a specific topic (“risks related to taking aspirin”) in reliably faster time than a competitors Web site (c) 2008, Emerald Strategies, Inc. All Rights Reserved 23

Slide 24: Usability Test Participants Participant Gender Age Education Web Experience Test Time P1 Male 40-59 Advanced degree 10 years or more 53:50 P2 Female 40-59 Some college 5 to 9 years 40:16 P3 Male 60-75 Bachelors degree 1 to 4 years 36:29 P4 Female 40-59 Some college 5 to 9 years 29:43 P5 Female 40-59 High school 5 to 9 years 36:47 P6 Male 18-39 Bachelors degree 5 to 9 years 38:18 P7 Female 40-59 Bachelors degree 10 years or more 49:51 P8 Male 18-39 Bachelors degree 10 years or more 36:28 P9 Female 40-59 Bachelors degree 5 to 9 years 40:26 P10 Female 40-59 Some college 10 years or more 45:27 P11 Male 40-59 Bachelors degree 10 years or more 18:55 P12 Male 18-39 Some college 10 years or more 54:29 P13 Male 60-75 Advanced degree 10 years or more 37:43 P14 Female 40-59 Advanced degree 10 years or more 21:25 P15 Male 18-39 High school 5 to 9 years 37:33 P16 Female 60-75 Advanced degree 10 years or more 40:05 (c) 2008, Emerald Strategies, Inc. All Rights Reserved 24

Slide 25: Age and Education • Gender KEY QUESTION: – Female: 50% Does the participant – Male: 50% profile match your • Age target audience? – 18 - 39: 25% – 40 - 59: 56% – 60 - 74: 19% • Education – High school: 13% – Some college: 25% – Bachelor’s degree: 37% – Advanced degree: 25% (c) 2008, Emerald Strategies, Inc. All Rights Reserved 25

Slide 27: What Indicated Success Performance Results Between the old HHS site and the Redesign Scenario Text Success Rate Baseline Final Test Prototype You want to find a nursing home for a relative. 38% 88% You want to know what diabetes is and how 73% 94% you can prevent it. You want to know what housing organizations 13% 94% are available to help assist the homeless in your area. You want to know what the Fiscal Year 2001 71% 94% budget for HHS was. Your cousin is considering a career in medical 8% 88% research and asked you if HHS offers financial aid to undergraduate students. Average success rate 41% 92% (c) 2008, Emerald Strategies, Inc. All Rights Reserved 27

Slide 28: Sample Reporting Format Scenario: You want to know what diabetes is and how you can prevent it? Where would you get this information? Pathways Task Completion Notes/observations and Time -Diseases and ____Successful Conditions ____Unsuccessful -CDC Health Topics ____Time (c) 2008, Emerald Strategies, Inc. All Rights Reserved 28

Slide 29: Task Analysis pros and cons Advantages Disadvantages - Software automates - Must set up software test measures of time, clicks correctly to collect data and success - Remote testing format - Think-aloud method hides visual cues reveals cues on user - Selecting no less than 8 preferences testers critical - Good results possible with - Selection bias possible as few as 8 testers without good screening - Results helps you focus - Deeper review of content on what matters most and navigation missing - Can be repeated over - One shot test limits impact time (c) 2008, Emerald Strategies, Inc. All Rights Reserved 29

Slide 30: 3 Phases of Web Management Emerald Strategies: Web Governance in 20 Steps PHASE ONE: PHASE TWO: PHASE THREE: DRAW THE BUILD THE MANAGE THE BLUEPRINT FOUNDATION LIFECYCLE 1. EVOLVE DESIGN 1. ENDORSEMENT 1. DRAFT CHARTER Tell creative teams to Get it in writing from the Launch a Web Council, present three options with organization’s top brass: the with roles & responsibilities same navigation; meld your visionary executive. pushed to frontlines. favorite ideas. 2. SET GOALS 2. TEST Baseline current activities and 2. INVENTORY Conduct user testing before define the “problem” your Web site Use an Excel spreadsheet to launching to the public. wants to tackle. scope existing content. 3. ANALYZE AUDIENCES 3. VET Talk to internal and external Afford internal powers an 3. SET STANDARDS audiences and define your market. opportunity to review. For content that will be migrated to your site; be ready to retire or change what 4. BENCHMARK doesn’t measure up! 4. ADJUST Examine the competition to Make necessary surface best practices, new ideas. adjustments using feedback. 4. NEW CONTENT Target and develop, keeping 5. DOCUMENT REQUIREMENTS it in tune with your navigation Develop list of high-level needs; and standards. 5. ROLLOUT STRATEGY validate with stakeholder team. Mobilize outreach activities to spread your message. 5. RECRUIT CHANNEL MANAGERS 6. SELECT NAVIGATION Your core team of data Lead stakeholder team to decide stewards, knowledge 6. SCORECARD on primary and secondary experts who may have been Create metrics framework navigation with meaningful labels. part of stakeholder team. to measure performance. 6. INFORMATION 7. BRAND IT ARCHITECTURE 7. MARKET Pick a tag line and theme to match Use selected content — old Have an ongoing program your value proposition and fit your and new — and retire what of activities and upgrades. market and objectives. doesn’t fit. www.emeraldstrategies.net | Emerald Strategies, Inc. © 2004 (c) 2008, Emerald Strategies, Inc. All Rights Reserved 30

Slide 31: Draw the blueprint Endorsement Document requirements • Get it in writing from the • Develop list of high-level organization's top brass: the needs; validate with visionary executive stakeholder team. Set goals Select navigation • Baseline current activities and • Lead stakeholder team to define the "problem" your Web decide on primary and site wants to tackle. secondary navigation with Analyze audiences meaningful labels. • Talk to internal and external Brand it audiences and define your • Pick a tag line and theme to market. match your value proposition Benchmark and fit your market and • Examine the competition and objectives. surface best practices, new ideas. (c) 2008, Emerald Strategies, Inc. All Rights Reserved 31

Slide 32: Web blueprint package • When? • What? – If you are at the start of – Lead a card-sorting a site redesign project exercise to validate – If you want to avoid an navigation expensive redesign – Facilitate a session for and simply improve stakeholder feedback navigation – Benchmark capabilities – If you need a against top competitors competitive analysis (c) 2008, Emerald Strategies, Inc. All Rights Reserved 32

Slide 33: Card-sorting definitions ...a user-centered design method for increasing a system’s findability. The process involves sorting a series of cards, each labeled with a piece of content or functionality into groups that make sense to users or participants – Source: Card-sorting: a Definitive Guide by Donna Maurer and Todd Warfel “...can provide insight into users’ mental models, illuminating the way that they often tacitly group sort and label tasks and content within their own heads” – Source: Information Architecture for the World Wide Web (c) 2008, Emerald Strategies, Inc. All Rights Reserved 33

Slide 34: Card-sorting What is it? • Method that generates an overall structure for your information, as well as suggestions for navigation, menus and possible taxonomies Benefits? • Explore how users group items – low cost + low tech • Develop structures that are logical to users • Maximize probability of users finding information (c) 2008, Emerald Strategies, Inc. All Rights Reserved 34

Slide 35: How do you do it? • Item names to be Options: categorized printed on • Use larger cards and ask note cards, 1 item/ card whether they understood the term (yes, no or • Participants are asked to unsure) or how frequently group cards in a way that the user uses info. makes sense to them; Participants may be asked • Ask participants to to... rename cards or think of – assign cards to groups and synonyms label groups by giving each – clarify taxonomy category names (open – unmask jargon sort) – sort cards into pre-defined categories (closed sort) (c) 2008, Emerald Strategies, Inc. All Rights Reserved 35

Slide 36: Card-sorting answers questions • Do users want to see information grouped by subject, process, business group or information type? • How many potential main categories are there? (typically relates to navigation) • What should these groups be called? • Which terms/labels are logical and intuitive to users? • How similar (or different) are the needs of user groups? (c) 2008, Emerald Strategies, Inc. All Rights Reserved 36

Slide 37: Card-sorting pros and cons Advantages Disadvantages - Easy and inexpensive - Does not consider users’ - Reliable tasks - User-centric - Not an evaluation - technique (like usability Helps to develop categories that are logical testing) to users - Can be time-consuming - Helps to identify items that - Sometimes difficult to need to be renamed analyze – tools have - Helps with terminology limitations - Limited number of participants (c) 2008, Emerald Strategies, Inc. All Rights Reserved 37

Slide 38: Facilitated feedback Organization feedback Stakeholder feedback • Where is the organization • How is our world changing? today? (Customers? • What are the new market Suppliers? Competitors? conditions and forces? Investors and shareholders?) • What are the new ways to • What are our distinct win? competencies? • What are others doing? • What are our vulnerabilities? Where could we easily be • What is threatening us or what attacked? are the most urgent things for us to do? • What could we improve easily? • Who do we want to be like? • What do we feel proud and sorry (rant and rave) about? • What are the issues, problems and complaints that we have? (c) 2008, Emerald Strategies, Inc. All Rights Reserved 38

Slide 39: What are YOU ready for? • Expert Review – with scorecard – with scorecard and report • Top Task Tune-up – with client recruiting participants – with participant recruitment • Web Blueprint Package (c) 2008, Emerald Strategies, Inc. All Rights Reserved 39

Slide 40: Contact us Kathy McShea Owner & Founder Emerald Strategies, Inc. 202.543.2112 Web Management & Marketing ~ http:// www.emeraldstrategies.net Media Training ~ http://www.prclinic.com Join my professional network ~ http:// www.linkedin.com/in/kathymcshea (c) 2008, Emerald Strategies, Inc. All Rights Reserved 40

Slideshare.net (beta)