Usability Testing Options

Usability Testing: Options to Fit Any Budget Kathy McShea 2008

Usability is: “the effectiveness and efficiency of a system to meet the user’s needs, resulting in user satisfaction and productivity .” Usability is an essential part of all great Web sites Usability Technology Content Usability Engineering is: “ tools , techniques and user centered processes applied to achieve usability.”

Satisfaction = Loyalty
Rational Loyalty
Are you satisfied?
Would you recommend?
Would you come back?
Emotional Loyalty
Confidence
This {brand} delivers what they promise
This {brand} is a name I can always trust
Integrity
{brand} always treats me fairly
If a problem arises, I can count on {brand} to reach a fair and satisfactory resolution
Pride
I feel proud to be a {brand} customer
{brand} always treats me with respect
Passion
{brand} is the perfect company for people like me
I can’t imagine a world without {brand}

The elephant in the room...
VISITORS
85% abandon a site due to poor design
83% leave a site if they have to make too many clicks to find what they are looking for
62% gave up looking for an item while shopping online
40% don’t return due to a negative experience like hard to use content
ORGANIZATIONS
50% of sales are lost because visitors can’t find content
51% of sites comply with simple usability principles

Why usability works...
Ensures your Web site is:
Easy to learn and use
Useful to the intended audience
Satisfying to use
Users can:
Easily find information on your site
Perform tasks on your site with confidence
Understand information (jargon, acronyms, wall of words)
Use the information to accomplish their goal
Leave satisfied with the experience, recommend and decide to return

Top 3 Benefits
Get actionable results to measure progress: are you succeeding or failing?
Align with industry best practices – don’t reinvent the wheel
Lead with facts not opinions; it is a research-based orientation

Who is your user?
User characteristics:
Who is the site for?
What are the users like?
Environmental characteristics:
When/where will they access the site
Goal and Task Characteristics:
Why will they come to the site?
What will they do on the site?

What are the right tasks?
Assess audience goals using...
Surveys
Call record data
Interviews
Search log analysis
Field studies
Assess business and organization goals...
Collect data from leadership
Surveys
Interviews
Review relevant business documents
Conduct competitive analysis

What’s the right test?
Inspection evaluations
Expert Review with scorecard and optional report (1 week)
Human performance testing
Top Task Tune-up (3 to 4 weeks)
Participant recruitment can be done by client
Software inspection
We can run the LIFT MACHINE test in conjunction with expert review to evaluate accessibility and usability issues
Qualitative research
Web blueprint package ; facilitating stakeholder and/or audience feedback with competitive benchmarking (4 to 6 weeks)

Expert Review
A scorecard maps the user experience to a “value” ladder
Prioritize: take it a step at a time , to rise above “discouraged”

Climb the ladder…
Awareness : Access to content
Tactics :
Speed and reliability of the framework must be high
Information must be accessible to qualified users
Adhere to guidelines for accessibility
Marketing your site
Satisfaction : Deliver an error-free experience to users – Does it work? Can I recover quickly?
Tactics :
Contact info for the Web team/Help should be visible in the expected location
Broken links are fixed quickly
Content is searchable
Indexes work as expected

Higher steps…
Confidence : Is the site intuitive and learnable? Can I find it?
Tactics :
Navigation
Link behavior
Ease of use
Readability
Packaging
Forms
Trust: Do you deliver what you promise? Are you transparent?
Tactics :
Use a privacy policy
Populate the “about us” programs on their pages
Show content is current by adopting a review schedule
Be authoritative, avoid duplication, use related links
Graphic display guidelines

We’ve completed over 200 Web Site evaluations using our custom scorecard to measure, usability, 508 compliance, content and policy rules.

Current ES Scorecard
Custom Scorecard reviews 36 factors on usability against best practice standards
Some items weighted: total score of 100
Providing valuable content is worth 46 points
Helping people find stuff is worth 31 points
Leveraging feedback is worth 23 points
feedback finding content NOTE: The 36 factors could form the basis for future revisions to your editorial guidelines or publishing rules

Publishing Rules We’ve created documentation for standards and publishing rules to govern Web sites with the House Democratic Caucus and HouseNet at the U.S. House of Representatives and the AF Portal Intranet at the USAF.

Expert Review pros and cons
Advantages
The yardstick you use is best practice standards and evidence-based research
Can pave the way for embracing standards across your organization
“ Grades” are familiar
Actionable report focused on problem spots
You can track improvement over time
You can compare score with competition
Disadvantages
Uniform understanding of terms and definitions needed
Expert reviewer track record is critical to success
You must avoid single point of failure: For two-sets of eyes use software inspection OR second reviewer
Binary report means no shades of success
Does not involve users
Not possible to review features, tasks, functionality

Why More Can be Less.... People judge their own satisfaction based on the peak moments and the end points... SOURCE: Barry Schwartz, author The Paradox of Choice, Why More is Less (2004) Too many choices = paralysis Imagined alternatives = regret Choice = escalation of expectations Today: no excuse for failure

Usability testing process
Recruit participants (NLT 8)
Create scenarios
Conduct the test
Bailey’s Usability Testing Environment
Remote testing with Go to Meeting
Analyze the results
Prepare the test report

Our favorite tool: UTE

Usability testing process
Create scenarios (user tasks)
Usually each person attempts to complete several tasks (up to 7)
The scenarios are presented in random order
Secure participants
Representative of typical users
Large enough number to represent typical users

First time
At least 95% of typical users will complete a specific task (“purchase a camera”) in less than three minutes
At least 90% of users will find information on a specific topic (“risks related to taking aspirin”) within 30 seconds
At least 95% of users will be able to find a particular report (“cancer mortality in the U.S.”) within 45 seconds
Objectives

Compared with a Baseline
The average time to complete a particular task (“make an airline reservation”) will be 10% faster when using the revised Web site than when using the original Web site
At least 95% of typical users will complete a specific task (“purchase a camera”) in 10% less time than the previous test
At least 95% of users will be able to find a particular report (“cancer mortality in the U.S.”) in 10% less time than the previous test
Objectives

Compared to the Competition
The average time to complete a particular task (“make an airline reservation”) will be reliably faster when using our Web site than when using a competitors Web site
At least 95% of typical users of our website will complete a specific task (“purchase a camera”) reliably faster than a competitors Web site
At least 90% of users will find information on a specific topic (“risks related to taking aspirin”) in reliably faster time than a competitors Web site
Objectives

Usability Test Participants Participants 40:05 10 years or more Advanced degree 60-75 Female P16 37:33 5 to 9 years High school 18-39 Male P15 21:25 10 years or more Advanced degree 40-59 Female P14 37:43 10 years or more Advanced degree 60-75 Male P13 54:29 10 years or more Some college 18-39 Male P12 18:55 10 years or more Bachelors degree 40-59 Male P11 45:27 10 years or more Some college 40-59 Female P10 40:26 5 to 9 years Bachelors degree 40-59 Female P9 36:28 10 years or more Bachelors degree 18-39 Male P8 49:51 10 years or more Bachelors degree 40-59 Female P7 38:18 5 to 9 years Bachelors degree 18-39 Male P6 36:47 5 to 9 years High school 40-59 Female P5 29:43 5 to 9 years Some college 40-59 Female P4 36:29 1 to 4 years Bachelors degree 60-75 Male P3 40:16 5 to 9 years Some college 40-59 Female P2 53:50 10 years or more Advanced degree 40-59 Male P1 Test Time Web Experience Education Age Gender Participant

Age and Education
Gender
Female: 50%
Male: 50%
Age
18 - 39: 25%
40 - 59: 56%
60 - 74: 19%
Education
High school: 13%
Some college: 25%
Bachelor’s degree: 37%
Advanced degree: 25%
Participants KEY QUESTION: Does the participant profile match your target audience?

We help you target common tasks users may find difficult and use task analysis studies to recommend improvements in usability.

Performance Results Between the old HHS site and the Redesign What Indicated Success 92% 41% Average success rate 88% 8% Your cousin is considering a career in medical research and asked you if HHS offers financial aid to undergraduate students. 94% 71% You want to know what the Fiscal Year 2001 budget for HHS was. 94% 13% You want to know what housing organizations are available to help assist the homeless in your area. 94% 73% You want to know what diabetes is and how you can prevent it. 88% 38% You want to find a nursing home for a relative. Final Prototype Baseline Test Success Rate Scenario Text

Sample Reporting Format
Scenario: You want to know what diabetes is and how you can prevent it? Where would you get this information?
____Successful ____Unsuccessful ____Time -Diseases and Conditions -CDC Health Topics Notes/observations Task Completion and Time Pathways

Task Analysis pros and cons
Advantages
Software automates measures of time, clicks and success
Think-aloud method reveals cues on user preferences
Good results possible with as few as 8 testers
Results helps you focus on what matters most
Can be repeated over time
Disadvantages
Must set up software test correctly to collect data
Remote testing format hides visual cues
Selecting no less than 8 testers critical
Selection bias possible without good screening
Deeper review of content and navigation missing
One shot test limits impact

3 Phases of Web Management

Draw the blueprint
Endorsement
Get it in writing from the organization's top brass: the visionary executive
Set goals
Baseline current activities and define the "problem" your Web site wants to tackle.
Analyze audiences
Talk to internal and external audiences and define your market.
Benchmark
Examine the competition and surface best practices, new ideas.
Document requirements
Develop list of high-level needs; validate with stakeholder team.
Select navigation
Lead stakeholder team to decide on primary and secondary navigation with meaningful labels.
Brand it
Pick a tag line and theme to match your value proposition and fit your market and objectives.

Web blueprint package
When?
If you are at the start of a site redesign project
If you want to avoid an expensive redesign and simply improve navigation
If you need a competitive analysis
What?
Lead a card-sorting exercise to validate navigation
Facilitate a session for stakeholder feedback
Benchmark capabilities against top competitors

Card-sorting definitions
...a user-centered design method for increasing a system’s findability. The process involves sorting a series of cards, each labeled with a piece of content or functionality into groups that make sense to users or participants
Source: Card-sorting: a Definitive Guide by Donna Maurer and Todd Warfel
“ ...can provide insight into users’ mental models, illuminating the way that they often tacitly group sort and label tasks and content within their own heads”
Source: Information Architecture for the World Wide Web

Card-sorting
What is it?
Method that generates an overall structure for your information, as well as suggestions for navigation, menus and possible taxonomies
Benefits?
Explore how users group items – low cost + low tech
Develop structures that are logical to users
Maximize probability of users finding information

How do you do it?
Item names to be categorized printed on note cards, 1 item/ card
Participants are asked to group cards in a way that makes sense to them; Participants may be asked to...
assign cards to groups and label groups by giving each category names (open sort)
sort cards into pre-defined categories (closed sort)
Options:
Use larger cards and ask whether they understood the term (yes, no or unsure) or how frequently the user uses info.
Ask participants to rename cards or think of synonyms
clarify taxonomy
unmask jargon

Do users want to see information grouped by subject, process, business group or information type?
How many potential main categories are there? (typically relates to navigation)
What should these groups be called?
Which terms/labels are logical and intuitive to users?
How similar (or different) are the needs of user groups?
Card-sorting answers questions

Card-sorting pros and cons
Advantages
Easy and inexpensive
Reliable
User-centric
Helps to develop categories that are logical to users
Helps to identify items that need to be renamed
Helps with terminology
Disadvantages
Does not consider users’ tasks
Not an evaluation technique (like usability testing)
Can be time-consuming
Sometimes difficult to analyze – tools have limitations
Limited number of participants

Facilitated feedback
Organization feedback
Where is the organization today? (Customers? Suppliers? Competitors? Investors and shareholders?)
What are our distinct competencies?
What are our vulnerabilities? Where could we easily be attacked?
What could we improve easily?
What do we feel proud and sorry (rant and rave) about?
What are the issues, problems and complaints that we have?
Stakeholder feedback
How is our world changing?
What are the new market conditions and forces?
What are the new ways to win?
What are others doing?
What is threatening us or what are the most urgent things for us to do?
Who do we want to be like?
Sample

What are YOU ready for?
Expert Review
with scorecard
with scorecard and report
Top Task Tune-up
with client recruiting participants
with participant recruitment
Web Blueprint Package

Contact us
Kathy McShea Owner & Founder Emerald Strategies, Inc. 202.543.2112 Web Management & Marketing ~ http:// www.emeraldstrategies.net Media Training ~ http:// www.prclinic.com Join my professional network ~ http:// www.linkedin.com/in/kathymcshea

http://www.emeraldstrategies.net	951
http://www.slideshare.net	30
http://www.prclinic.com	21
http://mtweblife.blogspot.com	14
http://feeds.feedburner.com	6
http://emeraldstrategies.net	4

Usability Testing Options

by Kathy McShea

Accessibility

Categories

Upload Details

Usage Rights

6 Embeds 1,026

Statistics

Usability Testing Options Webinar Transcript