The Usability Test Process: Steps, tips, and more! Dr. Jennifer L. Bowie For Digital Rhetoric
The Design Continuum System-Centered User-Friendly User-Centered
- Do what they can, not what they should
- Often draw on stereotypes
- Reasoning not necessarily supported
- Based on untested profiles and assumptions
- Users part of the design process
- Involves research of/with users
- Includes participatory design, contextual inquiry, ethnography, and usability testing
What is Usability?
- “ A function of particular users performing particular tasks in a particular environment” (Smith et al. 68)
- The “ people who use the product can do so quickly and easily to accomplish their own tasks ” (Dumas and Redish 4)
- User-centered design, not “user-friendly”
What is Usability Testing?
- An empirical study of a product’s usability by observing actual users do real tasks with the product
- Specific usability goals/concerns
- Observing and recording the testing
Step 1: User Analysis & Profiles
- Who are your actual users? You may need to break your users into typical user categories. Consider:
- Demographics: age, sex, race, education level, cultural background, socioeconomic status,…
- Experience level with the product, with products of the same genre, with required technology,...
- people with disabilities or impairments (from color blindness and learning disabilities to more severe disabilities)
Step 1: User Analysis & Profiles Con.
- Break users into clear subgroups
- Profile/Define the characteristics of each subgroup
- Choose user profiles to test:
- Ideally users from all major profiles will be tested
- If limited testing: Choose profiles based on highest number of users in that profile or profiles that you think may have the greatest usability issues
Step 2: Decide what to Test
- Choose an overall purpose
- Example: How useable is our new website?
- Determine objectives or what you are testing for. Examples:
- Does our search engine provide usable results in the first 5 links returned?
- Are search results clear to the users?
- Performance: Can they do it?
- Understandability: Can they understand it?
- Read-and-locate: Can they find it?
Step 2: Decide what to Test con.
- Consider tasks with a high chance of user failure (complex tasks, one-of-a-kind tasks, highly abstract or technical tasks)
- Consider tasks with a high cost of user failure (tasks that require support, like help or support calls, to complete; tasks where data could be damaged or lost )
- First impressions (look and feel)
- Select performance objectives (should be individualized for each task)
- Time : How long to complete tasks, to find things, to performance procedures
- Error/Success : user errors, attempts to do/find something, numbers of times section re-read, if the task was completed successfully
Step 3: Preparing for the Testing
- Choose order of tasks: start easy, go sequential, or be random
- Create written test materials:
- Written welcome speech/ Intro to be read to user
- Pre-task and post task questionnaires & interview questions
- Recruit participants & determine “payment”
- Define team member’s roles:
- Facilitator/Briefer (necessary): Often only team member to interact with users
- Observation recorder/note taker (necessary)
- Camera operator (optional)
- Help desk operator (optional)
- Test administer (optional)
- Practice: conduct walkthroughs of the testing and if possible pilot test (the pilot test users could even be a team member)
- Prepare test environment (day of test)
Step 4: Conducting the test
- Greet & Brief participant:
- Emphasize that you are not testing them, but the product and that they should act as natural as possible
- Explain think-aloud protocol (if using)
- Emphasize how user tells you she has completed a task
- Stress that the testing is “anonymous”
- Be unbiased (especially the Facilitator/Briefer)
- Intervene carefully (avoid as much as possible)
Step 5: Analyzing the Data
- Collate data into findings:
- Top-down approach: predetermine categories of findings (like navigation, design, terminology) and go through data looking for “hits”
- Bottom-up approach: put each observation on a sticky note/note card, sort into categories and label categories
- Determine time and errors/success
- Examine findings for each user, user profile, and task
- Use analysis techniques such as statistics (even averages help)
Step 5: Analyzing the Data con.
- Determine cause of problems
- Determine scope/severity of problems
- Make recommendations/changes
Good Luck & Have Fun!
- Barnum Usability Testing and Research
- Barker Writing Software Documentation, Chapter 6 “Conducting Usability Tests”
- Hom “General Concepts of Usability Testing” http://jthom.best.vwh.net/usability/general.htm