Your SlideShare is downloading. ×

Usability Test Process


Published on

  • Be the first to comment

  • Be the first to like this

No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

No notes for slide


  • 1. The Usability Test Process: Steps, tips, and more! Dr. Jennifer L. Bowie For Digital Rhetoric
  • 2. The Design Continuum System-Centered User-Friendly User-Centered
    • Users dumb
    • Users same as us
    • Will use regardless
    • Bells and whistles
    • Do what they can, not what they should
    • Consider the audiences
    • Users will like this
    • Often draw on stereotypes
    • Reasoning not necessarily supported
    • Based on untested profiles and assumptions
    • Users valuable
    • Users part of the design process
    • Early focus on users
    • Iterative
    • Involves research of/with users
    • Includes participatory design, contextual inquiry, ethnography, and usability testing
  • 3. What is Usability?
    • “ A function of particular users performing particular tasks in a particular environment” (Smith et al. 68)
    • The “ people who use the product can do so quickly and easily to accomplish their own tasks ” (Dumas and Redish 4)
    • User-centered design, not “user-friendly”
  • 4. What is Usability Testing?
    • An empirical study of a product’s usability by observing actual users do real tasks with the product
    • Involves:
      • Real users
      • Real tasks
      • Specific usability goals/concerns
      • Observing and recording the testing
      • Data analysis
  • 5. Step 1: User Analysis & Profiles
    • Who are your actual users? You may need to break your users into typical user categories. Consider:
      • Demographics: age, sex, race, education level, cultural background, socioeconomic status,…
      • Experience level with the product, with products of the same genre, with required technology,...
      • Other things:
        • motivation
        • learning style
        • subject matter knowledge
        • location of use
        • physical characteristics
        • people with disabilities or impairments (from color blindness and learning disabilities to more severe disabilities)
  • 6. Step 1: User Analysis & Profiles Con.
    • Create user profiles:
      • Break users into clear subgroups
      • Profile/Define the characteristics of each subgroup
    • Choose user profiles to test:
      • Ideally users from all major profiles will be tested
      • If limited testing: Choose profiles based on highest number of users in that profile or profiles that you think may have the greatest usability issues
  • 7. Step 2: Decide what to Test
    • Choose an overall purpose
        • Example: How useable is our new website?
    • Determine objectives or what you are testing for. Examples:
        • Does our search engine provide usable results in the first 5 links returned?
        • Are search results clear to the users?
    • Choose type of test:
        • Performance: Can they do it?
        • Understandability: Can they understand it?
        • Read-and-locate: Can they find it?
  • 8. Step 2: Decide what to Test con.
    • Select tasks:
      • Consider tasks with a high chance of user failure (complex tasks, one-of-a-kind tasks, highly abstract or technical tasks)
      • Consider tasks with a high cost of user failure (tasks that require support, like help or support calls, to complete; tasks where data could be damaged or lost )
      • Consider:
        • First impressions (look and feel)
        • First tasks
        • Tasks most performed
        • Critical tasks
        • Specific problem areas
        • New task for the product
    • Select performance objectives (should be individualized for each task)
      • Time : How long to complete tasks, to find things, to performance procedures
      • Error/Success : user errors, attempts to do/find something, numbers of times section re-read, if the task was completed successfully
  • 9. Step 3: Preparing for the Testing
    • Choose order of tasks: start easy, go sequential, or be random
    • Create written test materials:
        • Task list for users
        • Written welcome speech/ Intro to be read to user
        • Consent forms
        • Observation forms
        • Pre-task and post task questionnaires & interview questions
        • Other materials
    • Recruit participants & determine “payment”
    • Define team member’s roles:
        • Facilitator/Briefer (necessary): Often only team member to interact with users
        • Observation recorder/note taker (necessary)
        • Camera operator (optional)
        • Help desk operator (optional)
        • Test administer (optional)
    • Create written test plan
    • Practice: conduct walkthroughs of the testing and if possible pilot test (the pilot test users could even be a team member)
    • Prepare test environment (day of test)
  • 10. Step 4: Conducting the test
    • Greet & Brief participant:
        • Read/say welcome
        • Emphasize that you are not testing them, but the product and that they should act as natural as possible
        • Explain think-aloud protocol (if using)
        • Emphasize how user tells you she has completed a task
        • Stress that the testing is “anonymous”
    • Be unbiased (especially the Facilitator/Briefer)
    • Intervene carefully (avoid as much as possible)
    • Observe and record data
    • Debrief user
  • 11. Step 5: Analyzing the Data
    • Collate data into findings:
        • Choose an approach:
          • Top-down approach: predetermine categories of findings (like navigation, design, terminology) and go through data looking for “hits”
          • Bottom-up approach: put each observation on a sticky note/note card, sort into categories and label categories
        • Determine time and errors/success
          • Examine findings for each user, user profile, and task
          • Use analysis techniques such as statistics (even averages help)
  • 12. Step 5: Analyzing the Data con.
    • Analyze data:
        • Determine cause of problems
        • Determine scope/severity of problems
        • Make recommendations/changes
    • Report Findings
  • 13. Good Luck & Have Fun!
    • Where to find out more:
    • Barnum Usability Testing and Research
    • Barker Writing Software Documentation, Chapter 6 “Conducting Usability Tests”
    • Hom “General Concepts of Usability Testing”