Usability Test Process


Published on

  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Usability Test Process

  1. 1. The Usability Test Process: Steps, tips, and more! Dr. Jennifer L. Bowie For Digital Rhetoric
  2. 2. The Design Continuum System-Centered User-Friendly User-Centered <ul><li>Users dumb </li></ul><ul><li>Users same as us </li></ul><ul><li>Will use regardless </li></ul><ul><li>Bells and whistles </li></ul><ul><li>Do what they can, not what they should </li></ul><ul><li>Consider the audiences </li></ul><ul><li>Users will like this </li></ul><ul><li>Often draw on stereotypes </li></ul><ul><li>Reasoning not necessarily supported </li></ul><ul><li>Based on untested profiles and assumptions </li></ul><ul><li>Users valuable </li></ul><ul><li>Users part of the design process </li></ul><ul><li>Early focus on users </li></ul><ul><li>Iterative </li></ul><ul><li>Involves research of/with users </li></ul><ul><li>Includes participatory design, contextual inquiry, ethnography, and usability testing </li></ul>
  3. 3. What is Usability? <ul><li>“ A function of particular users performing particular tasks in a particular environment” (Smith et al. 68) </li></ul><ul><li>The “ people who use the product can do so quickly and easily to accomplish their own tasks ” (Dumas and Redish 4) </li></ul><ul><li>User-centered design, not “user-friendly” </li></ul>
  4. 4. What is Usability Testing? <ul><li>An empirical study of a product’s usability by observing actual users do real tasks with the product </li></ul><ul><li>Involves: </li></ul><ul><ul><li>Real users </li></ul></ul><ul><ul><li>Real tasks </li></ul></ul><ul><ul><li>Specific usability goals/concerns </li></ul></ul><ul><ul><li>Observing and recording the testing </li></ul></ul><ul><ul><li>Data analysis </li></ul></ul>
  5. 5. Step 1: User Analysis & Profiles <ul><li>Who are your actual users? You may need to break your users into typical user categories. Consider: </li></ul><ul><ul><li>Demographics: age, sex, race, education level, cultural background, socioeconomic status,… </li></ul></ul><ul><ul><li>Experience level with the product, with products of the same genre, with required technology,... </li></ul></ul><ul><ul><li>Other things: </li></ul></ul><ul><ul><ul><li>motivation </li></ul></ul></ul><ul><ul><ul><li>learning style </li></ul></ul></ul><ul><ul><ul><li>subject matter knowledge </li></ul></ul></ul><ul><ul><ul><li>location of use </li></ul></ul></ul><ul><ul><ul><li>physical characteristics </li></ul></ul></ul><ul><ul><ul><li>people with disabilities or impairments (from color blindness and learning disabilities to more severe disabilities) </li></ul></ul></ul>
  6. 6. Step 1: User Analysis & Profiles Con. <ul><li>Create user profiles: </li></ul><ul><ul><li>Break users into clear subgroups </li></ul></ul><ul><ul><li>Profile/Define the characteristics of each subgroup </li></ul></ul><ul><li>Choose user profiles to test: </li></ul><ul><ul><li>Ideally users from all major profiles will be tested </li></ul></ul><ul><ul><li>If limited testing: Choose profiles based on highest number of users in that profile or profiles that you think may have the greatest usability issues </li></ul></ul>
  7. 7. Step 2: Decide what to Test <ul><li>Choose an overall purpose </li></ul><ul><ul><ul><li>Example: How useable is our new website? </li></ul></ul></ul><ul><li>Determine objectives or what you are testing for. Examples: </li></ul><ul><ul><ul><li>Does our search engine provide usable results in the first 5 links returned? </li></ul></ul></ul><ul><ul><ul><li>Are search results clear to the users? </li></ul></ul></ul><ul><li>Choose type of test: </li></ul><ul><ul><ul><li>Performance: Can they do it? </li></ul></ul></ul><ul><ul><ul><li>Understandability: Can they understand it? </li></ul></ul></ul><ul><ul><ul><li>Read-and-locate: Can they find it? </li></ul></ul></ul>
  8. 8. Step 2: Decide what to Test con. <ul><li>Select tasks: </li></ul><ul><ul><li>Consider tasks with a high chance of user failure (complex tasks, one-of-a-kind tasks, highly abstract or technical tasks) </li></ul></ul><ul><ul><li>Consider tasks with a high cost of user failure (tasks that require support, like help or support calls, to complete; tasks where data could be damaged or lost ) </li></ul></ul><ul><ul><li>Consider: </li></ul></ul><ul><ul><ul><li>First impressions (look and feel) </li></ul></ul></ul><ul><ul><ul><li>First tasks </li></ul></ul></ul><ul><ul><ul><li>Tasks most performed </li></ul></ul></ul><ul><ul><ul><li>Critical tasks </li></ul></ul></ul><ul><ul><ul><li>Specific problem areas </li></ul></ul></ul><ul><ul><ul><li>New task for the product </li></ul></ul></ul><ul><li>Select performance objectives (should be individualized for each task) </li></ul><ul><ul><li>Time : How long to complete tasks, to find things, to performance procedures </li></ul></ul><ul><ul><li>Error/Success : user errors, attempts to do/find something, numbers of times section re-read, if the task was completed successfully </li></ul></ul>
  9. 9. Step 3: Preparing for the Testing <ul><li>Choose order of tasks: start easy, go sequential, or be random </li></ul><ul><li>Create written test materials: </li></ul><ul><ul><ul><li>Task list for users </li></ul></ul></ul><ul><ul><ul><li>Written welcome speech/ Intro to be read to user </li></ul></ul></ul><ul><ul><ul><li>Consent forms </li></ul></ul></ul><ul><ul><ul><li>Observation forms </li></ul></ul></ul><ul><ul><ul><li>Pre-task and post task questionnaires & interview questions </li></ul></ul></ul><ul><ul><ul><li>Other materials </li></ul></ul></ul><ul><li>Recruit participants & determine “payment” </li></ul><ul><li>Define team member’s roles: </li></ul><ul><ul><ul><li>Facilitator/Briefer (necessary): Often only team member to interact with users </li></ul></ul></ul><ul><ul><ul><li>Observation recorder/note taker (necessary) </li></ul></ul></ul><ul><ul><ul><li>Camera operator (optional) </li></ul></ul></ul><ul><ul><ul><li>Help desk operator (optional) </li></ul></ul></ul><ul><ul><ul><li>Test administer (optional) </li></ul></ul></ul><ul><li>Create written test plan </li></ul><ul><li>Practice: conduct walkthroughs of the testing and if possible pilot test (the pilot test users could even be a team member) </li></ul><ul><li>Prepare test environment (day of test) </li></ul>
  10. 10. Step 4: Conducting the test <ul><li>Greet & Brief participant: </li></ul><ul><ul><ul><li>Read/say welcome </li></ul></ul></ul><ul><ul><ul><li>Emphasize that you are not testing them, but the product and that they should act as natural as possible </li></ul></ul></ul><ul><ul><ul><li>Explain think-aloud protocol (if using) </li></ul></ul></ul><ul><ul><ul><li>Emphasize how user tells you she has completed a task </li></ul></ul></ul><ul><ul><ul><li>Stress that the testing is “anonymous” </li></ul></ul></ul><ul><li>Be unbiased (especially the Facilitator/Briefer) </li></ul><ul><li>Intervene carefully (avoid as much as possible) </li></ul><ul><li>Observe and record data </li></ul><ul><li>Debrief user </li></ul>
  11. 11. Step 5: Analyzing the Data <ul><li>Collate data into findings: </li></ul><ul><ul><ul><li>Choose an approach: </li></ul></ul></ul><ul><ul><ul><ul><li>Top-down approach: predetermine categories of findings (like navigation, design, terminology) and go through data looking for “hits” </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Bottom-up approach: put each observation on a sticky note/note card, sort into categories and label categories </li></ul></ul></ul></ul><ul><ul><ul><li>Determine time and errors/success </li></ul></ul></ul><ul><ul><ul><ul><li>Examine findings for each user, user profile, and task </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Use analysis techniques such as statistics (even averages help) </li></ul></ul></ul></ul>
  12. 12. Step 5: Analyzing the Data con. <ul><li>Analyze data: </li></ul><ul><ul><ul><li>Determine cause of problems </li></ul></ul></ul><ul><ul><ul><li>Determine scope/severity of problems </li></ul></ul></ul><ul><ul><ul><li>Make recommendations/changes </li></ul></ul></ul><ul><li>Report Findings </li></ul>
  13. 13. Good Luck & Have Fun! <ul><li>Where to find out more: </li></ul><ul><li>Barnum Usability Testing and Research </li></ul><ul><li>Barker Writing Software Documentation, Chapter 6 “Conducting Usability Tests” </li></ul><ul><li>Hom “General Concepts of Usability Testing” </li></ul>