Usability Testing Basics: What's it All About? at Web SIG Cleveland
Upcoming SlideShare
Loading in...5
×
 

Usability Testing Basics: What's it All About? at Web SIG Cleveland

on

  • 2,351 views

Presented to Web SIG Cleveland on May 21, 2011 at Notre Dame College in South Euclid (Cleveland), Ohio....

Presented to Web SIG Cleveland on May 21, 2011 at Notre Dame College in South Euclid (Cleveland), Ohio.

Learn all you need to get started:
- Where you can conduct studies (does it have to be in a lab?)
- Types of studies (RITE, think aloud, etc.)
- Tips for recruiting participants
- Tips for Interacting with participants without biasing the study
- Preparing for the study (materials needed, forms, etc.)
- Guidance for analyzing the study

Statistics

Views

Total Views
2,351
Views on SlideShare
2,346
Embed Views
5

Actions

Likes
2
Downloads
45
Comments
0

1 Embed 5

http://www.webheadneeded.com 5

Accessibility

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Usability Testing Basics: What's it All About? at Web SIG Cleveland Usability Testing Basics: What's it All About? at Web SIG Cleveland Presentation Transcript

    • Web SIG ClevelandMay 21, 2011
      Usability Testing Basics: What's it All About?
    • Talking About Today
      #WebSigCLE
      @carologic
      @sos_jr
    • Agenda
      What is Usability?
      Why we test usability
      Planning studies
      Facilitating sessions
      Analyzing data
    • What is Usability?
    • Usabilityis an important characteristicof what makes a good User Experience
    • Functional Aspects
      Effective
      Efficient
      Learnable
    • User’s Perspective
      Useful experience
      Feel in control and supported
      Supplements and enhances skills and expertise
      Satisfied Delighted
    • Usability Testing
      Measures users ability to achieve specific goals of effectiveness, efficiency, and satisfaction.
    • What is it?
      Real Users doing real tasks
      Being observed
      Using prototypes or live products
    • Can Test…
      Websites, Mobile, Blenders, Airport service
      Simulations or mockups
      Early prototypes (paper, low-fi)
      Production prototypes (html, hi-fi)
      Help documentation
      Processes (receipt of materials, purchase)
    • It is not…
      Quality testing
      Full accessibility testing
      System testing
      Acceptance testing
    • Do I need a lab?
      Computer / Concept
      Participant
      Facilitator
      Observer
      Timer
      Logger
      Rubin, Jeffrey. Handbook of Usability Testing. John Wiley & Sons, Inc.; 1994.
    • Just Do It!
      Anywhere (conference room, remotely)
      Any Stage (earlier in process the better)
      Anytime (un-moderated)
      Realistic test environment
      Photo by Roebot at http://www.flickr.com/photos/roebot/2964156413/
    • Why we test Usability
    • "The biggest waste of all is building something no one wants"
      - @ericries #LeanStartupMI via @MelBugai
    • You are not the user
      Designing for someone else
      Need to step back - may miss details
      May be perfect for you, but not the user
      Honest feedback from users
      Validate understanding of tasks and context
      Unforeseen requirements
    • Rationale
      Goals being met
      Content & purpose clear
      Match expectations
      Verify product meets customer needs
      Gather information for future product development
      Comparison testing
    • Save Time & Money
      Up front by testing prototypes
      Reduce maintenance issues
      Reduce Customer Service requests
    • Minimize Human Cost
      Tiredness
      Discomfort
      Embarrassment
      Frustration
      Effort
    • Find Design Problems
      System status available
      Wording choices clear
      Placement of content
      Consistency
      Recognition, Not Recall
    • Benefits of Good UX
      Increased Usefulness
      Increased Efficiency ($$$)
      Improved Productivity
    • Benefits (continued)
      Fewer Errors
      Reduced Training Time
      Improved Acceptance
      Happy Users!
    • Planning studies
    • Define Goals
      Specific
      Measurable
      Qualitative
      Quantitative
      What you need to learn about the product and its audience
    • Scope Effort
      Consider budget, resources
      Time
      Recruiting
      Facilitating
      Analyzing
      Adding participants increases budget & time
    • Design the Test
      Select Methodology
      Based on goals
      Identify participants
      Tasks to be completed
      Team roles
    • Types of Usability Tests
      Single participant
      Co-discovery (two)
      Group usability testing (3 or more)
      Rapid iterative testing
    • Facilitation Styles
      Talk aloud/Think aloud
      Task focused (limited/no discussion)
      Cooperative usability testing
      Video review with participant after study (Retrospective)
    • Location
      Formal Lab
      On-site
      Workplace, conference room
      In home
      At a conference
      Remote
      Moderated
      Un-moderated
    • Test Roles
      Facilitator
      Participant
      “Computer” (paper prototypes)
      Note taker / data logger / timer
      Software, recording operator
      Product / Technical Expert(s)
      Observers
    • Use A Script/Guide
      Memory tool for facilitator
      Promote consistency
      Questions
      Order of questions
      List out scenarios representative of typical tasks
    • Test Guide Includes
      Welcome to participants
      Steps in study(forms, tasks, questions)
      Notes to yourself
      Reset/configuration prompts
      Thank you to participants (incentives if any)
    • Participant Materials
      Provide with
      Usernames
      Passwords
      Text for forms
      Images to upload, etc.
      Tasks in writing if complex (3x5 cards)
    • Task Building
      Rubin, Jeffrey. Handbook of Usability Testing. John Wiley & Sons, Inc.; 1994.
    • Questions
      Quality of questions correlates to quality of answers:
      Don’t lead or make assumptions
      Use participant’s words
    • Stretch & Exercise
      36
    • Question 1
      Do you regularly book your travel online to save money?
    • Alternates – Question 1
      How often do you travel?
      <listen>
      What proportion of that do you book online?
      <listen>
      Why do you book travel online?
      <listen>
    • Rationale - Question 1
      Do you regularly book your travel online to save money?
      Address one issue at a time and avoid double-barreled questions.
    • Question 2
      What are your thoughts about a new feature that allows you to instant message a travel agent with any questions as you book your travel?
    • Alternates – Question 2
      Would you like to correspond with a travel agent while you are booking travel?
      <listen>
      What are some ways that you would like to correspond with a travel agent while you are booking travel?
      <listen>
    • Rationale – Question 2
      What are your thoughts about a new feature that allows you to instant message a travel agent with any questions as you book your travel?
      This question asked the participant to predict the future.
    • Schedule & Recruit
      Schedule location for pilot and study
      Leave time between sessions
      Sessions no longer than 2 hours
      Invite Observers (key stakeholders, project managers, etc.)
    • Pilot Study – Find Problems
      Verify
      Tasks are typical
      Concept is on-track
      Time needed to complete
      Practice before going live with participants
      New ideas for follow-on questions or things to observe
      Refine script and tasks
    • How Do I Find Participants?
    • Create a Screener
      List of questions to determines who will participate
      Describe, then get details:
      Computer activities
      Use of product/service
      People who pass screener = user group
    • Which Student?
      Rick
      Connie
      http://www.flickr.com/photos/mrjkbh/ via http://creativecommons.org/licenses/by-nc-nd/2.0/deed.en
      http://www.flickr.com/photos/caharley72/ (Christopher Alison Photography) via http://creativecommons.org/licenses/by-nc-nd/2.0/
    • Representative Users
      Two weeks on average to recruit
      Primary user population
      People with disabilities
      “We are all only temporarily able-bodied. Accessibility is good for us all.”
      Get to spirit of the law (Section 508, WCAG 2.0)
      -@mollydotcom at #stirtrek 2011 via @carologic
    • Hire a Recruiter
      Allows you to focus on activity.
      Can tell if person will be a good participant.
      May already have a list they can start with.
      Good recruiters:
      find right participants.
      give regular updates.
      take care of directions, confirmations, incentives, etc.
    • If You Must Do it Yourself...
      Go where users go and intercept
      Online user groups
      Professional organizations
      Craigslist
      Online tools thru your site:http://ethnio.com
    • Final Recruiting
      Final recruit by phone.
      Ask questions that force them to talk.
      Don’t recruit non-talkers.
      Confirm participant eligibility
      Note unusual issues
      Recruit for pilot test
    • Number of Participants
      As many as possible (rarely statistically significant)
      Usability Testing Research (in 1990’s)
      5 from distinct sub-group of the user population will yield 80% of the findings (Nielsen, Virzi, Lewis)
      Assumes expert has reviewed concept for obvious issues
      Recommend:
      Early tests with 8 – 12 users per user group
      Iterative testing (3 per day, iterate, 3 new users)
      Barnum, Carol M. (Jan. 2003). What’s in a Number? STC Usability SIG Newsletter, Usability Interface. http://www.stcsig.org/usability/newsletter/0301-number.html Retrieved: 20080323
    • Honorarium
      Pay them or give product credits
      Amount varies by:
      Amount of time needed
      Their role (doctors need more to persuade them)
      Their interest, devotion to product
    • Facilitating sessions
    • Welcome & Prepare
      Offer beverage
      Express appreciation for help
      Explain purpose of research
      Sign paperwork
      Consent Form
      Non-Disclosure Agreement(s)
    • Participant Reassurance
      Make sure they are comfortable
      Not testing them, rather testing…
      Encourage feedback (positive and negative)
      “I was not involved in the design of this so you can’t hurt my feelings”
    • During Session
      Remain passive (body, face)
      Don’t confirm or reject answers
      Use participant’s words
      Listen for vocalizations
      Watch non-verbal gestures
      Encourage participant to elaborate
      Ask your question and let them talk
    • Silence is GoldenUser’s Time to Think!
      58
    • Get Them Unstuck
      Progressively give assistance
      What are you trying to do right now?
      What do you think the next step is?
      What would you do in this situation if you were at work?
      Hints
      Do you see anything that might help you? (small hint)
      Have you checked the Help? (medium hint)
      What do you think the xxx button does? (large hint)
    • Debrief Participant
      Post-Study questionnaire
      Thank for participation
      Explain honorarium and delivery
    • Analyzing data
    • Look for Patterns
      Identify repetition
      Continuation of study
      Adds cost
      Delays reporting
      Low probability of many new findings
    • Measurements
      Success
      Time on task
      % of tasks completed/not completed
      Number of steps to accomplish task
      Learning time
      Number of errors
      Number of times help consulted
      Satisfaction
    • Transform Data
      Create Findings and Recommendations
      High level or detailed report
      Think about audience
      How will it be used?
    • Reporting Includes
      Executive summary
      Positive findings
      Provide solutions for negative findings
      Provide level of effort and prioritize
      Examples, screen shots
      Quotes
      Appendix with questionnaires, test materials
    • Do UX Early & Often
      Put it on the Wall as information radiators
      Test findings
      Artifacts
      Competitor info
    • Recommended Readings
      67
    • Contact
      Carol J. Smith
      @carologic
      carol@mw-research.com
      http://www.mw-research.com
    • References
      Cato, John. User-Centered Web Design. Addison Wesley Longman; 2001.
      Hackos, JoAnn T., PhD and Redish, Janice C. User and Task Analysis for Interface Design. Wiley; 1998.
      Henry, S.L. and Martinson, M. Evaluating for Accessibility, Usability Testing in Diverse Situations. Tutorial, 2003 UPA Conference. (Activity)
      Krug, Steve. Don't Make Me Think: A Common Sense Approach to Web Usability.
      Kuniavsky, Mike. Observing the User Experience: a Practitioner's Guide to User Research. Morgan Kaufmann, 2003.
      Nielsen, Jakob and Robert L. Mack. Usability Inspection Methods. John Wiley & Sons, Inc. 1994.
      Redish, Janice (Ginny). Letting Go of the Words: Writing Web Content that Works.
      Rubin, Jeffrey and Dana Chisnell. Handbook of Usability Testing: How to Plan, Design, and Conduct Effective Tests. John Wiley & Sons, Inc.
    • Tool Considerations
      • In person or remote?
      • Lab or on-site?
      • Prototype limitations (can it be online?, is it a document or a clickable site?)
      • Number of observers, number of participants?
      • Number of facilitators?
      • Logging and video editing needs (time on task, highlight video creation)?
      • Surveys before or after?
      • Eye tracking?
    • Usability Testing Software
      • Morae
      • Ovo
      • SilverBack (Mac only)
      • UserWorks
      • Noldus
      • Tobii (Eye-tracker)
      • SMI (Eye-tracker)
      • SurveyMonkey
    • Screen Sharing Software
      GoToMeeting – http://www.gotomeeting.com
      Lotus SametimeUnyte – http://www.unyte.com
      YuuGuu -- http://www.yuuguu.com
      WebEx – http://www.webex.com
      Yugma -- https://www.yugma.com/
      Trouble Shooting: CoPilot - https://www.copilot.com/
    • Satisfaction Questionnaires
      Standard Usability Measurement Inventory (SUMI)
      office/desktop software, purchase
      50 questions
      Website Analysis and MeasureMent Inventory (WHAMMI)
      Purchase
      20 questions
      System Usability Scale (SUS)
      Free
      10 questions
    • Recommended Sites
      Usability.gov
      W3C Web Accessibility Initiative
      http://www.w3.org/WAI/
      Accessibility Standards in US (Section 508)
      http://www.access-board.gov/sec508/508standards.htm
      Jakob Nielsen
      http://www.useit.com
      UPA – professional usability organization
      http://www.usabilityprofessionals.org/