Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Usability Testing for People w/ Disabilities

10,389 views

Published on

Learn best practices, tips and tricks to usability testing with people with disabilities.

Published in: Technology, Design
  • Slide 35 makes an excellent and key lesson learned - that when testing with PWDs, the app/website itself needs to be accessible first, and only then can it be tested for usability. If the app can't even be used, its not going to even be able to be "usability tested" - hence even more need to do usability design walkthrus BEFORE any prototype code is even written.
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Slide 24 assumes that note takers and observers understand how AT (and its strengths and weaknesses) plays a major role in the observed expereince of the PWDs, and perhaps more importantly, the expereince of the PWD user should also be considered and well understood early (before greeting them). Power AT users and novice AT users have very different expereinces and usability observed results, but otherwise have the same disability.
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Slide 16 should mention the need for "sponsor users" of specific applications, not just general users that use public web sites.
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Slide 13 & 14 do not take into consideration realities and conflicts of security, especially security of physical computer/phone network access. Many early walkthru and test need to be done inside a company's network and physically secure lab. Server and cloud based AT and testing/recording tools also have to be able to be used inside the secure network.
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Slide 10 is confusing. "Including PWDs " is not equal to "Testing with PWDs". PWDs can and should be included BEFORE it is coded and ready to "test". Inclusive Design Thinking includes PWDs as early as possible even with paper walk thru's.
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

Usability Testing for People w/ Disabilities

  1. 1. Usability Testing for People with DisabilitiesKathleen Wahlbin Mary Hunter UttInteractive Accessibility, Inc. The Paciello Group, LLCEmail: KathyW@ia11y.com Email: mary.h.utt@gmail.comOffice: (978) 443-0798 Office: (978) 618-9772 © 2012 Interactive Accessibility, Inc. All rights reserved.
  2. 2. “Web for All. Web on Everything” http://www.w3.org/Consortium/mission 2
  3. 3. Higher confidence, more satisfied, less frustratedSuccess 3 Source: Beyond Alt Text Jacob Nielsen 2002
  4. 4. Accessible ≠ Usable 4
  5. 5. the extent to which a product can beused by specified users toachieve specified goals witheffectiveness, efficiency and satisfactionin a specified context of useISO 9241-11 5
  6. 6. The Is an approach thatincorporates direct userfeedback throughout thedevelopment cycle in order to reducecosts and create products and toolsthat meet the users needsUsability Professionals Association 6
  7. 7. The “… making sure thatsomething works well: that aperson of average (or even below average)ability and experience can use the thing – withoutfor its intended purposegetting hopelesslyfrustrated Steve Krug, author of Don’t Make Me Think 7
  8. 8. Usability TestingQuantifying the user-friendliness of a product Satisfaction Error Learnability Prevention Efficiency Effectiveness 8
  9. 9. Usability testing has biggest impact earlyImpactonDesign Early Development Prototype Production Time
  10. 10. When to include people with disabilities (PWD)?• First, test with people without disabilities• After accessibility guidelines have been verified• When it makes sense to test 10
  11. 11. Planning is key to the success Prepare Analyze •Choose location • Conduct test • Prepare •Determine tasks sessions findings •Recruit • Analyze •Define audiences • Record results participants results •Environment •Materials Plan Test Report 11
  12. 12. Plan: Many location options• Lab• Conference room• Café• Remote testing• Informal test Picture Sources: http://www.flickr.com/photos/4483970 9@N07/5739824042 12 http://www.uqul.uq.edu.au/
  13. 13. Lab vs. RemoteTraditional Lab Remote Testing• Pros • Pros – Easy to observe users and body language – Cost effective – Controlled environment – No travel – easier scheduling / coordination – Easy to record user and screen / audio – Large pool of testers - easier to recruit – No communication glitches – Realistic, representative environment• Cons ○ Their hardware / software ○ Their AT – devices and applications – Transportation and coordination ○ Their settings – AT and settings might not match ○ Often, a helper set up their AT for them • Cons ○ AT differences can distract from real test – Cannot see the participants ○ User might combine AT – hard to match – Does not show magnified screen – Expensive – Problems downloading and installing – Can only draw from local user base desktop sharing – Lab and building must be accessible – Participants can be interrupted or distracted 13
  14. 14. Remote testing can be effective and economical• Moderator / Note-taker – Screen sharing: Acrobat Connect, Skype, Join.me – Recording: Morae, Camtasia, Acrobat Connect – Robust computer – Speakerphone• Participant – High-speed internet access – Speakerphone – Robust computer – Camera (optional) 14
  15. 15. Plan: Design good tasks “Log in and locate your inbox.”“Set up a distribution listof 5 recipients and sendthem a memo that is onyour desktop.” 15
  16. 16. Plan: Who should be included?• Determine the purpose of testing – What are the core tasks?• Define the user profiles – Who uses the product? – Determine which categories of disabilities apply to your site• Find suitable participants – Participant’s background and skills should represent the product’s intended user 16
  17. 17. Plan: How many participants? 17 Source: Jacob Nielsen; http://www.useit.com/alertbox/20000319.html
  18. 18. Prepare: Recruiting - the hard part• Organizations for specific disabilities or conditions• Local disability-related support groups• Rehabilitation or disability services (government, university, local programs)• AT providers• Client contacts: customer support or sales• Notice boards or mailing lists• Social media: Facebook, Twitter, LinkedIn• Classified ads: Craigslist• Recruitment agency or market research agency• Participants 18
  19. 19. Recruiting: Lessons LearnedDo Don’t – Allow 1-2 weeks for recruiting – Use the same participants – Create a screener and ask – Use participants who are familiar participants how they access with the site or application websites – Rely on family and friends – Get multiple contact methods – Explain too much about the – Send meeting invitation and purpose of the test follow up with participants the – Be afraid to turn them down if day before they do not fit the profile – Get consent over email – Ask for referrals – Get backup participants 19
  20. 20. Recruiting: What should we pay?• Usually $50-$100 depending on length of session• Gift cards are easy• Consider options for international participants 20
  21. 21. Prepare: Don’t forget the paperwork Script Ensure consistent instructionsConsent Forms Official acknowledgement for taking part NDA If needed Waivers Permission for recordingsQuestionnaires Gauge satisfaction 21
  22. 22. Prepare: Determine how to collect data 22 Source: UserFocus http://www.userfocus.co.uk/resources/datalogger.html
  23. 23. Prepare: Do a dry run, test the environment• Is the environment accessible?• How much time is needed for the setup?• Is the paperwork and test script material ok? 23
  24. 24. Test: Six steps for conducting a good test Greet the participant Explain the study, your role and their role Perform tasks Ask satisfaction questions Debrief participants Debrief note taker and observers 24
  25. 25. Conducting the Test: Be a good moderatorDo Don’t• Be friendly • Start the test unless they have• Give clear instructions to the signed a consent/release form participant prior to tasks • Assume what the participants• Stay objective and detached need• Give encouraging but non- • Give hints or ask leading questions committal feedback • Use industry jargon• Ask if you don’t understand why a • Accept just yes/no for answers users did an action• Know when to stop a task, but don’t rescue a person too soon 25
  26. 26. If the participant says “hmm”, “oops” or “I wonder” What made you say that? Do you have questions or comments? Describe the steps you’re going through here 26
  27. 27. If the participant is silent for 10-20 seconds… What are you What are you thinking? doing now? 27
  28. 28. If the participant decides to stop… What would you Is that what you do next? expected to happen? Walk me through what you did Do you have any comments? 28
  29. 29. Test: Take good notes 29
  30. 30. Conduct satisfaction questionnaire• Three standard sets of questions: – Standard usability measurement inventory (SUMI) – Website analysis and measurement inventory (WHAMMI) – System usability scale (SUS) 30
  31. 31. Test: Wrap up the session Review the participant’s test session Understand the issues, difficulties and omissions Debrief all observers and note takers Prepare short summary of the session and the results 31
  32. 32. Results: Analyze performance metrics Completion Are users able to complete the task?Time on Task How long does it take to complete the task? Page Views How many pages to complete the task? Errors How many per task and the severity? Satisfaction How does the user rate the product? 32
  33. 33. Results: Find patterns in the data 33
  34. 34. Report• Determine top items based on the data and observations, not opinion• Report results for product improvement• Provide recommendations by AT• Review recommendations with designers and developers 34
  35. 35. Lessons learned• Be prepared for technology issues for remote testing• Incomplete or buggy prototypes can prevent users from completing tasks – Code freeze – Not sufficiently tested for accessibility• Not scheduling enough time in between sessions – In case user takes longer than planned – To collect notes and results while it’s fresh 35
  36. 36. Key Points – Usable accessibility benefits all• You can test usability formally or informally• Doesn’t have to cost a lot of money• Provides a richer understanding of how people with disabilities use the product• Identifies issues that lead to more usable and accessible products for all users• Value comes from getting as close as possible to what real people are doing with your product 36
  37. 37. Questions? http://www.slideshare.net/kwahlbin 37

×