Inclusive Usability Testing - WordCamp London
•Download as PPTX, PDF•
3 likes•1,434 views
Running tests with real users is critical for so many organizations, whether when evaluating MVPs or just as part of iterative updates. For an organization that already has embraced inclusive design, the next step is to integrate it into user testing by incorporating users with disabilities into your normal testing process. Note that this is not the same as accessibility testing. Ideally your accessibility work is done so that you can test a fully functional and accessible site/application for usability regardless of disability. I will discuss how to plan for and execute these sessions as well as pitfalls to avoid. Ideally you will walk away with high-level understanding of where to start.
Recommended
Recommended
More Related Content
Similar to Inclusive Usability Testing - WordCamp London
Similar to Inclusive Usability Testing - WordCamp London (20)
More from Adrian Roselli
More from Adrian Roselli (20)
Recently uploaded
Recently uploaded (20)
Inclusive Usability Testing - WordCamp London
- 1. Inclusive Usability Testing Presented by Adrian Roselli (@aardrian) for WordCamp London. Slides from this talk will be available at rosel.li/WCLDN.
- 2. • I’ve written some stuff, • Member of W3C, • Building for the web since 1993, • Learn more at AdrianRoselli.com, • Avoid on Twitter @aardrian. About Adrian Roselli
- 3. Types of Disabilities • Hearing Impaired • Blind • Low Vision • Physical Impairment • Cognitive
- 5. “I encourage you to fail in interacting with people with disabilities because you will learn a bunch. You will learn what not to say and what people care about. You’ll learn about where the obstacles are—both the designed, physical barriers and the constructed emotional ones that exist within yourself.” — Wendy Chisholm, http://sp1ral.com/2014/04/
- 6. Overview • Concerns • Planning • Payment • Venue • Recruitment • Accommodation • Tech • Process • Privacy
- 7. CONCERNS Photo by Save the Dream, CC BY 2.0
- 8. Concerns • Accessibility remediation must be complete.
- 9. Concerns • Accessibility remediation must be complete. • This is not accessibility testing.
- 10. Concerns • Accessibility remediation must be complete. • This is not accessibility testing. • This is not disability tourism.
- 11. Concerns • Accessibility remediation must be complete. • This is not accessibility testing. • This is not disability tourism. • Be clear on that with all stakeholders.
- 12. PLANNING Photo by Save the Dream, CC BY 2.0 Photo by benjaminlansky, CC BY 2.0
- 13. Planning • Review your tests and format.
- 14. Planning • Review your tests and format. • Are they structured? Informal? Remote?
- 15. Planning • Review your tests and format. • Are they structured? Informal? Remote? • How many participants are you planning?
- 16. Planning • Review your tests and format. • Are they structured? Informal? Remote? • How many participants are you planning? • Have you budgeted for compensating participants?
- 17. COMPENSATION Photo by Mike Mozart, CC BY 2.0
- 18. Compensation • Expect to pay participants more than you usually do.
- 19. Compensation • Expect to pay participants more than you usually do. • Additional burdens to participate:
- 20. Compensation • Expect to pay participants more than you usually do. • Additional burdens to participate: • Transportation cost,
- 21. Compensation • Expect to pay participants more than you usually do. • Additional burdens to participate: • Transportation cost, • Time off work,
- 22. Compensation • Expect to pay participants more than you usually do. • Additional burdens to participate: • Transportation cost, • Time off work, • May be underemployed,
- 23. Compensation • Expect to pay participants more than you usually do. • Additional burdens to participate: • Transportation cost, • Time off work, • May be underemployed, • Aides.
- 24. Compensation • Expect to pay participants more than you usually do. • Additional burdens to participate: • Transportation cost, • Time off work, • May be underemployed, • Aides. • Gift cards that can be accepted where they shop.
- 25. VENUE Photo by Chris Waits, CC BY 2.0
- 26. Venue • It must be accessible.
- 27. Venue • It must be accessible. • Not just the building, but the entire route.
- 28. Venue • It must be accessible. • Not just the building, but the entire route. • Bus line, transport services, etc.
- 29. Venue • It must be accessible. • Not just the building, but the entire route. • Bus line, transport services, etc. • Meet them at the door.
- 30. Venue • It must be accessible. • Not just the building, but the entire route. • Bus line, transport services, etc. • Meet them at the door. • Relief area for service animals.
- 31. Venue • It must be accessible. • Not just the building, but the entire route. • Bus line, transport services, etc. • Meet them at the door. • Relief area for service animals. • Exiting the venue.
- 32. RECRUITMENT Photo by NASA Goddard Space Flight Center, CC BY 2.0
- 33. Recruitment • Now you can find participants!
- 34. Recruitment • Now you can find participants! • Community / support organizations:
- 35. Recruitment • Now you can find participants! • Community / support organizations: • Demographics,
- 36. Recruitment • Now you can find participants! • Community / support organizations: • Demographics, • Name recognition,
- 37. Recruitment • Now you can find participants! • Community / support organizations: • Demographics, • Name recognition, • Existing relationships,
- 38. Recruitment • Now you can find participants! • Community / support organizations: • Demographics, • Name recognition, • Existing relationships, • Contextual support.
- 39. Recruitment • Now you can find participants! • Community / support organizations: • Demographics, • Name recognition, • Existing relationships, • Contextual support. • Let the organization recruit.
- 40. ACCOMMODATION Photo by David Luders, CC BY 2.0
- 41. Accommodation • Build extra time for every task.
- 42. Accommodation • Build extra time for every task. • Allow them to be late.
- 43. Accommodation • Build extra time for every task. • Allow them to be late. • Service animals, canes, etc. do not play well with tripods and cables.
- 44. Accommodation • Build extra time for every task. • Allow them to be late. • Service animals, canes, etc. do not play well with tripods and cables. • Service animals need a clear space under the table.
- 45. Accommodation • Build extra time for every task. • Allow them to be late. • Service animals, canes, etc. do not play well with tripods and cables. • Service animals need a clear space under the table. • Different seating options: widths, arms, wheels, etc.
- 46. Accommodation • Build extra time for every task. • Allow them to be late. • Service animals, canes, etc. do not play well with tripods and cables. • Service animals need a clear space under the table. • Different seating options: widths, arms, wheels, etc. • A place to park scooters, chairs, etc.
- 47. TECH Photo by City University Interaction Lab, CC BY 2.0
- 48. Tech • Use the participant’s system (or community org system).
- 49. Tech • Use the participant’s system (or community org system). • For mobile testing, do not use mounts.
- 50. Tech • Use the participant’s system (or community org system). • For mobile testing, do not use mounts. • Do not mess with the user’s configuration.
- 51. Tech • Use the participant’s system (or community org system). • For mobile testing, do not use mounts. • Do not mess with the user’s configuration. • If necessary to modify, ask permission for any and every change.
- 52. Tech • Use the participant’s system (or community org system). • For mobile testing, do not use mounts. • Do not mess with the user’s configuration. • If necessary to modify, ask permission for any and every change. • Return it to the way you found it when done.
- 53. PROCESS Photo by Eelke, CC BY 2.0
- 54. Process • Be prepared to read agreements, instructions, etc. aloud.
- 55. Process • Be prepared to read agreements, instructions, etc. aloud. • Point a camera at the user and interactions.
- 56. Process • Be prepared to read agreements, instructions, etc. aloud. • Point a camera at the user and interactions. • Drive a second monitor from device and record it.
- 57. Process • Be prepared to read agreements, instructions, etc. aloud. • Point a camera at the user and interactions. • Drive a second monitor from device and record it. • Do not interrupt the user when using AT.
- 58. Process • Be prepared to read agreements, instructions, etc. aloud. • Point a camera at the user and interactions. • Drive a second monitor from device and record it. • Do not interrupt the user when using AT. • Reassure user none of the mistakes are his/her fault.
- 59. Process • Be prepared to read agreements, instructions, etc. aloud. • Point a camera at the user and interactions. • Drive a second monitor from device and record it. • Do not interrupt the user when using AT. • Reassure user none of the mistakes are his/her fault. • Users may apologize for finding errors.
- 60. PRIVACY Photo by frankieleon, CC BY 2.0
- 61. Privacy • Some personal health information may be revealed.
- 62. Privacy • Some personal health information may be revealed. • Be prepared to treat it as confidential.
- 63. Privacy • Some personal health information may be revealed. • Be prepared to treat it as confidential. • Where possible, anonymize data for reporting.
- 64. Privacy • Some personal health information may be revealed. • Be prepared to treat it as confidential. • Where possible, anonymize data for reporting. • Coordinate with recruiting organization.
- 65. Wrap-up Joe Dolson working with Léonie Watson to perform screen reader testing on Gutenberg at CSUN Assistive Technology Conference.
- 66. Resources • Tips For Conducting Usability Studies With Participants With Disabilities https://www.smashingmagazine.com/2018/03/tips-conducting-usability-studies-participants-disabilities/ • Microsoft Inclusive Design Toolkit https://www.microsoft.com/en-us/design/inclusive • Selfish Accessibility http://adrianroselli.com/2017/03/selfish-accessibility-slides-from-wordcamp-london-2017.html • W3C Web Accessibility Perspectives https://www.w3.org/WAI/perspectives/
- 67. Coming up… Saturday, 5 May CSS Display Properties versus HTML Semantics 14—16 June Prototyping Accessibility
- 68. Inclusive Usability Testing Presented by Adrian Roselli (@aardrian) for WordCamp London. Slides from this talk will be available at rosel.li/WCLDN.
Editor's Notes
- Increase with age Situational Temporary
- Now let’s watch my talk from last year, Selfish Accessibility. It is on WordPress.tv
- • Involved with WAI early on • Aspects of the work involves our own hesitation • Do I offer an elbow? What do I say? Do I ask? • It’s more important to start interacting
- Broken down into these general categories, with some overlap.
- • Save the Dream and SASOL. • National Sport Day 2017 to promote education and inclusivity through sport. • High-profile Qatari athletes and footballers from Al Sadd Football Club.
- 1 of 9 • Audits should have happened • Bugs should be fixed • You should be testing an accessible version
- 1 of 9 • You are wasting everyone’s time if it becomes that • It is compelling to wade into a11y testing when you have disabled participants, but do not • Be clear on the scope of the tests with all stakeholders
- 1 of 9 • Participants are there for their knowledge and expertise • Limit the scope to what is needed for the testing
- 1 of 9 • You don’t want to have to fight this after the fact • It’s easier to say that this was usability testing, not a11y testing if you are perceived as not flagging issues later • Be wary of those in the organization who want to help for PR reasons
- • Learning how to understand the signing of a floor plan in sign language • My teacher also forgets to give us the new reference point after each room • so I never know whether the kitchen is down the hall from the dining room or on the other side of the dining room. • It's rather confusing.
- 2 of 9 • This will help you prioritize questions and steps • As we will discuss, you may not get through everything • Testing theories? Gauging responses? • Live platform? Prototype? • You may think some steps are faster or easier • Can accidentally impart bias
- 2 of 9 • Important to know if certain tools are needed • Are URLs long, complex to type • Do you have a “landing” page? • Anecdote of having made a quick CodePen to provide complex URLs as just a list of links for participants at a short URL
- 2 of 9 • Useful for time and money budget • Plus recruiting efforts
- 2 of 9 • I hope so
- • Gift cards can be most useful when they have cash value. • Avoid cards at venues that require travel, do not support disabilities, etc. • Get cards that are more useful than just a free coffee but take $20 to get to use. • Can be used as signature guides • Avoid the term “payment”
- 3 of 9 I used “pay” for space reasons
- 3 of 9
- 3 of 9 • Door-to-door services • Bus routes that require longer waits • Borrowing or using a mobility aid
- 3 of 9 • Many are not in salaried positions • Hourly workers can lose an entire day of wages
- 3 of 9 • Many are not in salaried positions • Hourly workers can lose an entire day of wages
- 3 of 9 • Some may need additional help from aides • This can be to help with… • Going out in public • Keeping a schedule • Arranging transportation
- 3 of 9 • Avoid cards at venues that require travel, do not support disabilities, etc. • Get cards that are more useful than just a free coffee but take $20 to get to use. • Many participants may be overwhelmed with all the information you have provided • They may feel less pressured to spend all at once to use it up
- 4 of 9 • A token effort is unacceptable • pay attention to whether or not it is truly accessible • Kutna Hora, Czech Republic • It looks wrong because it is wrong • A participant will know at a glance, but after it is too late to pass
- 4 of 9 • Repeat
- 4 of 9 • Consider the bus line • What does the person have to navigate after getting off the bus? • Are there sidewalks? • Is there parking? • Does the person have to cross an open space like a parking lot?
- 4 of 9 • I talked about costs before, consider where a participant may want to cut costs as well
- 4 of 9 • Be prepared to meet participants at the door • Is there a front desk? • Is it a straightforward route? • Elevators? That can handle wheelchairs? Animals? • Bathrooms? Nearby? • Lighting? Noise? Smells?
- 4 of 9 • Even just a safe tree
- 4 of 9 • Walk the person out • Are you releasing the person at night? Rush hour? After buses stop on that route? • Are you prepared to walk the user to the bus stop? • Tell my story of walking the girl a half mile
- 5 of 9 • Blind Camp of Maryland Visits NASA Goddard • NASA's Goddard Space Flight Center in Greenbelt, Md. • hosted some 100 campers and volunteers from the Deaf-Blind Camp of Maryland. • Based in West River, Md., was established in 1998 • provides safe, fun, barrier-free week for people who have significant hearing and vision loss. • campers, aged 18 to 80, learned about space and NASA missions
- 5 of 9 • Define the characteristics of participants that the study seeks to include • The more specific, the harder it is to reruit • There are a host of challenges to finding the right participants • Understanding how to qualify participant skills • Some may report great skill, but it may only apply to certain tasks
- 5 of 9 • Many cities have a blind association, for example • Get familiar with what your city has to offer • Ask each one for names of others; they often have overlapping constituents, services, etc.
- 5 of 9 • An org can help you get the specific demographics you want • Age, gender, skill level, tools, job, etc.
- 5 of 9 • The known brand can make recruitment easier • Participants will feel the study is pre-vetted • Good to ally yourself this way
- 5 of 9 • It can recommend services and people • It can introduce you to other orgs • It can help find specific participants
- 5 of 9 • Maybe convey considerations for the audience you had not considered • ie, skill level, ability, etc • Can help with venue, messaging, structure, etc
- 5 of 9 • No need to expose members to you • Constituents more likely to notice the recruiting effort • Pre-validates the study to them
- 6 of 9 • May not anticipate some of the assistive tech that your users will bring. • A scooter and service dog combined will overwhelm even a spacious room.
- 6 of 9 • Build extra time into every step • Surfing with assistive technology is rarely as fast as not • Do not rush them • Allow for rests • Allow them to leave early, particularly if they start late • They may have discovered they need more time than planned to get out
- 6 of 9 • Many are at the whim of buses, poor signage, poor instructions, etc.
- 6 of 9 • Nor do people who want to pet them
- 6 of 9 • Make sure a dog can lay at its owner’s feet • Leave a large space free of cables, power bricks, etc under the table • Let the owner tell you where the animal will ultimately go
- 6 of 9 • Consider different body sizes and shapes • Some people will prefer stability, others the ability to roll around, etc. • For those who cannot see options, explain them
- 6 of 9 • Just because they come in a chair does not mean they want to use it • It should be nearby and in plain sight; this is their independence
- • This set-up may not work • For mobile testing that the user may hold device to ear
- 7 of 9 • It is already configured and ready to go anyway • You needn’t waste time getting participant acclimated to a new platform • If testing at community org with users who use those systems, then use those systems
- 7 of 9
- 7 of 9 • As soon as you mess with their configuration, you are creating friction • That alone can taint your results • Will almost definitely frustrate users
- 7 of 9 • If you are genuinely stuck, like turning plugging their system into an external monitor • Ask permission for each change • Make a note of each change
- 7 of 9 • Use those notes you made of each change • Remember that you are messing with some peoples’ lifelines • A tiny configuration change can be the difference between safe independence and being in danger
- 8 of 9 Nuthin’
- 8 of 9 • Sometimes participants have not read documents in advance • Could be lack of interest, could be an inaccessible document • If they are screen reader users, you can often read a faster rate
- 8 of 9 • This can help you see thing you might miss by watching • Leaning in and be off-putting for some participants • You may miss where the AT is guiding the user • The set-up pictured may not work
- 8 of 9 • This can allow you to frame a shot with participant face and screen in one • Very useful for mobile testing
- 8 of 9 • Especially do not speak while a screen reader is speaking • If participant is using dictation software, allow for less interaction • Allow user to parse a screen mentally before diving in; watch this
- 8 of 9
- 8 of 9 • Just as you would in a typical session, assure participant it is not his/her fault • Even if the tech is misbehaving, it is not the person’s fault
- 9 of 9
- 9 of 9 • The very nature of the participation means you will come to know health information • It may be very general, such as a person is blind, or is a wheelchair user • Some participants will volunteer more just in the course of conversation • Capture only what you need • Ask what AT they use, not their disability
- 9 of 9 • This is not a HIPAA issue • This is about respecting the participant’s privacy
- 9 of 9 • Maybe only use first names • Maybe consider what video clips you share • Maybe avoid photos that show unrelated parts of the scene
- 9 of 9 • It may have some familiarity with participant preferences • It may be able to guide you on best practices • Consider adopting its language in your materials