UXPA 2023 Poster: 5 Key Findings from Moderated Accessibility Testing with Screen-Reader Users

1. 5 Key Findings from Moderated Accessibility Testing
with Screen-Reader Users
Between 2021-2022, UserTesting conducted
25+ accessibility tests for customers. We
discovered that there were five key findings
that were most prevalent in this research.
Note: Poster examples are not from research
conducted for customers, but rather from
independent research conducted by the
authors.
Issue: The screen-reader is unable to navigate the page in a sequential
order (e.g., focus moves unexpectedly to a different part of the page).
Result: The user becomes frustrated because they have moved to a
different part of the page & must try to find where they were previously.
Naomi Enzinna, John Romadka, Hope Paulos - Accessibility Research Professional Services @ UserTesting
Method: Moderated accessibility/usability test
5 participants who are blind per study
All must use the same screen-reader, browser,
and device to reduce variation in findings.
Recommended due to popularity of use:
- Desktop: JAWS & Chrome
- Mobile: VoiceOver & Safari
3-4 Tasks where users interact with navigation,
form fields, and/or other elements to test the
accessibility of common problem areas.
Qualitative analysis focused on identifying
where users are unable to complete a task
without assistance or added effort/frustration.
Issues are tied to the Web Content
Accessibility Guidelines (WCAG) if relevant.
5 Key Findings
Overview Our Accessibility Testing Method
#1 Unexpected Location
Issue: An image or graphic does not have a text description, or the
image's description is not helpful for the user.
Result: The user feels less confident completing a task because they’re
missing information being conveyed by images.
#2 Missing Alt-Text
Issue: The user is not notified when a change occurs on the page (e.g.,
after adding an item to the cart, they should hear "item added to cart").
Result: The user is not confident an action was successful and must
search for confirmation. User may require sighted assistance.
#3 No Feedback
Issue: Labels or instructions for interactive elements are not
communicated to the user or are not sufficiently descriptive.
Result: This leads to user error and frustration. User may require
sighted assistance to complete the task.
#4 Insufficient Labels
Issue: The user is not made aware of error, has difficulty finding an
error message, or the error message is not sufficiently descriptive.
Result: The user is unable to fix the error. User may require sighted
assistance to complete the task.
#5 Unclear Error Messages
Benefits of Accessibility Research
Watch example clips of
each key finding here!
(1) Create inclusive products: Ensure your product is usable by everyone—including those with disabilities.
(2) Generate revenue: An additional 15-20% of the population can use your product, likely increasing profits.
(3) Reduce risk of costly litigation: Addressing accessibility issues early can help prevent million-dollar lawsuits.
Example: When the user selects
an autofill suggestion, the screen-
reader's focus jumps to the top of
the page unexpectedly.
Autofill:
Word
Field 1
Field 2
Enter
Wo
“It changed
focus on me”
Alt text = ???????
Example: The alt-text for an
image of a sandwich reads
"descriptive text" which is not
meaningful to the user.
Field 1*
Field 2*
*Required
“This doesn’t look
like it’s required”
“This looks like it’s
required”
“Required”
Example: The "required" label is
only read for some of the required
fields. The user doesn't know
there are required fields they
need to complete.
Example: After checking a
checkbox, the user does not hear
a message stating that the box
was checked. Also, the checkbox
is still labeled "not checked."
Example: When the user makes
an error, an error message visually
appears but the user is not
alerted. Later, the user is unable
to find and fix the error.
WCAG: 2.4.3 Focus Order
WCAG: 1.1.1 Non-text Content
WCAG: 4.1.3 Status Message
WCAG: 2.4.6 Headings and Labels; 3.3.2 Labels or Instructions; 4.1.2 Name, Role, Value
WCAG: 3.3.1 Error Identification, 3.3.3 Error Suggestion
**Crickets**
“Did anything happen?”
Field 1
Text !
Error message
**Crickets**
“From what I can tell there
are no errors on the page”