The document discusses the challenges and inconsistencies in interpreting WCAG 2.x success criteria during accessibility audits. It highlights common misunderstandings and subjective interpretations that can lead to incorrect failure reports, as well as the evolving nature of WCAG with the introduction of new criteria in 2.2. The author emphasizes the need for objective evaluations, auditor education, and a consistent approach to reporting failures to avoid misinterpretations and ensure compliance.

1. These (still) aren't the SCs you're looking for …
(mis)adventures in WCAG 2.x interpretation and audits
Patrick H. Lauke / a11yTO / 19 October 2023

2. about me …
▪ principal accessibility specialist at TetraLogical
▪ previously at The Paciello Group (now TPGi)
▪ occasional W3C AGWG member
▪ WCAG trash panda

3. déjà vu?
you may have already seen an earlier version of this talk …
even years later, the same question still bubbles up
(on mailing lists, a11ySlack, mastodon, …)

4. “which success criterion can I
fail this under?”

6. far too often, auditors clearly
dislike something, and look for
a justification to fail it …
and I get it … sometimes it's just to show how smart we are

7. overstep the boundaries of
WCAG SCs
claim something has to be fixed/changed "to pass WCAG"
when it normatively doesn't

9. we are not lawyers (or judges)
but our audits and evaluations often have some legal dimension to them.
▪ evaluations should be as objective as possible
▪ evaluations should be consistent
… of course, this is easier said than done

10. WCAG is built on the idea that
success criteria can be
evaluated clearly, unambiguously
and consistently…
… but that's not always the case

11. with WCAG 2.2 we now have
9 more SCs!
… just more potential for confusion and misinterpretation?

12. only a few
cherry-picked examples
but there's more where that came from…

13. WCAG success criteria are often
misunderstood and/or
misinterpreted
leads to wrong, or at least inconsistent, error reporting

15. 2.4.6 Headings and Labels (AA)
Headings and labels describe topic or purpose.
this doesn't mandate the use of headings and labels … only that if a
page uses headings and labels, they must be descriptive.
it also doesn't mandate that headings and labels be correctly marked-up -
that's the job of 1.3.1 Info and Relationships and (where it affects
"accessible name" of controls) 4.1.2 Name, Role, Value .
lastly, if labels aren't there, it's a 3.3.2 Labels or Instructions problem.
“

16. <p class="heading1">More information</p>
▪ passes 2.4.6 Headings and Labels
▪ fails 1.3.1 Info and Relationships

17. 3.3.2 Labels or Instructions (A)
Labels or instructions are provided when content requires user
input.
again, this doesn't mandate that labels be marked-up as <label> and
properly associated with form controls - that's covered by
1.3.1 Info and Relationships and (where it affects "accessible name" of
controls) 4.1.2 Name, Role, Value .
“

18. <input type="text">
▪ passes 2.4.6 Headings and Labels
▪ fails 3.3.2 Labels or Instructions
▪ fails 4.1.2 Name, Role, Value

19. <p>First name</p>
<input type="text">
▪ passes 2.4.6 Headings and Labels
▪ passes 3.3.2 Labels or Instructions
▪ fails 1.3.1 Info and Relationships
▪ fails 2.5.3 Label in Name
▪ fails 4.1.2 Name, Role, Value

20. <p>First name</p>
<input type="text" aria-label="First name" >
▪ passes 2.4.6 Headings and Labels
▪ passes 3.3.2 Labels or Instructions
▪ passes 2.5.3 Label in Name
▪ passes 4.1.2 Name, Role, Value
▪ passes (arguably) 1.3.1 Info and Relationships

21. 2.1.1 Keyboard (A)
All functionality of the content is operable through a keyboard
interface […]
doesn't say anything about which keys are needed to operate
controls/functionality
“

22. <a href="#" onclick="…" role="button">fake button</a>
▪ passes 2.1.1 Keyboard
even though it doesn't respond to SPACE like real button would

23. <a href="#" role="button">Activate using 'b'</a>
.addEventListener('keypress', function(e) {
if (e.key == 'b') { /* do something */ }
})
▪ passes 2.1.1 Keyboard
even though it can only be triggered using the letter b

24. 1.3.3 Sensory Characteristics (A)
Instructions provided for understanding and operating content do
not rely solely on sensory characteristics of components such as
shape, color, size, visual location, orientation, or sound.
this only relates specifically to instructions … and not whether or not
sensory characteristics are used for content - this is covered by other SCs,
like 1.4.1 Use of Color or even 1.1.1 Non-Text Content .
Fixes for Understanding 1.3.3 Sensory Characteristics #767
“

25. example: buttons with shapes or colours
▪ fail 1.1.1 Non-text Content (if there's no hidden text / accessible name)
▪ fail 1.4.1 Use of Color for the second set
▪ fail 4.1.2 Name, Role, Value (if there's no accessible name)
▪ pass 1.3.3 Sensory Characteristics as there are no instructions

26. example: buttons with shapes or colours
▪ fail 1.1.1 Non-text Content (if there's no hidden text / accessible name)
▪ fail 1.4.1 Use of Color for the second set
▪ fail 4.1.2 Name, Role, Value (if there's no accessible name)
▪ fail 1.3.3 Sensory Characteristics as there are instructions

27. SCs that are interpreted as their
opposites

28. 1.4.1 Use of Color (A)
Color is not used as the only visual means of conveying
information, indicating an action, prompting a response, or
distinguishing a visual element.
both lines contain a link. but only in the first line is colour used.
the first cases fails 1.4.1 Use of Color, but the second passes
auditors pondering failing the previous example for not using colour …
“

29. 1.3.1 Info and Relationships (A)
Information, structure, and relationships conveyed through
presentation can be programmatically determined or are available
in text.
auditors pondering failing the previous example for not conveying things
through presentation …
“

30. SCs are non-commutative
("if X, then Y" does not mean "if Y, then X")

31. WCAG success criteria and
cascades of fail …

32. cascade of fail
<a href="…"> <img src="…"> </a>
fails multiple criteria , in sequence:
▪ the image itself fails 1.1.1 Non-text content
▪ as a result, link lacks accessible name and fails 4.1.2 Name, Role, Value
▪ arguably, link also fails 2.4.4 Link Purpose (In Context)
need consistency in how this is reported

33. interrelated criteria …

34. 3.1.1 Language of Page (A)
The default human language of each Web page can be
programmatically determined.
3.1.2 Language of Parts (AA)
The human language of each passage or phrase in the content can
be programmatically determined […]
if you can't determine the language of the page, then you also can't
determine the language of each passage or phrase.
fail one, and you automatically fail the other …
“
“

35. minimum and enhanced
▪ 1.4.3 Contrast (Minimum) → 1.4.6 Contrast (Enhanced)
▪ 2.4.11 Focus Not Obscured (Minimum) → 2.4.12 Focus Not Obscured
(Enhanced)
▪ 2.5.8 Target Size (Minimum) → 2.5.5 Target Size (Enhanced)
▪ 3.3.8 Accessible Authentication (Minimum) → 3.3.9 Accessible
Authentication (Enhanced)
fail one, and you automatically fail the other …
need consistency in how this is reported (if you care about AAA)

36. auditor education / consistency
problems …
internal training and resources can help

37. more problematic are issues caused by
WCAG SCs that are vague ,
incomplete or otherwise
lacking

38. WCAG 2.x is not perfect
written by well-meaning, but fallible humans …

39. WCAG success criteria can be
subjective …

40. subjective interpretation?
▪ 1.1.1 All non-text content […] has a text alternative that serves the
equivalent purpose - but what's the purpose?
▪ 1.2.2 Captions are provided … - but do these captions need to be
exact, identify speakers, etc.?
▪ 1.3.1 Information, structure, and relationships conveyed through
presentation […] - where do you draw the line?
▪ 2.4.6 Headings and labels describe topic or purpose - what's
"descriptive" exactly?

41. what is a list?
<a href="/">home</a>
<a href="…">products</a>
<a href="…">contact</a>
do you fail 1.3.1 Info and Relationships because they didn't wrap this in a
<ul> , even if this isn't conveyed through presentation ?
if you look hard enough, it's lists all the way down
turtles

42. WCAG has a few strange
omissions , ambiguities ,
exemptions …

43. 2.4.4 Link Purpose (In Context) (AA)
The purpose of each link can be determined from the link text
alone or from the link text together with its programmatically
determined link context […]
what counts as "programmatically determined link context"? do preceding
headings count?
“

44. "I think what the
founding fathers/mothers
of WCAG meant to say …"

46. understanding documents and techniques try to clarify …
normative versus
non-normative

47. understanding / techniques can't
provide examples of
all possible scenarios

48. beyond the need for subjective interpretation
WCAG success criteria can have
odd loopholes …

49. 2.4.7 Focus Visible (AA)
Any keyboard operable user interface has a mode of operation
where the keyboard focus indicator is visible.
but what does visible mean? it's not normatively defined…
“

50. a single extra pixel shown on focus is arguably visible

51. WCAG 2.1 decided not to modify
2.0 SCs, patched loopholes
with more SCs
but these new SCs also ended up having some loopholes

52. 1.4.11 Non-text Contrast (AA)
The visual presentation of the following have a contrast ratio of at
least 3:1 against adjacent color(s):
▪ User Interface Components: Visual information required to identify
user interface components and states […]
▪ Graphical Objects: […]
“

53. much better … that pixel has a 3:1 contrast ratio now

54. … but we'll fix focus WCAG 2.2?
2.4.11 Focus Appearance (Minimum) (Level AA)
2.4.12 Focus Appearance (Enhanced) (Level AAA)

55. … but we'll fix focus WCAG 2.2!
2.4.13 Focus Appearance (Level AAA)
Only at Level AAA … where success criteria go to die …
2.4.11 Focus Appearance (Minimum) (Level AA)
2.4.12 Focus Appearance (Enhanced) (Level AAA)

56. 2.4.11 Focus Not Obscured (Minimum) (AA)
New in WCAG 2.2
When a user interface component receives keyboard focus, the
component is not entirely hidden due to author-created content.
“

57. classic example: sticky header …
this passes 2.4.11 Focus Not Obscured (Minimum) (AA)
but how far can we take the "not entirely hidden" aspect?

58. arguably … not entirely hidden
(some disagree whether or not focus indicator is part of the user interface component)

59. 1.4.1 Use of Color (A)
Color is not used as the only visual means of
conveying information , indicating an action, prompting a
response, or distinguishing a visual element.
“

60. if your focus indication just changes colour, 1.4.1 likely applies
If content is conveyed through the use of colors that differ not only
in their hue, but that also have a significant difference in lightness,
then this counts as an additional visual distinction, as long as the
difference in relative luminance between the colors leads to a
contrast ratio of 3:1 or greater.
used to be buried in technique F73, now in understanding for 1.4.1 (#1500)
“

61. beware: if you look hard enough, everything is a use of color .
is it essential for conveying information or just eye candy?

63. SCs that are overly specific …
and then end up only applying to very specific cases

64. 1.4.10 Reflow (AA)
Content can be presented without loss of information or
functionality, and without requiring scrolling in two dimensions for:
▪ Vertical scrolling content at a width equivalent to 320 CSS pixels
▪ Horizontal scrolling content at a height equivalent to 256 CSS
pixels
[…]
to help low vision users. should ideally have been "use responsive design"
instead, it was watered down to 400% zoom on a 1280×1024 display …
and now only normatively applies at those exact values
“

65. 1.4.12 Text Spacing (AA)
In content implemented using markup languages that support the
following text style properties, no loss of content or functionality
occurs by setting all of the following […]:
▪ Line height (line spacing) to at least 1.5 times the font size
▪ Spacing following paragraphs to at least 2 times the font size
▪ Letter spacing (tracking) to at least 0.12 times the font size
▪ Word spacing to at least 0.16 times the font size
[…]
only those exact values and over
do you test the "at least" part here, and up to which values?
what if user changes just a few (e.g. just letter spacing)
“

66. this bookmarklet didn't work to change text styles … can I fail this?
no you can't … the SC doesn't care if a user can or can't change the
metrics, or if a particular bookmarklet or method doesn't work.
only that if the metrics are indeed changed, then content doesn't become
unreadable or unusable.
Understanding 1.4.12: clarify the intent and author responsibility #1687
“

67. 3.3.8 Accessible Authentication (Minimum)
(AA)
New in WCAG 2.2
A cognitive function test (such as remembering a password or
solving a puzzle) is not required for any step
in an authentication process […]
“

68. what counts as an authentication process …

69. SCs that are confusingly titled

70. 2.5.8 Target Size (Minimum) (AA)
New in WCAG 2.2
The size of the target for pointer inputs is at least 24 by 24 CSS
pixels […]
at first glance, the SC defines a minimum target size (it says so in the title)
“

71. 2.5.8 Target Size (Minimum) (AA)
New in WCAG 2.2
The size of the target for pointer inputs is at least 24 by 24 CSS
pixels except where:
▪ Spacing: Undersized targets (those less than 24 by 24 CSS pixels)
are positioned so that if a 24 CSS pixel diameter circle is centered
on the bounding box of each, the circles do not intersect another
target or the circle for another undersized target
…
the first exception turns this into Target Spacing (Minimum)
“

72. no target too small – as long as it has spacing

73. 3.2.6 Consistent Help (A)
New in WCAG 2.2
If a Web page contains any of the following help mechanisms, and
those mechanisms are repeated on multiple Web pages within a
set of Web pages, they occur in
the same order relative to other page content , unless a change
is initiated by the user […]:
the understanding document goes in depth listing and explaining help
mechanisms, but the SC doesn't actually require any help mechanisms to
be present at all …
“

74. relative order here means just means a consistent "focus/reading order"
"If the help item is visually in a different location, but in the same
serial order, that is not helpful from a user's point of view, but it
would not fail this criterion."
“

76. even after years of auditing,
I sometimes have weird
moments of realisation
seeing SCs, and what they say/apply to, in a new light

78. same discussions about
applicability and interpretation
even happen within the AGWG

79. WCAG success criteria need to
be testable … but this doesn't
allow for nuance

80. WCAG SCs are binary
you either
pass or fail

81. sometimes the values/thresholds are just arbitrary lines in the sand …

82. no weighting given to impact or
frequency of a particular fail,
or how bad a failure is off the
mark
sometime, you just want to say something's a minor or
soft fail , but distinction doesn't exist

83. fail a single SC and you can't
really claim to be conformant

84. loopholes , omissions and
subjective requirements can
and will be exploited
auditors aren't the only ones who try to find these gaps …

86. … so what do we do then?

88. join my WCAG Trash Panda Web Ring
github.com/w3c/wcag/issues?q=is%3Aopen%20patrickhlauke
to look for discussions and pending clarifications

89. frustration: most efforts has been going towards new WCAG 2.2 SCs
rather than fixing what's broken in 2.0/2.1

90. w3.org/TR/wcag-3.0 … but still very early days
and it will be quite some time before legislation references 3.0 once it's actually a ratified recommendation …

91. but in the meantime …

92. don't creatively reinterpret
what an SC says
to fit your agenda
whether you're a developer or an auditor doing an evaluation

94. “but unless it's reported as a
WCAG failure
the client won't change it”
then you have more fundamental problem …

95. as auditor, you do your client a
disservice by not making clear
what is and isn't a
normative failure
what happens when a clued-up client rightly challenges your
claim? all your other results lose credibility!

96. be conservative in your
pass / fail assessments
document your hesitation, clearly state when something's
"more of a suggestion" than a hard failure

97. @patrick_h_lauke@mastodon.social
github.com/patrickhlauke
tetralogical.com