A guided tour of the native accessibility APIs on iOS and Android to help you understand what’s possible and learn how to speak accessibility to iOS and Android app developers.
2. Introduction
• Jon Gibbins (@dotjay)
– Accessibility consultant at Dig Inclusion
– Web developer since 1999
– Assistive technologist since 2002
– Mobile specialist since 2012
– Passionate about accessibility – why?
2
10. What we will learn
• AT on mobile devices in brief
• iOS UIAccessibility
• Android accessibility API
And if there’s time…
• Some remediation techniques
• Testing API on mobile platforms
10
16. Mobile screen readers
Two main interaction
methods
1. Explore by touch
– Drag finger over screen
– Items under your finger are
described by screen reader
– Tap with a second finger or
double tap to open/activate
2. Gesture navigation
16
17. Mobile screen readers
Two main interaction
methods
1. Explore by touch
2. Gesture navigation
– Swipe right/left moves focus
to next/previous content in
sequence
– Items are described by
screen reader as focus
moves
– Double tap to open/activate
17
18. VoiceOver
1. Triple click the Home
key to activate
2. Dial to open the Rotor
3. Swipe up/down to
navigate parts
4. Swipe right/left for
next/previous content
18
20. TalkBack
1. Activate using Power +
2 fingers shortcut
2. Swipe right/left for
next/previous content
3. Swipe up/down to
switch modes
4. Use Global/Local Menu
for other features
20
21. • Enable “Accessibility shortcut” in
Settings
• Long press volume to
suspend/resume (4.2+)
• Swipe down and right (L-shape) for
Global Context Menu
• Swipe down and left (backwards L) for
Local Context Menu
• “Read from top” is in the Global
Context Menu
TalkBack tips
21
26. A dialog takes place between an application and assistive technology.
26
Accessibility theory
Accessibility API fundamentals
27. • We know what assistive technology is,
but how does it work?
• Accessibility APIs
– Present interfaces as information rather
than purely visual
– Allow interfaces to be changed by AT
– Provide a common vocabulary
27
Accessibility theory
Accessibility API fundamentals
28. • Accessible Object Properties
– User interface is represented as a hierarchy of
accessible objects
– Each object has a variety of properties, such as:
• name: Defines a label. (“Hi, what’s your name?)
• role: Defines the behavior. (“So, what do you do?”)
• state: Defines the current condition. (“How are you?”)
28
Accessibility theory
Accessibility API fundamentals
29. • Accessible Events
– Accessibility APIs notify assistive
technologies of changes by broadcasting
events.
29
Accessibility theory
Accessibility API fundamentals
30. • Example: a checkbox in a desktop
application
– role: check box
– name: Open new windows in a new tab instead
– state: checked
30
Accessibility theory
Accessible object properties
31. • Examples in HTML
– role:
<img alt="Can of soda" src="soda.jpg">
<label for="last">Last name</label>
<input type="text" id="last" name="last" value="Bloggs">
– name:
<img alt="Can of soda" src="soda.jpg">
<label for="last">Last name</label>
<input type="text" id="last" name="last" value="Bloggs">
31
Accessibility theory
Accessible object properties
33. • An accessible name is required and
identifies an object. It is short and does
not necessarily describe the object.
<label for="date">Event date</label>
<input type="text" id="date" name="date">
33
Accessibility theory
Accessible object properties
34. • An accessible description is optional
text that provides additional information
about an object.
<label for="date">Event date</label>
<input type="text" id="date" name="date”
aria-describedby="instructions">
<p id="instructions">Must be in mm/dd/yyyy
format.</p>
34
Accessibility theory
Accessible object properties
36. source: Accessibility for iOS by Chris Fleizach, WWDC 2012 (47 mins)
https://developer.apple.com/videos/wwdc/2012/?id=210
The iOS Accessibility API
UIAccessibility
36
37. • UIKit mostly “just works”
– A lot of accessibility comes for free if you use
these UI components
• Much of the work involved is in setting labels
– accessibilityLabel
• Also look out for:
– isAccessibilityElement
– accessibilityTraits
– accessibilityHint
• This is about 80-90% of the work involved in
implementing accessibility on iOS
The iOS Accessibility API
UIAccessibility
37
38. • Back to our common API vocabulary:
– role = traits
– state = traits (again‽ multitasking superstar!)
– name = labels
– value = value
– desc = hint
The iOS Accessibility API
UIAccessibility
38
39. UIAccessibility
isAccessibilityElement
accessibilityLabel = name
accessibilityHint = description
accessibilityValue = value
accessibilityLanguage
accessibilityTraits = role and state
accessibilityFrame
accessibilityPath
accessibilityActivationPoint
accessibilityElementsHidden
accessibilityViewIsModal
accessibilityNavigationStyle
shouldGroupAccessibilityChildren
The iOS Accessibility API
UIAccessibility
39
42. UIAccessibilityTraits = role and state
let myCustomButton = UIView()
myCustomButton.isAccessibilityElement = true
myCustomButton.frame = self.view.frame
myCustomButton.accessibilityTraits
= UIAccessibilityTraitButton
myCustomButton.accessibilityLabel = "Press me!"
Best approach?
The iOS Accessibility API
UIAccessibility
42
43. UIAccessibilityTraits = role and state
let myButton = UIButton()
myButton.accessibilityLabel = "Press me!"
Use UIKit!
The iOS Accessibility API
UIAccessibility
43
44. accessibilityFrame
– Method implementation
– VoiceOver polls the API for accessibilityFrame info
• Why? Frames can change, e.g. on orientation
– Important when merging two elements or creating invisible
prompts (such as container headings)
The iOS Accessibility API
UIAccessibility
44
45. UIAccessibilityIsVoiceOverRunning
– Allows developers to detect is VoiceOver is running.
UIAccessibilityVoiceOverStatusChanged
– Allows developers to detect when VoiceOver is enabled.
• When is this appropriate?
– e.g. performance vs restart app
The iOS Accessibility API
UIAccessibility
45
48. Labels
• accessible name
– Describe the meaning or purpose of the interface
element
– Front-load important information
– Use commas to break up information when announced
by TalkBack
• android:contentDescription in XML layout
files
• setContentDescription() to dynamically
update descriptions
50. Labels
• Assign contentDescription to all user interface
components, e.g.
– ImageButton
– ImageView
– CheckBox
<ImageButton
android:id="@+id/add_note_button"
android:src="@drawable/add_note"
android:contentDescription="@string/add_note"/>
• Decorative images should not have descriptions
– android:contentDescription="@null"
51. EditText fields
• android:labelFor
= best approach
Most like HTML / ARIA (since API level 17, Jelly Bean MR1)
• android:hint
= not ideal accessibility
Previously, best practice to provide an attribute instead of a content
description. When the field is filled, TalkBack reads the entered content
to the user, instead of the hint text.
• android:contentDescription
= terrible accessibility
Only read before text is typed into the field. Once the field is filled, the
contentDescription is never read again
53. Roles
• Use built-in UI components / Material
Design for role and state:
– Button
– Text field
– Checkbox + setChecked(boolean)
– Radio button
– Switches / Toggle button
– Spinner
– Pickers
– Sliders
55. Focus
Two focus concepts on Android:
1. Accessibility Focus (or "traversal”)
The linear order in which elements receive focus
when using TalkBack gesture navigation.
2. Input Focus
The order interactive widgets (text fields, buttons,
etc) receive focus when navigating using a D-pad
(spatial directional navigation) or keyboard (linear
Tab navigation).
56. Accessibility Focus
Enabling accessibility
android:importantForAccessibility (API 16, Jelly Bean)
Like aria-hidden (in reverse)…
• auto (default)
The accessibility service determines whether the view is important for
accessibility. Recommended for most cases.
• yes
The view is important for accessibility. Useful if a view does not appear
to be exposed to TalkBack for any reason.
• no
The view is not important for accessibility. Useful if a view is announced
by TalkBack but you want to hide it from TalkBack users.
• noHideDescendants
The view is not important for accessibility, nor are any of its descendant
views.
57. Accessibility Focus
Traversal order (API 22+)
android:accessibilityTraversalAfter
Sets the next view to receive focus when using
TalkBack gesture navigation.
android:accessibilityTraversalBefore
Sets the previous view to receive focus when using
TalkBack gesture navigation.
Note: API 22 is Android 5.1, Lollipop MR1.
58. Input Focus
Enabling focus
android:focusable
Like tabindex in HTML…
If a view is not focusable by default:
• android:focusable="true"
• setFocusable("true")
Check state or request focus
• isFocusable( )
• requestFocus( )
59. Input Focus
Focus order for Tab navigation (API 1+)
android:nextFocusForward
Sets the next selectable view to receive focus
when the user navigates, for example, by pressing
Tab on the keyboard.
60. Input Focus
Focus order for directional navigation (API 1+)
android:nextFocusLeft / Right / Up / Down
Sets the next view to take focus when the user
navigates using the arrow keys on a keyboard or
directional controller.
61. Focus
• How do I move TalkBack focus?
targetView.sendAccessibilityEvent(
AccessibilityEvent.TYPE_VIEW_FOCUSED
)
66. • If focus does not move to a new screen, likely that a
screen changed notification has not been fired
iOS remediation tips
66
67. • If focus does not move to a new screen, likely that a
screen changed notification has not been fired
• If focus does not move to a modal, likely that a layout
changed notification has not been fired
iOS remediation tips
67
68. • Separating visual text from what is announced
– e.g. accessible date format announcements
iOS remediation tips
68
69. • Separating visual text from what is announced
– e.g. accessible date format announcements
• Add hidden content
– e.g. landmark names
– hidden heading solution
iOS remediation tips
69
70. • Separating visual text from what is announced
– e.g. accessible date format announcements
• Add hidden content
– e.g. landmark names
– hidden heading solution
• Arbitrary announcements
– e.g. dynamic messages
iOS remediation tips
70
74. Android
• name =
android:contentDescription or
setContentDescription()
• role and state = built-in controls
– Button
– Text field
– Checkbox ( e.g. setChecked(boolean) )
– Radio button
– Toggle button
– Spinner
– Pickers
75. Android
• Add hidden content
– e.g. landmark names
– LinearLayout with a contentDescription and
android:importantForAccessibility="yes"
– Hidden heading solution (no visible focus)
<TextView
android:layout_width="1dp"
android:layout_height="1dp"
android:contentDescription="My heading"/>
78. iOS testing
• Accessibility Inspector
– Check exposed accessibility properties
– Part of Xcode, but does not require Apple
Developer Program subscription to use
– Xcode > Open Developer Tool > Accessibility Inspector
– Older: General > Accessibility > Accessibility Inspector
78
79. iOS testing
• New Accessibility Inspector in Xcode 8
– Launched September 2016
– Use to identify potential accessibility issues
– Inspect app on attached iOS device
– Not a replacement for manual testing
– Automated accessibility audits
– Interactive inspection modes
– Accessibility settings live preview
Xcode > Open Developer Tool > Accessibility Inspector
79
83. iOS testing
• New Accessibility Inspector
•Select target
• Inspect tool
• Inspection
• Debug and analyze accessibility state
83
84. iOS testing
• New Accessibility Inspector
•Audit
• Find and report accessibility issues, and
suggests fixes
• Element has no description
• Potentially inaccessible element
• Image has no label
• Image name used in description
• Does not check contrast / color blindness
84
85. iOS testing
• New Accessibility Inspector
•Settings
• Test impact of accessibility setting change
85
86. iOS testing tips
• Sup' up your Rotor
– Speed up your testing
– In Rotor settings, include the setting for "Speech
Rate”
– Use Rotor to tweak speech rate on the fly
– I use around 75%
86
87. iOS testing tips
• Sup' up your Rotor
– You often don't have access to the code base, so
you can't inspect what is going on
– Switch the character mode and listen to the label
spelt out
– You should be able to ascertain if something is
human readable text or a piece of code, an ID of
some kind
– Typical for images to be "named" its file name.
87
95. iOS Accessibility
• iOS Accessibility Programming Guide
• Apple Developer videos on accessibility
– Accessibility for iOS – get started by watching Chris
Fleizach at WWDC 2012 (47 mins)
• Usability of iPad Apps and Websites by
Nielsen Norman Group
• Cheat sheet for learning the gestures used
on VoiceOver for iOS
95
96. Android accessibility
• Android Emulator
http://developer.android.com/tools/help/emulator.html
• Using the Android Emulator
http://developer.android.com/tools/devices/emulator.html
• Android Accessibility home page
http://developer.android.com/guide/topics/ui/accessibility/index.html
• Making Android Applications Accessible
http://developer.android.com/guide/topics/ui/accessibility/apps.html
• Android Accessibility Developer Checklist
http://developer.android.com/guide/topics/ui/accessibility/checklist.html
• Android Accessibility Testing Checklist
http://developer.android.com/tools/testing/testing_accessibility.html
• Using Android Lint (video, 1 min 45 secs)
http://www.youtube.com/watch?v=OtwCe-YlD5k
• Enabling TalkBack (video, 1 min 27 secs)
http://www.youtube.com/watch?v=VY6CZo7gADE
96