Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

iOS and Android accessibility APIs (AccessU 2017)

1,090 views

Published on

A guided tour of the native accessibility APIs on iOS and Android to help you understand what’s possible and learn how to speak accessibility to iOS and Android app developers.

Published in: Mobile
  • Easy and hassle free way to make money online! I have just registered with this site and straight away I was making money! It doesn't get any better than this. Thank you for taking out all the hassle and making money answering surveys as easy as possible even for non-techie guys like me!  http://t.cn/AieX2Loq
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Looking For A Job? Positions available now. FT or PT. $10-$30/hr. No exp required. ★★★ http://t.cn/AieXipTS
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • How Do Social Media Jobs Pay $35 Per Hour? ★★★ http://ishbv.com/socialpaid/pdf
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Hey guys! Who wants to chat with me? More photos with me here 👉 http://www.bit.ly/katekoxx
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

iOS and Android accessibility APIs (AccessU 2017)

  1. 1. iOS and Android accessibility APIs #AccessU2017 Austin, Texas 18 May 2017 (Happy #GAAD!) Jon Gibbins @dotjay
  2. 2. Introduction • Jon Gibbins (@dotjay) – Accessibility consultant at Dig Inclusion – Web developer since 1999 – Assistive technologist since 2002 – Mobile specialist since 2012 – Passionate about accessibility – why? 2
  3. 3. Introduction Photo credits: LG, Gould, Larson, DiC, Apple 3
  4. 4. Introduction 4
  5. 5. Why accessibility? 5 Photo credit: Jon Gibbins with thanks to Drake Music Project
  6. 6. Empathy Most of us have a connection to accessibility 6
  7. 7. Empathy is about understanding people 7
  8. 8. Inclusion is about understanding people and the barriers that they face 8
  9. 9. Mobile accessibility 9
  10. 10. What we will learn • AT on mobile devices in brief • iOS UIAccessibility • Android accessibility API And if there’s time… • Some remediation techniques • Testing API on mobile platforms 10
  11. 11. Questions are encouraged – If you have a question, or want to know more, please put your hand up 11
  12. 12. Mobile features In brief… 12
  13. 13. Mobile features iOS 13 Settings > General > Accessibility
  14. 14. Mobile features Android 14 Settings > Accessibility
  15. 15. Mobile screen readers 15
  16. 16. Mobile screen readers Two main interaction methods 1. Explore by touch – Drag finger over screen – Items under your finger are described by screen reader – Tap with a second finger or double tap to open/activate 2. Gesture navigation 16
  17. 17. Mobile screen readers Two main interaction methods 1. Explore by touch 2. Gesture navigation – Swipe right/left moves focus to next/previous content in sequence – Items are described by screen reader as focus moves – Double tap to open/activate 17
  18. 18. VoiceOver 1. Triple click the Home key to activate 2. Dial to open the Rotor 3. Swipe up/down to navigate parts 4. Swipe right/left for next/previous content 18
  19. 19. VoiceOver demo 19
  20. 20. TalkBack 1. Activate using Power + 2 fingers shortcut 2. Swipe right/left for next/previous content 3. Swipe up/down to switch modes 4. Use Global/Local Menu for other features 20
  21. 21. • Enable “Accessibility shortcut” in Settings • Long press volume to suspend/resume (4.2+) • Swipe down and right (L-shape) for Global Context Menu • Swipe down and left (backwards L) for Local Context Menu • “Read from top” is in the Global Context Menu TalkBack tips 21
  22. 22. TalkBack demo 22
  23. 23. Mobile zoom tools 23
  24. 24. Zoom demo 24
  25. 25. Accessibility API fundamentals
  26. 26. A dialog takes place between an application and assistive technology. 26 Accessibility theory Accessibility API fundamentals
  27. 27. • We know what assistive technology is, but how does it work? • Accessibility APIs – Present interfaces as information rather than purely visual – Allow interfaces to be changed by AT – Provide a common vocabulary 27 Accessibility theory Accessibility API fundamentals
  28. 28. • Accessible Object Properties – User interface is represented as a hierarchy of accessible objects – Each object has a variety of properties, such as: • name: Defines a label. (“Hi, what’s your name?) • role: Defines the behavior. (“So, what do you do?”) • state: Defines the current condition. (“How are you?”) 28 Accessibility theory Accessibility API fundamentals
  29. 29. • Accessible Events – Accessibility APIs notify assistive technologies of changes by broadcasting events. 29 Accessibility theory Accessibility API fundamentals
  30. 30. • Example: a checkbox in a desktop application – role: check box – name: Open new windows in a new tab instead – state: checked 30 Accessibility theory Accessible object properties
  31. 31. • Examples in HTML – role: <img alt="Can of soda" src="soda.jpg"> <label for="last">Last name</label> <input type="text" id="last" name="last" value="Bloggs"> – name: <img alt="Can of soda" src="soda.jpg"> <label for="last">Last name</label> <input type="text" id="last" name="last" value="Bloggs"> 31 Accessibility theory Accessible object properties
  32. 32. • Examples in HTML – state: <label for="last">Last name</label> <input type="text" id="last" name="last" value="Bloggs" disabled="disabled"> – value: <label for="last">Last name</label> <input type="text" id="last" name="last" value="Bloggs"> 32 Accessibility theory Accessible object properties
  33. 33. • An accessible name is required and identifies an object. It is short and does not necessarily describe the object. <label for="date">Event date</label> <input type="text" id="date" name="date"> 33 Accessibility theory Accessible object properties
  34. 34. • An accessible description is optional text that provides additional information about an object. <label for="date">Event date</label> <input type="text" id="date" name="date” aria-describedby="instructions"> <p id="instructions">Must be in mm/dd/yyyy format.</p> 34 Accessibility theory Accessible object properties
  35. 35. The iOS Accessibility API: UIAccessibility overview
  36. 36. source: Accessibility for iOS by Chris Fleizach, WWDC 2012 (47 mins) https://developer.apple.com/videos/wwdc/2012/?id=210 The iOS Accessibility API UIAccessibility 36
  37. 37. • UIKit mostly “just works” – A lot of accessibility comes for free if you use these UI components • Much of the work involved is in setting labels – accessibilityLabel • Also look out for: – isAccessibilityElement – accessibilityTraits – accessibilityHint • This is about 80-90% of the work involved in implementing accessibility on iOS The iOS Accessibility API UIAccessibility 37
  38. 38. • Back to our common API vocabulary: – role = traits – state = traits (again‽ multitasking superstar!) – name = labels – value = value – desc = hint The iOS Accessibility API UIAccessibility 38
  39. 39. UIAccessibility isAccessibilityElement accessibilityLabel = name accessibilityHint = description accessibilityValue = value accessibilityLanguage accessibilityTraits = role and state accessibilityFrame accessibilityPath accessibilityActivationPoint accessibilityElementsHidden accessibilityViewIsModal accessibilityNavigationStyle shouldGroupAccessibilityChildren The iOS Accessibility API UIAccessibility 39
  40. 40. UIAccessibilityTraits UIAccessibilityTraitNone; UIAccessibilityTraitButton; UIAccessibilityTraitLink; UIAccessibilityTraitSearchField; UIAccessibilityTraitImage; UIAccessibilityTraitSelected; UIAccessibilityTraitPlaysSound; UIAccessibilityTraitKeyboardKey; UIAccessibilityTraitStaticText; The iOS Accessibility API UIAccessibility 40
  41. 41. UIAccessibilityTraits (continued) UIAccessibilityTraitSummaryElement; UIAccessibilityTraitNotEnabled; UIAccessibilityTraitUpdatesFrequently; UIAccessibilityTraitStartsMediaSession; UIAccessibilityTraitAdjustable; UIAccessibilityTraitAllowsDirectInteraction; UIAccessibilityTraitCausesPageTurn; UIAccessibilityTraitHeader; The iOS Accessibility API UIAccessibility 41
  42. 42. UIAccessibilityTraits = role and state let myCustomButton = UIView() myCustomButton.isAccessibilityElement = true myCustomButton.frame = self.view.frame myCustomButton.accessibilityTraits = UIAccessibilityTraitButton myCustomButton.accessibilityLabel = "Press me!" Best approach? The iOS Accessibility API UIAccessibility 42
  43. 43. UIAccessibilityTraits = role and state let myButton = UIButton() myButton.accessibilityLabel = "Press me!" Use UIKit! The iOS Accessibility API UIAccessibility 43
  44. 44. accessibilityFrame – Method implementation – VoiceOver polls the API for accessibilityFrame info • Why? Frames can change, e.g. on orientation – Important when merging two elements or creating invisible prompts (such as container headings) The iOS Accessibility API UIAccessibility 44
  45. 45. UIAccessibilityIsVoiceOverRunning – Allows developers to detect is VoiceOver is running. UIAccessibilityVoiceOverStatusChanged – Allows developers to detect when VoiceOver is enabled. • When is this appropriate? – e.g. performance vs restart app The iOS Accessibility API UIAccessibility 45
  46. 46. Android accessibility API 46
  47. 47. Labelling (names)
  48. 48. Labels • accessible name – Describe the meaning or purpose of the interface element – Front-load important information – Use commas to break up information when announced by TalkBack • android:contentDescription in XML layout files • setContentDescription() to dynamically update descriptions
  49. 49. Labels
  50. 50. Labels • Assign contentDescription to all user interface components, e.g. – ImageButton – ImageView – CheckBox <ImageButton android:id="@+id/add_note_button" android:src="@drawable/add_note" android:contentDescription="@string/add_note"/> • Decorative images should not have descriptions – android:contentDescription="@null"
  51. 51. EditText fields • android:labelFor = best approach Most like HTML / ARIA (since API level 17, Jelly Bean MR1) • android:hint = not ideal accessibility Previously, best practice to provide an attribute instead of a content description. When the field is filled, TalkBack reads the entered content to the user, instead of the hint text. • android:contentDescription = terrible accessibility Only read before text is typed into the field. Once the field is filled, the contentDescription is never read again
  52. 52. Roles
  53. 53. Roles • Use built-in UI components / Material Design for role and state: – Button – Text field – Checkbox + setChecked(boolean) – Radio button – Switches / Toggle button – Spinner – Pickers – Sliders
  54. 54. Focus
  55. 55. Focus Two focus concepts on Android: 1. Accessibility Focus (or "traversal”) The linear order in which elements receive focus when using TalkBack gesture navigation. 2. Input Focus The order interactive widgets (text fields, buttons, etc) receive focus when navigating using a D-pad (spatial directional navigation) or keyboard (linear Tab navigation).
  56. 56. Accessibility Focus Enabling accessibility android:importantForAccessibility (API 16, Jelly Bean) Like aria-hidden (in reverse)… • auto (default) The accessibility service determines whether the view is important for accessibility. Recommended for most cases. • yes The view is important for accessibility. Useful if a view does not appear to be exposed to TalkBack for any reason. • no The view is not important for accessibility. Useful if a view is announced by TalkBack but you want to hide it from TalkBack users. • noHideDescendants The view is not important for accessibility, nor are any of its descendant views.
  57. 57. Accessibility Focus Traversal order (API 22+) android:accessibilityTraversalAfter Sets the next view to receive focus when using TalkBack gesture navigation. android:accessibilityTraversalBefore Sets the previous view to receive focus when using TalkBack gesture navigation. Note: API 22 is Android 5.1, Lollipop MR1.
  58. 58. Input Focus Enabling focus android:focusable Like tabindex in HTML… If a view is not focusable by default: • android:focusable="true" • setFocusable("true") Check state or request focus • isFocusable( ) • requestFocus( )
  59. 59. Input Focus Focus order for Tab navigation (API 1+) android:nextFocusForward Sets the next selectable view to receive focus when the user navigates, for example, by pressing Tab on the keyboard.
  60. 60. Input Focus Focus order for directional navigation (API 1+) android:nextFocusLeft / Right / Up / Down Sets the next view to take focus when the user navigates using the arrow keys on a keyboard or directional controller.
  61. 61. Focus • How do I move TalkBack focus? targetView.sendAccessibilityEvent( AccessibilityEvent.TYPE_VIEW_FOCUSED )
  62. 62. Other stuff…
  63. 63. Other • How do I get TalkBack to speak? View.announceForAccessibility( "Say something! Say something! Anything…");
  64. 64. Other • How do I detect TalkBack? – AccessibilityManager.isEnabled() (API 4) – isScreenReaderActive (old, don’t use)
  65. 65. iOS remediation 65
  66. 66. • If focus does not move to a new screen, likely that a screen changed notification has not been fired iOS remediation tips 66
  67. 67. • If focus does not move to a new screen, likely that a screen changed notification has not been fired • If focus does not move to a modal, likely that a layout changed notification has not been fired iOS remediation tips 67
  68. 68. • Separating visual text from what is announced – e.g. accessible date format announcements iOS remediation tips 68
  69. 69. • Separating visual text from what is announced – e.g. accessible date format announcements • Add hidden content – e.g. landmark names – hidden heading solution iOS remediation tips 69
  70. 70. • Separating visual text from what is announced – e.g. accessible date format announcements • Add hidden content – e.g. landmark names – hidden heading solution • Arbitrary announcements – e.g. dynamic messages iOS remediation tips 70
  71. 71. • … • Create VoiceOver specific behaviours – Demo: Stocks.app chart (UIAccessibilityContainer) – Demo: Mail.app swipe options (accessibilityCustomActions) iOS remediation tips 71
  72. 72. • … • Create VoiceOver specific behaviours – Demo: Stocks.app chart (UIAccessibilityContainer) – Demo: Mail.app swipe options (accessibilityCustomActions) • Reorder focusable elements – Accessibility Elements array (again UIAccessibilityContainer) iOS remediation tips 72
  73. 73. Android remediation 73
  74. 74. Android • name = android:contentDescription or setContentDescription() • role and state = built-in controls – Button – Text field – Checkbox ( e.g. setChecked(boolean) ) – Radio button – Toggle button – Spinner – Pickers
  75. 75. Android • Add hidden content – e.g. landmark names – LinearLayout with a contentDescription and android:importantForAccessibility="yes" – Hidden heading solution (no visible focus) <TextView android:layout_width="1dp" android:layout_height="1dp" android:contentDescription="My heading"/>
  76. 76. Android • Custom AccessibilityAction – Added to Local Context Menu • When all is broken, override accessibility tree – AccessibilityNode*
  77. 77. iOS testing 77
  78. 78. iOS testing • Accessibility Inspector – Check exposed accessibility properties – Part of Xcode, but does not require Apple Developer Program subscription to use – Xcode > Open Developer Tool > Accessibility Inspector – Older: General > Accessibility > Accessibility Inspector 78
  79. 79. iOS testing • New Accessibility Inspector in Xcode 8 – Launched September 2016 – Use to identify potential accessibility issues – Inspect app on attached iOS device – Not a replacement for manual testing – Automated accessibility audits – Interactive inspection modes – Accessibility settings live preview Xcode > Open Developer Tool > Accessibility Inspector 79
  80. 80. Accessibility Inspector 80
  81. 81. iOS testing • New Accessibility Inspector •Select target • Inspect tool • Inspection • Debug and analyze accessibility state 83
  82. 82. iOS testing • New Accessibility Inspector •Audit • Find and report accessibility issues, and suggests fixes • Element has no description • Potentially inaccessible element • Image has no label • Image name used in description • Does not check contrast / color blindness 84
  83. 83. iOS testing • New Accessibility Inspector •Settings • Test impact of accessibility setting change 85
  84. 84. iOS testing tips • Sup' up your Rotor – Speed up your testing – In Rotor settings, include the setting for "Speech Rate” – Use Rotor to tweak speech rate on the fly – I use around 75% 86
  85. 85. iOS testing tips • Sup' up your Rotor – You often don't have access to the code base, so you can't inspect what is going on – Switch the character mode and listen to the label spelt out – You should be able to ascertain if something is human readable text or a piece of code, an ID of some kind – Typical for images to be "named" its file name. 87
  86. 86. Android testing 88
  87. 87. Android testing • Accessibility Scanner – Launched March 2016 – Any Android app – Identifies potential issues – Suggests solutions • larger text • color choices • navigational tools • more 89
  88. 88. Android testing • Accessibility Scanner – Download – Enable in Settings > Accessibility – Open app to test – Start scan – Long-pressing to move screens 90
  89. 89. Accessibility Scanner 91 © Google
  90. 90. Questions?
  91. 91. Thank you! • Slides available later • I’m @dotjay on Twitter • Feedback gratefully received! Copyright 2017 Jon Gibbins/Dig Inclusion Contributions from The Paciello Group used with permission © Jon Gibbins 93
  92. 92. Resources 94
  93. 93. iOS Accessibility • iOS Accessibility Programming Guide • Apple Developer videos on accessibility – Accessibility for iOS – get started by watching Chris Fleizach at WWDC 2012 (47 mins) • Usability of iPad Apps and Websites by Nielsen Norman Group • Cheat sheet for learning the gestures used on VoiceOver for iOS 95
  94. 94. Android accessibility • Android Emulator http://developer.android.com/tools/help/emulator.html • Using the Android Emulator http://developer.android.com/tools/devices/emulator.html • Android Accessibility home page http://developer.android.com/guide/topics/ui/accessibility/index.html • Making Android Applications Accessible http://developer.android.com/guide/topics/ui/accessibility/apps.html • Android Accessibility Developer Checklist http://developer.android.com/guide/topics/ui/accessibility/checklist.html • Android Accessibility Testing Checklist http://developer.android.com/tools/testing/testing_accessibility.html • Using Android Lint (video, 1 min 45 secs) http://www.youtube.com/watch?v=OtwCe-YlD5k • Enabling TalkBack (video, 1 min 27 secs) http://www.youtube.com/watch?v=VY6CZo7gADE 96

×