Accessibility testing is an all too often forgotten part of test automation, typically left for the manual test team. Well now we can go back to our managers with some really great initiatives on how to save them time/money by automating these tests as well. Focusing on it will not only help those with disabilities who use the application you're testing, but optimizing for accessibility will also make automated testing so much easier to accomplish and maintain for your app. Test Automation and Selenium expert Manoj Kumar takes this in-depth look at accessibility testing, covering the essentials, including: • What is Accessibility testing • The Accessibility standards • Different tools available to automate Accessibility testing • How to automate Accessibility testing, and what resources are needed • How to implement an effective Accessibility strategy within your organization

1. AUTOMATING ACCESSIBILITY CHECKS
WEB IS FOR EVERYONE
Manoj Kumar Kumar
1

2. ABOUT ME
▸ Open source enthusiast
▸ Steering committee member - Selenium project.
▸ Organizer seleniumconf.com
▸ Author assertselenium.com
▸ Tweets: @manoj9788 | LinkedIn: manojkumar9788
▸ Accessibility enthusiast
2

3. LIST OF TOPICS TO BE COVERED
▸ What & Why of Accessibility testing
▸ Overview accessibility standards
▸ How & When’s of Testing for Accessibility
▸ Implementation strategy
▸ A11yTaaS (Accessibility Testing As A Service)
▸ a11y in Action
Accessibility Primer
3

4. STATS
About 20% of the world's population lives with some form of disability
4

5. ACCESSIBILITY
▸ Accessibility is about enabling people
▸ Disability only exists if we don’t provide ways to include people
▸ People have impairments NOT disabilities
WHAT?
TESTING APPLICATION WHETHER IT IS USABLE BY PEOPLE WITH DISABILITIES (EVERYONE)
5

6. EXAMPLES
6

7. 7

8. 8

9. 9

10. 10

11. ACCESSIBILITY
DISABILITY
≠
PHYSICAL
CONDITION
DISABILITY
=
LIMITED
HUMAN
INTERACTION
11

12. WHAT IS IT DO WITH APPS?
12

13. IF YOUR APP ISN’T ACCESSIBLE, YOU ARE CREATING BARRIERS
& MAKING THEIR IMPAIRMENT A DISABILITY.
ACCESSIBILITY
13

14. WHY?
Better Business: builds apps that are user-friendly and cater to those
20% population
ACCESSIBILITY 14

15. WHY?
Accessible website yield a good SEO(Search Engine Optimization)
▸ No text in images
▸ Use <H> heading tags
▸ Meaningful contextual links
ACCESSIBILITY
information-technology-directory.co.uk
15

16. WHY?
Abide by Legal legislation
ACCESSIBILITY 16

17. OVERVIEW - ACCESSIBILITY STANDARDS
WCAG 2.0 & GSA SECTION 508
▸ Web Content Accessibility Guidelines
▸ WCAG 1.0: May 1999. Superseded by WCAG 2.0
▸ WCAG 2.0: Dec 2008 - testable, technology agnostic.
▸ WCAG 2.1: (Working draft) accessible mobile devices, low vision, graphics
▸ General Services Administration Section 508 Standards
▸ United States Federal Act. Published in Dec 2000.
17

18. ▸ Level A - basic web accessibility features
▸ Level AA - deals with the biggest and most common barriers for
disabled users
▸ Level AAA – the highest level of web accessibility
SUCCESS CRITERIA
Refer: https://www.w3.org/TR/WCAG20/
18

19. 19

20. ▸ Blind
▸ Low-vision
▸ Colorblind
▸ Deaf
▸ Cognitive/Learning
ABILITIES IN DECLINE
Pic: Usability Matters
20

21. DEVELOP FOR THE FUTURE
Microsoft Inclusive Design Toolkit
21

22. ARIA ATTRIBUTES
ARIA is a set of attributes that you can add to HTML elements.
These attributes communicate role, state and property semantics to assistive
technologies via the accessibility APIs implemented in browsers.
22

23. TESTING
23

24. There is no substitute for real user feedback.
You need to include people with disabilities to Test.
MANUAL - ACCESSIBILITY TESTING 24

25. WHAT YOU CAN DO?
‣ Tab through a page with keyboard
DEMO: Try only to tab through
https://www.ssa.gov/
https://www.delta.com/
MANUAL - ACCESSIBILITY TESTING 25

26. ‣ Use screen reader for testing
▸ Screen readers use Heading structures to get an overall understanding of the page.
JAWS, NVDA
VoiceOver
OS X
ChromeVox Speak Screen Talk BackFang’s
WHAT YOU CAN DO?
MANUAL - ACCESSIBILITY TESTING 26

27. WHAT YOU CAN DO?
▸ Headings and Semantics
▸ Firefox web developer plugin
MANUAL - ACCESSIBILITY TESTING 27

28. Good for checking Headings and Semantic errors
28

29. chrome://accessibility
ACCESSIBILITY TREE 29

30. Google - Accessibility Plugin
30

31. WHAT YOU CAN DO?
▸ Browser Plugins
▸ Chrome Accessibility Inspection
▸ Computed Text
▸ Aria labels
▸ Roles
▸ Form criteria
MANUAL - ACCESSIBILITY TESTING 31

32. WHAT YOU CAN DO?
▸ Visual Testing is important for Accessibility to check if the
appear correctly on different browsers and OS
▸ Web driver css, Applitools(recommended)
▸ Simulate colour blind and check your apps
▸ Nice to have - hoping to see in Applitools, more native
with test automation.
MANUAL - ACCESSIBILITY TESTING 32

33. Accessibility Testing
33

34. HOW FAR IS IT HELPFUL ?
Tests can roughly catch upto 30% of a11y issues,
depending on the audit rules.
AUTOMATING ACCESSIBILITY CHECKS
*Something is better than Nothing*
34

35. Tools??
35

36. ACCESSIBILITY- INTEGRATION TESTS
36

37. ACCESSIBILITY - WEBDRIVER
WebDriver is a human-centric user-friendly bot.
WebDriver won't interact on elements that are hidden on the DOM
and can work only with elements that are visible to the human eye
Which means you are already testing for accessibility.
37

38. USE CASES
▸ Drive tests with Accessibility in mind
▸ WebDriver tests with images, animations turned off
▸ Tab through only tests flow
▸ Start to scope functional tests that navigate using only the keyboard
Report this message sent from Matt Brandt
Delete this message sent from Matt Brandt
ACCESSIBILITY - WEBDRIVER 38

39. DEMO
39
https://goo.gl/2NL74R
https://goo.gl/JZyYrB

40. 40
Code sample: https://goo.gl/2NL74R
Demo Video: https://youtu.be/GrmDuL4-VNk

41. 41
Code sample: https://goo.gl/JZyYrB
Demo Video: https://youtu.be/N52x8lzr-7s

42. var selector = '#menu-0-button';
driver.findElement(selenium.By.css(selector))
.then(function (element) {
element.sendKeys(Key.SPACE);
return element;
})
.then(function (element) {
return element.getAttribute('aria-expanded')
})
.then(function (attr) {
expect(attr).toEqual('true');
});
ACCESSIBILITY - WEBDRIVERJS 42

43. WEBDRIVER + PLUGINS(OPEN-SOURCE)
43

44. AXE - ACCESSIBILITY ENGINE
▸ Open source from Deque systems
▸ JavaScript module for a11y testing
▸ Axe-WebDriverJs
AXE-CORE - TOOLS SUITE 44

45. AXE - WEBDRIVERJS 45
DEMO

46. ▸ Java
▸ Cucumber-selenium
▸ Capybara
▸ Watir
AXE -TOOLS SUITE 46

47. ▸ Protractor Accessibility Plugin for Angular pages
▸ aXe engine
▸ Accessibility Developer tools
▸ tenon.io
NG-ACCESSIBLITY 47

48. ACCESSIBILITY UNIT-TESTS
48

49. ▸ By Addy Osmani
▸ JavaScript module for a11y testing
▸ https://addyosmani.com/a11y/
ACCESSIBILITY - DEVELOPERS- A11Y 49

50. React-A11y 50

51. 51
ACCESSIBILITY - DEVELOPERS- A11Y
To test shadow DOM

52. import {expect} from 'chai';
import App from '../app/components/App';
import a11yHelper from "./a11yHelper";
describe('Accessibility', function () {
this.timeout(10000);
it('Has no errors', function () {
let config = {};
a11yHelper.testEnzymeComponent(
<App/>, config, function (results) {
expect(results.violations.length).to.equal(0);
});
});
});
ACCESSIBILITY - DEVELOPER-ENZYME-TESTS 52

53. ‣ Pa11y dashboard => pronounce as ‘pally’
‣ Helps you run Accessibility testing as a service
GNU GPL License
Allowed to share, modify the software.
ACCESSIBILITY TESTING AS A SERVICE 53

54. Using Pa11y dashboard and Mongodb on a Docker
container
ACCESSIBILITY TESTING AS A SERVICE 54

55. PA11Y DASHBOARD
AUTOMATED ACCESSIBILITY TESTING AS A SERVICE 55

56. ▸ GOOD
▸ Scans a lot of code
▸ Good starting point of manual review
▸ BAD
▸ False positives
The Good and Bad 56

57. http://www.washington.edu/accesscomputing/AU/issues.html
LIST OF COMMON ACCESSIBILITY ISSUES 57

58. HOW CAN I IMPLEMENT A11Y STRATEGY IN MY
ORGANISATION?
Thumb Rules
➤ No single person can ensure an app is accessible on their own.
➤ Adopting Accessibility mindset takes time organisation wide.
➤ Develop Accessibility as a Culture
58

59. Involve people with disabilities early and understand
problem space and design per digital world resolving
those problems and thus meet business goals(those
20%)
STRATEGY
59

60. STRATEGY
➤ Establish an Org commitment with key stakeholders responsible for Accessibility.
➤ A checklist/questionnaire to include Accessibility for both new and existing apps
➤ Establish a benchmark (Say Level A, AA )
➤ Establish a standards for UX designers, (UI)Developers & Testers
➤ Start As early as possible from UX GEL(Global Experience Language)
➤ Color contrast
➤ Inclusive design ethos
➤ Conduct usability, User testing session outside org.
60

61. ▸ https://webaccessibility.withgoogle.com/course
▸ https://addyosmani.com/a11y/
▸ https://www.w3.org/WAI/intro/wcag
▸ https://dequeuniversity.com/
REFERENCES 61

62. LET’S TEST AND HELP BUILD A WEB THAT’S
ACCESSIBLE FOR EVERYONE
THANK YOU!
APPLITOOLS AND SAUCE LABS
Manoj Kumar