1.
Accessibility Metrics
Ted Drake and Sagar Barbhaya
Intuit Accessibility
CSUN 2017
Accessible version: http://www.last-
child.com/a11y-data-metrics/

2.
What Do You Need?
Write it on a post it note

3.
Three Stages
1. Accessibility Management
Goals
2. State of the Industry
3. Solutions to fill gaps

4.
Current State
• Manual Testing and Evaluations
• Continual improvements
• Documentation
• Community Outreach
• Stakeholders
• Centralized Accessibility Leadership

5.
What we have
• Product database with evaluations, scores,
stakeholders, and documents
• Bug Tracking
• Scattered Employee Resources
• Centralized documentation
• Non-integrated automation

6.
What we need
• Dashboards to surface data and trends
• Connections between platforms
• Automation and unit testing for developers
• Shared test results
• Connected resources
• Support channel integration

7.
Future Goal
• Measurable lifecycle from tests/evaluation to
production
• Deep consumer feedback integration
• Diverse workforce with support channels
• Developer ownership of testing and solutions
• Product ownership of process and success

8.
State of the Industry

9.
How are we doing?
- Product Manager

10.
What to Measure?

11.
Product Quality Metrics
• Issues by Priority – 77%
• Issues by WCAG category – 54%
• last time product had a manual evaluation – 46%
• Closed Captioning support – 38%
• Number of accessibility bugs opened per month/
year – 31%

12.
Priority Levels
• Impact on Customer
• Template / Module / Unique
• Density of Issues
• High Traffic, Key Flows

13.
Test to Ticket flow

14.
Company Metrics
•AODA compliance – 55%
•Number of media inquiries about product/company
accessibility – 45%
•Conference talks by employees on accessibility – 45%
•Product VPAT Coverage – 45%
•Compliance towards 21st Century Communications Act
(CVAA) – 36%

15.
Compliance and Legal

16.
Human Resources
• Section 503: Self-Identification
• Accommodations Requests
• Ergonomic Evaluations
• Service/Perk Requests

17.
Customer Metrics
•Usability participants with a disability – 63%
•Feedback via customer support – 50%
•Usability activities, including user personas,
with disabilities – 38%
•Customer support calls - 38%
•Tickets filed by customer support – 38%

18.
Net Promoter Score

19.
Storage and Sharing

20.
QuickBase

21.
Spreadsheets

22.
Pa11y
Testing +
Dashboard

23.
Google
Docs

24.
Cloud Storage
• DropBox
• Box
• Amazon Web Services
• Google Drive
• Slack

25.
WorldSpace

26.
Solutions

27.
Bug Tracking

28.
JIRA

29.
JIRA Queries
• labels = accessibility
• labels=accessibility and status=Open
• labels=accessibility and status = blocked
• labels=accessibility and status=Closed and
resolved > “-4w”

30.
DOMO

31.
Integrated Testing
• Developer testing: Tenon, aXe, Nemo
• Unit tests include accessibility
• Export test results to JIRA
• Tracking tool usage

32.
Finding Outliers
• Extremely critical to prioritize
• Outliers need to be
identified and fixed first
• Example - Turbo Tax in
season production bugs

33.
Customer Support Logs
• Search for Deaf and HOH customer requests:
[ LEVEL:SENTENCE TYPE:KEYWORD {deaf,
hearing}]
[ LEVEL:SENTENCE TYPE:AND_KEYWORD
{capability, impaired, challenged, hard}]
• “Accessibility” topic within Salesforce for reports
• Microsoft: Dedicated Support Channel

34.
What Did You Need?

35.
Questions?