Accessibility Metrics
Ted Drake and Sagar Barbhaya
Intuit Accessibility
CSUN 2017
Accessible version: http://www.last-
child.com/a11y-data-metrics/

What Do You Need?
Write it on a post it note

Three Stages
1. Accessibility Management
Goals
2. State of the Industry
3. Solutions to fill gaps

Current State
• Manual Testing and Evaluations
• Continual improvements
• Documentation
• Community Outreach
• Stakeholders
• Centralized Accessibility Leadership

What we have
• Product database with evaluations, scores,
stakeholders, and documents
• Bug Tracking
• Scattered Employee Resources
• Centralized documentation
• Non-integrated automation

What we need
• Dashboards to surface data and trends
• Connections between platforms
• Automation and unit testing for developers
• Shared test results
• Connected resources
• Support channel integration

Future Goal
• Measurable lifecycle from tests/evaluation to
production
• Deep consumer feedback integration
• Diverse workforce with support channels
• Developer ownership of testing and solutions
• Product ownership of process and success

State of the Industry

How are we doing?
- Product Manager

What to Measure?

Product Quality Metrics
• Issues by Priority – 77%
• Issues by WCAG category – 54%
• last time product had a manual evaluation – 46%
• Closed Captioning support – 38%
• Number of accessibility bugs opened per month/
year – 31%

Priority Levels
• Impact on Customer
• Template / Module / Unique
• Density of Issues
• High Traffic, Key Flows

Test to Ticket flow

Company Metrics
•AODA compliance – 55%
•Number of media inquiries about product/company
accessibility – 45%
•Conference talks by employees on accessibility – 45%
•Product VPAT Coverage – 45%
•Compliance towards 21st Century Communications Act
(CVAA) – 36%

Compliance and Legal

Human Resources
• Section 503: Self-Identification
• Accommodations Requests
• Ergonomic Evaluations
• Service/Perk Requests

Customer Metrics
•Usability participants with a disability – 63%
•Feedback via customer support – 50%
•Usability activities, including user personas,
with disabilities – 38%
•Customer support calls - 38%
•Tickets filed by customer support – 38%

Net Promoter Score

Storage and Sharing

QuickBase

Spreadsheets

Pa11y
Testing +
Dashboard

Google
Docs

Cloud Storage
• DropBox
• Box
• Amazon Web Services
• Google Drive
• Slack

WorldSpace

Solutions

Bug Tracking

JIRA

JIRA Queries
• labels = accessibility
• labels=accessibility and status=Open
• labels=accessibility and status = blocked
• labels=accessibility and status=Closed and
resolved > “-4w”

DOMO

Integrated Testing
• Developer testing: Tenon, aXe, Nemo
• Unit tests include accessibility
• Export test results to JIRA
• Tracking tool usage

Finding Outliers
• Extremely critical to prioritize
• Outliers need to be
identified and fixed first
• Example - Turbo Tax in
season production bugs

Customer Support Logs
• Search for Deaf and HOH customer requests:
[ LEVEL:SENTENCE TYPE:KEYWORD {deaf,
hearing}]
[ LEVEL:SENTENCE TYPE:AND_KEYWORD
{capability, impaired, challenged, hard}]
• “Accessibility” topic within Salesforce for reports
• Microsoft: Dedicated Support Channel

What Did You Need?

Questions?