Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Accessibility metrics Accessibility Data Metrics and Reporting – Industry Best Practices

1,742 views

Published on

Accessible version: http://www.last-child.com/a11y-data-metrics/
Learn how top companies are tracking and graphing product accessibility progress and incorporating data from automated, manual, and user testing to create management dashboards.

Published in: Internet
  • Be the first to comment

Accessibility metrics Accessibility Data Metrics and Reporting – Industry Best Practices

  1. 1. Accessibility Metrics Ted Drake and Sagar Barbhaya Intuit Accessibility CSUN 2017 Accessible version: http://www.last- child.com/a11y-data-metrics/
  2. 2. What Do You Need? Write it on a post it note
  3. 3. Three Stages 1. Accessibility Management Goals 2. State of the Industry 3. Solutions to fill gaps
  4. 4. Current State • Manual Testing and Evaluations • Continual improvements • Documentation • Community Outreach • Stakeholders • Centralized Accessibility Leadership
  5. 5. What we have • Product database with evaluations, scores, stakeholders, and documents • Bug Tracking • Scattered Employee Resources • Centralized documentation • Non-integrated automation
  6. 6. What we need • Dashboards to surface data and trends • Connections between platforms • Automation and unit testing for developers • Shared test results • Connected resources • Support channel integration
  7. 7. Future Goal • Measurable lifecycle from tests/evaluation to production • Deep consumer feedback integration • Diverse workforce with support channels • Developer ownership of testing and solutions • Product ownership of process and success
  8. 8. State of the Industry
  9. 9. How are we doing? - Product Manager
  10. 10. What to Measure?
  11. 11. Product Quality Metrics • Issues by Priority – 77% • Issues by WCAG category – 54% • last time product had a manual evaluation – 46% • Closed Captioning support – 38% • Number of accessibility bugs opened per month/ year – 31%
  12. 12. Priority Levels • Impact on Customer • Template / Module / Unique • Density of Issues • High Traffic, Key Flows
  13. 13. Test to Ticket flow
  14. 14. Company Metrics •AODA compliance – 55% •Number of media inquiries about product/company accessibility – 45% •Conference talks by employees on accessibility – 45% •Product VPAT Coverage – 45% •Compliance towards 21st Century Communications Act (CVAA) – 36%
  15. 15. Compliance and Legal
  16. 16. Human Resources • Section 503: Self-Identification • Accommodations Requests • Ergonomic Evaluations • Service/Perk Requests
  17. 17. Customer Metrics •Usability participants with a disability – 63% •Feedback via customer support – 50% •Usability activities, including user personas, with disabilities – 38% •Customer support calls - 38% •Tickets filed by customer support – 38%
  18. 18. Net Promoter Score
  19. 19. Storage and Sharing
  20. 20. QuickBase
  21. 21. Spreadsheets
  22. 22. Pa11y Testing + Dashboard
  23. 23. Google Docs
  24. 24. Cloud Storage • DropBox • Box • Amazon Web Services • Google Drive • Slack
  25. 25. WorldSpace
  26. 26. Solutions
  27. 27. Bug Tracking
  28. 28. JIRA
  29. 29. JIRA Queries • labels = accessibility • labels=accessibility and status=Open • labels=accessibility and status = blocked • labels=accessibility and status=Closed and resolved > “-4w”
  30. 30. DOMO
  31. 31. Integrated Testing • Developer testing: Tenon, aXe, Nemo • Unit tests include accessibility • Export test results to JIRA • Tracking tool usage
  32. 32. Finding Outliers • Extremely critical to prioritize • Outliers need to be 
 identified and fixed first • Example - Turbo Tax in 
 season production bugs
  33. 33. Customer Support Logs • Search for Deaf and HOH customer requests: 
 [ LEVEL:SENTENCE TYPE:KEYWORD {deaf, hearing}] 
 [ LEVEL:SENTENCE TYPE:AND_KEYWORD {capability, impaired, challenged, hard}] • “Accessibility” topic within Salesforce for reports • Microsoft: Dedicated Support Channel
  34. 34. What Did You Need?
  35. 35. Questions?

×