Accessibility metrics Accessibility Data Metrics and Reporting – Industry Best Practices

1,130 views

Published on

Accessible version: http://www.last-child.com/a11y-data-metrics/
Learn how top companies are tracking and graphing product accessibility progress and incorporating data from automated, manual, and user testing to create management dashboards.

Published in: Internet
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,130
On SlideShare
0
From Embeds
0
Number of Embeds
441
Actions
Shares
0
Downloads
25
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

Accessibility metrics Accessibility Data Metrics and Reporting – Industry Best Practices

  1. 1. Accessibility Metrics Ted Drake and Sagar Barbhaya Intuit Accessibility CSUN 2017 Accessible version: http://www.last- child.com/a11y-data-metrics/
  2. 2. What Do You Need? Write it on a post it note
  3. 3. Three Stages 1. Accessibility Management Goals 2. State of the Industry 3. Solutions to fill gaps
  4. 4. Current State • Manual Testing and Evaluations • Continual improvements • Documentation • Community Outreach • Stakeholders • Centralized Accessibility Leadership
  5. 5. What we have • Product database with evaluations, scores, stakeholders, and documents • Bug Tracking • Scattered Employee Resources • Centralized documentation • Non-integrated automation
  6. 6. What we need • Dashboards to surface data and trends • Connections between platforms • Automation and unit testing for developers • Shared test results • Connected resources • Support channel integration
  7. 7. Future Goal • Measurable lifecycle from tests/evaluation to production • Deep consumer feedback integration • Diverse workforce with support channels • Developer ownership of testing and solutions • Product ownership of process and success
  8. 8. State of the Industry
  9. 9. How are we doing? - Product Manager
  10. 10. What to Measure?
  11. 11. Product Quality Metrics • Issues by Priority – 77% • Issues by WCAG category – 54% • last time product had a manual evaluation – 46% • Closed Captioning support – 38% • Number of accessibility bugs opened per month/ year – 31%
  12. 12. Priority Levels • Impact on Customer • Template / Module / Unique • Density of Issues • High Traffic, Key Flows
  13. 13. Test to Ticket flow
  14. 14. Company Metrics •AODA compliance – 55% •Number of media inquiries about product/company accessibility – 45% •Conference talks by employees on accessibility – 45% •Product VPAT Coverage – 45% •Compliance towards 21st Century Communications Act (CVAA) – 36%
  15. 15. Compliance and Legal
  16. 16. Human Resources • Section 503: Self-Identification • Accommodations Requests • Ergonomic Evaluations • Service/Perk Requests
  17. 17. Customer Metrics •Usability participants with a disability – 63% •Feedback via customer support – 50% •Usability activities, including user personas, with disabilities – 38% •Customer support calls - 38% •Tickets filed by customer support – 38%
  18. 18. Net Promoter Score
  19. 19. Storage and Sharing
  20. 20. QuickBase
  21. 21. Spreadsheets
  22. 22. Pa11y Testing + Dashboard
  23. 23. Google Docs
  24. 24. Cloud Storage • DropBox • Box • Amazon Web Services • Google Drive • Slack
  25. 25. WorldSpace
  26. 26. Solutions
  27. 27. Bug Tracking
  28. 28. JIRA
  29. 29. JIRA Queries • labels = accessibility • labels=accessibility and status=Open • labels=accessibility and status = blocked • labels=accessibility and status=Closed and resolved > “-4w”
  30. 30. DOMO
  31. 31. Integrated Testing • Developer testing: Tenon, aXe, Nemo • Unit tests include accessibility • Export test results to JIRA • Tracking tool usage
  32. 32. Finding Outliers • Extremely critical to prioritize • Outliers need to be 
 identified and fixed first • Example - Turbo Tax in 
 season production bugs
  33. 33. Customer Support Logs • Search for Deaf and HOH customer requests: 
 [ LEVEL:SENTENCE TYPE:KEYWORD {deaf, hearing}] 
 [ LEVEL:SENTENCE TYPE:AND_KEYWORD {capability, impaired, challenged, hard}] • “Accessibility” topic within Salesforce for reports • Microsoft: Dedicated Support Channel
  34. 34. What Did You Need?
  35. 35. Questions?

×