If there is no formal program in place, chances are accessibility of your systems will be low
Often, the answer is “nobody”.In cases where such a person exists, they might not have a firm grasp of what compliance isHuman Resources/ EEO staffUX team
The PMO is the best starting point for making sure accessibility progress is tracked during projects – especially new ones.“No PMO” means a formalized process for accessibility compliance may not exist.
An established SDLC may (hopefully) contain touch points during the various phases where accessibility can be verified and tested for.If not, at the very least it gives us place to put it.
This is sort of the “proof in the pudding”. You can nearly guarantee that if it isn’t in the SDLC then it is unlikely that the company will have any meaningful accessibility program.After all, if it isn’t formalized there, then where else can it be formalized?
Any time an IT product/ service is developed or procured, various specific requirements are included in the procurement and/ or requirements documents. If these documents do not discuss the accessibility requirements, then accessibility is not likely to be a consideration for the developers (and the final deliverable will not be compliant)
No control over what gets in means anything gets in.Typically organizations validate for security, functional performance, but not accessibility.
Most developers are self-taught in their trade.Formal training (at computer schools, college comp. sci. programs, etc.) often do not discuss accessibility to any meaningful degreeTherefore if the company doesn’t do it, it doesn’t happen.
Typically, only web accessibility has much attention by customers.Or, they assume software to be inherently accessibleOr, they don’t give much concern over other technologies
The more that a website/web application diverges from standard HTML & CSS, the more likely that developers are ill-prepared to address accessibilityAlso, the more likely that they’ll be given to misunderstanding the facts regarding accessibility.
Typically none of these will have strong accessibility auditing experience.Developers’ jobs should include testing their code, therefore they should be doing *some* testing for accessibility. However, unless they’re well educated in accessibility they probably don’t know much about it and therefore don’t know how to test for it.QA staff generally don’t have much accessibility knowledge – but they should.
Use of free tools/toolbars/ plugins and out of date products (AccVerify, InFocus, A-Prompt, Bobby/Watchfire) indicates a lack of formal accessibility process in the org. and lack of knowledge in this domain.(At the very least, it betrays a lack of organizational maturity with respect to accessibility)
If they use an enterprise-class tool (AMP, Compliance Sherriff, Rational AppScan, Worldspace): Have they had any formal training in how to use the product?If they have not been trained on the product, there’s a strong chance they’re not getting much benefit from using it. They may have even given up on the tool altogether.
Lacking a documented methodology will mean that results may be: InaccurateIncompleteNot repeatable during Regression
“None”Obviously BadWCAGWhat version?What Level/ Priority?Section 508What technical provisions? Do they test for 1194.31?(Gov’t/ Integrators) If they develop Flash/ Flex and don’t test for 1194.21, they don’t understand accessibility
Most developers are self-taught in their trade.Formal training (at computer schools, college comp. sci. programs, etc.) often do not discuss accessibility to any meaningful degreeTherefore if the company doesn’t do it, it doesn’t happen.
If so, which assistive technologies?Use of free AT or simulators is not likely to yield valid results Use of only one AT is not considered complete coverage. What is their level of proficiency with these assistive technologies?Less than “daily use” is considered equivalent to “not proficient”
Each provision of a given standard needs documented conformance criteriaOtherwise subjective interpretation occurs.Test results will be:IncompleteInvalidNot Repeatable
The single best way to measure accessibility is to test with users with disabilities.Beware that users who are highly technical are not good representations of typical users Success with one user does not correlate to success with all usersOne guy with JAWS being successful at a task is not a blanket blessing upon the entire system
Fact: Nobody has ever been sued because they did not meet technical conformance criteria.They were sued because of functional performance issuesTechnical conformance is meant as a way to meet functional performance (and to prosecute/ defend the system)
Fundamentally, the primary reason why accessibility isn’t more well-placed in the enterprise is because
The bottom end of estimates of cost-per-defect is about $500 per bug. This amount is mostly dependent on two factors: time to fix the average bug and the number of bugs fixed. Less bugs fixed will raise the cost-per-bug, as the typical case is that developers can (and will) be able to fix multiple bugs of the same type rather quickly.The thing to take away from this is the fact that this is time and money that does not need to be spent. If all staff are properly trained and accessibility is properly integrated into the development lifecycle, the project budget and timeline are protected from this unnecessary expense
During all projects dealing with the purchase, development, or implementation of a software or web project, accessibility needs to be included in every stage – from planning to disposition. For instance, during the planning phase accessibility stakeholders must be identified and included. Later these stakeholders are able to assist in requirements analysis and development. During the design and development phases, accessibility requirements must be included and accessibility testing should be included in the testing phase. Finally, accessibility should be included into the deployment and maintenance phases to ensure as the project is released and maintained.
Testing for accessibility compliance is vital for determining the status of compliance for both the individual projects of an organization as well as the organization as a whole. Without thorough testing, any statements relating to the organization’s compliance will be based purely on conjecture. It is only through the implementation of testing and auditing processes that the necessary data can be captured to understand the organization’s compliance status. Organizations should develop detailed policies with respect to their accessibility compliance and part of these policies should outline the criteria against which systems will be tested and how that criterion will be tested. The organization should then develop or purchase the necessary tools or services to perform that testing.
When it comes to accessibility during project management, there is really little difference. In both cases, you will want to ensure that accessibility has been included in all phases it is appropriate to do so.The primary difference comes with how this is managed. In a waterfall model, accessibility is handled like all other traits of the product: everything is planned as much as possible up front, whereas in an Agile model only those accessibility concerns during this iteration are relevant.
Accessibility user stories can help ensure accessibility is kept in mind when determining what will be included in the iteration. In the first example, we see a user story that may result in an entirely new feature (or characteristic of a feature) that probably would not exist were it not for the consideration of accessibility during story development. In the second case, we see a modification or extension of a typical user story which could have a big impact on how that feature evolves. “Everyone” visiting the site would benefit from the ability to compare colleges. However, the “using a screen reader” part may help steer the characteristics of how the college-comparison feature is designed & developed.Identifying a Customer Representative who is disabled (or is a SME on accessibility) can be an invaluable resource during development cycle. Team members can turn to the Customer Representative with questions regarding how the goal(s) in the user story can be met – in an accessible manner.
Based on features under development this cycle:Identify any applicable standards. Once the team has come to a determination of what features are going to be included in this iteration, it is vital to then come to a determination of which standards are applicable to those features. The company itself may have determined which standards it is going to adhere to (such as WCAG 1.0, WCAG 2.0, Section 508, or something else). However, when it comes to this iteration, not all of the items in the standard will apply. For example, imagine our company has chosen WCAG 2.0. If there are no features which include audio or video during this iteration, then we don’t need to worry about Guideline 2 of WCAG 2.0 (Time Based Media). So the first step here is to determine which standards and provisions of those standards are applicable to the work we’re doing in this iteration.For those standards, identify conformance criteria. Each provision of our chosen standard can further be broken down into conformance criteria. For example, WCAG 2.0 Success Criterion 1.1.1 (Non-text content) can be broken down into several conformance criteria. For example “All images must have alternate text”, “All alternate text for images should be informative”, and so on. (Generally the conformance criteria can be borrowed by or born directly from the standard language itself. See: http://www.w3.org/TR/UNDERSTANDING-WCAG20/text-equiv-all.html_For each conformance criteria, identify best practices to develop requirements. Once we know what the conformance criteria are known, we can then draw together a list of best practices aimed at meeting the conformance criteria. Again, in consideration of WCAG 2.0 1.1.1, there could be as many as 3-dozen best practices – however, not all 36 will apply in this iteration, depending upon what features are under development.Note: this probably sounds like a lot of work to be doing – the likes of which appears counter to the spirit of what Agile is all about. The reality, however, is that much of the work above can be done at the organization level once and can exist as internal best practices from which to draw when working on this iteration’s features.Include these requirements in Definition of Done. Once we have our best practices outlined, we know what it takes to determine what “Done” means. In many cases, our “Done” list won’t look too different than it does today. It will still include things like “All Code Checked In”, “All Unit Tests Passed”, etc. But what might be different is what it takes to get there. In other words, our list of tests may be different. Or, on the other hand, we could actually include a new line “All Accessibility Tests Passed” or “All disabled user test cases passed”. However you as a company choose to approach this, just be sure that you keep in mind that if accessibility isn’t part of what you as the PM considers done, it won’t be what the developer thinks of as done, either.
During the active development during each iteration, developers will create tests to verify the code is working and meets the requirements for the User Story. In the process of creating these tests, the developers should be sure to include any applicable accessibility tests as well.We recognize that accessibility is primarily a user-interface concern and therefore developers may not be developing unit tests for the UI. However, it is up to the team to decide whether any tests can be included. For instance, if you’re testing the code that retrieves a video file from a database, the test could be extended to include a verification that the related SMIL or TimedText file is pulled as well.In cases where actual Unit Tests aren’t being written, developers should still utilize some other form of testing. For example, a developer could utilize FireEyes to test for accessibility. Automated testing, in and of itself, is not sufficient for making definitive judgments regarding accessibility, but this should be regarded as bare minimum before committing changes.
Accessibility is not unlike any other requirements. There will invariably be some accessibility-related requirements which just can’t make it through in this iteration. So, like any other unmet requirement, we place unmet accessibility items into the backlog for inclusion during the next sprint.
Remediation of accessibility issues should take into consideration how long it will take to fix a bug and what sort of positive impact fixing the bug will have on users with disabilities. We can infer from the image in this slide that issues which can be fixed fast and have high positive impact should be the ones which get our more immediate attention. Issues which take a long time to fix and don’t really make much impact should be deferred for later – with the understanding that all issues deserve consideration.Although this image shows only two axes: Time and Impact, there is also one more which should be considered, which is Volume. In this scenario, we’ll want to ensure that we fix repetitive pattern issues en masse. This provides a level of efficiency which will assist in mitigating risk (for the pattern violations) rather quickly. Either way, the image above gets to the point rather well.
In order for all relevant parties to understand their roles in ensuring accessibility in the enterprise, they must be trained. Executives, managers, project managers, procurement staff, human resources personnel and even most web & software developers typically do not get significant training in accessibility compliance. Even most computer science or human factors programs in college do not offer detailed instruction in development of accessible systems. Therefore the inclusion of new accessibility-related requirements into their duties will result in failure unless they have been sufficiently trained in what their duties are with respect to accessibility.
Managing Accessibility Compliance in the Enterprise
Managing Accessibility Compliance in the Enterprise<br />Karl Groves<br />Director of Training, Deque Systems<br />Phone: 443-517-9280 <br />E-mail: firstname.lastname@example.org<br />Twitter: @karlgroves<br />
Agenda<br />Defining the problem<br />Solving the Problem<br />Gauging Organizational Maturity<br />Managing Compliance<br />Project Management Approaches<br />Waterfall<br />Agile<br />Training<br />Center of Excellence<br />Recap & Questions<br />
Defining the Problem<br />Objective: Understand the ways in which accessibility is often mishandled in large organizations, thus leading to risk exposure<br />
Defining the Problem<br />Accessibility compliance is often backwards<br />Testing & compliance efforts often happen after the fact<br />Post-deployment remediation is often expensive, time-consuming, and incapable of addressing high impact issues<br />This damages profitability, timelines, and quality<br />
Defining the Problem<br />Accessibility is often not part of the process<br />Should be included in all phases of the lifecycle<br />Planning<br />Requirements<br />Procurement/ Design & Development<br />Release<br />Maintenance<br />
Defining the Problem<br />Staff often lack training on accessibility<br />Executives<br />Human Resources<br />Project Managers<br />Developers<br />Content Creators<br />QA<br />
Defining the Problem<br />Accessibility policy & procedure not formalized<br />Not part of ELC/ SDLC<br />No formal conformance criteria<br />No teeth to acceptance process<br />No enterprise tools provided to staff<br />
Gauging Organizational Maturity<br />Solving the Problem<br />
Gauging Organizational Maturity<br />Where does the organization stand nowwith respect to Accessibility Policy & Procedure?<br />This gives us our path moving forward.<br />
Gauging Organizational Maturity<br />Is there a formal program in place to manage accessibility compliance?<br />
Gauging Organizational Maturity<br />Who is tasked with coordinating accessibility compliance?<br />
Gauging Organizational Maturity<br />Does the org. have a PMO (Project Management Office)?<br />PMO might also be Program Management Office<br />
Gauging Organizational Maturity<br />Does the organization have a documented SDLC/ ELC?<br />
Gauging Organizational Maturity<br />Has accessibility been placed into your ELC/ SDLC processes?<br />
Gauging Organizational Maturity<br />Does language exist in your procurement/ specification development documents which discuss accessibility compliance?<br />If so, is it specific enough to be followed<br />
Gauging Organizational Maturity<br />Are deliverables validated for accessibility before acceptance?<br />Is code validated for accessibility before acceptance into source control?<br />
Gauging Organizational Maturity<br />What internal training is in place to educate QA/ development staff in accessibility?<br />
Gauging Organizational Maturity<br />What technologies are under development by the company?<br />Web?<br />Software?<br />Documents? <br />Multimedia?<br />
Gauging Organizational Maturity<br />What software/ tools are in use by the development team to assess accessibility?<br />
Gauging Organizational Maturity<br />If they use an enterprise-class tool, have they had any formal training in how to use the product?<br />
Gauging Organizational Maturity<br />Is there a formalized (documented) accessibility auditing methodology in place?<br />
Gauging Organizational Maturity<br />To what standards are the company’s products developed/ tested against?<br />
Gauging Organizational Maturity<br />What formal training does the typical developer have in accessibility?<br />What formal training does the typical QA tester have in accessibility?<br />
Gauging Organizational Maturity<br />Do the testers use assistive technologies to perform tests?<br />
Gauging Organizational Maturity<br />Have they documented the conformance criteria for the standards against which they’ve chosen to comply?<br />
Gauging Organizational Maturity<br />Does the company test their system(s) using users with disabilities?<br />
Gauging Organizational Maturity<br />Does the company test for functional performance?<br />
Organizational Maturity: A Customer Story<br />Conducted an Accessibility Skills Assessment Survey for a client with about 200 staff representing Content Creators, Design/UI, QA and Project Management members. The client’s goal was to determine their accessibility related knowledge. The results were:<br /><ul><li> 79% had formal Computer Science training
55% of the skills questions were answered incorrectly across all 4 areas
23% of the respondents had some formal training in accessibility
22% had training* in the workplace on Accessibility (not formal training)
21% seek out accessibility knowledge online through web sites and blogs
3% of those tested attended an accessibility related event
0% have purchased books on the topic</li></li></ul><li>Managing Compliance<br />Solving the Problem<br />
Managing Compliance<br />How can we address the shortcomings found in our organization’s level of maturity with regard to accessibility?<br />
Managing Compliance: High Level<br />Train, train, train<br />Institutionalize conformance<br />Plan compliance a head of time<br />Include a SME throughout all project phases<br />Monitor compliance at all phases<br />Implement Center of Excellence<br />Prevention is preferable to inspection & rework.<br />Remediation can add up to 40% more time to front-end development if not done right in the first place<br />
Remediation vs. Doing it Right<br />Avg. cost per defect = (num of devs * num of hours) * cost per dev per hour<br /> --------------------------------------------------<br /> (number of fixed defects)<br />Some estimates in QA community calculate cost around $500 per defect to find & fix defects and deploy remediated code<br />Dependent upon #of bugs, etc.<br />
Planning Phase<br />Determine what risk is involved re: accessibility<br />Determine the overall impact accessibility may have on project timeline<br />Determine whether any extra funding or resources are needed for accessibility<br />Include accessibility assets needed for project<br />Determine what accessibility related activities are necessary in each phase<br />
Requirements Phase<br />Identify accessibility stakeholders<br />For each feature/ technology in use, determine what standards and guidelines will apply<br />Include typical use cases/ user stories to generate accessibility requirements<br />
Procurement<br />Investigate what conformance requirements exist for deliverable<br />Communicate this in all solicitations<br />Research available market offerings<br />Determine which product/ service offers highest level of compliance while fitting business need<br />Validate vendor claims of conformance, they will often be inaccurate or incomplete<br />Ensure final award documents cite conformance requirements<br />
Design Phase<br />Utilize deliverables from planning & requirements phases to inform design phase<br />Revisit/ revise conformance criteria based on technologies in use<br />Validate design prototypes and comps with stakeholders and SMEs<br />Audit functional mockups for accessibility<br />Utilize formal best practices to gauge compliance<br />
Development Phase<br />Get ahead of accessibility issues. This is the last viable chance to prevent problems<br />Revisit/ revise conformance criteria based on technologies in use<br />Perform iterative testing as system is developed<br />Developers should test code as they develop, just as they would for browser compatibility<br />
Testing<br />Thorough testing required<br />Test against formal standard with well-defined conformance criteria<br />Ensure testing involves functional performance with assistive technologies<br />
Deployment Phase<br />Ensure system is deployed with any accessibility-related configuration in place<br />
Maintenance Phase<br />Provide a method to identify and track accessibility-related problems (pref. as bugs)<br />Assign appropriate priority to issues<br />
Milestone 1<br />Milestone 2<br />Milestone 3<br />Milestone 4A<br />Milestone 4B<br />Milestone 5<br />Work ProductComponent<br />Applicable Provision Evaluation<br />Initial<br />Final<br />Update<br />Update<br />Accessibility Risk Information Document<br />Initial<br />Final<br />Update<br />Update<br />Initial<br />Final<br />Update<br />Accessibility Compliance Approach<br />Initial<br />Final<br />Update<br />Update<br />Integration Plan forAccessible Support<br />Initial<br />Final<br />Final<br />Initial<br />Accessibility Test Plan<br />Accessibility Test Results<br />Identify the Applicable 508 Provision<br />Create a Test Plan<br />Identify 508 Issuesand Make Corrections<br />Enterprise Life Cycle (ELC) Section 508 Work Product - to - Milestone Cross-Reference Matrix<br />
Agile vs. Waterfall<br />Both methodologies have:<br />Planning<br />Requirements<br />Design<br />Develop<br />Implementation<br />Difference is in approach<br />No difference regarding accessibility<br />
Agile - Planning<br />Develop accessibility user stories<br />“I want to be able to access audio description for online videos”<br />“I want to be able to compare products”…”using a screen reader”<br />Identify disabled Customer Representative<br />“Customer collaborationover contract negotiation”<br />(Agile Manifesto)<br />
Agile – Planning<br />Based on features under development this cycle:<br />Identify any applicable standards.<br />For those standards, identify conformance criteria<br />For each conformance criteria, identify best practices to develop requirements<br />Include these requirements in Definition of Done<br />
Agile - Development<br />Developers should create accessibility tests during test development<br />Developers should utilize automated testing (inc. tools like FireEyes) during development prior to committing changes<br />
Agile - Development<br />Ensure any unmet accessibility requirements are put into sprint backlog for reinclusion next iteration<br />
Remediation<br />Treat accessibility errors as you would any other bug<br />Prioritize based on impact, time to fix<br />
Benefits of Training<br />Addresses disparities in level of understanding<br />Addresses inaccuracies/ deficiencies in understanding<br />Reduces risk of non-compliant interfaces & content<br />Avoids costly post release remediation<br />Protects project timelines and budgets<br />
Training Philosophy<br />Train people to understand disabilities<br />A firm grasp of “Why” can always lead you to discover “how”. Technology is always changing. Challenges faced by disabled users do not change.<br />Train people to understand their specific impact on end users<br />
Training<br />All involved in design & dev, plus HR & execs should get high level understanding of:<br />Laws<br />Standards<br />Understanding Disability<br />
Training<br />Human Resources<br />Skill set(s) to look for in future applicants<br />Training requirements for current staff<br />
Training<br />Procurement<br />Legal implications of accessibility compliance<br />How different technologies impact accessibility<br />How, when, and which standards apply<br />
Training<br />Project Management<br />Understanding requirements & how to define them<br />Integrating accessibility into lifecycle: what & where<br />
Training<br />Designers<br />Specific BPs relating to interaction & visual design<br />What they design gets implemented<br />
Training<br />Developers<br />Specific BPs relating to production of accessible interfaces<br />Specific advanced techniques based on technologies under development. <br />
Training<br />Content Creators<br />Specific BPs relating to production of accessible content<br />Techniques & Procedures on use of content creation tools (i.e. content management systems) so accessible output is ensured<br />
Training<br />QA<br />Need to understand how to test for accessibility<br />Need to understand how to use accessibility testing tools & interpret their output<br />
Accessibility Center of Excellence<br />Managing Compliance<br />
Center of Excellence: What it is<br />Centralized location for knowledge, training, support, and expertise in accessibility.<br />Provides communication between knowledge domains<br />Develops, maintains, and shares accessibility resources, and assets<br />
COE: The Promise<br />Support for individuals and enterprise <br />Standards for consistent implementation<br />Training to improve individual and enterprise execution<br />Measurements to the expectation<br />Governance for consistent implementation by the agency and contractors<br />
COE: Support <br />Design Support<br />Prototype Validation<br />Development Support<br />Accessibility User Stories<br />Customer Advocate<br />Subject matter expertise<br />Testing Support<br />Testing/ Conformance<br />Continuous Monitoring<br />Use Case/ Usability Test Support<br />
COE: Standards<br />Broad, Organizational Standards<br />Interpretation of Ind. Standards<br />Development Guides<br />Global Testing to determine areas of improvement<br />
COE: Training<br />Establish an agency/corporate curriculum<br />Testing/ Conformance guides based on technologies in use<br />New hire assets<br />
COE: Measurements<br />Dashboard reporting throughout all levels of the enterprise<br />Establish your benchmark and measure improvements<br />Assist PM in measuring success<br />Gather metrics<br />
COE: Governance<br />Ensure consistent contract language<br />Ensure compliance of deliverables by vendors<br />Gatekeeper to acceptance/ release/ milestone exit<br />Rules which are not enforced don’t get followed<br />
Recap<br />The Problem<br />Accessibility Compliance is Backwards<br />Accessibility Not Part of the Process<br />Staff are not trained<br />Accessibility Policy & Procedure not formalized<br />The Solution<br />Train, train, train<br />Institutionalize conformance<br />Plan compliance<br />Include a SME throughout all project phases<br />Monitor compliance<br />Implement Center of Excellence<br />