Cynthia C. Shelly, Microsoft  Mike Barta, University of Washington
<ul><li>Web Applications </li></ul><ul><ul><li>Are interactive applications first </li></ul></ul><ul><ul><li>“ Web” is an ...
<ul><li>User Testing </li></ul><ul><li>Automated Validation </li></ul><ul><li>Auditing </li></ul><ul><li>Traditional Softw...
<ul><li>What is it </li></ul><ul><ul><li>End users </li></ul></ul><ul><ul><li>Influence or validate design </li></ul></ul>...
<ul><li>Uses </li></ul><ul><ul><li>Understanding design issues </li></ul></ul><ul><ul><li>How the product feels </li></ul>...
<ul><li>What is it </li></ul><ul><ul><li>Static analysis of code looking for common problems </li></ul></ul><ul><ul><li>Do...
<ul><li>Uses </li></ul><ul><ul><li>Quickly find common, known problems in markup </li></ul></ul><ul><ul><li>Find regressio...
<ul><li>What is it </li></ul><ul><ul><li>Done on finished or mostly finished product </li></ul></ul><ul><ul><li>By experts...
<ul><li>Uses </li></ul><ul><ul><li>Evaluates the state of a product at a given point in time </li></ul></ul><ul><ul><li>Us...
<ul><li>What is it </li></ul><ul><ul><li>Focus is on assuring a level of quality </li></ul></ul><ul><ul><li>Types of testi...
<ul><li>Uses </li></ul><ul><ul><li>Change the output on a quality measure </li></ul></ul><ul><ul><li>Test throughout the s...
<ul><li>We can improve the accessibility of a Web application by treating accessibility the same as other quality metrics,...
Upcoming SlideShare
Loading in …5
×

Application of Traditional Software Testing Methodologies to Web Accessibility

1,340 views

Published on

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,340
On SlideShare
0
From Embeds
0
Number of Embeds
5
Actions
Shares
0
Downloads
13
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Application of Traditional Software Testing Methodologies to Web Accessibility

  1. 1. Cynthia C. Shelly, Microsoft Mike Barta, University of Washington
  2. 2. <ul><li>Web Applications </li></ul><ul><ul><li>Are interactive applications first </li></ul></ul><ul><ul><li>“ Web” is an implementation detail </li></ul></ul><ul><ul><li>History of the Web as a collection of interconnected static documents obscures this </li></ul></ul><ul><li>Accessibility is an aspect of software quality </li></ul><ul><ul><li>The same methods used to find security or performance or functionality bugs can be used to find accessibility bugs </li></ul></ul>
  3. 3. <ul><li>User Testing </li></ul><ul><li>Automated Validation </li></ul><ul><li>Auditing </li></ul><ul><li>Traditional Software Testing? </li></ul>
  4. 4. <ul><li>What is it </li></ul><ul><ul><li>End users </li></ul></ul><ul><ul><li>Influence or validate design </li></ul></ul><ul><ul><li>Focused on discoverability and ease of use </li></ul></ul><ul><li>Types </li></ul><ul><ul><li>Usability Testing </li></ul></ul><ul><ul><li>Beta Testing </li></ul></ul><ul><ul><li>Reviews by end users outside the product team </li></ul></ul><ul><li>Application to Accessibility </li></ul><ul><ul><li>Testing with ‘real’ people with disabilities </li></ul></ul>
  5. 5. <ul><li>Uses </li></ul><ul><ul><li>Understanding design issues </li></ul></ul><ul><ul><li>How the product feels </li></ul></ul><ul><ul><li>Interaction with a particular AT product </li></ul></ul><ul><ul><ul><li>User who is comfortable using the AT product </li></ul></ul></ul><ul><li>Issues </li></ul><ul><ul><li>Isolating technical issues so developers can fix them </li></ul></ul><ul><ul><li>Usability testing uses small samples of like users </li></ul></ul><ul><ul><ul><li>Difficult to cover all disabilities </li></ul></ul></ul><ul><ul><ul><li>Difficult to get a group of users with same disability and AT configuration </li></ul></ul></ul><ul><ul><li>Can be expensive, in time or money </li></ul></ul><ul><ul><li>Beta testing and user reviews have response bias, skew towards more technical users </li></ul></ul><ul><ul><li>Need working code for AT interop testing, prototypes less useful for accessibility user testing </li></ul></ul>
  6. 6. <ul><li>What is it </li></ul><ul><ul><li>Static analysis of code looking for common problems </li></ul></ul><ul><ul><li>Done on markup, generally not on script </li></ul></ul><ul><li>Types </li></ul><ul><ul><li>General purpose tools test against a technology specification </li></ul></ul><ul><ul><li>Product-specific tools test against the product specification </li></ul></ul><ul><li>Application to Accessibility </li></ul><ul><ul><li>Test against an accessibility standard (WCAG, 508, etc.) </li></ul></ul>
  7. 7. <ul><li>Uses </li></ul><ul><ul><li>Quickly find common, known problems in markup </li></ul></ul><ul><ul><li>Find regressions </li></ul></ul><ul><ul><li>Spot trends, find areas where manual testing resources would be best spent </li></ul></ul><ul><li>Issues </li></ul><ul><ul><li>Many accessibility issues are difficult to automate </li></ul></ul><ul><ul><li>The issues that are easy to automate aren’t always the most important to test </li></ul></ul><ul><ul><li>General purpose tools have a lot of warnings and false postitives </li></ul></ul><ul><ul><li>Real bugs can get lost in the noise </li></ul></ul><ul><ul><li>It’s a tool, it’s not testing </li></ul></ul>
  8. 8. <ul><li>What is it </li></ul><ul><ul><li>Done on finished or mostly finished product </li></ul></ul><ul><ul><li>By experts not involved in development </li></ul></ul><ul><ul><li>Combination of automated testing and expert evaluation </li></ul></ul><ul><ul><li>Measures conformance to a set standard </li></ul></ul><ul><li>Application to Accessibility </li></ul><ul><ul><li>Expert evaluation against accessibility standards </li></ul></ul>
  9. 9. <ul><li>Uses </li></ul><ul><ul><li>Evaluates the state of a product at a given point in time </li></ul></ul><ul><ul><li>Useful for reporting </li></ul></ul><ul><ul><li>Useful for development teams without the expertise to do their own testing </li></ul></ul><ul><li>Issues </li></ul><ul><ul><li>Measures quality, does not assure it </li></ul></ul><ul><ul><li>Late in product cycle </li></ul></ul><ul><ul><li>Limited ramp-up time </li></ul></ul><ul><ul><li>Limited time to conduct audit </li></ul></ul><ul><ul><li>Organizational dynamics can work against outside auditors </li></ul></ul><ul><ul><li>Difficult to convince organizations to fix technically compliant but poorly accessible/usable sites </li></ul></ul>
  10. 10. <ul><li>What is it </li></ul><ul><ul><li>Focus is on assuring a level of quality </li></ul></ul><ul><ul><li>Types of testing typically done during the process of developing software </li></ul></ul><ul><ul><li>Discrete tools such as Unit, Functional, System, Integration, Regression, etc. </li></ul></ul><ul><ul><li>Procedural tools, such as exit criteria and design validation </li></ul></ul><ul><ul><li>The goal of testing is not to perform evaluations, but to pass them </li></ul></ul><ul><li>Application to Accessibility </li></ul><ul><ul><li>Test accessibility as one of many metrics of software quality </li></ul></ul><ul><ul><li>Test the functional accessibility of an interactive application </li></ul></ul><ul><ul><li>Test at various architectural levels for the exposure of information needed for accessibility </li></ul></ul>
  11. 11. <ul><li>Uses </li></ul><ul><ul><li>Change the output on a quality measure </li></ul></ul><ul><ul><li>Test throughout the software life cycle: requirements gathering, design, coding, testing, release </li></ul></ul><ul><ul><li>Isolate technical issues so developers can fix them </li></ul></ul><ul><ul><li>Find issues early, and educate the team, to prevent issues </li></ul></ul><ul><ul><li>Targeted: picking test cases most likely to uncover bugs </li></ul></ul><ul><ul><li>Understand the most important bugs, based on the purpose of the product </li></ul></ul><ul><ul><li>Constant stream of information allows adjustments can be made during the development process for the current release </li></ul></ul><ul><ul><li>Work with the organizational dynamics and rhythm of the business to impact organizational behavior with respect to quality metrics </li></ul></ul><ul><ul><ul><li>Help managers plan for resource allocation to impact accessibility at various points in the software life cycle </li></ul></ul></ul><ul><ul><ul><li>Raise the expertise of organization </li></ul></ul></ul><ul><ul><ul><li>Devote significant time to testing </li></ul></ul></ul><ul><ul><ul><li>Test accessibility the same way other quality metrics are tested </li></ul></ul></ul><ul><ul><ul><li>Leverage existing processes to reduce marginal cost </li></ul></ul></ul>
  12. 12. <ul><li>We can improve the accessibility of a Web application by treating accessibility the same as other quality metrics, and the application the same as other applications </li></ul><ul><li>Traditional Software Testing has a goal of changing the quality of a product, not just measuring it </li></ul><ul><li>There is much to be gained by applying the knowledge in the software testing field to web accessibility </li></ul>

×