Work presented during the W4A 2013. http://www.youtube.com/watch?feature=player_embedded&v=h5sX6KeR1WM

1. © 2013 IBM Corporation
Web Accessibility Snapshot:
An Effort to Reveal Coding Guidelines
Conformance
Vagner Figueredo de Santana and Rogério Abreu de Paula
IBM Research Brazil, Social Enterprise Technology Group, Service Systems
May 13, 2013
IBM Research

2. © 2013 IBM Corporation
Agenda
 Introduction and background
 Motivation
 Approach
 Results
 Discussion and conclusion

3. © 2013 IBM Corporation
Introduction and background
The Web has at least 14 billions of pages (www.worldwidewebsize.com)

4. © 2013 IBM Corporation
Introduction and background
 World Health Organization says that 15% have some for of disability
 Web Accessibility
– Perceive
– Understand
– Navigate
– Interact
– Contribute
 Correct Web pages markup is essential to semantics

5. © 2013 IBM Corporation
Introduction and background
 Level of comprehension required to understand the guidelines might be
the cause of its low use by web developers
– Sloat et al.2006
 Automated tests count on 35% of false negatives
– Brajnik, 2004
 Guidelines are not the “silver bullet” for Web accessibility
 Coding guidelines set the minimum standards
 Conformance with coding guidelines is not sufficient
 … but it is necessary

6. © 2013 IBM Corporation
Motivation
 Inspiration: Nomensa's report to the United Nation
 The Nomensa's report:
– Run in 2006
– Counted on 100 websites
– Involved sites from 20 countries
– Results: 97% of the websites did not reach WCAG 1.0 (Level A)
 Objective:
– To draw a broader and updated picture of coding guidelines
conformance
– Considering
• The top 1,000 popular websites
• Sample of 1,000 websites

7. © 2013 IBM Corporation
Approach
 To provide a snapshot
– Alexa.com's top 1,000 popular websites (homepages)
– Sample of 1,000 websites (homepages)
 To automatically evaluate
– Selenium IDE
• www.seleniumhq.org
– AChecker
• www.achecker.ca
 Metrics considered
– WCAG 2.0 Level A
• Minimum level to which a Web page should conform
– Known problems reported by AChecker
• Identified with certainty as accessibility barriers
• Developers must fix these problems

8. © 2013 IBM Corporation
Approach
 Sampling the Web
– Input
• One alphanumeric array (a, b, c, …, 0, …, 9)
• One array containing top level domains (ac, ad, …, com, …, net, …)
• List size (e.g., 1000)
– Procedure
• While the list's length is less then 1000
– Generates the length of the host name
– While host name is less than the generate length
Select a random position and concatenates it to the host name
– Selects a random top level domain and concatenates it to the host
– Tries to connect to the generated address
– If the address is valid, then it is added to the list of addresses
– Output
• List of working addresses

9. © 2013 IBM Corporation
Results
 Sampled (n=987)
- Mean known problems
7.00 (SD=35.48)
- One or more known problems: 85.11%
- Zero know problems: 14.89%
 Top popular (n=922)
- Mean known problems:
32.97 (SD=60.47)
- One or more known problems: 95.66%
- Zero know problems: 4.34%

10. © 2013 IBM Corporation
Results
 Sampled vs Top 1,000
- Results do not follow a normal
distribution
Shapiro-Wilk → p-value < 0.001
- Result sets are different
Wilcoxom Rank Sum → p-value 0.001
- Popularity is not correlated to the
number of known problems
Pearson correlation coefficient of 0.09

11. © 2013 IBM Corporation
Discussion and conclusion
 Homepages with know problems ranges form 85-95%
– Why homepages?
 Conformance is necessary... not sufficient
 A lot of work is beyond of automatic evaluation.... and this snapshot
 Commonly found problems
– Missing alt attribute
– Form with missing labels
 Web Accessibility Snapshot project aims at
– Providing updates in this subject
– Invite organizations to solve issues of higher impact with easy fixes
– For the future
• To increase the data set
• To highlight more and more an action-centered point of view

12. © 2013 IBM Corporation
If you want to know more about the project,
just let me know.... vagsant@br.ibm.com
Thank you!