1.
A macroscopic Web accessibility
evaluation at different
processing phases
Nádia Fernandes, Luís Carriço
9th International Cross-Disciplinary Conference on Web Accessibility –
16/17th April 2012 – Lyon, France.

2.
Motivation
9th International Cross-Disciplinary Conference on Web Accessibility –
2
16/17th April 2012 – Lyon, France.

3.
Introduction
• The Web is being used by all kinds of people;
• Web sites must be accessible;
• Modern Web development transcends static HTML;
• A more real evaluation is necessary.
9th International Cross-Disciplinary Conference on Web Accessibility –
3
16/17th April 2012 – Lyon, France.

4.
Previous work
9th International Cross-Disciplinary Conference on Web Accessibility –
4
16/17th April 2012 – Lyon, France.

5.
QualWeb evaluator
• Accessibility evaluation of Web
pages:
– using WCAG 2.0 ,
• 18 HTML techniques;
– at different processing
Before
phases
• BEFORE and AFTER
Browser processing;
After
9th International Cross-Disciplinary Conference on Web Accessibility –
5
16/17th April 2012 – Lyon, France.

6.
QualWeb evaluator: execution
9th International Cross-Disciplinary Conference on Web Accessibility –
6
16/17th April 2012 – Lyon, France.

7.
Some improvements...
9th International Cross-Disciplinary Conference on Web Accessibility –
7
16/17th April 2012 – Lyon, France.

8.
PhantomJs
• is a command-line tool that uses WebKit ,
• works like a WebKit-based Web browser (simulation);
• can be controlled using Javascript.
9th International Cross-Disciplinary Conference on Web Accessibility –
8
16/17th April 2012 – Lyon, France.

9.
Problems Solved!
1. It avoids data injection at the browser level;
2. Evaluated Web page before and after browser processing is
exactly the same;
3. Integration with a crawler.
9th International Cross-Disciplinary Conference on Web Accessibility –
9
16/17th April 2012 – Lyon, France.

10.
Research Question
How do the macroscopic properties emerging from Web
accessibility change in respect to the processing phase of
delivery?
9th International Cross-Disciplinary Conference on Web Accessibility –
10
16/17th April 2012 – Lyon, France.

11.
Experimental Study
• We evaluated a set of Web pages from a list provided by the
Portuguese Web Archive (version of 2008);
• We used the QualWeb evaluator.
9th International Cross-Disciplinary Conference on Web Accessibility –
11
16/17th April 2012 – Lyon, France.

12.
Metrics
• Results of the evaluation are presented in terms of:
– PASS, WARN, and FAIL (Applicable = PASS + WARN + FAIL)
• Metrics used:
– rate conservative =
– rate optimistic =
– rate strict =
• The results are between accessible (100%) and not accessible (0%).
9th International Cross-Disciplinary Conference on Web Accessibility –
12
16/17th April 2012 – Lyon, France.

13.
Results
• 24, 462 Web pages evaluated.
• Total Number of HTML elements:
– before processing - 24,918,720
– after processing - 41,967,072
– (Ratio ≈ 1.7).
• Average Number of HTML elements per page:
– before processing - 1010
– after processing – 1710
– (Ratio ≈1.7).
9th International Cross-Disciplinary Conference on Web Accessibility –
13
16/17th April 2012 – Lyon, France.

14.
The differences of an HTML
document between both
Processing Phases were observed.
9th International Cross-Disciplinary Conference on Web Accessibility –
14
16/17th April 2012 – Lyon, France.

15.
Results: Average Outcomes
• Successes
– before processing - 9 elements
– after processing - 87 elements
– (Ratio ≈ 9.7).
• Failures
– before processing - 46 elements
– after processing – 176 elements
– (Ratio ≈ 3.8).
• Warnings
– before processing - 262 elements
– after processing – 451 elements
– (Ratio ≈ 1.7).
9th International Cross-Disciplinary Conference on Web Accessibility –
15
16/17th April 2012 – Lyon, France.

16.
Results: rates Before and After processing
• Conservative rate:
– The average quality increases after
processing.
– Accessibility quality between 60% and 90%
disappeared after processing.
• Optimistic rate:
– The average quality decreases after
processing.
– Some results lower than 20% disappeared
after processing.
– Decrease of higher accessibility quality,
leading to a lower accessibility average.
9th International Cross-Disciplinary Conference on Web Accessibility –
16
16/17th April 2012 – Lyon, France.

17.
Results: rates Before and After processing
• Strict rate:
– The average increased after processing.
– Results higher than 85% disappeared
after processing.
– Worse pages before processing get higher
scores after processing.
– Better pages before processing are
ranked lower after processing.
9th International Cross-Disciplinary Conference on Web Accessibility –
17
16/17th April 2012 – Lyon, France.

18.
Limitations
1. Techniques coverage
2. Dynamic content
3. Automatic evaluation
9th International Cross-Disciplinary Conference on Web Accessibility –
18
16/17th April 2012 – Lyon, France.

19.
Discussion
• Evaluations before processing clearly is not the best option!
• We used/percept/interact with the after processed version.
• Considering the rates…
– Web pages possess higher uniformity
– That can be explained with reusable code
9th International Cross-Disciplinary Conference on Web Accessibility –
19
16/17th April 2012 – Lyon, France.

20.
Discussion
• Impact on Designing Accessible Web Pages
– Importance of sharing reusable code;
– High quality reusable code produce better quality pages.
• Impact on the Perception of Accessibility
– Each metric identify different perspectives;
– It is important that evaluations assess what user perceive.
9th International Cross-Disciplinary Conference on Web Accessibility –
20
16/17th April 2012 – Lyon, France.

21.
Conclusion
• We presented a large-scale study of accessibility on the Web.
• We were able to characterize some accessibility properties of the
Web, pointing some differences between processing phases.
• The results obtained on the evaluation of pages after browser
processing tend to be more homogeneous than before.
• Considering that the end-user interacts with the after processed
pages then most studies about Web quality should be redone.
9th International Cross-Disciplinary Conference on Web Accessibility –
21
16/17th April 2012 – Lyon, France.

22.
Future Work
1. Enlarge the coverage of WCAG 2.0 implemented tests
2. Evaluate Rich Internet applications
3. Perform a comparative set of studies
9th International Cross-Disciplinary Conference on Web Accessibility –
22
16/17th April 2012 – Lyon, France.

23.
nadiaf@di.fc.ul.pt
9th International Cross-Disciplinary Conference on Web Accessibility –
23
16/17th April 2012 – Lyon, France.