1.
A macroscopic Web accessibility evaluation at different processing phases Nádia Fernandes, Luís Carriço 9th International Cross-Disciplinary Conference on Web Accessibility – 16/17th April 2012 – Lyon, France.

2.
Motivation 9th International Cross-Disciplinary Conference on Web Accessibility – 2 16/17th April 2012 – Lyon, France.

3.
Introduction• The Web is being used by all kinds of people;• Web sites must be accessible;• Modern Web development transcends static HTML;• A more real evaluation is necessary. 9th International Cross-Disciplinary Conference on Web Accessibility – 3 16/17th April 2012 – Lyon, France.

4.
Previous work 9th International Cross-Disciplinary Conference on Web Accessibility –4 16/17th April 2012 – Lyon, France.

5.
QualWeb evaluator• Accessibility evaluation of Web pages: – using WCAG 2.0 , • 18 HTML techniques; – at different processing Before phases • BEFORE and AFTER Browser processing; After 9th International Cross-Disciplinary Conference on Web Accessibility – 5 16/17th April 2012 – Lyon, France.

6.
QualWeb evaluator: execution 9th International Cross-Disciplinary Conference on Web Accessibility – 6 16/17th April 2012 – Lyon, France.

7.
Some improvements... 9th International Cross-Disciplinary Conference on Web Accessibility –7 16/17th April 2012 – Lyon, France.

8.
PhantomJs• is a command-line tool that uses WebKit ,• works like a WebKit-based Web browser (simulation);• can be controlled using Javascript. 9th International Cross-Disciplinary Conference on Web Accessibility – 8 16/17th April 2012 – Lyon, France.

9.
Problems Solved!1. It avoids data injection at the browser level;2. Evaluated Web page before and after browser processing is exactly the same;3. Integration with a crawler. 9th International Cross-Disciplinary Conference on Web Accessibility – 9 16/17th April 2012 – Lyon, France.

10.
Research Question How do the macroscopic properties emerging from Web accessibility change in respect to the processing phase of delivery? 9th International Cross-Disciplinary Conference on Web Accessibility – 10 16/17th April 2012 – Lyon, France.

11.
Experimental Study• We evaluated a set of Web pages from a list provided by the Portuguese Web Archive (version of 2008);• We used the QualWeb evaluator. 9th International Cross-Disciplinary Conference on Web Accessibility – 11 16/17th April 2012 – Lyon, France.

12.
Metrics• Results of the evaluation are presented in terms of: – PASS, WARN, and FAIL (Applicable = PASS + WARN + FAIL)• Metrics used: – rate conservative = – rate optimistic = – rate strict =• The results are between accessible (100%) and not accessible (0%). 9th International Cross-Disciplinary Conference on Web Accessibility – 12 16/17th April 2012 – Lyon, France.

13.
Results• 24, 462 Web pages evaluated.• Total Number of HTML elements: – before processing - 24,918,720 – after processing - 41,967,072 – (Ratio ≈ 1.7).• Average Number of HTML elements per page: – before processing - 1010 – after processing – 1710 – (Ratio ≈1.7). 9th International Cross-Disciplinary Conference on Web Accessibility – 13 16/17th April 2012 – Lyon, France.

14.
The differences of an HTML document between bothProcessing Phases were observed. 9th International Cross-Disciplinary Conference on Web Accessibility – 14 16/17th April 2012 – Lyon, France.

15.
Results: Average Outcomes• Successes – before processing - 9 elements – after processing - 87 elements – (Ratio ≈ 9.7).• Failures – before processing - 46 elements – after processing – 176 elements – (Ratio ≈ 3.8).• Warnings – before processing - 262 elements – after processing – 451 elements – (Ratio ≈ 1.7). 9th International Cross-Disciplinary Conference on Web Accessibility – 15 16/17th April 2012 – Lyon, France.

16.
Results: rates Before and After processing• Conservative rate: – The average quality increases after processing. – Accessibility quality between 60% and 90% disappeared after processing.• Optimistic rate: – The average quality decreases after processing. – Some results lower than 20% disappeared after processing. – Decrease of higher accessibility quality, leading to a lower accessibility average. 9th International Cross-Disciplinary Conference on Web Accessibility – 16 16/17th April 2012 – Lyon, France.

17.
Results: rates Before and After processing• Strict rate: – The average increased after processing. – Results higher than 85% disappeared after processing. – Worse pages before processing get higher scores after processing. – Better pages before processing are ranked lower after processing. 9th International Cross-Disciplinary Conference on Web Accessibility – 17 16/17th April 2012 – Lyon, France.

18.
Limitations1. Techniques coverage2. Dynamic content3. Automatic evaluation 9th International Cross-Disciplinary Conference on Web Accessibility – 18 16/17th April 2012 – Lyon, France.

19.
Discussion• Evaluations before processing clearly is not the best option!• We used/percept/interact with the after processed version.• Considering the rates… – Web pages possess higher uniformity – That can be explained with reusable code 9th International Cross-Disciplinary Conference on Web Accessibility – 19 16/17th April 2012 – Lyon, France.

20.
Discussion• Impact on Designing Accessible Web Pages – Importance of sharing reusable code; – High quality reusable code produce better quality pages.• Impact on the Perception of Accessibility – Each metric identify different perspectives; – It is important that evaluations assess what user perceive. 9th International Cross-Disciplinary Conference on Web Accessibility – 20 16/17th April 2012 – Lyon, France.

21.
Conclusion• We presented a large-scale study of accessibility on the Web.• We were able to characterize some accessibility properties of the Web, pointing some differences between processing phases.• The results obtained on the evaluation of pages after browser processing tend to be more homogeneous than before.• Considering that the end-user interacts with the after processed pages then most studies about Web quality should be redone. 9th International Cross-Disciplinary Conference on Web Accessibility – 21 16/17th April 2012 – Lyon, France.

22.
Future Work1. Enlarge the coverage of WCAG 2.0 implemented tests2. Evaluate Rich Internet applications3. Perform a comparative set of studies 9th International Cross-Disciplinary Conference on Web Accessibility – 22 16/17th April 2012 – Lyon, France.

23.
nadiaf@di.fc.ul.pt 9th International Cross-Disciplinary Conference on Web Accessibility –23 16/17th April 2012 – Lyon, France.