Universidad Politécnica de Madrid School of Computing José Luis Fuertes Ricardo González Emmanuelle Gutiérrez Loïc Martínez SIDAR Foundation International Cross-Disciplinary Conference on Web Accessibility Madrid, Spain April 20-21, 2009 Hera-FFX: a Firefox add-on for semi-automatic web accessibility evaluation

Contents Introduction Features of Web Accessibility Tools HERA HERA-FFX Design Overview Internal structure Results of analysis Evaluation process Conclusions Future Work

Introduction

Features of Web Accessibility Tools

HERA

HERA-FFX

Design

Overview

Internal structure

Results of analysis

Evaluation process

Conclusions

Future Work

Introduction Web accessibility evaluation Complex task, requiring Human expertise Software support Human evaluators need Sound knowledge Experience in web development WCAG proficiency Software tools should Provide support for manual evaluation Automate as much of the work as possible

Web accessibility evaluation

Complex task, requiring

Human expertise

Software support

Human evaluators need

Sound knowledge

Experience in web development

WCAG proficiency

Software tools should

Provide support for manual evaluation

Automate as much of the work as possible

Features of web accessibility tools Key features of a web accessibility evaluation tool Automated preliminary evaluation ( AE ) Support for manual entry of checkpoint results ( ME ) Modification of page rendering to assist checkpoint evaluation ( PM ) Annotated code view to assist checkpoint evaluation ( CV ) Evaluation of local pages ( LP ) Evaluation of restricted-access pages ( RA ) Evaluation of rendered pages ( RP ) Report generation ( RG ) Support for training ( ST ) Multi-session capacity ( MS ) Flexibility to integrate other accessibility guidelines ( FL )

Key features of a web accessibility evaluation tool

Automated preliminary evaluation ( AE )

Support for manual entry of checkpoint results ( ME )

Modification of page rendering to assist checkpoint evaluation ( PM )

Annotated code view to assist checkpoint evaluation ( CV )

Evaluation of local pages ( LP )

Evaluation of restricted-access pages ( RA )

Evaluation of rendered pages ( RP )

Report generation ( RG )

Support for training ( ST )

Multi-session capacity ( MS )

Flexibility to integrate other accessibility guidelines ( FL )

Features of web accessibility tools Comparison of free web accessibility evaluation tools Tool Type AE MF LP CV LP RA RP RG ST MS FL A-Checker online     Access. Check online   EvalAccess online    Foxability extension       Functional Access. Evaluator online    Hera online        Cynthia Says online   Firefox Access. Extension extension      TAW online online   TAW standalone application         Torquemada online   Total Validator online   WAVE online    WAVE Firefox toolbar extension      Web Access. Inspector application    Web Access. Self-evaluation report     

Comparison of free web accessibility evaluation tools

Features of web accessibility tools Results All the desirable features are covered by at least one tool No one tool has all the desirable features There is a need for a web accessibility evaluation tool that has Strong support for manual evaluation Ability to evaluate all types of web pages Flexibility to incorporate new accessibility guidelines

Results

All the desirable features are covered by at least one tool

No one tool has all the desirable features

There is a need for a web accessibility evaluation tool that has

Strong support for manual evaluation

Ability to evaluate all types of web pages

Flexibility to incorporate new accessibility guidelines

HERA Free on-line tool from 2003-2005 Developed by the Sidar Foundation Support for the semi-manual evaluation of Web page accessibility HERA “ H ojas de E stilo para la R evisión de la A ccesibilidad” “ Accessibility Evaluation Style Sheets” Generates new versions of the analysed page, highlighting interesting elements Strengths: Minimisation of the need to inspect source code to directly evaluate checkpoints in the original rendering Positive feedback received from users Valuable tool for educational purposes

Free on-line tool from 2003-2005

Developed by the Sidar Foundation

Support for the semi-manual evaluation of Web page accessibility

HERA

“ H ojas de E stilo para la R evisión de la A ccesibilidad”

“ Accessibility Evaluation Style Sheets”

Generates new versions of the analysed page, highlighting interesting elements

Strengths:

Minimisation of the need

to inspect source code

to directly evaluate checkpoints in the original rendering

Positive feedback received from users

Valuable tool for educational purposes

HERA-FFX Hera-FFX Design Main goals : Be as usable as Hera Be flexible enough to incorporate additional requirements and new standards Prototype decisions : Firefox add-on WCAG 1.0 Spanish

Hera-FFX Design

Main goals :

Be as usable as Hera

Be flexible enough to incorporate additional requirements and new standards

Prototype decisions :

Firefox add-on

WCAG 1.0

Spanish

HERA-FFX Hera-FFX overview Stores guideline definition, checkpoints and tests in an XML-based configuration file Firefox loads and interprets Web pages Builds a DOM representation of the web page. It is used by Hera-FFX to perform the accessibility evaluation Performs a preliminary automatic accessibility evaluation User performs a manual accessibility evaluation Accessibility report: XHTML + CSS document containing detailed information about the status of the evaluation Saved evaluation: this XML file can divide evaluations into several sessions

Hera-FFX overview

HERA-FFX Internal structure for storing the required information Evaluation Guideline Checkpoint Test Element DOM-Element

Internal structure for storing the required information

HERA-FFX Test analysis results Hera-FFX inspects the Web page and automatically assigns a value for each test Pass : the web page passes the test Fail : the web page fails the test Not applicable : the test is not applicable Verify : the tool cannot decide what the result is and the user is asked to run the test manually The user can manually assign two more values for each test Partial : the web page fails the test on a minor point Unknown : the evaluator cannot decide what the result of the test should be

Test analysis results

Hera-FFX inspects the Web page and automatically assigns a value for each test

Pass : the web page passes the test

Fail : the web page fails the test

Not applicable : the test is not applicable

Verify : the tool cannot decide what the result is and the user is asked to run the test manually

The user can manually assign two more values for each test

Partial : the web page fails the test on a minor point

Unknown : the evaluator cannot decide what the result of the test should be

HERA-FFX Results of checkpoint analysis verify  i / eval i = verify   i ≠ j (eval j ≠ fail  eval j ≠ partial) Condition Result  i / eval i = fail fail  i / eval i = partial   i ≠ j ( eval j ≠ fail ) partial  i / eval i = pass   i ≠ j (eval j = pass  eval j = N/A ) pass  i / eval i = N/A not applicable  i / eval i = unknown   i ≠ j ( eval j = unknown  eval j = N/A) unknown

Results of checkpoint analysis

HERA-FFX Detailed Evaluation Process Current web page Manual Evaluation Support Help on Checkpoints Instructions for Evaluation Modified Page View Code View Evaluation Results Result Summary Automated Preliminary Analysis Report Generation <hmtl lang=&quot;es&quot;> <head> <title>HERA </title> </head> <body>

Detailed Evaluation Process

HERA-FFX Automated Preliminary Analysis The browser loads a page Hera-FFX applies the relevant tests to evaluate each DOM-element An icon on the Firefox status bar shows the progress and the final aggregated result Users can get more details as a tool tip

Automated Preliminary Analysis

The browser loads a page

Hera-FFX applies the relevant tests to evaluate each DOM-element

An icon on the Firefox status bar shows the progress and the final aggregated result

Users can get more details as a tool tip

Summary of Results HERA-FFX User can choose the navigation style: Table-based Guideline-based General information URL, configuration, date, time, number of errors, number of manual checks needed, browser…

Summary of Results

Table-based

Guideline-based

General information

URL, configuration, date, time, number of errors, number of manual checks needed, browser…

HERA-FFX Manual Evaluation Support User interface designed to evaluate point by point according to the selected navigation style Interface control: keyboard and mouse Colour contrast Colours and icons Interface elements: relative units

Manual Evaluation Support

User interface designed to evaluate point by point according to the selected navigation style

Interface control: keyboard and mouse

Colour contrast

Colours and icons

Interface elements: relative units

HERA-FFX Manual Evaluation Support Simulated view of the modified page Indicates which elements have to be inspected Element fails Element should be evaluated

Manual Evaluation Support

Simulated view of the modified page

Indicates which elements have to be inspected

Element fails

Element should be evaluated

HERA-FFX Manual Evaluation Support Generated HTML code view Indicates which elements have to be inspected Element fails Element should be evaluated

Manual Evaluation Support

Generated HTML code view

Indicates which elements have to be inspected

Element fails

Element should be evaluated

HERA-FFX Manual Evaluation Support Visualization of the result of a call to an external service W3C HTML validation service

Manual Evaluation Support

Visualization of the result of a call to an external service

W3C HTML validation service

Conclusions Both the Sidar Foundation and the Technical University of Madrid are using Hera in their Web accessibility courses with very encouraging results Hera-FFX is a Firefox add-on designed to perform a semi-automatic evaluation of web accessibility Automatic preliminary evaluation Provides guidance for manual evaluation Includes the desirable features of a web accessibility evaluation tool

Both the Sidar Foundation and the Technical University of Madrid are using Hera in their Web accessibility courses with very encouraging results

Hera-FFX is a Firefox add-on designed to perform a semi-automatic evaluation of web accessibility

Automatic preliminary evaluation

Provides guidance for manual evaluation

Includes the desirable features of a web accessibility evaluation tool

Conclusions Hera-FFX – Hera comparison            Hera-FFX Feature Hera Automated preliminary evaluation  Support for manual entry  Modification of page presentation  Annotated code view  Evaluation of local pages Evaluation or restricted-access pages Evaluation of rendered pages Report generation  Support for training  Multi-session capacity  Flexibility

Hera-FFX – Hera comparison

Conclusions Hera-FFX – Hera comparison Increased efficiency Each manual evaluation is automatically updated Same effectiveness No relevant differences Increased user satisfaction Users can evaluate pages that they were unable to with Hera Same completeness 65 WCAG 1.0 checkpoints Slightly increased correctness Hera-FFX inspects the rendered-page, which is closer to user experience

Hera-FFX – Hera comparison

Increased efficiency

Each manual evaluation is automatically updated

Same effectiveness

No relevant differences

Increased user satisfaction

Users can evaluate pages that they were unable to with Hera

Same completeness

65 WCAG 1.0 checkpoints

Slightly increased correctness

Hera-FFX inspects the rendered-page, which is closer to user experience

Future Work Report generation in different formats PDF, EARL… Internationalization of the add-on Update to cover the Spanish web content accessibility standard (UNE 139803:2004) Update to cover WCAG 2.0 Already started Major structural changes are needed To cover principles, guidelines, success criteria, techniques and failures To handle WCAG 2.0 flexibility and choose between multiple techniques to satisfy a criteria Consideration of “situations” (the evaluator will have to decide which situation applies to each element) Boolean technique expressions

Report generation in different formats

PDF, EARL…

Internationalization of the add-on

Update to cover the Spanish web content accessibility standard (UNE 139803:2004)

Update to cover WCAG 2.0

Already started

Major structural changes are needed

To cover principles, guidelines, success criteria, techniques and failures

To handle WCAG 2.0 flexibility and choose between multiple techniques to satisfy a criteria

Consideration of “situations” (the evaluator will have to decide which situation applies to each element)

Boolean technique expressions

Future Work Develop a comprehensive stand-alone evaluation tool including workgroup tools Project creation and management Evaluation work distribution Aggregation of results from different evaluators Assessment of evaluator reliability …

Develop a comprehensive stand-alone evaluation tool including workgroup tools

Project creation and management

Evaluation work distribution

Aggregation of results from different evaluators

Assessment of evaluator reliability

…

Universidad Politécnica de Madrid School of Computing José Luis Fuertes Ricardo González Emmanuelle Gutiérrez Loïc Martínez SIDAR Foundation International Cross-Disciplinary Conference on Web Accessibility Madrid, Spain April 20-21, 2009 Hera-FFX: a Firefox add-on for semi-automatic web accessibility evaluation