Quantitative Evaluation for Web Accessibility with Respect to Disabled Groups
The Web Accessibility Initiative (WAI) proposed Web Content Accessibility Guidelines (WCAG) in order to describe qualifications of accessible web pages.
The level conformance evaluation of WCAG (Levels A, AA and AAA) cannot indicate accessibility for specific disabled groups.
In this paper, we propose a methodology for evaluating web accessibility with respect to disabled users, especially the vision-impaired and the hearing-impaired.
Related Work (1)
Automatic tools for qualitative evaluation such as ATRC, A-Prompt,
WAVE4.0, EvalAccess2.0 and Tidy
Sullivan et al., 2000
Related Work (2)
Zeng et al., 2004
Web Accessibility Benchmarking Cluster introduce UWEM 0.5 (Unified Web Evaluation Methodology 0.5), 2005
Related Work (3)
Web Accessibility Benchmarking Cluster introduce UWEM 1.0 (Unified Web Evaluation Methodology 1.0), 2006
Related Work (4)
Bühler et al., 2006
where Nb is the number of potential barriers; Bb is the number of failures for one barrier type; B is the number of all failures.
Related Work (5)
Brajnik et al., 2006
They propose the Barrier Walkthrough method (BW), adapted from the heuristic walkthrough method which is used for usability investigation.
A barrier is derived from the usage scenario of users.
In this work, each barrier is grouped by user category, and WCAG checkpoints are listed as its causes.
But severity evaluation for barriers in this method is depended upon the expertise of assessors
An Improved Approach (1)
We adopt the BW to find possible barriers of assessed web pages and further analyze severity from related checkpoints of the barriers , which is based on two assumptions
The first assumption
The barriers which relate to several checkpoints should have severity values greater than those which relate to few checkpoints.
The second assumption
The barriers which relate to Priority 1 checkpoints should have severity values greater than those which relate to Priority 2 and Priority 3 checkpoints.
An Improved Approach (2)
Our approach has two stages
The first stage :
We adopt the BW to find possible barriers of assessed web pages.
All barriers will be transformed into their related checkpoints for each disabled group.
An Improved Approach (3)
Our approach has two stages
The second stage :
All violated checkpoints will be calculated to find severity value from our formula, T1
Our formula is derived from WAB_SCORE
An Improved Approach (4) where is number of potential barrier types on a web page is number of actual barriers for barrier type i is number of potential barriers for barrier type i is number of checkpoints related to priority level p and barrier type i.
An Improved Approach (5)
In addition, the result of the formula is normalized in the range of [0, 1] by tuning weight used in the calculation.
By following WAB_SCORE, Priority 1 is twice as important as Priority 2 and three times as important as Priority 3 (W2 = W1/2 and W3 = W1/3)
W1 + W2+ W3 = 1
W1 +(1/2) W1+ (1/3)W1 = 1
Therefore W1= 6/11; W2= 3/11 and W3= 2/11
In our experiment, we used web pages on ".com", ".gov" and ".edu" domains.
All pages had "Level A conformance icon" WAI logos on the pages.
To acquire these pages, we used “Level A conformance icon, W3C-WAI Web Content Accessibility Guidelines 1.0” as search keywords using the Google search
Model Evaluation Comparison of values from T1 and A3 from all data
Results and Discussion
The pages which comply with Level A satisfy only 11 checkpoints out of 37 for the vision-impaired.
They omit 26 checkpoints of Level AA and Level AAA, which may be the cause of 15 types of possible barriers for that group.
From the experimental results, although all pages claim they conform to LevelA,
only 64.41% of those actually conform when re-checked by EvalAccess 2.0, while 19.46% of these are suitable for the vision-impaired.
Our evaluation methodology can help complement the level conformance evaluation offered by WCAG.
From our study, we found that Level A conformance pages may be not suitable for some groups of users such as the vision-impaired
Violated checkpoints are computed to be a single value representing the accessibility of a web page.
The correlation of results from our formula, T1 and baseline, A3 is rather strong.
The strong correlation shows quality of our approach that it can indicate accessibility for the specific disabled group.