Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Are Users the Gold Standard for Accessibility Evaluation?

468 views

Published on

User testing is considered a key part of web accessibility evaluation. However, little is known about how effective is for identifying accessibility problems. Our experience, informed by a series of studies with blind users, corroborates that a website with a significant number of guideline violations can be perceived as accessible, and on the contrary, some participants may not perceive a highly accessible website as accessible. Accessibility guidelines are often criticised by their partial coverage and questionable validity. However, we should be very careful about making categorical statements in this regard as there are a number of variables that may introduce biases in user tests. We identify sources of bias related to user expertise, the experimental setting, employed language and reporting that, if not adequately controlled, may influence on the validity and reliability of the evaluation results. We discuss the limitations and practical implications of user testing with blind users for web accessibility evaluation.

Published in: Science, Design, Technology
  • Be the first to comment

  • Be the first to like this

Are Users the Gold Standard for Accessibility Evaluation?

  1. 1. Are Users the Gold Standard for Accessibility Evaluation?   7th April - W4A’14 Amaia Aizpurua1, Myriam Arrue Simon Harper2, Markel Vigo3 U. of the Basque Country University of Manchester http://dx.doi.org/10.1145/2596695.2596705 1:  @amaiaaizpurua     2:  @sharpic   3:  @markelvigo  
  2. 2. Motivation •  User testing for accessibility evaluation •  Encouraged by the community •  No established procedures – What? – When? – How? 27th April - W4A’14 Are Users the Gold Standard for Accessibility Evaluation?
  3. 3. In practice •  Evaluating web accessibility for blind users – User testing method •  Effectiveness in identifying accessibility problems [Mankoff et al. 2005] – Accessibility guidelines •  Partial coverage of user problems [Power et al. 2012] 37th April - W4A’14 Are Users the Gold Standard for Accessibility Evaluation?
  4. 4. Study results •  Accessibility: conformance vs. perceived Website AA conformance level Perceived 47th April - W4A’14 Are Users the Gold Standard for Accessibility Evaluation? Median   Mode   SD   6 7 1.95 6 6 1.42 2 1 1.62 6 6 1.95 1: very inaccessible 7: very accessible Sa+sfied  SC   Non-­‐sa+sfied  SC   73% 27% 69% 31% 52% 48% 36% 64% !
  5. 5. Why this mismatch? •  Guidelines vs. user problems •  Eliciting users’ problems – Experience/report problems differently: web expertise – Interaction context, tasks type •  Identifying accessibility issues – Combination of causes, contextual information – Under the influence of the evaluator effect 57th April - W4A’14 Are Users the Gold Standard for Accessibility Evaluation?
  6. 6. Discussion •  Inclusive participatory evaluation – Following the considerations of co-operative evaluation – Evaluator and user evaluating a website collaboratively – Instead of interpreting data afterwards, work on problems during the session •  Improve effectiveness 67th April - W4A’14 Are Users the Gold Standard for Accessibility Evaluation?
  7. 7. Conclusions •  User testing may not be the gold standard for web accessibility evaluation •  Users are a gold mine •  Co-operative accessibility evaluation •  Bridge the gap? 77th April - W4A’14 Are Users the Gold Standard for Accessibility Evaluation?
  8. 8. Thank you! 87th April - W4A’14 Contact amaia.aizpurua@ehu.es | @amaiaaizpurua ! myriam.arrue@ehu.es ! simon.harper@manchester.ac.uk | @sharpic! markel.vigo@manchester.ac.uk | @markelvigo! !

×