Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Enriching Web Information Scent for Blind Users<br />Markel Vigo1, Barbara Leporini2, and Fabio Paternò2<br />1 Laboratory...
Hypothesis: annotating links with the accessibility level of the page where they point would increase user orientation 	<b...
Hypothesis: annotating links with the accessibility level of the page that the link point to increases user orientation 	<...
…calls for <br />2. Challenges<br />Goal: annotation of links with accessibility assessment results in navigation scenario...
3. User-Tailored Assessment: Evaluation	<br />Accessibility is measured in terms of conformance to web guidelines for blin...
 following the classification by Brajnik for the BW method</li></ul>- developed the Accessibility Checker for Blind users ...
 4 principles: structure and arrangement, content appropriateness, multimodal output, consistency
 automatic guideline review tool: Magenta
Magenta checks adequate content of tags and attributes, arrangement of headings or shortcuts</li></ul>Enriching Web Inform...
3. User-Tailored Assessment: Evaluation	<br />Accessibility is measured in terms of conformance to web guidelines for blin...
 Addressed
 Addressed but not implemented</li></ul>- Complementary<br />- Contradictory<br />Enriching Web Information Scent for Blin...
3. User-Tailored Assessment: Measurement	<br />Metrics Calculation Component<br />ACB<br />Accessibility report<br />Exclu...
 Using aggregation methods
 Considering issue typology:
automatic issues (earl:automatic) yield earl:passed or earl:fail
 recommendations
 semi-automatic issues (earl:semiAuto)</li></ul>Enriching Web Information Scent for Blind Users<br />ACM ASSETS2009<br />
3. User-Tailored Assessment: Measurement	<br />Traditional aggregation:<br />where W: weights and E: evaluation results<br...
3. User-Tailored Assessment: Measurement	<br /><ul><li> Example: a checkpoint implements 3 test cases</li></ul>T1earl:auto...
Upcoming SlideShare
Loading in …5
×

Enriching Web Information Scent for Blind Users

1,198 views

Published on

Link annotation with the accessibility level of the target Web page is an adaptive navigation support technique aimed at increasing blind users’ orientation in Web sites. In this work, the accessibility level of a page is measured by exploiting data from evaluation reports produced by two automatic assessment tools. These tools support evaluation of accessibility and usability guideline-sets. As a result, links are annotated with a score that indicates the conformance of the target Web page to blind user accessibility and usability guidelines. A user test with 16 users was conducted in order to observe the strategies they followed when links were annotated with these scores. With annotated links, the navigation paradigm changed from sequential to browsing randomly through the subset of those links with high scores. Even if there was not a general agreement on the correspondence between scores and user perception of accessibility, users found annotations helpful when browsing through links related to a given topic.

Published in: Technology, Design
  • Be the first to comment

  • Be the first to like this

Enriching Web Information Scent for Blind Users

  1. 1. Enriching Web Information Scent for Blind Users<br />Markel Vigo1, Barbara Leporini2, and Fabio Paternò2<br />1 Laboratory of HCI for Special Needs<br />2 Human Interfaces in Information Systems<br />1 University of the Basque Country<br />2 Italian National Research Council<br />
  2. 2. Hypothesis: annotating links with the accessibility level of the page where they point would increase user orientation <br /> “Visually impaired users need to be warned of obstacles because their reliance on cues is higher than for sighted users”- Goble et al.<br /> ”Detecting and notifying users about barriers improves user orientation”- Harper et al.<br />Goal: use of accessibility assessment results in web navigation scenarios<br />Information Scent:<br />Thus, we aim at enriching information scent using accessibility assessment results for screen reader users.<br />1. Introduction<br />1. Visual or textual cues provided on a Web site to suggest what information its links may contain.<br />2. The perceived usefulness of a page based on such information.<br />Enriching Web Information Scent for Blind Users<br />ACM ASSETS2009<br />
  3. 3. Hypothesis: annotating links with the accessibility level of the page that the link point to increases user orientation <br /> “Visually impaired users need to be warned of obstacles because their reliance on cues is higher than for sighted users”- Goble et al.<br /> ”Detecting and notifying users about barriers improves user orientation”- Harper et al.<br />Goal: deployment of accessibility assessment results in navigation scenarios<br />Information Scent:<br />Thus, we aim at enriching information scent using accessibility assessment results<br />1. Introduction<br />not accessible<br />highly accessible<br />1. Visual or textual cues provided on a Web site to suggest what information it or its links may contain.<br />2. The perceived usefulness of a page based on such information.<br />fairly accessible<br />Enriching Web Information Scent for Blind Users<br />ACM ASSETS2009<br />
  4. 4. …calls for <br />2. Challenges<br />Goal: annotation of links with accessibility assessment results in navigation scenarios.<br /><ul><li> automatic evaluation due to efficiency needs</li></ul> - based on guideline review tools<br /> - beaware of tool limitations<br /> - make assumptions, take risks<br /><ul><li> user-tailored assessment</li></ul> - current assessment techniques address all user groups<br /> - adaptive evaluation and measurement<br /> - quantitative scores for accuracy and discrimination power<br />Enriching Web Information Scent for Blind Users<br />ACM ASSETS2009<br />
  5. 5. 3. User-Tailored Assessment: Evaluation <br />Accessibility is measured in terms of conformance to web guidelines for blind users.<br /><ul><li>Web Accessibility: subset of WCAG 1.0
  6. 6. following the classification by Brajnik for the BW method</li></ul>- developed the Accessibility Checker for Blind users (ACB)<br />- ACB checks lack of tags, attributes and appropriate combination of them<br /><ul><li>Web Usability: Usability Guidelines for the Blind (UGB)
  7. 7. 4 principles: structure and arrangement, content appropriateness, multimodal output, consistency
  8. 8. automatic guideline review tool: Magenta
  9. 9. Magenta checks adequate content of tags and attributes, arrangement of headings or shortcuts</li></ul>Enriching Web Information Scent for Blind Users<br />ACM ASSETS2009<br />
  10. 10. 3. User-Tailored Assessment: Evaluation <br />Accessibility is measured in terms of conformance to web guidelines for blind users.<br />ACB<br />Accessibility report<br />Exclusively accessibility issues<br />Dependencies solver<br />www.foo.com<br />Exclusively usability issues<br />Usability<br />report<br />Magenta<br /><ul><li> Resolving guideline set conflicts/overlap
  11. 11. Addressed
  12. 12. Addressed but not implemented</li></ul>- Complementary<br />- Contradictory<br />Enriching Web Information Scent for Blind Users<br />ACM ASSETS2009<br />
  13. 13. 3. User-Tailored Assessment: Measurement <br />Metrics Calculation Component<br />ACB<br />Accessibility report<br />Exclusively accessibility issues<br />accesibility score<br />Dependencies solver<br />www.foo.com<br />Exclusively usability issues<br />Usability<br />report<br />Magenta<br /><ul><li> Calculate failure-rates for each test case, earl:TestCase
  14. 14. Using aggregation methods
  15. 15. Considering issue typology:
  16. 16. automatic issues (earl:automatic) yield earl:passed or earl:fail
  17. 17. recommendations
  18. 18. semi-automatic issues (earl:semiAuto)</li></ul>Enriching Web Information Scent for Blind Users<br />ACM ASSETS2009<br />
  19. 19. 3. User-Tailored Assessment: Measurement <br />Traditional aggregation:<br />where W: weights and E: evaluation results<br />Logic Scores Preferences:<br />where ρ(d) are values selected upon the required logical relationship between evaluation results.<br />d=0 conjunction <br />0&lt; d &lt;0.5 quasiconjunction: simultaneity in satisfying all the evaluations<br />d=0.5  arithmetic mean<br />0.5&lt; d &lt;1 quasidisjunction: penalizes only when all evaluations are not satisfied<br />d=1 disjunction<br />Only intermediate values are applied<br />Enriching Web Information Scent for Blind Users<br />ACM ASSETS2009<br />
  20. 20. 3. User-Tailored Assessment: Measurement <br /><ul><li> Example: a checkpoint implements 3 test cases</li></ul>T1earl:automatic= 1<br />T2earl:automatic= 0.25<br />T3earl:semi-automatic= 0<br />0.3<br />0.25<br /><ul><li>strongquasiconjuctionamong earl:automatic</li></ul>- medium quasidisjunctionamong earl:semi-automatic<br /><ul><li> weak quasiconjunctionis applied at checkpoint level
  21. 21. between guidelines, mean value</li></ul>final score<br />?<br />guidelines<br />0.25<br />checkpoints<br />test cases<br />T2<br />T3<br />T1<br />1<br />0.25<br />0<br />Enriching Web Information Scent for Blind Users<br />ACM ASSETS2009<br />
  22. 22. 4. User-testing <br /><ul><li> Preliminary evaluation for annotated links
  23. 23. Experimental settings
  24. 24. 16 experienced blind users. Age M=43, sd=11
  25. 25. JAWS on Internet Explorer
  26. 26. Remote usability testing</li></ul>- Log analysis of interaction sequence, timing, keyboard and mouse actions<br /> - Post-task forms and post-test questionnaire<br /><ul><li> Two tasks
  27. 27. Browsing by navigating
  28. 28. Searching by navigating</li></ul>Enriching Web Information Scent for Blind Users<br />ACM ASSETS2009<br />
  29. 29. 4. User-testing: browsing by navigating<br /><ul><li> Definition: casual/aimless browsing deciding at each step where to go next
  30. 30. Goal: observe users with no/vague target in mind
  31. 31. Two sites with 10 links
  32. 32. Top ten search results for “Pisa” and “Firenze” keywords
  33. 33. Results were heterogeneous links wrt topic,
  34. 34. Following a pattern: wikipedia, local university, soccer team and so on
  35. 35. One site was manually annotated with accessibility scores and relevance scores
  36. 36. Relevance based on ranking {very relevant, relevant, medium, low, irrelevant}</li></ul>- 5 min free browsing. They had to write a report about what they learned<br />Enriching Web Information Scent for Blind Users<br />ACM ASSETS2009<br />
  37. 37. 4. Results for browsing by navigating<br /><ul><li> In the page without annotations,
  38. 38. 9 users proceeded sequentially. Kendall τ=[0.8-1.0] at most p<0.03
  39. 39. In the page with annotations,
  40. 40. Only 2 users proceeded sequentially. Kendall τ=1.0 at most p<0.05
  41. 41. None followed the sequence of most accessible links
  42. 42. None followed the path based on relevance
  43. 43. Some proceeded dichotomously
  44. 44. However, when aggregating accessibility scores of visited pages, 7 points over the median are obtained
  45. 45. This, can be interpreted as if the users browsed within the subset of more accessible pages according to random/preference criteria</li></ul>Enriching Web Information Scent for Blind Users<br />ACM ASSETS2009<br />
  46. 46. 4. User-testing: searching by navigating<br /><ul><li> Definition: look for a target by sequentially deciding at each step where to go next
  47. 47. Goal: observe users with a specific target in mind
  48. 48. Two sites with 10 links
  49. 49. Top ten search results for “Accommodations in Pisa”
  50. 50. Results were homogeneous links wrt topic
  51. 51. One site was manually annotated with accessibility scores
  52. 52. Relevance was not considered
  53. 53. Two tasks: (1) given a telephone number (2) address
  54. 54. They had to write down the name of the hotel</li></ul>Enriching Web Information Scent for Blind Users<br />ACM ASSETS2009<br />
  55. 55. 4. Results for searching by navigating<br /><ul><li> In the page without annotations,
  56. 56. Only 2 users proceeded sequentially. Kendall τ=1.0 at most p<0.05
  57. 57. One user proceeded inversely. Kendal τ=-1.0 at most p<0.02
  58. 58. In the page with annotations,
  59. 59. Only 2 users proceeded sequentially. Kendall τ=1.0 at most p<0.05
  60. 60. One user followed the most accessible path. Kendal τ=1.0 at most p<0.02
  61. 61. Again, when aggregating accessibility scores of visited pages 6 points over the median are obtained</li></ul>Enriching Web Information Scent for Blind Users<br />ACM ASSETS2009<br />
  62. 62. 5. Results: gathered informal comments <br /><ul><li> 8 users appreciated accessibility scores in links
  63. 63. The suitability of scores was more controversial</li></ul> …some were satisfied: <br /><ul><li>“values adequately reflect accessibility level”
  64. 64. “scores are useful”, “scores are interesting”
  65. 65. “accessibility scores seem correct”
  66. 66. “navigation is better if scores are included”</li></ul> …while other were not: <br /><ul><li>“strange validation”
  67. 67. “scores are not very coherent”</li></ul> …other changed their minds: <br /><ul><li>“I’m doubtful about accessibility criteria” “links with accessibility scores are useful”
  68. 68. “scores don’t convey the actual difference in accessibility level” “scores make navigation smoother and more instinctive”
  69. 69. “scores seem random””I missed accessibility scores in this task” </li></ul>Enriching Web Information Scent for Blind Users<br />ACM ASSETS2009<br />
  70. 70. 5. Results: post-test questionnaire <br /><ul><li> 5 point Likert-scale {1: totally disagree – 5: totally agree}
  71. 71. “Scores are useful for the browsing task”
  72. 72. “Scores are useful for the searching task”
  73. 73. “Scores are correlated with actual accessibility level in browsing task”
  74. 74. “Scores are correlated with actual accessibility level in searching task”</li></ul>Enriching Web Information Scent for Blind Users<br />ACM ASSETS2009<br />
  75. 75. 6. Conclusions: scores <br /><ul><li> Most users state that scores are useful to a certain extent
  76. 76. Although there is not an agreement
  77. 77. “The perception of accessibility depends on each user and their particular computer settings”
  78. 78. There is no agreement on the type of scores they prefer
  79. 79. 50% for qualitative and quantitative
  80. 80. Considering the informal comments it seems that the annotation technique prevails over scores</li></ul>Enriching Web Information Scent for Blind Users<br />ACM ASSETS2009<br />
  81. 81. 6. Conclusions: annotation technique <br /><ul><li> In the searching scenario users do search within the subset of most accessible links
  82. 82. In the browsing scenario users change paradigm
  83. 83. From sequential browsing to random in the subset of most accessible links
  84. 84. Subjective scores are more balanced than in the searching scenario
  85. 85. When directly enquired, users state that accessibility annotations would be useful in those scenarios where the topic of the linked pages would be similar</li></ul>Enriching Web Information Scent for Blind Users<br />
  86. 86. 6. Conclusions: annotation technique <br /><ul><li> It seems that annotation technique would better fit in an scenario where:</li></ul> - They are browsing casually<br /> - and topics of linked pages are similar<br /><ul><li> For instance on the leaf nodes of a web directory</li></ul>Enriching Web Information Scent for Blind Users<br />
  87. 87. Enriching Web Information Scent for Blind Users<br />Questions?<br />Markel Vigo1, Barbara Leporini2, and Fabio Paternò2<br />1 Laboratory of HCI for Special Needs<br />2 Human Interfaces in Information Systems<br />1 University of the Basque Country<br />2 Italian National Research Council<br />

×