Enriching Web Information Scent for Blind Users

928 views
859 views

Published on

Link annotation with the accessibility level of the target Web page is an adaptive navigation support technique aimed at increasing blind users’ orientation in Web sites. In this work, the accessibility level of a page is measured by exploiting data from evaluation reports produced by two automatic assessment tools. These tools support evaluation of accessibility and usability guideline-sets. As a result, links are annotated with a score that indicates the conformance of the target Web page to blind user accessibility and usability guidelines. A user test with 16 users was conducted in order to observe the strategies they followed when links were annotated with these scores. With annotated links, the navigation paradigm changed from sequential to browsing randomly through the subset of those links with high scores. Even if there was not a general agreement on the correspondence between scores and user perception of accessibility, users found annotations helpful when browsing through links related to a given topic.

Published in: Technology, Design
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
928
On SlideShare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
11
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • or added values
  • assumptions: all checkpoints, test cases have the same impact on the user
  • ACC: tags: summary for tables, attributes: alt for images and combination: formssome studies on: even if web pages are accessible they are not usableUSA: “click here” statements, shortcuts to skip to main content
  • there’s certainly an overlap
  • atomic rulesrecommendations: highly subjective issues, AT dependant issues. Violating them does not have a strong effect on accessibility but maybe on usability
  • LSP overcomes traditional aggregation methodstrong, medium, weakρ(d) values are obtained elsewhere –rho-INTUITIVE AND EXPLICIT WAY
  • Example to illustrate how it worksAll test cases have to be satisfied to meet the whole checkpointHowever the uncertainty on semi-automatic issues leads us to consider the medium quasidisjunctionamong earl:semi-automaticBetween checkpoints there also a sense of simultaneity to satisfy the whole guideline
  • capture ofget their impressions, free to comment .subjective data
  • as expected by blind users
  • why relevance was not considered: in this scenario it would lead to the solution
  • as expected by blind users
  • Even if they were not told to, in the after task forms users gave their opinion.changed their minds those that did first the task with annotated links in the first scenario. it was curious. surprisinglyTECHNIQUE PREVAILS OVER THE SCORES
  • At first sightEven if there’s a peak for mid values there’s not a general agreementUsefulness and perception of score validity is higher for the browsing task, at least it is more balanced
  • results are inconclusiveand how some user changed their minds about scores
  • Enriching Web Information Scent for Blind Users

    1. 1. Enriching Web Information Scent for Blind Users<br />Markel Vigo1, Barbara Leporini2, and Fabio Paternò2<br />1 Laboratory of HCI for Special Needs<br />2 Human Interfaces in Information Systems<br />1 University of the Basque Country<br />2 Italian National Research Council<br />
    2. 2. Hypothesis: annotating links with the accessibility level of the page where they point would increase user orientation <br /> “Visually impaired users need to be warned of obstacles because their reliance on cues is higher than for sighted users”- Goble et al.<br /> ”Detecting and notifying users about barriers improves user orientation”- Harper et al.<br />Goal: use of accessibility assessment results in web navigation scenarios<br />Information Scent:<br />Thus, we aim at enriching information scent using accessibility assessment results for screen reader users.<br />1. Introduction<br />1. Visual or textual cues provided on a Web site to suggest what information its links may contain.<br />2. The perceived usefulness of a page based on such information.<br />Enriching Web Information Scent for Blind Users<br />ACM ASSETS2009<br />
    3. 3. Hypothesis: annotating links with the accessibility level of the page that the link point to increases user orientation <br /> “Visually impaired users need to be warned of obstacles because their reliance on cues is higher than for sighted users”- Goble et al.<br /> ”Detecting and notifying users about barriers improves user orientation”- Harper et al.<br />Goal: deployment of accessibility assessment results in navigation scenarios<br />Information Scent:<br />Thus, we aim at enriching information scent using accessibility assessment results<br />1. Introduction<br />not accessible<br />highly accessible<br />1. Visual or textual cues provided on a Web site to suggest what information it or its links may contain.<br />2. The perceived usefulness of a page based on such information.<br />fairly accessible<br />Enriching Web Information Scent for Blind Users<br />ACM ASSETS2009<br />
    4. 4. …calls for <br />2. Challenges<br />Goal: annotation of links with accessibility assessment results in navigation scenarios.<br /><ul><li> automatic evaluation due to efficiency needs</li></ul> - based on guideline review tools<br /> - beaware of tool limitations<br /> - make assumptions, take risks<br /><ul><li> user-tailored assessment</li></ul> - current assessment techniques address all user groups<br /> - adaptive evaluation and measurement<br /> - quantitative scores for accuracy and discrimination power<br />Enriching Web Information Scent for Blind Users<br />ACM ASSETS2009<br />
    5. 5. 3. User-Tailored Assessment: Evaluation <br />Accessibility is measured in terms of conformance to web guidelines for blind users.<br /><ul><li>Web Accessibility: subset of WCAG 1.0
    6. 6. following the classification by Brajnik for the BW method</li></ul>- developed the Accessibility Checker for Blind users (ACB)<br />- ACB checks lack of tags, attributes and appropriate combination of them<br /><ul><li>Web Usability: Usability Guidelines for the Blind (UGB)
    7. 7. 4 principles: structure and arrangement, content appropriateness, multimodal output, consistency
    8. 8. automatic guideline review tool: Magenta
    9. 9. Magenta checks adequate content of tags and attributes, arrangement of headings or shortcuts</li></ul>Enriching Web Information Scent for Blind Users<br />ACM ASSETS2009<br />
    10. 10. 3. User-Tailored Assessment: Evaluation <br />Accessibility is measured in terms of conformance to web guidelines for blind users.<br />ACB<br />Accessibility report<br />Exclusively accessibility issues<br />Dependencies solver<br />www.foo.com<br />Exclusively usability issues<br />Usability<br />report<br />Magenta<br /><ul><li> Resolving guideline set conflicts/overlap
    11. 11. Addressed
    12. 12. Addressed but not implemented</li></ul>- Complementary<br />- Contradictory<br />Enriching Web Information Scent for Blind Users<br />ACM ASSETS2009<br />
    13. 13. 3. User-Tailored Assessment: Measurement <br />Metrics Calculation Component<br />ACB<br />Accessibility report<br />Exclusively accessibility issues<br />accesibility score<br />Dependencies solver<br />www.foo.com<br />Exclusively usability issues<br />Usability<br />report<br />Magenta<br /><ul><li> Calculate failure-rates for each test case, earl:TestCase
    14. 14. Using aggregation methods
    15. 15. Considering issue typology:
    16. 16. automatic issues (earl:automatic) yield earl:passed or earl:fail
    17. 17. recommendations
    18. 18. semi-automatic issues (earl:semiAuto)</li></ul>Enriching Web Information Scent for Blind Users<br />ACM ASSETS2009<br />
    19. 19. 3. User-Tailored Assessment: Measurement <br />Traditional aggregation:<br />where W: weights and E: evaluation results<br />Logic Scores Preferences:<br />where ρ(d) are values selected upon the required logical relationship between evaluation results.<br />d=0 conjunction <br />0&lt; d &lt;0.5 quasiconjunction: simultaneity in satisfying all the evaluations<br />d=0.5  arithmetic mean<br />0.5&lt; d &lt;1 quasidisjunction: penalizes only when all evaluations are not satisfied<br />d=1 disjunction<br />Only intermediate values are applied<br />Enriching Web Information Scent for Blind Users<br />ACM ASSETS2009<br />
    20. 20. 3. User-Tailored Assessment: Measurement <br /><ul><li> Example: a checkpoint implements 3 test cases</li></ul>T1earl:automatic= 1<br />T2earl:automatic= 0.25<br />T3earl:semi-automatic= 0<br />0.3<br />0.25<br /><ul><li>strongquasiconjuctionamong earl:automatic</li></ul>- medium quasidisjunctionamong earl:semi-automatic<br /><ul><li> weak quasiconjunctionis applied at checkpoint level
    21. 21. between guidelines, mean value</li></ul>final score<br />?<br />guidelines<br />0.25<br />checkpoints<br />test cases<br />T2<br />T3<br />T1<br />1<br />0.25<br />0<br />Enriching Web Information Scent for Blind Users<br />ACM ASSETS2009<br />
    22. 22. 4. User-testing <br /><ul><li> Preliminary evaluation for annotated links
    23. 23. Experimental settings
    24. 24. 16 experienced blind users. Age M=43, sd=11
    25. 25. JAWS on Internet Explorer
    26. 26. Remote usability testing</li></ul>- Log analysis of interaction sequence, timing, keyboard and mouse actions<br /> - Post-task forms and post-test questionnaire<br /><ul><li> Two tasks
    27. 27. Browsing by navigating
    28. 28. Searching by navigating</li></ul>Enriching Web Information Scent for Blind Users<br />ACM ASSETS2009<br />
    29. 29. 4. User-testing: browsing by navigating<br /><ul><li> Definition: casual/aimless browsing deciding at each step where to go next
    30. 30. Goal: observe users with no/vague target in mind
    31. 31. Two sites with 10 links
    32. 32. Top ten search results for “Pisa” and “Firenze” keywords
    33. 33. Results were heterogeneous links wrt topic,
    34. 34. Following a pattern: wikipedia, local university, soccer team and so on
    35. 35. One site was manually annotated with accessibility scores and relevance scores
    36. 36. Relevance based on ranking {very relevant, relevant, medium, low, irrelevant}</li></ul>- 5 min free browsing. They had to write a report about what they learned<br />Enriching Web Information Scent for Blind Users<br />ACM ASSETS2009<br />
    37. 37. 4. Results for browsing by navigating<br /><ul><li> In the page without annotations,
    38. 38. 9 users proceeded sequentially. Kendall τ=[0.8-1.0] at most p<0.03
    39. 39. In the page with annotations,
    40. 40. Only 2 users proceeded sequentially. Kendall τ=1.0 at most p<0.05
    41. 41. None followed the sequence of most accessible links
    42. 42. None followed the path based on relevance
    43. 43. Some proceeded dichotomously
    44. 44. However, when aggregating accessibility scores of visited pages, 7 points over the median are obtained
    45. 45. This, can be interpreted as if the users browsed within the subset of more accessible pages according to random/preference criteria</li></ul>Enriching Web Information Scent for Blind Users<br />ACM ASSETS2009<br />
    46. 46. 4. User-testing: searching by navigating<br /><ul><li> Definition: look for a target by sequentially deciding at each step where to go next
    47. 47. Goal: observe users with a specific target in mind
    48. 48. Two sites with 10 links
    49. 49. Top ten search results for “Accommodations in Pisa”
    50. 50. Results were homogeneous links wrt topic
    51. 51. One site was manually annotated with accessibility scores
    52. 52. Relevance was not considered
    53. 53. Two tasks: (1) given a telephone number (2) address
    54. 54. They had to write down the name of the hotel</li></ul>Enriching Web Information Scent for Blind Users<br />ACM ASSETS2009<br />
    55. 55. 4. Results for searching by navigating<br /><ul><li> In the page without annotations,
    56. 56. Only 2 users proceeded sequentially. Kendall τ=1.0 at most p<0.05
    57. 57. One user proceeded inversely. Kendal τ=-1.0 at most p<0.02
    58. 58. In the page with annotations,
    59. 59. Only 2 users proceeded sequentially. Kendall τ=1.0 at most p<0.05
    60. 60. One user followed the most accessible path. Kendal τ=1.0 at most p<0.02
    61. 61. Again, when aggregating accessibility scores of visited pages 6 points over the median are obtained</li></ul>Enriching Web Information Scent for Blind Users<br />ACM ASSETS2009<br />
    62. 62. 5. Results: gathered informal comments <br /><ul><li> 8 users appreciated accessibility scores in links
    63. 63. The suitability of scores was more controversial</li></ul> …some were satisfied: <br /><ul><li>“values adequately reflect accessibility level”
    64. 64. “scores are useful”, “scores are interesting”
    65. 65. “accessibility scores seem correct”
    66. 66. “navigation is better if scores are included”</li></ul> …while other were not: <br /><ul><li>“strange validation”
    67. 67. “scores are not very coherent”</li></ul> …other changed their minds: <br /><ul><li>“I’m doubtful about accessibility criteria” “links with accessibility scores are useful”
    68. 68. “scores don’t convey the actual difference in accessibility level” “scores make navigation smoother and more instinctive”
    69. 69. “scores seem random””I missed accessibility scores in this task” </li></ul>Enriching Web Information Scent for Blind Users<br />ACM ASSETS2009<br />
    70. 70. 5. Results: post-test questionnaire <br /><ul><li> 5 point Likert-scale {1: totally disagree – 5: totally agree}
    71. 71. “Scores are useful for the browsing task”
    72. 72. “Scores are useful for the searching task”
    73. 73. “Scores are correlated with actual accessibility level in browsing task”
    74. 74. “Scores are correlated with actual accessibility level in searching task”</li></ul>Enriching Web Information Scent for Blind Users<br />ACM ASSETS2009<br />
    75. 75. 6. Conclusions: scores <br /><ul><li> Most users state that scores are useful to a certain extent
    76. 76. Although there is not an agreement
    77. 77. “The perception of accessibility depends on each user and their particular computer settings”
    78. 78. There is no agreement on the type of scores they prefer
    79. 79. 50% for qualitative and quantitative
    80. 80. Considering the informal comments it seems that the annotation technique prevails over scores</li></ul>Enriching Web Information Scent for Blind Users<br />ACM ASSETS2009<br />
    81. 81. 6. Conclusions: annotation technique <br /><ul><li> In the searching scenario users do search within the subset of most accessible links
    82. 82. In the browsing scenario users change paradigm
    83. 83. From sequential browsing to random in the subset of most accessible links
    84. 84. Subjective scores are more balanced than in the searching scenario
    85. 85. When directly enquired, users state that accessibility annotations would be useful in those scenarios where the topic of the linked pages would be similar</li></ul>Enriching Web Information Scent for Blind Users<br />
    86. 86. 6. Conclusions: annotation technique <br /><ul><li> It seems that annotation technique would better fit in an scenario where:</li></ul> - They are browsing casually<br /> - and topics of linked pages are similar<br /><ul><li> For instance on the leaf nodes of a web directory</li></ul>Enriching Web Information Scent for Blind Users<br />
    87. 87. Enriching Web Information Scent for Blind Users<br />Questions?<br />Markel Vigo1, Barbara Leporini2, and Fabio Paternò2<br />1 Laboratory of HCI for Special Needs<br />2 Human Interfaces in Information Systems<br />1 University of the Basque Country<br />2 Italian National Research Council<br />

    ×