1.
Enriching Web Information Scent for Blind Users<br />Markel Vigo1, Barbara Leporini2, and Fabio Paternò2<br />1 Laboratory of HCI for Special Needs<br />2 Human Interfaces in Information Systems<br />1 University of the Basque Country<br />2 Italian National Research Council<br />

2.
Hypothesis: annotating links with the accessibility level of the page where they point would increase user orientation <br /> “Visually impaired users need to be warned of obstacles because their reliance on cues is higher than for sighted users”- Goble et al.<br /> ”Detecting and notifying users about barriers improves user orientation”- Harper et al.<br />Goal: use of accessibility assessment results in web navigation scenarios<br />Information Scent:<br />Thus, we aim at enriching information scent using accessibility assessment results for screen reader users.<br />1. Introduction<br />1. Visual or textual cues provided on a Web site to suggest what information its links may contain.<br />2. The perceived usefulness of a page based on such information.<br />Enriching Web Information Scent for Blind Users<br />ACM ASSETS2009<br />

3.
Hypothesis: annotating links with the accessibility level of the page that the link point to increases user orientation <br /> “Visually impaired users need to be warned of obstacles because their reliance on cues is higher than for sighted users”- Goble et al.<br /> ”Detecting and notifying users about barriers improves user orientation”- Harper et al.<br />Goal: deployment of accessibility assessment results in navigation scenarios<br />Information Scent:<br />Thus, we aim at enriching information scent using accessibility assessment results<br />1. Introduction<br />not accessible<br />highly accessible<br />1. Visual or textual cues provided on a Web site to suggest what information it or its links may contain.<br />2. The perceived usefulness of a page based on such information.<br />fairly accessible<br />Enriching Web Information Scent for Blind Users<br />ACM ASSETS2009<br />

4.
…calls for <br />2. Challenges<br />Goal: annotation of links with accessibility assessment results in navigation scenarios.<br /><ul><li> automatic evaluation due to efficiency needs</li></ul> - based on guideline review tools<br /> - beaware of tool limitations<br /> - make assumptions, take risks<br /><ul><li> user-tailored assessment</li></ul> - current assessment techniques address all user groups<br /> - adaptive evaluation and measurement<br /> - quantitative scores for accuracy and discrimination power<br />Enriching Web Information Scent for Blind Users<br />ACM ASSETS2009<br />

5.
3. User-Tailored Assessment: Evaluation <br />Accessibility is measured in terms of conformance to web guidelines for blind users.<br /><ul><li>Web Accessibility: subset of WCAG 1.0

6.
following the classification by Brajnik for the BW method</li></ul>- developed the Accessibility Checker for Blind users (ACB)<br />- ACB checks lack of tags, attributes and appropriate combination of them<br /><ul><li>Web Usability: Usability Guidelines for the Blind (UGB)

7.
4 principles: structure and arrangement, content appropriateness, multimodal output, consistency

8.
automatic guideline review tool: Magenta

9.
Magenta checks adequate content of tags and attributes, arrangement of headings or shortcuts</li></ul>Enriching Web Information Scent for Blind Users<br />ACM ASSETS2009<br />

10.
3. User-Tailored Assessment: Evaluation <br />Accessibility is measured in terms of conformance to web guidelines for blind users.<br />ACB<br />Accessibility report<br />Exclusively accessibility issues<br />Dependencies solver<br />www.foo.com<br />Exclusively usability issues<br />Usability<br />report<br />Magenta<br /><ul><li> Resolving guideline set conflicts/overlap

11.
Addressed

12.
Addressed but not implemented</li></ul>- Complementary<br />- Contradictory<br />Enriching Web Information Scent for Blind Users<br />ACM ASSETS2009<br />

13.
3. User-Tailored Assessment: Measurement <br />Metrics Calculation Component<br />ACB<br />Accessibility report<br />Exclusively accessibility issues<br />accesibility score<br />Dependencies solver<br />www.foo.com<br />Exclusively usability issues<br />Usability<br />report<br />Magenta<br /><ul><li> Calculate failure-rates for each test case, earl:TestCase

14.
Using aggregation methods

15.
Considering issue typology:

16.
automatic issues (earl:automatic) yield earl:passed or earl:fail

17.
recommendations

18.
semi-automatic issues (earl:semiAuto)</li></ul>Enriching Web Information Scent for Blind Users<br />ACM ASSETS2009<br />

19.
3. User-Tailored Assessment: Measurement <br />Traditional aggregation:<br />where W: weights and E: evaluation results<br />Logic Scores Preferences:<br />where ρ(d) are values selected upon the required logical relationship between evaluation results.<br />d=0 conjunction <br />0&lt; d &lt;0.5 quasiconjunction: simultaneity in satisfying all the evaluations<br />d=0.5  arithmetic mean<br />0.5&lt; d &lt;1 quasidisjunction: penalizes only when all evaluations are not satisfied<br />d=1 disjunction<br />Only intermediate values are applied<br />Enriching Web Information Scent for Blind Users<br />ACM ASSETS2009<br />

20.
3. User-Tailored Assessment: Measurement <br /><ul><li> Example: a checkpoint implements 3 test cases</li></ul>T1earl:automatic= 1<br />T2earl:automatic= 0.25<br />T3earl:semi-automatic= 0<br />0.3<br />0.25<br /><ul><li>strongquasiconjuctionamong earl:automatic</li></ul>- medium quasidisjunctionamong earl:semi-automatic<br /><ul><li> weak quasiconjunctionis applied at checkpoint level

21.
between guidelines, mean value</li></ul>final score<br />?<br />guidelines<br />0.25<br />checkpoints<br />test cases<br />T2<br />T3<br />T1<br />1<br />0.25<br />0<br />Enriching Web Information Scent for Blind Users<br />ACM ASSETS2009<br />

22.
4. User-testing <br /><ul><li> Preliminary evaluation for annotated links

23.
Experimental settings

24.
16 experienced blind users. Age M=43, sd=11

25.
JAWS on Internet Explorer

26.
Remote usability testing</li></ul>- Log analysis of interaction sequence, timing, keyboard and mouse actions<br /> - Post-task forms and post-test questionnaire<br /><ul><li> Two tasks

27.
Browsing by navigating

28.
Searching by navigating</li></ul>Enriching Web Information Scent for Blind Users<br />ACM ASSETS2009<br />

29.
4. User-testing: browsing by navigating<br /><ul><li> Definition: casual/aimless browsing deciding at each step where to go next

30.
Goal: observe users with no/vague target in mind

31.
Two sites with 10 links

32.
Top ten search results for “Pisa” and “Firenze” keywords

33.
Results were heterogeneous links wrt topic,

34.
Following a pattern: wikipedia, local university, soccer team and so on

35.
One site was manually annotated with accessibility scores and relevance scores

36.
Relevance based on ranking {very relevant, relevant, medium, low, irrelevant}</li></ul>- 5 min free browsing. They had to write a report about what they learned<br />Enriching Web Information Scent for Blind Users<br />ACM ASSETS2009<br />

37.
4. Results for browsing by navigating<br /><ul><li> In the page without annotations,

38.
9 users proceeded sequentially. Kendall τ=[0.8-1.0] at most p<0.03

39.
In the page with annotations,

40.
Only 2 users proceeded sequentially. Kendall τ=1.0 at most p<0.05

41.
None followed the sequence of most accessible links

42.
None followed the path based on relevance

43.
Some proceeded dichotomously

44.
However, when aggregating accessibility scores of visited pages, 7 points over the median are obtained

45.
This, can be interpreted as if the users browsed within the subset of more accessible pages according to random/preference criteria</li></ul>Enriching Web Information Scent for Blind Users<br />ACM ASSETS2009<br />

46.
4. User-testing: searching by navigating<br /><ul><li> Definition: look for a target by sequentially deciding at each step where to go next

47.
Goal: observe users with a specific target in mind

48.
Two sites with 10 links

49.
Top ten search results for “Accommodations in Pisa”

50.
Results were homogeneous links wrt topic

51.
One site was manually annotated with accessibility scores

52.
Relevance was not considered

53.
Two tasks: (1) given a telephone number (2) address

54.
They had to write down the name of the hotel</li></ul>Enriching Web Information Scent for Blind Users<br />ACM ASSETS2009<br />

55.
4. Results for searching by navigating<br /><ul><li> In the page without annotations,

56.
Only 2 users proceeded sequentially. Kendall τ=1.0 at most p<0.05

57.
One user proceeded inversely. Kendal τ=-1.0 at most p<0.02

58.
In the page with annotations,

59.
Only 2 users proceeded sequentially. Kendall τ=1.0 at most p<0.05

60.
One user followed the most accessible path. Kendal τ=1.0 at most p<0.02

61.
Again, when aggregating accessibility scores of visited pages 6 points over the median are obtained</li></ul>Enriching Web Information Scent for Blind Users<br />ACM ASSETS2009<br />

62.
5. Results: gathered informal comments <br /><ul><li> 8 users appreciated accessibility scores in links

63.
The suitability of scores was more controversial</li></ul> …some were satisfied: <br /><ul><li>“values adequately reflect accessibility level”

64.
“scores are useful”, “scores are interesting”

65.
“accessibility scores seem correct”

66.
“navigation is better if scores are included”</li></ul> …while other were not: <br /><ul><li>“strange validation”

67.
“scores are not very coherent”</li></ul> …other changed their minds: <br /><ul><li>“I’m doubtful about accessibility criteria” “links with accessibility scores are useful”

68.
“scores don’t convey the actual difference in accessibility level” “scores make navigation smoother and more instinctive”

69.
“scores seem random””I missed accessibility scores in this task” </li></ul>Enriching Web Information Scent for Blind Users<br />ACM ASSETS2009<br />

70.
5. Results: post-test questionnaire <br /><ul><li> 5 point Likert-scale {1: totally disagree – 5: totally agree}

71.
“Scores are useful for the browsing task”

72.
“Scores are useful for the searching task”

73.
“Scores are correlated with actual accessibility level in browsing task”

74.
“Scores are correlated with actual accessibility level in searching task”</li></ul>Enriching Web Information Scent for Blind Users<br />ACM ASSETS2009<br />

75.
6. Conclusions: scores <br /><ul><li> Most users state that scores are useful to a certain extent

76.
Although there is not an agreement

77.
“The perception of accessibility depends on each user and their particular computer settings”

78.
There is no agreement on the type of scores they prefer

79.
50% for qualitative and quantitative

80.
Considering the informal comments it seems that the annotation technique prevails over scores</li></ul>Enriching Web Information Scent for Blind Users<br />ACM ASSETS2009<br />

81.
6. Conclusions: annotation technique <br /><ul><li> In the searching scenario users do search within the subset of most accessible links

82.
In the browsing scenario users change paradigm

83.
From sequential browsing to random in the subset of most accessible links

84.
Subjective scores are more balanced than in the searching scenario

85.
When directly enquired, users state that accessibility annotations would be useful in those scenarios where the topic of the linked pages would be similar</li></ul>Enriching Web Information Scent for Blind Users<br />

86.
6. Conclusions: annotation technique <br /><ul><li> It seems that annotation technique would better fit in an scenario where:</li></ul> - They are browsing casually<br /> - and topics of linked pages are similar<br /><ul><li> For instance on the leaf nodes of a web directory</li></ul>Enriching Web Information Scent for Blind Users<br />

87.
Enriching Web Information Scent for Blind Users<br />Questions?<br />Markel Vigo1, Barbara Leporini2, and Fabio Paternò2<br />1 Laboratory of HCI for Special Needs<br />2 Human Interfaces in Information Systems<br />1 University of the Basque Country<br />2 Italian National Research Council<br />