-
1.
Enriching Web Information Scent for Blind Users<br />Markel Vigo1, Barbara Leporini2, and Fabio Paternò2<br />1 Laboratory of HCI for Special Needs<br />2 Human Interfaces in Information Systems<br />1 University of the Basque Country<br />2 Italian National Research Council<br />
-
2.
Hypothesis: annotating links with the accessibility level of the page where they point would increase user orientation <br /> “Visually impaired users need to be warned of obstacles because their reliance on cues is higher than for sighted users”- Goble et al.<br /> ”Detecting and notifying users about barriers improves user orientation”- Harper et al.<br />Goal: use of accessibility assessment results in web navigation scenarios<br />Information Scent:<br />Thus, we aim at enriching information scent using accessibility assessment results for screen reader users.<br />1. Introduction<br />1. Visual or textual cues provided on a Web site to suggest what information its links may contain.<br />2. The perceived usefulness of a page based on such information.<br />Enriching Web Information Scent for Blind Users<br />ACM ASSETS2009<br />
-
3.
Hypothesis: annotating links with the accessibility level of the page that the link point to increases user orientation <br /> “Visually impaired users need to be warned of obstacles because their reliance on cues is higher than for sighted users”- Goble et al.<br /> ”Detecting and notifying users about barriers improves user orientation”- Harper et al.<br />Goal: deployment of accessibility assessment results in navigation scenarios<br />Information Scent:<br />Thus, we aim at enriching information scent using accessibility assessment results<br />1. Introduction<br />not accessible<br />highly accessible<br />1. Visual or textual cues provided on a Web site to suggest what information it or its links may contain.<br />2. The perceived usefulness of a page based on such information.<br />fairly accessible<br />Enriching Web Information Scent for Blind Users<br />ACM ASSETS2009<br />
-
4.
…calls for <br />2. Challenges<br />Goal: annotation of links with accessibility assessment results in navigation scenarios.<br /><ul><li> automatic evaluation due to efficiency needs</li></ul> - based on guideline review tools<br /> - beaware of tool limitations<br /> - make assumptions, take risks<br /><ul><li> user-tailored assessment</li></ul> - current assessment techniques address all user groups<br /> - adaptive evaluation and measurement<br /> - quantitative scores for accuracy and discrimination power<br />Enriching Web Information Scent for Blind Users<br />ACM ASSETS2009<br />
-
5.
3. User-Tailored Assessment: Evaluation <br />Accessibility is measured in terms of conformance to web guidelines for blind users.<br /><ul><li>Web Accessibility: subset of WCAG 1.0
-
6.
following the classification by Brajnik for the BW method</li></ul>- developed the Accessibility Checker for Blind users (ACB)<br />- ACB checks lack of tags, attributes and appropriate combination of them<br /><ul><li>Web Usability: Usability Guidelines for the Blind (UGB)
-
7.
4 principles: structure and arrangement, content appropriateness, multimodal output, consistency
-
8.
automatic guideline review tool: Magenta
-
9.
Magenta checks adequate content of tags and attributes, arrangement of headings or shortcuts</li></ul>Enriching Web Information Scent for Blind Users<br />ACM ASSETS2009<br />
-
10.
3. User-Tailored Assessment: Evaluation <br />Accessibility is measured in terms of conformance to web guidelines for blind users.<br />ACB<br />Accessibility report<br />Exclusively accessibility issues<br />Dependencies solver<br />www.foo.com<br />Exclusively usability issues<br />Usability<br />report<br />Magenta<br /><ul><li> Resolving guideline set conflicts/overlap
-
11.
Addressed
-
12.
Addressed but not implemented</li></ul>- Complementary<br />- Contradictory<br />Enriching Web Information Scent for Blind Users<br />ACM ASSETS2009<br />
-
13.
3. User-Tailored Assessment: Measurement <br />Metrics Calculation Component<br />ACB<br />Accessibility report<br />Exclusively accessibility issues<br />accesibility score<br />Dependencies solver<br />www.foo.com<br />Exclusively usability issues<br />Usability<br />report<br />Magenta<br /><ul><li> Calculate failure-rates for each test case, earl:TestCase
-
14.
Using aggregation methods
-
15.
Considering issue typology:
-
16.
automatic issues (earl:automatic) yield earl:passed or earl:fail
-
17.
recommendations
-
18.
semi-automatic issues (earl:semiAuto)</li></ul>Enriching Web Information Scent for Blind Users<br />ACM ASSETS2009<br />
-
19.
3. User-Tailored Assessment: Measurement <br />Traditional aggregation:<br />where W: weights and E: evaluation results<br />Logic Scores Preferences:<br />where ρ(d) are values selected upon the required logical relationship between evaluation results.<br />d=0 conjunction <br />0< d <0.5 quasiconjunction: simultaneity in satisfying all the evaluations<br />d=0.5 arithmetic mean<br />0.5< d <1 quasidisjunction: penalizes only when all evaluations are not satisfied<br />d=1 disjunction<br />Only intermediate values are applied<br />Enriching Web Information Scent for Blind Users<br />ACM ASSETS2009<br />
-
20.
3. User-Tailored Assessment: Measurement <br /><ul><li> Example: a checkpoint implements 3 test cases</li></ul>T1earl:automatic= 1<br />T2earl:automatic= 0.25<br />T3earl:semi-automatic= 0<br />0.3<br />0.25<br /><ul><li>strongquasiconjuctionamong earl:automatic</li></ul>- medium quasidisjunctionamong earl:semi-automatic<br /><ul><li> weak quasiconjunctionis applied at checkpoint level
-
21.
between guidelines, mean value</li></ul>final score<br />?<br />guidelines<br />0.25<br />checkpoints<br />test cases<br />T2<br />T3<br />T1<br />1<br />0.25<br />0<br />Enriching Web Information Scent for Blind Users<br />ACM ASSETS2009<br />
-
22.
4. User-testing <br /><ul><li> Preliminary evaluation for annotated links
-
23.
Experimental settings
-
24.
16 experienced blind users. Age M=43, sd=11
-
25.
JAWS on Internet Explorer
-
26.
Remote usability testing</li></ul>- Log analysis of interaction sequence, timing, keyboard and mouse actions<br /> - Post-task forms and post-test questionnaire<br /><ul><li> Two tasks
-
27.
Browsing by navigating
-
28.
Searching by navigating</li></ul>Enriching Web Information Scent for Blind Users<br />ACM ASSETS2009<br />
-
29.
4. User-testing: browsing by navigating<br /><ul><li> Definition: casual/aimless browsing deciding at each step where to go next
-
30.
Goal: observe users with no/vague target in mind
-
31.
Two sites with 10 links
-
32.
Top ten search results for “Pisa” and “Firenze” keywords
-
33.
Results were heterogeneous links wrt topic,
-
34.
Following a pattern: wikipedia, local university, soccer team and so on
-
35.
One site was manually annotated with accessibility scores and relevance scores
-
36.
Relevance based on ranking {very relevant, relevant, medium, low, irrelevant}</li></ul>- 5 min free browsing. They had to write a report about what they learned<br />Enriching Web Information Scent for Blind Users<br />ACM ASSETS2009<br />
-
37.
4. Results for browsing by navigating<br /><ul><li> In the page without annotations,
-
38.
9 users proceeded sequentially. Kendall τ=[0.8-1.0] at most p<0.03
-
39.
In the page with annotations,
-
40.
Only 2 users proceeded sequentially. Kendall τ=1.0 at most p<0.05
-
41.
None followed the sequence of most accessible links
-
42.
None followed the path based on relevance
-
43.
Some proceeded dichotomously
-
44.
However, when aggregating accessibility scores of visited pages, 7 points over the median are obtained
-
45.
This, can be interpreted as if the users browsed within the subset of more accessible pages according to random/preference criteria</li></ul>Enriching Web Information Scent for Blind Users<br />ACM ASSETS2009<br />
-
46.
4. User-testing: searching by navigating<br /><ul><li> Definition: look for a target by sequentially deciding at each step where to go next
-
47.
Goal: observe users with a specific target in mind
-
48.
Two sites with 10 links
-
49.
Top ten search results for “Accommodations in Pisa”
-
50.
Results were homogeneous links wrt topic
-
51.
One site was manually annotated with accessibility scores
-
52.
Relevance was not considered
-
53.
Two tasks: (1) given a telephone number (2) address
-
54.
They had to write down the name of the hotel</li></ul>Enriching Web Information Scent for Blind Users<br />ACM ASSETS2009<br />
-
55.
4. Results for searching by navigating<br /><ul><li> In the page without annotations,
-
56.
Only 2 users proceeded sequentially. Kendall τ=1.0 at most p<0.05
-
57.
One user proceeded inversely. Kendal τ=-1.0 at most p<0.02
-
58.
In the page with annotations,
-
59.
Only 2 users proceeded sequentially. Kendall τ=1.0 at most p<0.05
-
60.
One user followed the most accessible path. Kendal τ=1.0 at most p<0.02
-
61.
Again, when aggregating accessibility scores of visited pages 6 points over the median are obtained</li></ul>Enriching Web Information Scent for Blind Users<br />ACM ASSETS2009<br />
-
62.
5. Results: gathered informal comments <br /><ul><li> 8 users appreciated accessibility scores in links
-
63.
The suitability of scores was more controversial</li></ul> …some were satisfied: <br /><ul><li>“values adequately reflect accessibility level”
-
64.
“scores are useful”, “scores are interesting”
-
65.
“accessibility scores seem correct”
-
66.
“navigation is better if scores are included”</li></ul> …while other were not: <br /><ul><li>“strange validation”
-
67.
“scores are not very coherent”</li></ul> …other changed their minds: <br /><ul><li>“I’m doubtful about accessibility criteria” “links with accessibility scores are useful”
-
68.
“scores don’t convey the actual difference in accessibility level” “scores make navigation smoother and more instinctive”
-
69.
“scores seem random””I missed accessibility scores in this task” </li></ul>Enriching Web Information Scent for Blind Users<br />ACM ASSETS2009<br />
-
70.
5. Results: post-test questionnaire <br /><ul><li> 5 point Likert-scale {1: totally disagree – 5: totally agree}
-
71.
“Scores are useful for the browsing task”
-
72.
“Scores are useful for the searching task”
-
73.
“Scores are correlated with actual accessibility level in browsing task”
-
74.
“Scores are correlated with actual accessibility level in searching task”</li></ul>Enriching Web Information Scent for Blind Users<br />ACM ASSETS2009<br />
-
75.
6. Conclusions: scores <br /><ul><li> Most users state that scores are useful to a certain extent
-
76.
Although there is not an agreement
-
77.
“The perception of accessibility depends on each user and their particular computer settings”
-
78.
There is no agreement on the type of scores they prefer
-
79.
50% for qualitative and quantitative
-
80.
Considering the informal comments it seems that the annotation technique prevails over scores</li></ul>Enriching Web Information Scent for Blind Users<br />ACM ASSETS2009<br />
-
81.
6. Conclusions: annotation technique <br /><ul><li> In the searching scenario users do search within the subset of most accessible links
-
82.
In the browsing scenario users change paradigm
-
83.
From sequential browsing to random in the subset of most accessible links
-
84.
Subjective scores are more balanced than in the searching scenario
-
85.
When directly enquired, users state that accessibility annotations would be useful in those scenarios where the topic of the linked pages would be similar</li></ul>Enriching Web Information Scent for Blind Users<br />
-
86.
6. Conclusions: annotation technique <br /><ul><li> It seems that annotation technique would better fit in an scenario where:</li></ul> - They are browsing casually<br /> - and topics of linked pages are similar<br /><ul><li> For instance on the leaf nodes of a web directory</li></ul>Enriching Web Information Scent for Blind Users<br />
-
87.
Enriching Web Information Scent for Blind Users<br />Questions?<br />Markel Vigo1, Barbara Leporini2, and Fabio Paternò2<br />1 Laboratory of HCI for Special Needs<br />2 Human Interfaces in Information Systems<br />1 University of the Basque Country<br />2 Italian National Research Council<br />
or added values
assumptions: all checkpoints, test cases have the same impact on the user
ACC: tags: summary for tables, attributes: alt for images and combination: formssome studies on: even if web pages are accessible they are not usableUSA: “click here” statements, shortcuts to skip to main content
there’s certainly an overlap
atomic rulesrecommendations: highly subjective issues, AT dependant issues. Violating them does not have a strong effect on accessibility but maybe on usability
LSP overcomes traditional aggregation methodstrong, medium, weakρ(d) values are obtained elsewhere –rho-INTUITIVE AND EXPLICIT WAY
Example to illustrate how it worksAll test cases have to be satisfied to meet the whole checkpointHowever the uncertainty on semi-automatic issues leads us to consider the medium quasidisjunctionamong earl:semi-automaticBetween checkpoints there also a sense of simultaneity to satisfy the whole guideline
capture ofget their impressions, free to comment .subjective data
as expected by blind users
why relevance was not considered: in this scenario it would lead to the solution
as expected by blind users
Even if they were not told to, in the after task forms users gave their opinion.changed their minds those that did first the task with annotated links in the first scenario. it was curious. surprisinglyTECHNIQUE PREVAILS OVER THE SCORES
At first sightEven if there’s a peak for mid values there’s not a general agreementUsefulness and perception of score validity is higher for the browsing task, at least it is more balanced
results are inconclusiveand how some user changed their minds about scores