Enriching Web Information Scent for Blind Users
Upcoming SlideShare
Loading in...5
×
 

Enriching Web Information Scent for Blind Users

on

  • 1,071 views

Link annotation with the accessibility level of the target Web page is an adaptive navigation support technique aimed at increasing blind users’ orientation in Web sites. In this work, the ...

Link annotation with the accessibility level of the target Web page is an adaptive navigation support technique aimed at increasing blind users’ orientation in Web sites. In this work, the accessibility level of a page is measured by exploiting data from evaluation reports produced by two automatic assessment tools. These tools support evaluation of accessibility and usability guideline-sets. As a result, links are annotated with a score that indicates the conformance of the target Web page to blind user accessibility and usability guidelines. A user test with 16 users was conducted in order to observe the strategies they followed when links were annotated with these scores. With annotated links, the navigation paradigm changed from sequential to browsing randomly through the subset of those links with high scores. Even if there was not a general agreement on the correspondence between scores and user perception of accessibility, users found annotations helpful when browsing through links related to a given topic.

Statistics

Views

Total Views
1,071
Views on SlideShare
1,069
Embed Views
2

Actions

Likes
0
Downloads
9
Comments
0

1 Embed 2

http://www.slideshare.net 2

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • or added values
  • assumptions: all checkpoints, test cases have the same impact on the user
  • ACC: tags: summary for tables, attributes: alt for images and combination: formssome studies on: even if web pages are accessible they are not usableUSA: “click here” statements, shortcuts to skip to main content
  • there’s certainly an overlap
  • atomic rulesrecommendations: highly subjective issues, AT dependant issues. Violating them does not have a strong effect on accessibility but maybe on usability
  • LSP overcomes traditional aggregation methodstrong, medium, weakρ(d) values are obtained elsewhere –rho-INTUITIVE AND EXPLICIT WAY
  • Example to illustrate how it worksAll test cases have to be satisfied to meet the whole checkpointHowever the uncertainty on semi-automatic issues leads us to consider the medium quasidisjunctionamong earl:semi-automaticBetween checkpoints there also a sense of simultaneity to satisfy the whole guideline
  • capture ofget their impressions, free to comment .subjective data
  • as expected by blind users
  • why relevance was not considered: in this scenario it would lead to the solution
  • as expected by blind users
  • Even if they were not told to, in the after task forms users gave their opinion.changed their minds those that did first the task with annotated links in the first scenario. it was curious. surprisinglyTECHNIQUE PREVAILS OVER THE SCORES
  • At first sightEven if there’s a peak for mid values there’s not a general agreementUsefulness and perception of score validity is higher for the browsing task, at least it is more balanced
  • results are inconclusiveand how some user changed their minds about scores

Enriching Web Information Scent for Blind Users Enriching Web Information Scent for Blind Users Presentation Transcript

  • Enriching Web Information Scent for Blind Users
    Markel Vigo1, Barbara Leporini2, and Fabio Paternò2
    1 Laboratory of HCI for Special Needs
    2 Human Interfaces in Information Systems
    1 University of the Basque Country
    2 Italian National Research Council
  • Hypothesis: annotating links with the accessibility level of the page where they point would increase user orientation
    “Visually impaired users need to be warned of obstacles because their reliance on cues is higher than for sighted users”- Goble et al.
    ”Detecting and notifying users about barriers improves user orientation”- Harper et al.
    Goal: use of accessibility assessment results in web navigation scenarios
    Information Scent:
    Thus, we aim at enriching information scent using accessibility assessment results for screen reader users.
    1. Introduction
    1. Visual or textual cues provided on a Web site to suggest what information its links may contain.
    2. The perceived usefulness of a page based on such information.
    Enriching Web Information Scent for Blind Users
    ACM ASSETS2009
  • Hypothesis: annotating links with the accessibility level of the page that the link point to increases user orientation
    “Visually impaired users need to be warned of obstacles because their reliance on cues is higher than for sighted users”- Goble et al.
    ”Detecting and notifying users about barriers improves user orientation”- Harper et al.
    Goal: deployment of accessibility assessment results in navigation scenarios
    Information Scent:
    Thus, we aim at enriching information scent using accessibility assessment results
    1. Introduction
    not accessible
    highly accessible
    1. Visual or textual cues provided on a Web site to suggest what information it or its links may contain.
    2. The perceived usefulness of a page based on such information.
    fairly accessible
    Enriching Web Information Scent for Blind Users
    ACM ASSETS2009
  • …calls for
    2. Challenges
    Goal: annotation of links with accessibility assessment results in navigation scenarios.
    • automatic evaluation due to efficiency needs
    - based on guideline review tools
    - beaware of tool limitations
    - make assumptions, take risks
    • user-tailored assessment
    - current assessment techniques address all user groups
    - adaptive evaluation and measurement
    - quantitative scores for accuracy and discrimination power
    Enriching Web Information Scent for Blind Users
    ACM ASSETS2009
  • 3. User-Tailored Assessment: Evaluation
    Accessibility is measured in terms of conformance to web guidelines for blind users.
    • Web Accessibility: subset of WCAG 1.0
    • following the classification by Brajnik for the BW method
    - developed the Accessibility Checker for Blind users (ACB)
    - ACB checks lack of tags, attributes and appropriate combination of them
    • Web Usability: Usability Guidelines for the Blind (UGB)
    • 4 principles: structure and arrangement, content appropriateness, multimodal output, consistency
    • automatic guideline review tool: Magenta
    • Magenta checks adequate content of tags and attributes, arrangement of headings or shortcuts
    Enriching Web Information Scent for Blind Users
    ACM ASSETS2009
  • 3. User-Tailored Assessment: Evaluation
    Accessibility is measured in terms of conformance to web guidelines for blind users.
    ACB
    Accessibility report
    Exclusively accessibility issues
    Dependencies solver
    www.foo.com
    Exclusively usability issues
    Usability
    report
    Magenta
    • Resolving guideline set conflicts/overlap
    • Addressed
    • Addressed but not implemented
    - Complementary
    - Contradictory
    Enriching Web Information Scent for Blind Users
    ACM ASSETS2009
  • 3. User-Tailored Assessment: Measurement
    Metrics Calculation Component
    ACB
    Accessibility report
    Exclusively accessibility issues
    accesibility score
    Dependencies solver
    www.foo.com
    Exclusively usability issues
    Usability
    report
    Magenta
    • Calculate failure-rates for each test case, earl:TestCase
    • Using aggregation methods
    • Considering issue typology:
    • automatic issues (earl:automatic) yield earl:passed or earl:fail
    • recommendations
    • semi-automatic issues (earl:semiAuto)
    Enriching Web Information Scent for Blind Users
    ACM ASSETS2009
  • 3. User-Tailored Assessment: Measurement
    Traditional aggregation:
    where W: weights and E: evaluation results
    Logic Scores Preferences:
    where ρ(d) are values selected upon the required logical relationship between evaluation results.
    d=0 conjunction
    0< d <0.5 quasiconjunction: simultaneity in satisfying all the evaluations
    d=0.5  arithmetic mean
    0.5< d <1 quasidisjunction: penalizes only when all evaluations are not satisfied
    d=1 disjunction
    Only intermediate values are applied
    Enriching Web Information Scent for Blind Users
    ACM ASSETS2009
  • 3. User-Tailored Assessment: Measurement
    • Example: a checkpoint implements 3 test cases
    T1earl:automatic= 1
    T2earl:automatic= 0.25
    T3earl:semi-automatic= 0
    0.3
    0.25
    • strongquasiconjuctionamong earl:automatic
    - medium quasidisjunctionamong earl:semi-automatic
    • weak quasiconjunctionis applied at checkpoint level
    • between guidelines, mean value
    final score
    ?
    guidelines
    0.25
    checkpoints
    test cases
    T2
    T3
    T1
    1
    0.25
    0
    Enriching Web Information Scent for Blind Users
    ACM ASSETS2009
  • 4. User-testing
    • Preliminary evaluation for annotated links
    • Experimental settings
    • 16 experienced blind users. Age M=43, sd=11
    • JAWS on Internet Explorer
    • Remote usability testing
    - Log analysis of interaction sequence, timing, keyboard and mouse actions
    - Post-task forms and post-test questionnaire
    • Two tasks
    • Browsing by navigating
    • Searching by navigating
    Enriching Web Information Scent for Blind Users
    ACM ASSETS2009
  • 4. User-testing: browsing by navigating
    • Definition: casual/aimless browsing deciding at each step where to go next
    • Goal: observe users with no/vague target in mind
    • Two sites with 10 links
    • Top ten search results for “Pisa” and “Firenze” keywords
    • Results were heterogeneous links wrt topic,
    • Following a pattern: wikipedia, local university, soccer team and so on
    • One site was manually annotated with accessibility scores and relevance scores
    • Relevance based on ranking {very relevant, relevant, medium, low, irrelevant}
    - 5 min free browsing. They had to write a report about what they learned
    Enriching Web Information Scent for Blind Users
    ACM ASSETS2009
  • 4. Results for browsing by navigating
    • In the page without annotations,
    • 9 users proceeded sequentially. Kendall τ=[0.8-1.0] at most p<0.03
    • In the page with annotations,
    • Only 2 users proceeded sequentially. Kendall τ=1.0 at most p<0.05
    • None followed the sequence of most accessible links
    • None followed the path based on relevance
    • Some proceeded dichotomously
    • However, when aggregating accessibility scores of visited pages, 7 points over the median are obtained
    • This, can be interpreted as if the users browsed within the subset of more accessible pages according to random/preference criteria
    Enriching Web Information Scent for Blind Users
    ACM ASSETS2009
  • 4. User-testing: searching by navigating
    • Definition: look for a target by sequentially deciding at each step where to go next
    • Goal: observe users with a specific target in mind
    • Two sites with 10 links
    • Top ten search results for “Accommodations in Pisa”
    • Results were homogeneous links wrt topic
    • One site was manually annotated with accessibility scores
    • Relevance was not considered
    • Two tasks: (1) given a telephone number (2) address
    • They had to write down the name of the hotel
    Enriching Web Information Scent for Blind Users
    ACM ASSETS2009
  • 4. Results for searching by navigating
    • In the page without annotations,
    • Only 2 users proceeded sequentially. Kendall τ=1.0 at most p<0.05
    • One user proceeded inversely. Kendal τ=-1.0 at most p<0.02
    • In the page with annotations,
    • Only 2 users proceeded sequentially. Kendall τ=1.0 at most p<0.05
    • One user followed the most accessible path. Kendal τ=1.0 at most p<0.02
    • Again, when aggregating accessibility scores of visited pages 6 points over the median are obtained
    Enriching Web Information Scent for Blind Users
    ACM ASSETS2009
  • 5. Results: gathered informal comments
    • 8 users appreciated accessibility scores in links
    • The suitability of scores was more controversial
    …some were satisfied:
    • “values adequately reflect accessibility level”
    • “scores are useful”, “scores are interesting”
    • “accessibility scores seem correct”
    • “navigation is better if scores are included”
    …while other were not:
    • “strange validation”
    • “scores are not very coherent”
    …other changed their minds:
    • “I’m doubtful about accessibility criteria” “links with accessibility scores are useful”
    • “scores don’t convey the actual difference in accessibility level” “scores make navigation smoother and more instinctive”
    • “scores seem random””I missed accessibility scores in this task”
    Enriching Web Information Scent for Blind Users
    ACM ASSETS2009
  • 5. Results: post-test questionnaire
    • 5 point Likert-scale {1: totally disagree – 5: totally agree}
    • “Scores are useful for the browsing task”
    • “Scores are useful for the searching task”
    • “Scores are correlated with actual accessibility level in browsing task”
    • “Scores are correlated with actual accessibility level in searching task”
    Enriching Web Information Scent for Blind Users
    ACM ASSETS2009
  • 6. Conclusions: scores
    • Most users state that scores are useful to a certain extent
    • Although there is not an agreement
    • “The perception of accessibility depends on each user and their particular computer settings”
    • There is no agreement on the type of scores they prefer
    • 50% for qualitative and quantitative
    • Considering the informal comments it seems that the annotation technique prevails over scores
    Enriching Web Information Scent for Blind Users
    ACM ASSETS2009
  • 6. Conclusions: annotation technique
    • In the searching scenario users do search within the subset of most accessible links
    • In the browsing scenario users change paradigm
    • From sequential browsing to random in the subset of most accessible links
    • Subjective scores are more balanced than in the searching scenario
    • When directly enquired, users state that accessibility annotations would be useful in those scenarios where the topic of the linked pages would be similar
    Enriching Web Information Scent for Blind Users
  • 6. Conclusions: annotation technique
    • It seems that annotation technique would better fit in an scenario where:
    - They are browsing casually
    - and topics of linked pages are similar
    • For instance on the leaf nodes of a web directory
    Enriching Web Information Scent for Blind Users
  • Enriching Web Information Scent for Blind Users
    Questions?
    Markel Vigo1, Barbara Leporini2, and Fabio Paternò2
    1 Laboratory of HCI for Special Needs
    2 Human Interfaces in Information Systems
    1 University of the Basque Country
    2 Italian National Research Council