ASSESSING THE EFFORT OF REPAIRING THEACCESSIBILITY OF WEB SITES  Nádia Fernandes, Luís Carriço  University of Lisbon  ICCH...
ICCHP, Linz, Austria, July 13, 2012   2
ICCHP, Linz, Austria, July 13, 2012   3Introduction• 40-50% of the Web content uses templates.• Evaluations are performed ...
ICCHP, Linz, Austria, July 13, 2012   4Objectives1.   Assess the effort of repairing a site’s accessibility that was     o...
ICCHP, Linz, Austria, July 13, 2012   5The Accessibility Repairing Effort Metric (AREM) The metric considers the sum of th...
ICCHP, Linz, Austria, July 13, 2012   6The Accessibility Repairing Effort Metric (AREM)The purpose of this metric is:1.   ...
ICCHP, Linz, Austria, July 13, 2012   7The Platform for Accessibility EvaluationBasis:  • QualWeb evaluator  • Fast Match ...
ICCHP, Linz, Austria, July 13, 2012   8Procedure1.    The DOM trees are obtained2.    The pages are compared using the Fas...
ICCHP, Linz, Austria, July 13, 2012   9Experimental Study• Objective: understand the advantage of using AREM and validate ...
ICCHP, Linz, Austria, July 13, 2012   10Experimental Study• QualWeb was applied with the template aware option set.• The v...
ICCHP, Linz, Austria, July 13, 2012              11Metrics Results – AREM (I)   100000    80000    60000    40000    20000...
ICCHP, Linz, Austria, July 13, 2012                  12Metrics Results - Conservative rate (I)    0.40    0.35    0.30    ...
ICCHP, Linz, Austria, July 13, 2012              13Metrics Results – AREM (II)   100000    80000    60000    40000    2000...
ICCHP, Linz, Austria, July 13, 2012                  14Metrics Results - Conservative rate (II)    0.40    0.35    0.30   ...
ICCHP, Linz, Austria, July 13, 2012                  15Metrics Results - Conservative rate (III)    0.40    0.35    0.30  ...
ICCHP, Linz, Austria, July 13, 2012   16Validating the Approach                                                          %...
ICCHP, Linz, Austria, July 13, 2012   17Discussion• The validation of the template detection yielded a deviation.• The alg...
ICCHP, Linz, Austria, July 13, 2012   18Conclusion• Templates can be very important, reducing the effort of correction.• T...
ICCHP, Linz, Austria, July 13, 2012   19Future Work1.   Improvements of Fast-Match algorithm to guarantee a     higher acc...
ICCHP, Linz, Austria, July 13, 2012   20Thank younadiaf@di.fc.ul.pt
Upcoming SlideShare
Loading in …5
×

Assessing the Effort of Repairing the Accessibility of Web Sites

293
-1

Published on

Published in: Technology, Design
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
293
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
1
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Assessing the Effort of Repairing the Accessibility of Web Sites

  1. 1. ASSESSING THE EFFORT OF REPAIRING THEACCESSIBILITY OF WEB SITES Nádia Fernandes, Luís Carriço University of Lisbon ICCHP, Linz, Austria, July 11-13, 2012
  2. 2. ICCHP, Linz, Austria, July 13, 2012 2
  3. 3. ICCHP, Linz, Austria, July 13, 2012 3Introduction• 40-50% of the Web content uses templates.• Evaluations are performed in pages as a whole.• Obfuscating results.• Available metrics may be misleading.
  4. 4. ICCHP, Linz, Austria, July 13, 2012 4Objectives1. Assess the effort of repairing a site’s accessibility that was originally developed using templates Requirements: • Develop a new metric, • Template detection algorithm2. Introducing accessible templates as a form of rapidly repairing a page
  5. 5. ICCHP, Linz, Austria, July 13, 2012 5The Accessibility Repairing Effort Metric (AREM) The metric considers the sum of the number of fails and warnings reported by accessibility evaluation techniques, excluding repeated instances.“Primitive elements”- elements of a page, when an element thatis part of a template is only considered once.
  6. 6. ICCHP, Linz, Austria, July 13, 2012 6The Accessibility Repairing Effort Metric (AREM)The purpose of this metric is:1. assess the quality measurement of the accessibility of site construction (effort person.month);2. and not the perceived quality of the site towards end-users.
  7. 7. ICCHP, Linz, Austria, July 13, 2012 7The Platform for Accessibility EvaluationBasis: • QualWeb evaluator • Fast Match algorithm
  8. 8. ICCHP, Linz, Austria, July 13, 2012 8Procedure1. The DOM trees are obtained2. The pages are compared using the Fast Match algorithm • Comparative function • Result: The “primitive” nodes3. The accessibility evaluation is executed (QualWeb evaluator)4. Reports generation (new reports)5. The repairing effort estimation according to AREM is computed.
  9. 9. ICCHP, Linz, Austria, July 13, 2012 9Experimental Study• Objective: understand the advantage of using AREM and validate the template detection method.• Object of study: 15 sites with templates (Alexa Top 100) 5 high 5 low level level 5 medium level
  10. 10. ICCHP, Linz, Austria, July 13, 2012 10Experimental Study• QualWeb was applied with the template aware option set.• The values for the AREM using the element primitiveness.• We used a conservative rate metric to obtain values of accessibility quality to compare with AREM. rate conservative =
  11. 11. ICCHP, Linz, Austria, July 13, 2012 11Metrics Results – AREM (I) 100000 80000 60000 40000 20000 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Web sites Non-primitive Primitive 1 – Higher level of template usage, 15 – Lower level of template usage
  12. 12. ICCHP, Linz, Austria, July 13, 2012 12Metrics Results - Conservative rate (I) 0.40 0.35 0.30 0.25 0.20 0.15 0.10 0.05 0.00 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Web sites Non-primitive Primitive 1 – Higher level of template usage, 15 – Lower level of template usage
  13. 13. ICCHP, Linz, Austria, July 13, 2012 13Metrics Results – AREM (II) 100000 80000 60000 40000 20000 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Web sites Non-primitive Primitive 1 – Higher level of template usage, 15 – Lower level of template usage
  14. 14. ICCHP, Linz, Austria, July 13, 2012 14Metrics Results - Conservative rate (II) 0.40 0.35 0.30 0.25 0.20 0.15 0.10 0.05 0.00 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Web sites Non-primitive Primitive 1 – Higher level of template usage, 15 – Lower level of template usage
  15. 15. ICCHP, Linz, Austria, July 13, 2012 15Metrics Results - Conservative rate (III) 0.40 0.35 0.30 0.25 0.20 0.15 0.10 0.05 0.00 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Web sites Non-primitive Primitive 1 – Higher level of template usage, 15 – Lower level of template usage
  16. 16. ICCHP, Linz, Austria, July 13, 2012 16Validating the Approach % errors on template Web sites % template detectionHigher level of 56% 14%template detection 51% 8%Mid level of template 33% 6%detection 31% 12%Lower level of 18% 7%template detection 15% 1% The number of incorrect elements detected by Fast Match algorithm is less than 10% (average).
  17. 17. ICCHP, Linz, Austria, July 13, 2012 17Discussion• The validation of the template detection yielded a deviation.• The algorithm had more failures in sites which use more template based components.• Regarding the AREM metric: • The difference between the computed values is high; • Less 30% of repairing issues (primitive elements); • Depending on the sites this value can decrease substantially.• These results confirm and support our previous experiment’s results.
  18. 18. ICCHP, Linz, Austria, July 13, 2012 18Conclusion• Templates can be very important, reducing the effort of correction.• The metric defined is a real indicator of the work that have to be done, unlike certain quality metric that can be misleading.• We performed a validation experiment of both metric and framework and conclude that the template detection algorithm has a high efficacy.
  19. 19. ICCHP, Linz, Austria, July 13, 2012 19Future Work1. Improvements of Fast-Match algorithm to guarantee a higher accuracy level;2. A large-scale evaluation of the fast match algorithm.
  20. 20. ICCHP, Linz, Austria, July 13, 2012 20Thank younadiaf@di.fc.ul.pt
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×