Using Acceptance Tests to Validate Accessibility Requirements in RIA

1,483 views
1,412 views

Published on

These slides were presented in W4A.

Published in: Technology, Education
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,483
On SlideShare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
7
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide
  • Good morning! My name is Ana Luiza and I will talk about “ Using Acceptance Tests to Validate Accessibility Requirements in RIA ”
  • I will talk about: Research context, State of art, Hypothesis, Acceptance Tests, Preliminary results, Advantages and limitations and Next steps.
  • I will start discussing about Research context.
  • Nowadays, there are many web applications available, such as: amazon, google, twitter, youtube and facebook.
  • These Web applications have been developing considering the Web 2.0, that have many characteristics.
  • One of them is Perpetual Beta, because these Web applications are always under development.
  • It happens because it is necessary to consider users, and user`s needs can change.
  • Than, it`s necessary to implement new features and to improve Web applications.
  • But, there is a challenge. It`s necessary to do these, spending less than an hour.
  • To implement new features and to improve Web applications, it`s common to follow all these steps: Requirements, Design, Implementation, Tests and Deployment.
  • In this context, it`s important to consider Continuous Integration…
  • … because in each steps, it`s necessary to check the quality.
  • Then in this case, when developers insert any code in repository it`s necessary to make tests.
  • The result of theses tests is a report…
  • … and this report, can show if the quality is OK…
  • … or not OK. This part of Continuous Integration is related to tests.
  • The idea is to follow these steps spending less than an hour, because user’s needs change very fast. So, Web application need to change as well; and automatic tools can validate if the changes are OK or NOT faster.
  • But, we have to consider how these automatic tools evaluate accessibility.
  • Nowadays, there are researches that check accessibility features like user testing, but if use them, you will spend more than one hour.
  • That`s why it`s useful to use automatic accessibility tests.
  • But, there is a problem because when we develop a Web application, we use many technologies, mainly if we will develop rich interface applications.
  • This kind of automatic tools usually checks just HTML…
  • … and it`s not enough to evaluate the accessibility.
  • So, our hypothesis…
  • … is to consider user`s context, rich applications features….
  • … and also accessibility in rich applications.
  • … . and this report will consider the accessibility too, because these acceptance tests consider the whole technologies.
  • Acceptance tests is “a formal test that determines if a system satisfies its acceptance criteria and allow the users to identity when a system will be accepted or not”.
  • This test doesn’t consider external interface because usually people with disabilities don’t use it, it’s more common to use assistive technologies. This test reflects User Stories because it considers how users can interact with Web applications.
  • And these Users Stories are represented by DOM structure witch there are the possibilities that people can navigate on Web applications and, we consider these possibilities to create User Scenario.
  • For example, we created a scenario considering this Web application. In this case, the user needs to find tennis news. What the user needs to do?
  • Firsty, user needs to go to Yahoo page.
  • In order to observe how our hypothesis works, we implemented the same Web application twice. One had keyboard accessible and another one we inserted some accessibility errors. To do the test, we considered only navigation and keyboard accessibility issues.
  • Theses Web applications was implemented using some interactions design patterns from Welie like flyout menu, accordion menu, overlay menu and tabbed menu.
  • So to do this test, we used our hypothesis and automatic tools recommended by W3C, as DaSilva, EvalAccess and others.
  • But some of them were not available to use in that moment.
  • Therefore, we used DaSilva, EvalAccess, WAVE and FAE report in contrast with our hypothesis.
  • Firstly, we checked if the tools could identify if the Web application is accessible or not. It was possible to notice that acceptance tests identify just one incorrect assertion.
  • In incorrect assertions there are false positives and false negatives.
  • False positives represent menus that were not accessible. And, these tools didn`t identify this.
  • So, the results could be OK, but on the other hand, it`s not OK, because the tools didn`t identify that the menus were not accessible.
  • In this case, it`s possible to put this Web application in production and it is not good because there are errors.
  • In contrast to false positives, false negatives represent menus that were accessible.
  • But these tools didn`t identify this.
  • So, the results couldn`t be OK.
  • In this case, it isn`t possible to put this Web application in production.
  • It`s necessary to say that many results considered…
  • … only warnings, because the tools didn`t identify errors of accessibility.
  • As well as, these tools didn`t identify any distinction between Web application which presented accessible design solutions and the one that did not.
  • Some references used in this presentation...
  • Using Acceptance Tests to Validate Accessibility Requirements in RIA

    1. 1. Using Acceptance Tests to Validate Accessibility Requirements in RIA Willian Massami Watanabe Renata Pontin de Mattos Fortes Ana Luiza Dias9th International Cross-Disciplinary Conference on Web Accessibility – 16/17th April 2012 –
    2. 2. Agenda• Research context• State of art• Hypothesis• Acceptance Tests• Preliminary results• Advantages and limitations• Next steps 2/35
    3. 3. Agenda• Research context• State of art• Hypothesis• Acceptance Tests• Preliminary results• Advantages and limitations• Next steps 2/35
    4. 4. Research Context 3/35
    5. 5. Research Context Web 2.0 4/35
    6. 6. Research Context Perpetual beta 4/35
    7. 7. Research ContextPerpetual beta Users 4/35
    8. 8. Research Context • New featuresPerpetual • Web applications improvements beta Users 4/35
    9. 9. Research ContextPerpetual beta Less than an hour 5/35
    10. 10. Research ContextPerpetual Less than an hour beta Requirements Design Implementation Tests Deployment 5/35
    11. 11. Research ContextContinous IntegrationContinuouslly compile and test the software to ensure quality [Kim et al. 2008] 5/35
    12. 12. Research Context Continuous IntegrationDevelopers 6/35
    13. 13. Research Context Continuous Integration Source code repositoryDevelopers 6/35
    14. 14. Research Context Continuous Integration • Unit tests • Integration tests • Compile • Acceptance tests Source code repositoryDevelopers Report 6/35
    15. 15. Research Context Continuous Integration • Unit tests • Integration tests • Compile • Acceptance tests Source code repositoryDevelopers Report 6/35
    16. 16. Research Context Continuous Integration • Unit tests • Integration tests • Compile • Acceptance tests Source code repositoryDevelopers Report 6/35
    17. 17. Research ContextPerpetual Less than an hour beta Requirements Design Implementation Tests Quality Deployment 7/35
    18. 18. Research Context Less than an hour TestsHow to evaluate accessibility? 8/35
    19. 19. Research Context• User testing [Romen e Svanaes, 2008]• WCAG conformance [W3C, 2008]• Barrier walkthrough [Brajnik, 2006]• Automatic accessibility tests 9/35
    20. 20. Research Context Less than an hour Tests• User tests• WCAG conformance• Barrier walkthrough• Automatic accessibility tests 9/35
    21. 21. Agenda• Research context• State of art• Hypothesis• Acceptance Tests• Preliminary results• Advantages and limitations• Next steps 10/35
    22. 22. State of artHTMLJavaScript CSS RIA 11/35
    23. 23. State of art [Velasco et al. 2008]HTML HTML Parser Static HTML content analysis 11/35
    24. 24. State of art Not evaluated HTMLJavaScript CSS 11/35
    25. 25. Agenda• Research context• State of art• Hypothesis• Acceptance Tests• Preliminary results• Advantages and limitations• Next steps 12/35
    26. 26. Hypothesis UsersRIA 13/35
    27. 27. Hypothesis UsersA RIA 13/35
    28. 28. Hypothesis Continuous Integration • Unit tests • Integration tests • Compile • Acceptance tests Source code repositoryDevelopers Report 14/35
    29. 29. Agenda• Research context• State of art• Hipothesis• Acceptance Tests• Preliminary results• Advantages and limitations• Next steps 15/35
    30. 30. Acceptance Test Definition“A formal test that determines if a system satisfies its acceptance criteria and allow the user to identify when a system will be accepted or not.” 16/35
    31. 31. Acceptance TestBehaviour Acceptance tests • Assertions made against the external interface • Test cases reflects User Stories 17/35
    32. 32. Acceptance Test 18/35
    33. 33. Acceptance Test 19/35
    34. 34. Acceptance Test [W3C,Focus management 2011] 20/35
    35. 35. Acceptance Test TAB key navigation to theelement containing the text “Sports” 20/35
    36. 36. Acceptance Test TAB key navigation to the element containing the text “Tennis” 20/35
    37. 37. Acceptance Test [W3C,Focus management 2011] 20/35
    38. 38. Acceptance Test [W3C,Focus management 2011] 20/35
    39. 39. Acceptance Test [W3C,Focus management 2011] 20/35
    40. 40. Agenda• Research context• State of art• Hipothesis• Acceptance Tests• Preliminary results• Advantages and limitations• Next steps 21/35
    41. 41. Preliminary results [W3C, Focus management 2011] HTML Acceptance validators tests flyout menu flyout menu accordion menu accordion menuoverlay menu overlay menu tabbed menu tabbed menu [Welie, 2008] 22/35
    42. 42. Preliminary results [W3C,Focus management 2011] All web tools that test navigation HTMLvalidators accessibility requirements http://www.w3.org/WAI/ER/tools/completeCynthia Accessibility check DaSilva Achecker EvalAccess FAE report WAVE Hera 23/35
    43. 43. Preliminary results [W3C,Focus management 2011] All web tools that test navigation HTMLvalidators accessibility requirements http://www.w3.org/WAI/ER/tools/completeCynthia Accessibility check DaSilva Achecker EvalAccess FAE report WAVE Hera 23/35
    44. 44. Preliminary results [W3C, Focus management 2011] HTML Acceptance validators testsDaSilva EvalAccess Our hypothesis WAVE FAE report 24/35
    45. 45. Preliminary results [W3C,Focus management 2011] Acceptance HTML Validators tests 25/35
    46. 46. Preliminary results [W3C,Focus management 2011] Acceptance HTML Validators tests 26/35
    47. 47. Preliminary results Projects that present defects that [W3C,Focus management not2011] were identified by the Tool Acceptance HTML Validators tests 26/35
    48. 48. Preliminary results Continuous Integration • Unit tests • Integration tests • Compile • Acceptance tests Source code repositoryDevelopers Report 27/35
    49. 49. Preliminary results Projects that might be put Integração Contínua in production event though they contain accessibility tests • Unit errors • Integraiton tests • Compile • Acceptance tests Source code repositoryDevelopers Report 27/35
    50. 50. Preliminary results [W3C,Focus management 2011] Acceptance HTML Validators tests 28/35
    51. 51. Preliminary results [W3C,Focus management 2011] Projects that present no Acceptance accessibility errors however are HTML Validators tests not put into production 28/35
    52. 52. Preliminary results Continuous Integration • Unit tests • Integration tests • Compile • Acceptance tests Source code repositoryDevelopers Report 29/35
    53. 53. Preliminary results Integração Contínua of Prevent the deployment a stable release of the software • Unit tets • TestesIntegration tests • Compile • Acceptance tests Source code repositoryDevelopers Report 29/35
    54. 54. Preliminary results [W3C,Focus management 2011] Acceptance HTML Validators tests 30/35
    55. 55. Preliminary results [W3C,Focus management 2011] Acceptance Warnings only HTML Validators HTML tests 30/35
    56. 56. Preliminary results [W3C,Focus management Do not make distinction between interface 2011] components that were accessible and the ones that were not Acceptance Validadores HTML tests 30/35
    57. 57. Agenda• Research context• State of art• Hipothesis• Acceptance Tests• Preliminary results• Advantages and limitations• Next steps 31/35
    58. 58. Advantages• In the experiment, the use of acceptance tests presented a higher number of correct assertions• Higher effort required in order to execute the tests• Assertions made agains the DOM structure inside of an use context makes it possible to evaluate ARIA requirements 32/35
    59. 59. Limitation• The limitation of this hyphotesis is the higher cost to describe all contexts of the users 33/35
    60. 60. Agenda• Research context• State of art• Hipothesis• Acceptance Tests• Preliminary results• Advantages and limitations• Next steps 34/35
    61. 61. Next steps• Implement new use scenarios for the acceptance tests• Evaluate ARIA requirements 35/35
    62. 62. REFERENCES• Seojin Kim, Sungjin Park, Jeonghyun Yun, and Younghoo Lee. 2008. Automated Continuous Integration of Component-Based Software: An Industrial Experience. In Proceedings of the 2008 23rd IEEE/ACM International Conference on Automated Software Engineering (ASE 08). IEEE Computer Society, Washington, DC, USA, 423-426. DOI=10.1109/ASE.2008.64 http://dx.doi.org/10.1109/ASE.2008.64• ROMEN, D.; SVANAES, D. Evaluating web site accessibility: validating the wai guidelines through usability testing with disabled users. In: NordiCHI ’08: Proceedings of the 5th Nordic conference on Human-computer interaction, New York, NY, USA: ACM, 2008, p. 535–538.• W3C Conformance evaluation of web sites for accessibility. 2008a. Available at: http://www.w3.org/WAI/eval/conformance.html
    63. 63. REFERENCES• BRAJNIK, G. Web accessibility testing: When the method is the culprit. In: Computers Helping People with Special Needs, Springer Berlin / Heidelberg, 2006, p. 156–163 (Lecture Notes in Computer Science, v.4061). Available at: http://www.springerlink.com/content/ h2573t61gu3n8533/• C. A. Velasco, D. Denev, D. Stegemann, and Y. Mohamad. A web compliance engineering framework to support the development of accessible rich internet applications. In Proceedings of the 2008 international cross- disciplinary conference on Web accessibility (W4A), W4A ’08, pages 45–49, New York, NY, USA, 2008. ACM.• W3C Accessible Rich Internet Applications (WAI-ARIA) 1.0. 2011. Available at: http://www.w3.org/TR/wai-aria/.• Welie. 2008. Available at: http://www.welie.com/patterns/.
    64. 64. Thank you very much! Contacts: watanabe willian@yahoo.com renata@icmc.usp.br anadias@icmc.usp.br URLs: http://www.pyccuracy.org https://github.com/watinha/Pyccuracy-Accessibility-Actions http://watinha.com/pyccuracy_test_1/templates/9th International Cross-Disciplinary Conference on Web Accessibility – 16/17th April 2012 –

    ×