Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Using Acceptance Tests to           Validate Accessibility           Requirements in RIA                            Willia...
Agenda• Research context• State of art• Hypothesis• Acceptance Tests• Preliminary results• Advantages and limitations• Nex...
Agenda• Research context• State of art• Hypothesis• Acceptance Tests• Preliminary results• Advantages and limitations• Nex...
Research Context                   3/35
Research Context           Web 2.0                     4/35
Research Context         Perpetual           beta                     4/35
Research ContextPerpetual  beta                Users                        4/35
Research Context            • New featuresPerpetual            • Web applications improvements  beta                      ...
Research ContextPerpetual  beta            Less than an hour                                5/35
Research ContextPerpetual   Less than an hour  beta                Requirements                   Design               Imp...
Research ContextContinous IntegrationContinuouslly compile and test the  software to ensure quality                       ...
Research Context         Continuous IntegrationDevelopers                                  6/35
Research Context         Continuous Integration             Source code              repositoryDevelopers                 ...
Research Context         Continuous Integration                            • Unit tests                            • Integ...
Research Context         Continuous Integration                            • Unit tests                            • Integ...
Research Context         Continuous Integration                            • Unit tests                            • Integ...
Research ContextPerpetual     Less than an hour  beta                  Requirements                     Design            ...
Research Context       Less than an hour                  TestsHow to evaluate accessibility?                           8/35
Research Context• User testing [Romen e Svanaes, 2008]• WCAG conformance [W3C, 2008]• Barrier walkthrough [Brajnik, 2006]•...
Research Context                Less than an hour                             Tests• User tests• WCAG conformance• Barrier...
Agenda• Research context• State of art• Hypothesis• Acceptance Tests• Preliminary results• Advantages and limitations• Nex...
State of artHTMLJavaScript    CSS                            RIA                                  11/35
State of art                       [Velasco et al.                           2008]HTML       HTML Parser        Static HTM...
State of art             Not evaluated  HTMLJavaScript    CSS                             11/35
Agenda• Research context• State of art• Hypothesis• Acceptance Tests• Preliminary results• Advantages and limitations• Nex...
Hypothesis             UsersRIA                     13/35
Hypothesis             UsersA RIA                     13/35
Hypothesis         Continuous Integration                            • Unit tests                            • Integration...
Agenda• Research context• State of art• Hipothesis• Acceptance Tests• Preliminary results• Advantages and limitations• Nex...
Acceptance Test                   Definition“A formal test that determines if a system satisfies its  acceptance criteria ...
Acceptance TestBehaviour Acceptance tests • Assertions made against the external     interface •   Test cases reflects Use...
Acceptance Test                  18/35
Acceptance Test                  19/35
Acceptance Test                   [W3C,Focus management    2011]                            20/35
Acceptance Test TAB key navigation to theelement containing the text         “Sports”                                 20/35
Acceptance Test            TAB key navigation to the             element containing the                 text “Tennis”     ...
Acceptance Test                   [W3C,Focus management    2011]                            20/35
Acceptance Test                   [W3C,Focus management    2011]                            20/35
Acceptance Test                   [W3C,Focus management    2011]                            20/35
Agenda• Research context• State of art• Hipothesis• Acceptance Tests• Preliminary results• Advantages and limitations• Nex...
Preliminary results                      [W3C,   Focus management    2011]       HTML             Acceptance      validato...
Preliminary results                            [W3C,Focus management             2011]               All web tools that te...
Preliminary results                            [W3C,Focus management             2011]               All web tools that te...
Preliminary results                       [W3C,  Focus management      2011]     HTML               Acceptance    validato...
Preliminary results                           [W3C,Focus management            2011]                                    Ac...
Preliminary results                            [W3C,Focus management             2011]                                    ...
Preliminary results             Projects that present defects that                       [W3C,Focus management not2011]   ...
Preliminary results         Continuous Integration                            • Unit tests                            • In...
Preliminary results         Projects that might be put           Integração Contínua        in production event though    ...
Preliminary results                           [W3C,Focus management            2011]                                    Ac...
Preliminary results                          [W3C,Focus management            2011]                  Projects that present...
Preliminary results         Continuous Integration                            • Unit tests                            • In...
Preliminary results          Integração Contínua of        Prevent the deployment          a stable release of the        ...
Preliminary results                           [W3C,Focus management            2011]                                    Ac...
Preliminary results                        [W3C,Focus management         2011]                                     Accepta...
Preliminary results                        [W3C,Focus management    Do not make distinction between interface             ...
Agenda• Research context• State of art• Hipothesis• Acceptance Tests• Preliminary results• Advantages and  limitations• Ne...
Advantages• In the experiment, the use of acceptance tests  presented a higher number of correct  assertions• Higher effor...
Limitation• The limitation of this hyphotesis is the higher  cost to describe all contexts of the users                   ...
Agenda• Research context• State of art• Hipothesis• Acceptance Tests• Preliminary results• Advantages and limitations• Nex...
Next steps• Implement new use scenarios for the  acceptance tests• Evaluate ARIA requirements                             ...
REFERENCES•   Seojin Kim, Sungjin Park, Jeonghyun Yun, and Younghoo Lee. 2008.    Automated Continuous Integration of Comp...
REFERENCES•   BRAJNIK, G. Web accessibility testing: When the method is the culprit. In:    Computers Helping People with ...
Thank you very much! Contacts:   watanabe willian@yahoo.com   renata@icmc.usp.br   anadias@icmc.usp.br URLs:  http://www.p...
Upcoming SlideShare
Loading in …5
×

Using Acceptance Tests to Validate Accessibility Requirements in RIA

1,876 views

Published on

These slides were presented in W4A.

Published in: Technology, Education
  • Be the first to comment

Using Acceptance Tests to Validate Accessibility Requirements in RIA

  1. 1. Using Acceptance Tests to Validate Accessibility Requirements in RIA Willian Massami Watanabe Renata Pontin de Mattos Fortes Ana Luiza Dias9th International Cross-Disciplinary Conference on Web Accessibility – 16/17th April 2012 –
  2. 2. Agenda• Research context• State of art• Hypothesis• Acceptance Tests• Preliminary results• Advantages and limitations• Next steps 2/35
  3. 3. Agenda• Research context• State of art• Hypothesis• Acceptance Tests• Preliminary results• Advantages and limitations• Next steps 2/35
  4. 4. Research Context 3/35
  5. 5. Research Context Web 2.0 4/35
  6. 6. Research Context Perpetual beta 4/35
  7. 7. Research ContextPerpetual beta Users 4/35
  8. 8. Research Context • New featuresPerpetual • Web applications improvements beta Users 4/35
  9. 9. Research ContextPerpetual beta Less than an hour 5/35
  10. 10. Research ContextPerpetual Less than an hour beta Requirements Design Implementation Tests Deployment 5/35
  11. 11. Research ContextContinous IntegrationContinuouslly compile and test the software to ensure quality [Kim et al. 2008] 5/35
  12. 12. Research Context Continuous IntegrationDevelopers 6/35
  13. 13. Research Context Continuous Integration Source code repositoryDevelopers 6/35
  14. 14. Research Context Continuous Integration • Unit tests • Integration tests • Compile • Acceptance tests Source code repositoryDevelopers Report 6/35
  15. 15. Research Context Continuous Integration • Unit tests • Integration tests • Compile • Acceptance tests Source code repositoryDevelopers Report 6/35
  16. 16. Research Context Continuous Integration • Unit tests • Integration tests • Compile • Acceptance tests Source code repositoryDevelopers Report 6/35
  17. 17. Research ContextPerpetual Less than an hour beta Requirements Design Implementation Tests Quality Deployment 7/35
  18. 18. Research Context Less than an hour TestsHow to evaluate accessibility? 8/35
  19. 19. Research Context• User testing [Romen e Svanaes, 2008]• WCAG conformance [W3C, 2008]• Barrier walkthrough [Brajnik, 2006]• Automatic accessibility tests 9/35
  20. 20. Research Context Less than an hour Tests• User tests• WCAG conformance• Barrier walkthrough• Automatic accessibility tests 9/35
  21. 21. Agenda• Research context• State of art• Hypothesis• Acceptance Tests• Preliminary results• Advantages and limitations• Next steps 10/35
  22. 22. State of artHTMLJavaScript CSS RIA 11/35
  23. 23. State of art [Velasco et al. 2008]HTML HTML Parser Static HTML content analysis 11/35
  24. 24. State of art Not evaluated HTMLJavaScript CSS 11/35
  25. 25. Agenda• Research context• State of art• Hypothesis• Acceptance Tests• Preliminary results• Advantages and limitations• Next steps 12/35
  26. 26. Hypothesis UsersRIA 13/35
  27. 27. Hypothesis UsersA RIA 13/35
  28. 28. Hypothesis Continuous Integration • Unit tests • Integration tests • Compile • Acceptance tests Source code repositoryDevelopers Report 14/35
  29. 29. Agenda• Research context• State of art• Hipothesis• Acceptance Tests• Preliminary results• Advantages and limitations• Next steps 15/35
  30. 30. Acceptance Test Definition“A formal test that determines if a system satisfies its acceptance criteria and allow the user to identify when a system will be accepted or not.” 16/35
  31. 31. Acceptance TestBehaviour Acceptance tests • Assertions made against the external interface • Test cases reflects User Stories 17/35
  32. 32. Acceptance Test 18/35
  33. 33. Acceptance Test 19/35
  34. 34. Acceptance Test [W3C,Focus management 2011] 20/35
  35. 35. Acceptance Test TAB key navigation to theelement containing the text “Sports” 20/35
  36. 36. Acceptance Test TAB key navigation to the element containing the text “Tennis” 20/35
  37. 37. Acceptance Test [W3C,Focus management 2011] 20/35
  38. 38. Acceptance Test [W3C,Focus management 2011] 20/35
  39. 39. Acceptance Test [W3C,Focus management 2011] 20/35
  40. 40. Agenda• Research context• State of art• Hipothesis• Acceptance Tests• Preliminary results• Advantages and limitations• Next steps 21/35
  41. 41. Preliminary results [W3C, Focus management 2011] HTML Acceptance validators tests flyout menu flyout menu accordion menu accordion menuoverlay menu overlay menu tabbed menu tabbed menu [Welie, 2008] 22/35
  42. 42. Preliminary results [W3C,Focus management 2011] All web tools that test navigation HTMLvalidators accessibility requirements http://www.w3.org/WAI/ER/tools/completeCynthia Accessibility check DaSilva Achecker EvalAccess FAE report WAVE Hera 23/35
  43. 43. Preliminary results [W3C,Focus management 2011] All web tools that test navigation HTMLvalidators accessibility requirements http://www.w3.org/WAI/ER/tools/completeCynthia Accessibility check DaSilva Achecker EvalAccess FAE report WAVE Hera 23/35
  44. 44. Preliminary results [W3C, Focus management 2011] HTML Acceptance validators testsDaSilva EvalAccess Our hypothesis WAVE FAE report 24/35
  45. 45. Preliminary results [W3C,Focus management 2011] Acceptance HTML Validators tests 25/35
  46. 46. Preliminary results [W3C,Focus management 2011] Acceptance HTML Validators tests 26/35
  47. 47. Preliminary results Projects that present defects that [W3C,Focus management not2011] were identified by the Tool Acceptance HTML Validators tests 26/35
  48. 48. Preliminary results Continuous Integration • Unit tests • Integration tests • Compile • Acceptance tests Source code repositoryDevelopers Report 27/35
  49. 49. Preliminary results Projects that might be put Integração Contínua in production event though they contain accessibility tests • Unit errors • Integraiton tests • Compile • Acceptance tests Source code repositoryDevelopers Report 27/35
  50. 50. Preliminary results [W3C,Focus management 2011] Acceptance HTML Validators tests 28/35
  51. 51. Preliminary results [W3C,Focus management 2011] Projects that present no Acceptance accessibility errors however are HTML Validators tests not put into production 28/35
  52. 52. Preliminary results Continuous Integration • Unit tests • Integration tests • Compile • Acceptance tests Source code repositoryDevelopers Report 29/35
  53. 53. Preliminary results Integração Contínua of Prevent the deployment a stable release of the software • Unit tets • TestesIntegration tests • Compile • Acceptance tests Source code repositoryDevelopers Report 29/35
  54. 54. Preliminary results [W3C,Focus management 2011] Acceptance HTML Validators tests 30/35
  55. 55. Preliminary results [W3C,Focus management 2011] Acceptance Warnings only HTML Validators HTML tests 30/35
  56. 56. Preliminary results [W3C,Focus management Do not make distinction between interface 2011] components that were accessible and the ones that were not Acceptance Validadores HTML tests 30/35
  57. 57. Agenda• Research context• State of art• Hipothesis• Acceptance Tests• Preliminary results• Advantages and limitations• Next steps 31/35
  58. 58. Advantages• In the experiment, the use of acceptance tests presented a higher number of correct assertions• Higher effort required in order to execute the tests• Assertions made agains the DOM structure inside of an use context makes it possible to evaluate ARIA requirements 32/35
  59. 59. Limitation• The limitation of this hyphotesis is the higher cost to describe all contexts of the users 33/35
  60. 60. Agenda• Research context• State of art• Hipothesis• Acceptance Tests• Preliminary results• Advantages and limitations• Next steps 34/35
  61. 61. Next steps• Implement new use scenarios for the acceptance tests• Evaluate ARIA requirements 35/35
  62. 62. REFERENCES• Seojin Kim, Sungjin Park, Jeonghyun Yun, and Younghoo Lee. 2008. Automated Continuous Integration of Component-Based Software: An Industrial Experience. In Proceedings of the 2008 23rd IEEE/ACM International Conference on Automated Software Engineering (ASE 08). IEEE Computer Society, Washington, DC, USA, 423-426. DOI=10.1109/ASE.2008.64 http://dx.doi.org/10.1109/ASE.2008.64• ROMEN, D.; SVANAES, D. Evaluating web site accessibility: validating the wai guidelines through usability testing with disabled users. In: NordiCHI ’08: Proceedings of the 5th Nordic conference on Human-computer interaction, New York, NY, USA: ACM, 2008, p. 535–538.• W3C Conformance evaluation of web sites for accessibility. 2008a. Available at: http://www.w3.org/WAI/eval/conformance.html
  63. 63. REFERENCES• BRAJNIK, G. Web accessibility testing: When the method is the culprit. In: Computers Helping People with Special Needs, Springer Berlin / Heidelberg, 2006, p. 156–163 (Lecture Notes in Computer Science, v.4061). Available at: http://www.springerlink.com/content/ h2573t61gu3n8533/• C. A. Velasco, D. Denev, D. Stegemann, and Y. Mohamad. A web compliance engineering framework to support the development of accessible rich internet applications. In Proceedings of the 2008 international cross- disciplinary conference on Web accessibility (W4A), W4A ’08, pages 45–49, New York, NY, USA, 2008. ACM.• W3C Accessible Rich Internet Applications (WAI-ARIA) 1.0. 2011. Available at: http://www.w3.org/TR/wai-aria/.• Welie. 2008. Available at: http://www.welie.com/patterns/.
  64. 64. Thank you very much! Contacts: watanabe willian@yahoo.com renata@icmc.usp.br anadias@icmc.usp.br URLs: http://www.pyccuracy.org https://github.com/watinha/Pyccuracy-Accessibility-Actions http://watinha.com/pyccuracy_test_1/templates/9th International Cross-Disciplinary Conference on Web Accessibility – 16/17th April 2012 –

×