Using Acceptance Tests to Validate Accessibility Requirements in RIA
Report
analuizadiasFollow
Apr. 16, 2012•0 likes•1,539 views
Using Acceptance Tests to Validate Accessibility Requirements in RIA
Apr. 16, 2012•0 likes•1,539 views
Download to read offline
Report
Technology
Education
These slides were presented in W4A.
analuizadiasFollow
Recommended
Synthesizing Knowledge from Software Development ArtifactsJeongwhan Choi
Positive FlowSathishkumar Vasudevan
Rob Sabourin: On TestingTechWell
An Overview of User Acceptance Testing (UAT)Usersnap
A Pragmatic Approach to REDCap ValidationRick Watts
Implementing a Performance Centre of ExcellenceRichard Bishop
More Related Content
Similar to Using Acceptance Tests to Validate Accessibility Requirements in RIA
Continuous testing in agile projects 2015Fabricio Epaminondas
Exploratory Testing Basics and Future Kari Kakkonen Moscow 2018Kari Kakkonen
Agile testing - Principles and best practicesDr Ganesh Iyer
BbWorld 2012 AppDynamics Presentationfarkasp
Mobile Video Games Testing Principles - Benjamin PoirrierHo Chi Minh City Software Testing Club
NERC CIP Version 5 and Beyond – Compliance and the Vendor’s RoleEnergySec
Recently uploaded
"Stateful app as an efficient way to build dispatching for riders and drivers...Fwdays
Dennis Wendland_The i4Trust Collaboration Programme.pptxFIWARE
Recommendation Modeling with Impression Data at NetflixJiangwei Pan
"Building Asynchronous SOA for Modern Applications", Sai Pragna Etikyala Fwdays
Solving today’s Traffic Problems with Sustainable Ride Hailing SolutionOn Demand Clone
Accelerating Data Science through Feature Platform, Transformers and GenAIFeatureByte
Using Acceptance Tests to Validate Accessibility Requirements in RIA
- 1. Using Acceptance Tests to Validate Accessibility Requirements in RIA Willian Massami Watanabe Renata Pontin de Mattos Fortes Ana Luiza Dias 9th International Cross-Disciplinary Conference on Web Accessibility – 16/17th April 2012 –
- 2. Agenda • Research context • State of art • Hypothesis • Acceptance Tests • Preliminary results • Advantages and limitations • Next steps 2/35
- 3. Agenda • Research context • State of art • Hypothesis • Acceptance Tests • Preliminary results • Advantages and limitations • Next steps 2/35
- 4. Research Context 3/35
- 5. Research Context Web 2.0 4/35
- 6. Research Context Perpetual beta 4/35
- 7. Research Context Perpetual beta Users 4/35
- 8. Research Context • New features Perpetual • Web applications improvements beta Users 4/35
- 9. Research Context Perpetual beta Less than an hour 5/35
- 10. Research Context Perpetual Less than an hour beta Requirements Design Implementation Tests Deployment 5/35
- 11. Research Context Continous Integration Continuouslly compile and test the software to ensure quality [Kim et al. 2008] 5/35
- 12. Research Context Continuous Integration Developers 6/35
- 13. Research Context Continuous Integration Source code repository Developers 6/35
- 14. Research Context Continuous Integration • Unit tests • Integration tests • Compile • Acceptance tests Source code repository Developers Report 6/35
- 15. Research Context Continuous Integration • Unit tests • Integration tests • Compile • Acceptance tests Source code repository Developers Report 6/35
- 16. Research Context Continuous Integration • Unit tests • Integration tests • Compile • Acceptance tests Source code repository Developers Report 6/35
- 17. Research Context Perpetual Less than an hour beta Requirements Design Implementation Tests Quality Deployment 7/35
- 18. Research Context Less than an hour Tests How to evaluate accessibility? 8/35
- 19. Research Context • User testing [Romen e Svanaes, 2008] • WCAG conformance [W3C, 2008] • Barrier walkthrough [Brajnik, 2006] • Automatic accessibility tests 9/35
- 20. Research Context Less than an hour Tests • User tests • WCAG conformance • Barrier walkthrough • Automatic accessibility tests 9/35
- 21. Agenda • Research context • State of art • Hypothesis • Acceptance Tests • Preliminary results • Advantages and limitations • Next steps 10/35
- 22. State of art HTML JavaScript CSS RIA 11/35
- 23. State of art [Velasco et al. 2008] HTML HTML Parser Static HTML content analysis 11/35
- 24. State of art Not evaluated HTML JavaScript CSS 11/35
- 25. Agenda • Research context • State of art • Hypothesis • Acceptance Tests • Preliminary results • Advantages and limitations • Next steps 12/35
- 26. Hypothesis Users RIA 13/35
- 27. Hypothesis Users A RIA 13/35
- 28. Hypothesis Continuous Integration • Unit tests • Integration tests • Compile • Acceptance tests Source code repository Developers Report 14/35
- 29. Agenda • Research context • State of art • Hipothesis • Acceptance Tests • Preliminary results • Advantages and limitations • Next steps 15/35
- 30. Acceptance Test Definition “A formal test that determines if a system satisfies its acceptance criteria and allow the user to identify when a system will be accepted or not.” 16/35
- 31. Acceptance Test Behaviour Acceptance tests • Assertions made against the external interface • Test cases reflects User Stories 17/35
- 32. Acceptance Test 18/35
- 33. Acceptance Test 19/35
- 34. Acceptance Test [W3C, Focus management 2011] 20/35
- 35. Acceptance Test TAB key navigation to the element containing the text “Sports” 20/35
- 36. Acceptance Test TAB key navigation to the element containing the text “Tennis” 20/35
- 37. Acceptance Test [W3C, Focus management 2011] 20/35
- 38. Acceptance Test [W3C, Focus management 2011] 20/35
- 39. Acceptance Test [W3C, Focus management 2011] 20/35
- 40. Agenda • Research context • State of art • Hipothesis • Acceptance Tests • Preliminary results • Advantages and limitations • Next steps 21/35
- 41. Preliminary results [W3C, Focus management 2011] HTML Acceptance validators tests flyout menu flyout menu accordion menu accordion menu overlay menu overlay menu tabbed menu tabbed menu [Welie, 2008] 22/35
- 42. Preliminary results [W3C, Focus management 2011] All web tools that test navigation HTML validators accessibility requirements http://www.w3.org/WAI/ER/tools/complete Cynthia Accessibility check DaSilva Achecker EvalAccess FAE report WAVE Hera 23/35
- 43. Preliminary results [W3C, Focus management 2011] All web tools that test navigation HTML validators accessibility requirements http://www.w3.org/WAI/ER/tools/complete Cynthia Accessibility check DaSilva Achecker EvalAccess FAE report WAVE Hera 23/35
- 44. Preliminary results [W3C, Focus management 2011] HTML Acceptance validators tests DaSilva EvalAccess Our hypothesis WAVE FAE report 24/35
- 45. Preliminary results [W3C, Focus management 2011] Acceptance HTML Validators tests 25/35
- 46. Preliminary results [W3C, Focus management 2011] Acceptance HTML Validators tests 26/35
- 47. Preliminary results Projects that present defects that [W3C, Focus management not2011] were identified by the Tool Acceptance HTML Validators tests 26/35
- 48. Preliminary results Continuous Integration • Unit tests • Integration tests • Compile • Acceptance tests Source code repository Developers Report 27/35
- 49. Preliminary results Projects that might be put Integração Contínua in production event though they contain accessibility tests • Unit errors • Integraiton tests • Compile • Acceptance tests Source code repository Developers Report 27/35
- 50. Preliminary results [W3C, Focus management 2011] Acceptance HTML Validators tests 28/35
- 51. Preliminary results [W3C, Focus management 2011] Projects that present no Acceptance accessibility errors however are HTML Validators tests not put into production 28/35
- 52. Preliminary results Continuous Integration • Unit tests • Integration tests • Compile • Acceptance tests Source code repository Developers Report 29/35
- 53. Preliminary results Integração Contínua of Prevent the deployment a stable release of the software • Unit tets • TestesIntegration tests • Compile • Acceptance tests Source code repository Developers Report 29/35
- 54. Preliminary results [W3C, Focus management 2011] Acceptance HTML Validators tests 30/35
- 55. Preliminary results [W3C, Focus management 2011] Acceptance Warnings only HTML Validators HTML tests 30/35
- 56. Preliminary results [W3C, Focus management Do not make distinction between interface 2011] components that were accessible and the ones that were not Acceptance Validadores HTML tests 30/35
- 57. Agenda • Research context • State of art • Hipothesis • Acceptance Tests • Preliminary results • Advantages and limitations • Next steps 31/35
- 58. Advantages • In the experiment, the use of acceptance tests presented a higher number of correct assertions • Higher effort required in order to execute the tests • Assertions made agains the DOM structure inside of an use context makes it possible to evaluate ARIA requirements 32/35
- 59. Limitation • The limitation of this hyphotesis is the higher cost to describe all contexts of the users 33/35
- 60. Agenda • Research context • State of art • Hipothesis • Acceptance Tests • Preliminary results • Advantages and limitations • Next steps 34/35
- 61. Next steps • Implement new use scenarios for the acceptance tests • Evaluate ARIA requirements 35/35
- 62. REFERENCES • Seojin Kim, Sungjin Park, Jeonghyun Yun, and Younghoo Lee. 2008. Automated Continuous Integration of Component-Based Software: An Industrial Experience. In Proceedings of the 2008 23rd IEEE/ACM International Conference on Automated Software Engineering (ASE '08). IEEE Computer Society, Washington, DC, USA, 423-426. DOI=10.1109/ASE.2008.64 http://dx.doi.org/10.1109/ASE.2008.64 • ROMEN, D.; SVANAES, D. Evaluating web site accessibility: validating the wai guidelines through usability testing with disabled users. In: NordiCHI ’08: Proceedings of the 5th Nordic conference on Human-computer interaction, New York, NY, USA: ACM, 2008, p. 535–538. • W3C Conformance evaluation of web sites for accessibility. 2008a. Available at: http://www.w3.org/WAI/eval/conformance.html
- 63. REFERENCES • BRAJNIK, G. Web accessibility testing: When the method is the culprit. In: Computers Helping People with Special Needs, Springer Berlin / Heidelberg, 2006, p. 156–163 (Lecture Notes in Computer Science, v.4061). Available at: http://www.springerlink.com/content/ h2573t61gu3n8533/ • C. A. Velasco, D. Denev, D. Stegemann, and Y. Mohamad. A web compliance engineering framework to support the development of accessible rich internet applications. In Proceedings of the 2008 international cross- disciplinary conference on Web accessibility (W4A), W4A ’08, pages 45–49, New York, NY, USA, 2008. ACM. • W3C Accessible Rich Internet Applications (WAI-ARIA) 1.0. 2011. Available at: http://www.w3.org/TR/wai-aria/. • Welie. 2008. Available at: http://www.welie.com/patterns/.
- 64. Thank you very much! Contacts: watanabe willian@yahoo.com renata@icmc.usp.br anadias@icmc.usp.br URLs: http://www.pyccuracy.org https://github.com/watinha/Pyccuracy-Accessibility-Actions http://watinha.com/pyccuracy_test_1/templates/ 9th International Cross-Disciplinary Conference on Web Accessibility – 16/17th April 2012 –
Editor's Notes
- Good morning! My name is Ana Luiza and I will talk about “ Using Acceptance Tests to Validate Accessibility Requirements in RIA ”
- I will talk about: Research context, State of art, Hypothesis, Acceptance Tests, Preliminary results, Advantages and limitations and Next steps.
- I will start discussing about Research context.
- Nowadays, there are many web applications available, such as: amazon, google, twitter, youtube and facebook.
- These Web applications have been developing considering the Web 2.0, that have many characteristics.
- One of them is Perpetual Beta, because these Web applications are always under development.
- It happens because it is necessary to consider users, and user`s needs can change.
- Than, it`s necessary to implement new features and to improve Web applications.
- But, there is a challenge. It`s necessary to do these, spending less than an hour.
- To implement new features and to improve Web applications, it`s common to follow all these steps: Requirements, Design, Implementation, Tests and Deployment.
- In this context, it`s important to consider Continuous Integration…
- … because in each steps, it`s necessary to check the quality.
- Then in this case, when developers insert any code in repository it`s necessary to make tests.
- The result of theses tests is a report…
- … and this report, can show if the quality is OK…
- … or not OK. This part of Continuous Integration is related to tests.
- The idea is to follow these steps spending less than an hour, because user’s needs change very fast. So, Web application need to change as well; and automatic tools can validate if the changes are OK or NOT faster.
- But, we have to consider how these automatic tools evaluate accessibility.
- Nowadays, there are researches that check accessibility features like user testing, but if use them, you will spend more than one hour.
- That`s why it`s useful to use automatic accessibility tests.
- But, there is a problem because when we develop a Web application, we use many technologies, mainly if we will develop rich interface applications.
- This kind of automatic tools usually checks just HTML…
- … and it`s not enough to evaluate the accessibility.
- So, our hypothesis…
- … is to consider user`s context, rich applications features….
- … and also accessibility in rich applications.
- … . and this report will consider the accessibility too, because these acceptance tests consider the whole technologies.
- Acceptance tests is “a formal test that determines if a system satisfies its acceptance criteria and allow the users to identity when a system will be accepted or not”.
- This test doesn’t consider external interface because usually people with disabilities don’t use it, it’s more common to use assistive technologies. This test reflects User Stories because it considers how users can interact with Web applications.
- And these Users Stories are represented by DOM structure witch there are the possibilities that people can navigate on Web applications and, we consider these possibilities to create User Scenario.
- For example, we created a scenario considering this Web application. In this case, the user needs to find tennis news. What the user needs to do?
- Firsty, user needs to go to Yahoo page.
- In order to observe how our hypothesis works, we implemented the same Web application twice. One had keyboard accessible and another one we inserted some accessibility errors. To do the test, we considered only navigation and keyboard accessibility issues.
- Theses Web applications was implemented using some interactions design patterns from Welie like flyout menu, accordion menu, overlay menu and tabbed menu.
- So to do this test, we used our hypothesis and automatic tools recommended by W3C, as DaSilva, EvalAccess and others.
- But some of them were not available to use in that moment.
- Therefore, we used DaSilva, EvalAccess, WAVE and FAE report in contrast with our hypothesis.
- Firstly, we checked if the tools could identify if the Web application is accessible or not. It was possible to notice that acceptance tests identify just one incorrect assertion.
- In incorrect assertions there are false positives and false negatives.
- False positives represent menus that were not accessible. And, these tools didn`t identify this.
- So, the results could be OK, but on the other hand, it`s not OK, because the tools didn`t identify that the menus were not accessible.
- In this case, it`s possible to put this Web application in production and it is not good because there are errors.
- In contrast to false positives, false negatives represent menus that were accessible.
- But these tools didn`t identify this.
- So, the results couldn`t be OK.
- In this case, it isn`t possible to put this Web application in production.
- It`s necessary to say that many results considered…
- … only warnings, because the tools didn`t identify errors of accessibility.
- As well as, these tools didn`t identify any distinction between Web application which presented accessible design solutions and the one that did not.
- Some references used in this presentation...