ATDD in Practice


Published on

A gentle introduction to Acceptance Test Driven Development

Published in: Technology, Education

ATDD in Practice

  1. 1. Acceptance Test Driven Development in practice Steven Mak
  2. 2. What are we up to now? Lost in translation Do not explain why Gaps discovered only until coding started Cumulative effects of small misunderstandings Inadequate and essentially flawed requirements and specifications
  3. 3. Failing to meet actual needs are obvious things really obvious? Imperative requirements Fulfilling specifications does not guarantee success
  4. 4. Meeting the needs with Acceptance TDD Acceptance Test Implementation Requirement Feedback Drive implementation of a requirement through a set of automated, executable acceptance tests
  5. 5. ATDD in a Nutshell Real-world examples to build a shared understanding of the domain Select a set of these examples to be a specification and an acceptance test suite Automate the verification of acceptance tests Focus the software development effort on the acceptance tests Use the set of acceptance tests to facilitate discussion about future change requests.
  6. 6. The ATDD cycle •Implementation of tests usually coding occur before the code is done testing • Feature not done until test passes Feature Specification architecture Done Workshop customer documentation other activities • Activities happen in parallel Developers Testers •Team clarifies and implement the feature Example Product Owner Architect tests together Technical writers •Tests are executable at the end of the ATDD Workshop
  7. 7. Benefits of ATDD Comprehensible examples over complex formulas Close Collaboration Definition of Done Trust and Commitment Testing on system level
  8. 8. Specification by Examples Use realistic examples to demonstrate differences in possibilities instead of abstract requirements Write specifications down as tables Workflows: Preconditions Processing steps Verifications
  9. 9. Examples, Tests, and Spec can become Examples Tests ela rify bo ve ra te Requirements
  10. 10. Specification workshop Ask the domain experts Developers and testers should suggest examples of edges or important issues for discussion Ubiquitous language Organise feedback to ensure shared understanding Use facilitator to stay focused if needed
  11. 11. Distilling the specifications Write tests collaboratively Examples in a form close to what your automation tool can understand Keep tests in a form that is human-readable Specification over Scripting, describe WHAT, not how Acceptance tests to prevent defects, not to discover Not necessarily automate anything Acceptance tests only work when we can discuss them
  12. 12. Some considerations User Interface Easy? Fragile? Performance issues? Boundary of Stub Sufficiently close Simulators? Business logic Not from developer perspective
  13. 13. Acceptance Test smells Long tests Parameters of calculation tests that always have the same value Similar test with minor differences Tests that reflect the way code was written Tests fail intermittently even though you didn’t change any code Interdependent tests, e.g. setup for others
  14. 14. Change Use existing acceptance tests to discuss future changes Seek advices from customer to determine if it specifies obsolete functionality when test fails Automate periodic execution of regression tests with CI Keep tests in the same version control as code
  15. 15. Tools Table-based frameworks FIT, RobotFramework, Text-based frameworks Exactor, TextTest,
  16. 16. FIT FIT stands for “Framework for Integrated Tests” Most popular framework in-use Table-based Supporting languages like Java, C/C++, C#, Python, or Ruby FitLibrary to extend FIT
  17. 17. FIT in practice Test doc with tables Customer writes a Technical staff enhance test document the tables in the doc containing examples Test doc with sanitised tables Technical staff Executable Test implements fixture classes Test doc and backing code (e.g. Java)
  18. 18. FIT Document Customers write acceptance tests in their own language using their own tools.
  19. 19. Fixtures Take information from the tables and turn them into method calls on the actual application. ColumnFixture - checking rules and calculations ActionFixture - Step-by-step processing RowFixture - Checking sets of data
  20. 20. Fixture example
  21. 21. Robot Framework Python-based Keyword-driven test automation framework Test libraries implemented either in Python or Java (with remote library available in many other languages thru XML- RPC) Test cases are written in tabular format, save in HTML or TSV files Syntax similar to natural language Users can create new keywords from existing ones and contribute to the project
  22. 22. Preparing Test cases Workflow tests - takes in initial state, action, verified that the system behaved as expected Test procedure using keywords Keyword arguments Test case name Test Case Action Argument Valid Login Open Login Page Input Name demo Input Password mode Submit Credentials
  23. 23. Higher-level test cases Free text suitable for communication even with non- technical customers Support given-when-then format in BDD
  24. 24. Data-driven test cases Define a keyword which will take the input data and prepare a table with test cases
  25. 25. Test case organisation Simple way: Single HTML file containing all test cases Test case tagging
  26. 26. Execution Gathering test cases, reading and setting variables Executing all actions for every test case Providing global statistics Writing the output in XML format Generating report and log in HTML format
  27. 27. Sample execution result
  28. 28. Sample Test Report in HTML format, showing all actions executed up to the failing action, with fail message
  29. 29. Tested application Interface Command line OperatingSystem SSHLibrary Telnet library Web Robot Selenium GUI Swing GUI library
  30. 30. Adoption Sense of achievement Integrity Openness Right timing
  31. 31. Facilitate Adoption Evangelise - increase awareness Lower the bar - quick path to the first success Train and educate - from general training to specific needs Share and infect - sit together and sharing sessions Coach and facilitate - learn about yourselves and cope with the findings Involve others - know the stake-holder groups and give them roles
  32. 32. Ideal candidate to work with Shared interest in success Authority to make decision Ability to understand implications Ability to explain the domain
  33. 33. Variations - an escape? Behaviour-driven development Example-driven development Executable specifications Names do not matter, but underlying practices matter Worthwhile to try if your business people do not like “testing”
  34. 34. Organisational Challenges Business Analysts Great choice for the facilitator of the spec workshop Working with Acceptance tests makes BA’s work easier Testers Help people avoid problems, not to discover them Automation of acceptance test gives testers more time to test manual things Developers Spec workshop gives a good chance to discuss with domain experts Both unit tests and acceptance tests are important
  35. 35. References Bridging the Communication Gap Gojko Adzic Practical TDD and ATDD for Java Developers Lasse Koskela Agile Testing Lisa Crispin and Janet Gregory
  36. 36. Thank you! Steven Mak Email: Twitter: