Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Acceptance Test Driven
Development in practice
Steven Mak
steven@odd-e.com
What are we up to now?
Lost in translation
Do not explain why
Gaps discovered only until coding started
Cumulative effects...
Failing to meet actual needs
 are obvious things really obvious?




 Imperative requirements




 Fulfilling specifications...
Meeting the needs with
Acceptance TDD

                   Acceptance Test              Implementation
    Requirement     ...
ATDD in a Nutshell
Real-world examples to build a shared understanding of the
domain
Select a set of these examples to be ...
The ATDD cycle
                                                           •Implementation of tests usually
               ...
Benefits of ATDD
Comprehensible examples over complex formulas
Close Collaboration
Definition of Done
Trust and Commitment
T...
Specification by Examples
Use realistic examples to demonstrate differences in
possibilities instead of abstract requiremen...
Examples, Tests, and Spec
             can become
 Examples                    Tests
  ela




                           ...
Specification workshop
Ask the domain experts
Developers and testers should suggest examples of
edges or important issues f...
Distilling the specifications
 Write tests collaboratively
 Examples in a form close to what your automation tool can
 unde...
Some considerations
User Interface
  Easy?

  Fragile?

  Performance issues?

Boundary of Stub
  Sufficiently close

  Sim...
Acceptance Test smells
Long tests
Parameters of calculation tests that always have the same
value
Similar test with minor ...
Change
Use existing acceptance tests to discuss future
changes
Seek advices from customer to determine if it specifies
obso...
Tools
Table-based frameworks
  FIT, http://fit.c2.com

  RobotFramework, http://robotframework.org

Text-based frameworks
 ...
FIT
 FIT stands for “Framework for Integrated Tests”
 Most popular framework in-use
 Table-based
 Supporting languages lik...
FIT in practice
                          Test doc
                         with tables

    Customer writes a
           ...
FIT Document
Customers write acceptance tests in their own
language using their own tools.
Fixtures
 Take information from the tables and turn them into
 method calls on the actual application.
 ColumnFixture - ch...
Fixture example
Robot Framework
Python-based Keyword-driven test automation framework
Test libraries implemented either in Python or Java ...
Preparing Test cases
Workflow tests - takes in initial state, action, verified
that the system behaved as expected
         ...
Higher-level test cases
 Free text suitable for communication even with non-
 technical customers
 Support given-when-then...
Data-driven test cases
 Define a keyword which will take the input data and
 prepare a table with test cases
Test case organisation
 Simple way: Single HTML file containing all test cases
 Test case tagging
Execution
Gathering test cases, reading and setting variables
Executing all actions for every test case
Providing global s...
Sample execution result
Sample Test Report
in HTML format, showing all actions executed up to the
failing action, with fail message
Tested application Interface
 Command line
   OperatingSystem
   SSHLibrary
   Telnet library
 Web
   Robot Selenium
 GUI
...
Adoption
Sense of achievement

Integrity

Openness

Right timing
Facilitate Adoption
 Evangelise - increase awareness
 Lower the bar - quick path to the first success
 Train and educate - ...
Ideal candidate to work with
 Shared interest in success
 Authority to make decision
 Ability to understand implications
 ...
Variations - an escape?
 Behaviour-driven development
 Example-driven development
 Executable specifications
 Names do not ...
Organisational Challenges
Business Analysts
  Great choice for the facilitator of the spec workshop

  Working with Accept...
References
Bridging the Communication Gap
  Gojko Adzic

Practical TDD and ATDD for Java Developers
  Lasse Koskela

Agile...
Thank you!
                                  Steven Mak
                   Email: steven@odd-e.com
       Twitter: http://...
Upcoming SlideShare
Loading in …5
×

ATDD in Practice

43,961 views

Published on

A gentle introduction to Acceptance Test Driven Development

Published in: Technology, Education

ATDD in Practice

  1. 1. Acceptance Test Driven Development in practice Steven Mak steven@odd-e.com
  2. 2. What are we up to now? Lost in translation Do not explain why Gaps discovered only until coding started Cumulative effects of small misunderstandings Inadequate and essentially flawed requirements and specifications
  3. 3. Failing to meet actual needs are obvious things really obvious? Imperative requirements Fulfilling specifications does not guarantee success
  4. 4. Meeting the needs with Acceptance TDD Acceptance Test Implementation Requirement Feedback Drive implementation of a requirement through a set of automated, executable acceptance tests
  5. 5. ATDD in a Nutshell Real-world examples to build a shared understanding of the domain Select a set of these examples to be a specification and an acceptance test suite Automate the verification of acceptance tests Focus the software development effort on the acceptance tests Use the set of acceptance tests to facilitate discussion about future change requests.
  6. 6. The ATDD cycle •Implementation of tests usually coding occur before the code is done testing • Feature not done until test passes Feature Specification architecture Done Workshop customer documentation other activities • Activities happen in parallel Developers Testers •Team clarifies and implement the feature Example Product Owner Architect tests together Technical writers •Tests are executable at the end of the ATDD Workshop
  7. 7. Benefits of ATDD Comprehensible examples over complex formulas Close Collaboration Definition of Done Trust and Commitment Testing on system level
  8. 8. Specification by Examples Use realistic examples to demonstrate differences in possibilities instead of abstract requirements Write specifications down as tables Workflows: Preconditions Processing steps Verifications
  9. 9. Examples, Tests, and Spec can become Examples Tests ela rify bo ve ra te Requirements
  10. 10. Specification workshop Ask the domain experts Developers and testers should suggest examples of edges or important issues for discussion Ubiquitous language Organise feedback to ensure shared understanding Use facilitator to stay focused if needed
  11. 11. Distilling the specifications Write tests collaboratively Examples in a form close to what your automation tool can understand Keep tests in a form that is human-readable Specification over Scripting, describe WHAT, not how Acceptance tests to prevent defects, not to discover Not necessarily automate anything Acceptance tests only work when we can discuss them
  12. 12. Some considerations User Interface Easy? Fragile? Performance issues? Boundary of Stub Sufficiently close Simulators? Business logic Not from developer perspective
  13. 13. Acceptance Test smells Long tests Parameters of calculation tests that always have the same value Similar test with minor differences Tests that reflect the way code was written Tests fail intermittently even though you didn’t change any code Interdependent tests, e.g. setup for others
  14. 14. Change Use existing acceptance tests to discuss future changes Seek advices from customer to determine if it specifies obsolete functionality when test fails Automate periodic execution of regression tests with CI Keep tests in the same version control as code
  15. 15. Tools Table-based frameworks FIT, http://fit.c2.com RobotFramework, http://robotframework.org Text-based frameworks Exactor, http://exactor.sourceforge.net TextTest, http://texttest.carmen.se
  16. 16. FIT FIT stands for “Framework for Integrated Tests” Most popular framework in-use Table-based Supporting languages like Java, C/C++, C#, Python, or Ruby FitLibrary to extend FIT
  17. 17. FIT in practice Test doc with tables Customer writes a Technical staff enhance test document the tables in the doc containing examples Test doc with sanitised tables Technical staff Executable Test implements fixture classes Test doc and backing code (e.g. Java)
  18. 18. FIT Document Customers write acceptance tests in their own language using their own tools.
  19. 19. Fixtures Take information from the tables and turn them into method calls on the actual application. ColumnFixture - checking rules and calculations ActionFixture - Step-by-step processing RowFixture - Checking sets of data
  20. 20. Fixture example
  21. 21. Robot Framework Python-based Keyword-driven test automation framework Test libraries implemented either in Python or Java (with remote library available in many other languages thru XML- RPC) Test cases are written in tabular format, save in HTML or TSV files Syntax similar to natural language Users can create new keywords from existing ones and contribute to the project
  22. 22. Preparing Test cases Workflow tests - takes in initial state, action, verified that the system behaved as expected Test procedure using keywords Keyword arguments Test case name Test Case Action Argument Valid Login Open Login Page Input Name demo Input Password mode Submit Credentials
  23. 23. Higher-level test cases Free text suitable for communication even with non- technical customers Support given-when-then format in BDD
  24. 24. Data-driven test cases Define a keyword which will take the input data and prepare a table with test cases
  25. 25. Test case organisation Simple way: Single HTML file containing all test cases Test case tagging
  26. 26. Execution Gathering test cases, reading and setting variables Executing all actions for every test case Providing global statistics Writing the output in XML format Generating report and log in HTML format
  27. 27. Sample execution result
  28. 28. Sample Test Report in HTML format, showing all actions executed up to the failing action, with fail message
  29. 29. Tested application Interface Command line OperatingSystem SSHLibrary Telnet library Web Robot Selenium GUI Swing GUI library
  30. 30. Adoption Sense of achievement Integrity Openness Right timing
  31. 31. Facilitate Adoption Evangelise - increase awareness Lower the bar - quick path to the first success Train and educate - from general training to specific needs Share and infect - sit together and sharing sessions Coach and facilitate - learn about yourselves and cope with the findings Involve others - know the stake-holder groups and give them roles
  32. 32. Ideal candidate to work with Shared interest in success Authority to make decision Ability to understand implications Ability to explain the domain
  33. 33. Variations - an escape? Behaviour-driven development Example-driven development Executable specifications Names do not matter, but underlying practices matter Worthwhile to try if your business people do not like “testing”
  34. 34. Organisational Challenges Business Analysts Great choice for the facilitator of the spec workshop Working with Acceptance tests makes BA’s work easier Testers Help people avoid problems, not to discover them Automation of acceptance test gives testers more time to test manual things Developers Spec workshop gives a good chance to discuss with domain experts Both unit tests and acceptance tests are important
  35. 35. References Bridging the Communication Gap Gojko Adzic Practical TDD and ATDD for Java Developers Lasse Koskela Agile Testing Lisa Crispin and Janet Gregory
  36. 36. Thank you! Steven Mak Email: steven@odd-e.com Twitter: http://twitter.com/stevenmak

×