Acceptance Test Driven
Development in practice
Steven Mak
steven@odd-e.com
What are we up to now?
Lost in translation
Do not explain why
Gaps discovered only until coding started
Cumulative effects...
Failing to meet actual needs
 are obvious things really obvious?




 Imperative requirements




 Fulfilling specifications...
Meeting the needs with
Acceptance TDD

                   Acceptance Test              Implementation
    Requirement     ...
ATDD in a Nutshell
Real-world examples to build a shared understanding of the
domain
Select a set of these examples to be ...
The ATDD cycle
                                                           •Implementation of tests usually
               ...
Benefits of ATDD
Comprehensible examples over complex formulas
Close Collaboration
Definition of Done
Trust and Commitment
T...
Specification by Examples
Use realistic examples to demonstrate differences in
possibilities instead of abstract requiremen...
Examples, Tests, and Spec
             can become
 Examples                    Tests
  ela




                           ...
Specification workshop
Ask the domain experts
Developers and testers should suggest examples of
edges or important issues f...
Distilling the specifications
 Write tests collaboratively
 Examples in a form close to what your automation tool can
 unde...
Some considerations
User Interface
  Easy?

  Fragile?

  Performance issues?

Boundary of Stub
  Sufficiently close

  Sim...
Acceptance Test smells
Long tests
Parameters of calculation tests that always have the same
value
Similar test with minor ...
Change
Use existing acceptance tests to discuss future
changes
Seek advices from customer to determine if it specifies
obso...
Tools
Table-based frameworks
  FIT, http://fit.c2.com

  RobotFramework, http://robotframework.org

Text-based frameworks
 ...
FIT
 FIT stands for “Framework for Integrated Tests”
 Most popular framework in-use
 Table-based
 Supporting languages lik...
FIT in practice
                          Test doc
                         with tables

    Customer writes a
           ...
FIT Document
Customers write acceptance tests in their own
language using their own tools.
Fixtures
 Take information from the tables and turn them into
 method calls on the actual application.
 ColumnFixture - ch...
Fixture example
Robot Framework
Python-based Keyword-driven test automation framework
Test libraries implemented either in Python or Java ...
Preparing Test cases
Workflow tests - takes in initial state, action, verified
that the system behaved as expected
         ...
Higher-level test cases
 Free text suitable for communication even with non-
 technical customers
 Support given-when-then...
Data-driven test cases
 Define a keyword which will take the input data and
 prepare a table with test cases
Test case organisation
 Simple way: Single HTML file containing all test cases
 Test case tagging
Execution
Gathering test cases, reading and setting variables
Executing all actions for every test case
Providing global s...
Sample execution result
Sample Test Report
in HTML format, showing all actions executed up to the
failing action, with fail message
Tested application Interface
 Command line
   OperatingSystem
   SSHLibrary
   Telnet library
 Web
   Robot Selenium
 GUI
...
Adoption
Sense of achievement

Integrity

Openness

Right timing
Facilitate Adoption
 Evangelise - increase awareness
 Lower the bar - quick path to the first success
 Train and educate - ...
Ideal candidate to work with
 Shared interest in success
 Authority to make decision
 Ability to understand implications
 ...
Variations - an escape?
 Behaviour-driven development
 Example-driven development
 Executable specifications
 Names do not ...
Organisational Challenges
Business Analysts
  Great choice for the facilitator of the spec workshop

  Working with Accept...
References
Bridging the Communication Gap
  Gojko Adzic

Practical TDD and ATDD for Java Developers
  Lasse Koskela

Agile...
Thank you!
                                  Steven Mak
                   Email: steven@odd-e.com
       Twitter: http://...
Upcoming SlideShare
Loading in...5
×

ATDD in Practice

34,905

Published on

A gentle introduction to Acceptance Test Driven Development

Published in: Technology, Education
2 Comments
38 Likes
Statistics
Notes
No Downloads
Views
Total Views
34,905
On Slideshare
0
From Embeds
0
Number of Embeds
10
Actions
Shares
0
Downloads
956
Comments
2
Likes
38
Embeds 0
No embeds

No notes for slide




































  • ATDD in Practice

    1. 1. Acceptance Test Driven Development in practice Steven Mak steven@odd-e.com
    2. 2. What are we up to now? Lost in translation Do not explain why Gaps discovered only until coding started Cumulative effects of small misunderstandings Inadequate and essentially flawed requirements and specifications
    3. 3. Failing to meet actual needs are obvious things really obvious? Imperative requirements Fulfilling specifications does not guarantee success
    4. 4. Meeting the needs with Acceptance TDD Acceptance Test Implementation Requirement Feedback Drive implementation of a requirement through a set of automated, executable acceptance tests
    5. 5. ATDD in a Nutshell Real-world examples to build a shared understanding of the domain Select a set of these examples to be a specification and an acceptance test suite Automate the verification of acceptance tests Focus the software development effort on the acceptance tests Use the set of acceptance tests to facilitate discussion about future change requests.
    6. 6. The ATDD cycle •Implementation of tests usually coding occur before the code is done testing • Feature not done until test passes Feature Specification architecture Done Workshop customer documentation other activities • Activities happen in parallel Developers Testers •Team clarifies and implement the feature Example Product Owner Architect tests together Technical writers •Tests are executable at the end of the ATDD Workshop
    7. 7. Benefits of ATDD Comprehensible examples over complex formulas Close Collaboration Definition of Done Trust and Commitment Testing on system level
    8. 8. Specification by Examples Use realistic examples to demonstrate differences in possibilities instead of abstract requirements Write specifications down as tables Workflows: Preconditions Processing steps Verifications
    9. 9. Examples, Tests, and Spec can become Examples Tests ela rify bo ve ra te Requirements
    10. 10. Specification workshop Ask the domain experts Developers and testers should suggest examples of edges or important issues for discussion Ubiquitous language Organise feedback to ensure shared understanding Use facilitator to stay focused if needed
    11. 11. Distilling the specifications Write tests collaboratively Examples in a form close to what your automation tool can understand Keep tests in a form that is human-readable Specification over Scripting, describe WHAT, not how Acceptance tests to prevent defects, not to discover Not necessarily automate anything Acceptance tests only work when we can discuss them
    12. 12. Some considerations User Interface Easy? Fragile? Performance issues? Boundary of Stub Sufficiently close Simulators? Business logic Not from developer perspective
    13. 13. Acceptance Test smells Long tests Parameters of calculation tests that always have the same value Similar test with minor differences Tests that reflect the way code was written Tests fail intermittently even though you didn’t change any code Interdependent tests, e.g. setup for others
    14. 14. Change Use existing acceptance tests to discuss future changes Seek advices from customer to determine if it specifies obsolete functionality when test fails Automate periodic execution of regression tests with CI Keep tests in the same version control as code
    15. 15. Tools Table-based frameworks FIT, http://fit.c2.com RobotFramework, http://robotframework.org Text-based frameworks Exactor, http://exactor.sourceforge.net TextTest, http://texttest.carmen.se
    16. 16. FIT FIT stands for “Framework for Integrated Tests” Most popular framework in-use Table-based Supporting languages like Java, C/C++, C#, Python, or Ruby FitLibrary to extend FIT
    17. 17. FIT in practice Test doc with tables Customer writes a Technical staff enhance test document the tables in the doc containing examples Test doc with sanitised tables Technical staff Executable Test implements fixture classes Test doc and backing code (e.g. Java)
    18. 18. FIT Document Customers write acceptance tests in their own language using their own tools.
    19. 19. Fixtures Take information from the tables and turn them into method calls on the actual application. ColumnFixture - checking rules and calculations ActionFixture - Step-by-step processing RowFixture - Checking sets of data
    20. 20. Fixture example
    21. 21. Robot Framework Python-based Keyword-driven test automation framework Test libraries implemented either in Python or Java (with remote library available in many other languages thru XML- RPC) Test cases are written in tabular format, save in HTML or TSV files Syntax similar to natural language Users can create new keywords from existing ones and contribute to the project
    22. 22. Preparing Test cases Workflow tests - takes in initial state, action, verified that the system behaved as expected Test procedure using keywords Keyword arguments Test case name Test Case Action Argument Valid Login Open Login Page Input Name demo Input Password mode Submit Credentials
    23. 23. Higher-level test cases Free text suitable for communication even with non- technical customers Support given-when-then format in BDD
    24. 24. Data-driven test cases Define a keyword which will take the input data and prepare a table with test cases
    25. 25. Test case organisation Simple way: Single HTML file containing all test cases Test case tagging
    26. 26. Execution Gathering test cases, reading and setting variables Executing all actions for every test case Providing global statistics Writing the output in XML format Generating report and log in HTML format
    27. 27. Sample execution result
    28. 28. Sample Test Report in HTML format, showing all actions executed up to the failing action, with fail message
    29. 29. Tested application Interface Command line OperatingSystem SSHLibrary Telnet library Web Robot Selenium GUI Swing GUI library
    30. 30. Adoption Sense of achievement Integrity Openness Right timing
    31. 31. Facilitate Adoption Evangelise - increase awareness Lower the bar - quick path to the first success Train and educate - from general training to specific needs Share and infect - sit together and sharing sessions Coach and facilitate - learn about yourselves and cope with the findings Involve others - know the stake-holder groups and give them roles
    32. 32. Ideal candidate to work with Shared interest in success Authority to make decision Ability to understand implications Ability to explain the domain
    33. 33. Variations - an escape? Behaviour-driven development Example-driven development Executable specifications Names do not matter, but underlying practices matter Worthwhile to try if your business people do not like “testing”
    34. 34. Organisational Challenges Business Analysts Great choice for the facilitator of the spec workshop Working with Acceptance tests makes BA’s work easier Testers Help people avoid problems, not to discover them Automation of acceptance test gives testers more time to test manual things Developers Spec workshop gives a good chance to discuss with domain experts Both unit tests and acceptance tests are important
    35. 35. References Bridging the Communication Gap Gojko Adzic Practical TDD and ATDD for Java Developers Lasse Koskela Agile Testing Lisa Crispin and Janet Gregory
    36. 36. Thank you! Steven Mak Email: steven@odd-e.com Twitter: http://twitter.com/stevenmak
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×