Acceptance tests

Uploaded on

Main concepts about ATDD (Acceptance Test Driven Development)

Main concepts about ATDD (Acceptance Test Driven Development)

More in: Education
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads


Total Views
On Slideshare
From Embeds
Number of Embeds



Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

    No notes for slide


  • 1. Acceptance Tests"no codes goes in until the tests go on"MSc. Dragan Tomić
  • 2. Content- what are acceptance tests- why do we need them- who creates them- when create- where use- Details...- Examples- Summary
  • 3. What are acceptance tests● Acceptance tests ○ users point of view ○ external view of system● Examine externally visible effects ○ input and output ○ state changes ○ external interfaces● Implementation independent (there automation may not be)
  • 4. Why do we need them?- Rework down from 60% to 20%- Workflow working first time- Getting business rules right (preventingRework)- Tighter cross-functional team integration- Crisp visible story completion criteria
  • 5. Classical development Elicit Analyze Design requirements requirements Code Test Deploy Why go back?- Misunderstandings...
  • 6. When we create ATs Analyze Elicit requirements with Design requirements Tests Code with Deploy Tests- AT are created when die Requirements are Analysed- Developers code with AT (Acceptance Tests)- Failing AT provides quick feedback
  • 7. Who creates them● Author the test ○ customer, tester, developer together● Connect tests to the system Customer Tester ○ developer or technical testers Developer● Run the tests ○ developer than tester ○ Manual or... ○ Automated (allows regression tests)
  • 8. Tests- Acceptance tests not substitute for interactivecommunication ○ provides focus for this communication- Tests in business domain terms ○ shared between customer and developer (common language)
  • 9. Details- Unit tests are not enough- Objective of acceptance tests (business-facing, notdeveloper-facing, deliver value to Applications user)- Too expensive or not?- Automated Acceptance Test best practices- The whole team owns the acceptance tests
  • 10. Creating and maintaining effective AATs● Four sections: ○ creating acceptance tests ○ creating an application driver layer ○ implementing acceptance tests ○ maintaining acceptance test suites● INVEST principles Independent, Negotiable, Valuable, Estimable, Small, Testable
  • 11. Creating AATs Given - state of app when test case begins When - interaction between app and user Then - state of app after that interaction is complete (Cucumber, JBehave, Concordion, Twist or xUnit) DSL - domain-specific language API for the application driver layer translates "high-level" commands to real application dataAdvantages: AT independance, simple test data creation
  • 12. Example 1 (Acceptance Criteria)Feature: Placing an orderScenario:User order should debit account correctlyGiven there is an instrument called bondAnd there is a user called Dave with 50 dollars in his accountWhen I log in as DaveAnd I select the instrument bondAnd I place an order to buy 4 at 10 dollars eachAnd the order is successfulThen I have 10 dollars left in my accountIn Cucumber we would save this AC in a file features/placing_an_order.features
  • 13. Example 1 (Implementation layer)Then we creating Ruby file features/step_definitions/placing_an_order_steps.rbrequire application_driver/admin_apirequire application_driver/trading_uiBefore do @admin_api = @trading_ui = TradingUi.newendGiven /^there is an instrument called (w+)$/ do |instrument| @admin_api.create_instrument(instrument) endGiven /^there is a user called (w+) with (w+) dollars in his account$/ do |user, amount| @admin_api.create_user(user, amount) endWhen /^I log in as (w+)$/ do |user| @trading_ui.login(user) endWhen /^I select the instrument (w+)$/ do |instrument| @trading_ui.select_instrument(instrument) endWhen /^I place an order to buy (d+) at (d+) dollars each$/ do |quantity, amount| @trading_ui.place_order(quantity, amount) endWhen /^the order for (d+) of (w+) at (d+) dollars each is successful$/ do |quantity, instrument, amount| @trading_ui.confirm_order_success(instrument, quantity, amount) endThen /^I have (d+) dollars left in my account$/ do |balance| @trading_ui.confirm_account_balance(balance) end
  • 14. Example 1 (Application driver layer)Feature: Placing an order Scenario: User order debits account correctly # features/placing_an_order.feature:3 Given there is an instrument called bond # features/step_definitions/placing_an_order_steps.rb:9 And there is a user called Dave with 50 dollars in his account # features/step_definitions/placing_an_order_steps.rb:13 When I log in as Dave # features/step_definitions/placing_an_order_steps.rb:17 And I select the instrument bond # features/step_definitions/placing_an_order_steps.rb:21 And I place an order to buy 4 at 10 dollars each # features/step_definitions/placing_an_order_steps.rb:25 And the order for 4 of bond at 10 dollars each is successful # features/step_definitions/placing_an_order_steps.rb:29 Then I have 10 dollars left in my account # features/step_definitions/placing_an_order_steps.rb:331 scenario (1 passed)7 steps (7 passed)0m0.016s
  • 15. Example 1 - RecapThis is the process for creating acceptance tests:● Discuss acceptance criteria with your customer● Write them down in the executable format described in the Example 1● Write an implementation for the test which uses only the domain language, accessing the application driver layer● Create an application driver layer which talks to the system under test
  • 16. Example 1 expressed as JUnit testpublic class PlacingAnOrderAcceptanceTest extends DSLTestCase { @Test public void userOrderShouldDebitAccountCorrectly() { adminAPI.createInstrument("name: bond"); adminAPI.createUser("Dave", "balance: 50.00"); tradingUI.login("Dave"); tradingUI.selectInstrument("bond"); tradingUI.placeOrder("price: 10.00", "quantity: 4"); tradingUI.confirmOrderSuccess("instrument: bond", "price: 10.00", "quantity: 4"); tradingUI.confirmBalance("balance: 10.00"); }}
  • 17. Managing Asynchrony and TimeoutsHas the test failed, or are we just waiting for the result to arrive?Example:[Test]public void ShouldSendEmailOnFileReceipt() { ClearAllFilesFromInbox(); DropFileToInbox(); ConfirmEmailWasReceived();}The email wont have been received by the time we check for its arrival!Solution:DELAY PERIOD or some service that handles incoming emails
  • 18. The Acceptance Test stage- A build that fails the acceptance tests will not be deployable- AT threshold is extremely important- Without excellent AT one of 3 things happens: ○ lot of time is spent fixing bugs ○ too much time and money spent on manual acceptance and regression testing ○ end up releasing poor-quality sofware- Keeping acceptance tests green (when breaks -> team STOP)- Fail fast (if it is going to fail)
  • 19. Acceptance Test Performance- Performance is not a primary concern(its more important to have a comprehensive automated test suite)- Typically, they take hours to complete- But, there is a spectrum of techniques forimprovement ○ parallel testing ○ refactor common tasks ○ share expensive resources ○ using compute grids (use cloud computing)
  • 20. Summary- AT simulate user interaction with the system in order toensure business requirements- Providing protection against large-scale changes tosystem- Significantly improving quality through comprehensiveautomated regression testing- Providing fast and reliable feedback whenever a defectoccurs so that it can be fixed immediately
  • 21. Thank you for your attention!!!