Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Specification-By-Example with Gherkin

17,750 views

Published on

Many teams struggle with the implementation of user story acceptance criteria and having a shared understanding about the expected story outcomes. This often results in missed stakeholder expectations, ad-hoc assumptions made by the team during implementation and conflict between team members and the product owner around testing.

This session shows how specification-by-example and acceptance test driven development will address team conflict, missed stakeholder expectations and overall increasing the level of clarity on the project end-to-end. The presentation will cover the theory behind ATDD and case-studies and practical experience from real projects.

The talk was held at the ALM summit 3 in Redmond, January 2013. Recording of the talk can be found here: http://channel9.msdn.com/Events/ALM-Summit/ALM-Summit-3/Implementing-ATDD-and-Specification-By-Example

Specification-By-Example with Gherkin

  1. 1. Specification-By-Example with Gherkin Christian Hassa,TechTalk (ch@techtalk.at, @chr99ha)
  2. 2. Almost 4 years later … #106 inVisual StudioGallery (30.1.2013) #135 in NuGet (30.1.2013) ~800 visits daily ~ 4000 active users > 30 contributors
  3. 3. Goals Make you curious about Specification-By-Example New perspective for SpecFlow users
  4. 4. About me • Managing Partner atTechTalk • Agile consulting and delivery • Based in Europe: Vienna, Budapest, Zurich
  5. 5. Terminology • Specification-By-Example • AcceptanceTest Driven Development (ATDD) • Behaviour Driven Development (BDD)
  6. 6. Agile requirements
  7. 7. Purpose of user stories? • Describe user needs • Unit of prioritization • Unit of planning • Reminder for a discussion • Mechanism to defer detail
  8. 8. Requirement levels Impact Mapping Story Mapping Specification-By-Example Why? How? Acceptance Criterion Epic Deliverable, Outcome Impact, Goal Easier to define upfront Harder to define upfront Reminder for a discussion Bug report Isolated, formalized example User Activity User Story Example Code
  9. 9. Collecting acceptance criteria “I would try to put a book into the shopping cart …” “I would try to remove a book from the shopping cart…” “I’d check whether the shopping cart is empty, when I enter the shop …” “I would try to add the same book again to the shopping cart …” Books can be placed into shopping cart. Books can be removed from shopping cart. Shopping cart should be empty when entering the shop. Adding same book again to shopping cart should increase quantity. As a potential customer I want to collect books in a shopping cart So that I can order several books at once. “Imagine this story is already implemented: What would you try out?”
  10. 10. Using examples
  11. 11. Examples for user stories UI wire frames, existing UI rules, key examples existing artifacts, mock-ups
  12. 12. Discussion of acceptance criteria Original idea for the illustration:George Dinwiddie http://blog.gdinwidiee.com public void TestInitialOrderDiscount() { Customer newCustomer = new Customer(); Order newOrder = new Order(newCustomer); newOrder.AddBook( Catalog.Find(“ISBN-0955683610”) ); Assert.Equals(33.75, newOrder.Subtotal); } Register as “bart_bookworm” Go to “/catalog/search” Enter “ISBN-0955683610” Click “Search” Click “Add to Cart” Click “View Cart” Verify “Subtotal” is “$33.75” We would like to encourage new users to buy in our shop. Therefore we offer 10% discount for their first order.
  13. 13. Specification-By-Example Examples … – make abstract descriptions better understandable – are usually not formally exchanged or documented Brian Marick Examples Tests Requirements consist of describe verify fulfillment of
  14. 14. Discussion of acceptance criteria Original idea for the illustration:George Dinwiddie http://blog.gdinwidiee.com public void TestInitialOrderDiscount() { Customer newCustomer = new Customer(); Order newOrder = new Order(newCustomer); newOrder.AddBook( Catalog.Find(“ISBN-0955683610”) ); Assert.Equals(33.75, newOrder.Subtotal); } Register as “bart_bookworm” Go to “/catalog/search” Enter “ISBN-0955683610” Click “Search” Click “Add to Cart” Click “View Cart” Verify “Subtotal” is “$33.75” We would like to encourage new users to buy in our shop. Therefore we offer 10% discount for their first order.
  15. 15. Illustrated with shared examples Original idea for the illustration:George Dinwiddie http://blog.gdinwidiee.com Given the user has not ordered yet When the user adds a book with the price of USD 37.50 into the shopping cart Then the shopping cart sub-total should be USD 33.75.
  16. 16. Discover new aspects Original idea for the illustration:George Dinwiddie http://blog.gdinwidiee.com Actually, this not quite right: Books on sale should be excluded.
  17. 17. Collaboration: 3 amigos “Happy Path” Technical feasability Exceptions, border cases
  18. 18. Abstract acceptance criteria As a shop visitor I want to collect books in my shopping basket so that I can purchase multiple books at once. Books can be added to the shopping basket Books can be removed from the shopping basket Shopping basket is initially empty The same book can be added multiple times to the shopping basket
  19. 19. Formalizing examples As a shop visitor I want to collect books in my shopping basket so that I can purchase multiple books at once. Books can be added to the shopping basket Given my shopping basket is empty When I add the book “Harry Potter” to my shopping basket Then my shopping basket should contain 1 copy of “Harry Potter”
  20. 20. Formalizing examples As a shop visitor I want to collect books in my shopping basket so that I can purchase multiple books at once. Books can be added to the shopping basketThe same book can be added multiple times to the shopping basket When I add the book “Harry Potter” to my shopping basket Then my shopping basket should contain 2 copies of “Harry Potter” Given my shopping basket contains 1 copy of “Harry Potter”
  21. 21. Example structure The same book can be added multiple times to the shopping basket When I add the book “Harry Potter” to my shopping basket Then my shopping basket should contain 2 copies of “Harry Potter” Given my shopping basket contains 1 copy of “Harry Potter” Title: Describes isolated, focused intention=abstract acceptance criterion Arrange: Context, initial state of the system Act: Execution of the feature Assert: Assertion of observable behavior And I should see the warning: “Book already existed in basket” Triple-A constraint “Checks” Steps can be chained up Evolution of a domain specific test and specification language.
  22. 22. Life time of examples
  23. 23. Purpose of examples • Shared understanding: acceptance criteria • Documentation: system details • Regression-tests: violated assumptions
  24. 24. Continuous validation When I add the book “Harry Potter” to my shopping basket Then my shopping basket should contain 2 copies of “Harry Potter” Given my shopping basket contains 1 copy of “Harry Potter” „Step Definitions“ are binding individual steps to an automatable interface of the application. Automation does not necessarily have to bind to the UI. Automation of system is supported/evolving with development. SystemUI Automation
  25. 25. Demo Gherkin automation for .NET • Visual Studio plugin (VS-Gallery) • NuGet Package http://www.specflow.org
  26. 26. Impact on testing
  27. 27. Test automation becomes expensive, when … • trying to automate manual tests • making tests unreadable when automating them • automating only after completing implementation structure readability when to test
  28. 28. Structure Manual tests Asserts Multiple combined features Structure ACT-ASSERT- ACT-ASSERT- ACT-ASSERT- … Dependent on existence of other features Long test path with high chance to break Cause and impact of an error hard to trace. Automated Check Single aspect of a single feature ARRANGE – ACT – ASSERT Independent of other features Short test path with lower chance to break Cause and impact of an error easier to trace.
  29. 29. Test automation pyramid User journeys Acceptance- criteria Units many harder easier Automation few exploratory testing Source: Mike Cohn
  30. 30. Readability // Go to web page 'http://localhost:40001/' using new browser instance BrowserWindow localhostBrowser = BrowserWindow.Launch( new System.Uri(this.RecordedMethod1Params.Url)); // Click 'Register found item' link Mouse.Click(uIRegisterFoundItemHyperlink, new Point(56, 9)); // Click 'Save' button Mouse.Click(uISaveButton, new Point(44, 14)); int foundItemNo1 = int.Parse(uIFundNr127Pane.InnerText.Substring(9)); // Click 'Register found item' link Mouse.Click(uIRegisterFoundItemHyperlink, new Point(63, 7)); // Click 'Save' button Mouse.Click(uISaveButton, new Point(34, 11)); int foundItemNo2 = int.Parse(uIFundNr128Pane.InnerText.Substring(9)); Assert.IsTrue(foundItemNo1 + 1 == foundItemNo2); // Click 'Close' button Mouse.Click(uICloseButton, new Point(26, 11));
  31. 31. A readable test case Scenario: New found items should receive a consecutive number for the current year Given the previous found item of the current year had the number 145 When I register a new found item Then the last found item of the current year should have the number 146
  32. 32. When to test Acceptance criteria (ATDD, BDD) UnitTests (TDD) business view technical view Exploratory tests Workflow tests Performance, Scalability, Usability,Security, … definingtheproduct criticizingtheproduct New dimension: defining the product Synergy: Specification of requirements and tests AgileTesting Quadrants: Brian Marick
  33. 33. Manual testing is still necessary! User journeys Acceptance- criteria Units exploratory testing Manual Check after story done Main success paths Undiscovered acceptance criteria No/(few) manual regression checks Few paths are enough More time for exploration
  34. 34. Cross-functional work Sprint 1 Sprint 2 Sprint 3 short iteration US4 Plan Implement & autom. test US5 Plan Implement & autom. test US2 Plan Implement & autom. test US3 Plan Implement & autom. test US6 Plan Implement & autom. test US1 Plan Implement & autom. test US7 Plan Implement & autom. test US8 Plan Implement & autom. test US9 Plan Implement & autom. test ExploratoryTests Specification and test Collaboration for defining acceptance criteria Collaboration for automation Preventing bugs instead of finding them! Extension of “Test Cases” Limit WIP Collaboration in manual testing
  35. 35. Impact on development
  36. 36. TDD workflow
  37. 37. Extending TDD for business
  38. 38. Transparency for stakeholders In Progress
  39. 39. Current sprint report: all scenarios
  40. 40. Starting with first scenario
  41. 41. Finishing first scenario
  42. 42. Progressing on scenarios
  43. 43. Progressing on scenarios
  44. 44. Progressing on scenarios
  45. 45. Progressing on scenarios
  46. 46. First user story ready for testing
  47. 47. Earlier start of manual testing
  48. 48. Done work can break again
  49. 49. Done work can break again
  50. 50. Business readable error reports
  51. 51. Living documentation
  52. 52. Higher level scenario structure Story Mapping Why? How? Acceptance Criterion Epic Deliverable, Outcome Impact, Goal User Activity User Story Example Code Easier to define upfront Harder to define upfront
  53. 53. Story Maps • Original concept by Jeff Patton • Support iterative product design • Optimized for desired outcome or deliverable the system should support
  54. 54. Building story maps Get popular books delivered fast and conveniently Find book Collect books Order Wait for delivery Receive delivery time browse best sellers shopping basket enter address delivery slip delivery notification pay with credit card search book by title wish list inquiry order status desired outcome or deliverable user activities system features necessity
  55. 55. Walking skeleton Enabling build–measure-learn Get popular books delivered fast and conveniently Find book Collect books Order Wait for delivery Receive delivery time browse best sellers enter address delivery slip pay with credit card search book by title wish list inquiry order status necessity shopping basket delivery notification manual work- around omitted steps desired outcome or deliverable user activities system features
  56. 56. Story Map example
  57. 57. Sprint 1
  58. 58. Sprint 2
  59. 59. Sprint 3
  60. 60. Sprint 4
  61. 61. Dropped Scope
  62. 62. Added scope
  63. 63. User Stories vs. Features Product/Sprint Backlog User Story 1 AccCrit 1 AccCrit 2 User Story 2 AccCrit 3 AccCrit 4 Living Documentation Feature 1 AccCrit 1 AccCrit 2 Feature n AccCrit 4 AccCrit m User Story n AccCrit 5 AccCrit m AccCrit 3 AccCrit 5 „Done“ • Future options of the system • Organized/refined according to priority, value, effort, risk, ... • Next possible increments of the product (units of work) • Current state of the system • Organized/refined for functional overview • Versioned and maintained together with source code
  64. 64. Living documentation • Link business readable automation from source code to story maps • Feed execution results into story maps • Sync Story Story Map requirements withTFS work items
  65. 65. Lessons learned
  66. 66. Collaboration over specification Documentation after conversation • Collaborative discovery • Should trigger new questions • Infinite number of examples • Readability and ability to automate
  67. 67. Level of automation Controller Business Layer Data Layer Model View Browser automation Trigger behavior through controller Assert behavior on model, db, .. Setup pre-conditions through service interfaces Out-of-process In-process
  68. 68. Test execution performance • Grouping tests – Current WIP – Completed work • Database – In-memory – Templates for setup • Parallel execution • Smart execution order • Level of automation
  69. 69. Internal vs. external DSL Example Source: Liz Keogh https://github.com/lunivore/tictactoe-java/blob/master/ scenarios/com/lunivore/tictactoe/scenarios/Three_in_a_row_wins.java
  70. 70. Non-functional acceptance criteria Given there are 100,000 users registered on the system When I create a new account Then I should be taken to my dashboard within 5ms When 1000 users are hitting the homepage simultaneously Then each user should get a response within 2ms Matt Wynne http://blog.mattwynne.net/2012/03/13/using-cucumber-for-load-testing
  71. 71. Tools
  72. 72. Gherkin automation tools www.cukes.info www.behat.org www.specflow.org Ruby, Java, JavaScript, C++ .NET, Mono, Silverlight,WP7 PHP
  73. 73. Books Bridiging the Communication Gap Gojko Adzic Specification by Example Gojko Adzic Explore It! Elisabeth Hendrickson Exploratory Software Testing James Whittaker
  74. 74. Summary Specification-by-example • Collaborative discovery and specification • Impact on development and test • Combine with other agile reqmt methods • Can be supported by tools
  75. 75. Questions Got curious about Specification-By-Example? Learned something new about using SpecFlow? Christian Hassa,TechTalk (ch@techtalk.at, @chr99ha) Join the BoF Session „Specification-By-Example (ATDD, BDD) in practice“ Today at 15h15 (right after this talk)

×