Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Tutorial: Implementing Specification-By-Example with Gherkin

27,721 views

Published on

1/2 day Tutorial held at XP 2013 conference in Vienna

Many teams struggle with the implementation of user story acceptance criteria and having a shared understanding about the expected story outcomes. This often results in missed stakeholder expectations, ad-hoc assumptions made by the team during implementation and conflict between team members and the product owner around testing.

In this tutorial, you will learn how specification-by-example and acceptance test driven development will address team conflict, missed stakeholder expectations and overall increasing the level of clarity on the project end-to-end. The presentation will cover the theory behind ATDD, case-studies and practical experience from real projects and several hands-on exercises to try out the presented concepts.

You will leave this tutorial with a fundamental understanding of specification-by-example and its benefits, as well as concrete pointers on how to get started using it in your own projects.

Published in: Technology

Tutorial: Implementing Specification-By-Example with Gherkin

  1. 1. CHRISTIAN HASSA (CH@TECHTALK.AT) TWITTER: @CHRISHASSA XP 2013 Vienna, June 3rd 2013 Specification-By-Example with Gherkin
  2. 2. About me • Managing Partner at TechTalk • Working as/coaching Product Owners TechTalk office Vienna/Austria
  3. 3. • Agile consulting and delivery • Offices: Vienna, Budapest, Zurich TechTalk Team About my Team
  4. 4. 4 Refining User Stories
  5. 5. 5 Impact Mapping Story Mapping Specification-By-Example Establishing a shared understanding Why? How? Code Acceptance Criterion Epic Capability Impact, Goal Easier to define upfront Harder to define upfront Bug report Isolated, formalized example Feature User Story Example Reminder for a discussion
  6. 6. 6 Collecting Acceptance Criteria “I would try to put a book into the shopping cart …” “I would try to remove a book from the shopping cart…” “I’d check whether the shopping cart is empty, when I enter the shop …” “I would try to add the same book again to the shopping cart …” Books can be placed into shopping cart. Books can be removed from shopping cart. Shopping cart should be empty when entering the shop. Adding same book again to shopping cart should increase quantity. As a potential customer I want to collect books in a shopping cart So that I can order several books at once. “Imagine this story is already implemented: how would you verify it?”
  7. 7. 7 Example: Mobile Ticket for Wiener Linien
  8. 8. 8 Exercise: Smart Alarm • What would you try out when the following user story was done? • Collect and write down the list of discussed acceptance criteria • Work in groups of 2-3 As a regular traveler I want to setup a smart alarm so that I can be warned about service disruptions I want to setup multiple smart alarms so that I can plan ahead for different travel plans I have
  9. 9. 9 Using examples
  10. 10. 10 UI wire frames, existing UI rules, key examples existing artifacts, samples Examples for user stories
  11. 11. 11 Specification-by-Example Examples … • make abstract descriptions better understandable However … • examples are usually not formally exchanged or documented Brian Marick Examples Tests Requirements consist of describe verify fulfillment of
  12. 12. 12 Discussion of acceptance criteria public void TestInitialOrderDiscount() { Customer newCustomer = new Customer(); Order newOrder = new Order(newCustomer); newOrder.AddBook( Catalog.Find(“ISBN-0955683610”) ); Assert.Equals(33.75, newOrder.Subtotal); } Register as “bart_bookworm” Go to “/catalog/search” Enter “ISBN-0955683610” Click “Search” Click “Add to Cart” Click “View Cart” Verify “Subtotal” is “$33.75” We would like to encourage new users to buy in our shop. Therefore we offer 10% discount for their first order. Original idea for the illustration: George Dinwiddie http://blog.gdinwidiee.com
  13. 13. 13 … illustrated with formalized examples Given the user has not ordered yet When the user adds a book with the price of EUR 37.5 into the shopping cart Then the shopping cart sub-total is EUR 33.75. Original idea for the illustration: George Dinwiddie http://blog.gdinwidiee.com
  14. 14. 14 Discover hidden assumptions Actually, this is not quite right: Books on sale should be excluded. Original idea for the illustration: George Dinwiddie http://blog.gdinwidiee.com
  15. 15. 15 Collaboration: 3 amigos “Happy Path” Technical feasability Exceptions, border cases Original idea for the illustration: George Dinwiddie http://blog.gdinwidiee.com
  16. 16. 16 Abstract acceptance criteria As a shop visitor I want to collect books in my shopping basket so that I can purchase multiple books at once. Books can be added to the shopping basket Books can be removed from the shopping basket Shopping basket is initially empty The same book can be added multiple times to the shopping basket
  17. 17. 17 Examples for acceptance criteria As a shop visitor I want to collect books in my shopping basket so that I can purchase multiple books at once. Books can be added to the shopping basket Given my shopping basket is empty When I add the book “Harry Potter” to my shopping basket Then my shopping basket should contain 1 copy of “Harry Potter”
  18. 18. 18 As a shop visitor I want to collect books in my shopping basket so that I can purchase multiple books at once. Books can be added to the shopping basket Examples for acceptance criteria Given my shopping basket contains 1 copy of “Harry Potter” When I add the book “Harry Potter” to my shopping basket Then my shopping basket should contain 2 copies of “Harry Potter” The same book can be added multiple times to the shopping basket
  19. 19. 19 The same book can be added multiple times to the shopping basket Structure of examples Given my shopping basket contains 1 copy of “Harry Potter” When I add the book “Harry Potter” to my shopping basket Then my shopping basket should contain 2 copies of “Harry Potter” Title: Describes intention/abstract acceptance criterion Arrange: Context, initial state of the system Act: Execution of the feature Assert: Assertion of observable behaviour And I should see the warning: “Book already existed in basket” Triple-A constraint “Checks” Chaining up steps
  20. 20. 20 Gherkin
  21. 21. 21 Feature: Description of feature or user story Szenariogrundriss: Beschreibung des Akzeptanzkriteriums Szenario: Beschreibung des Akzeptanzkriteriums Gherkin Feature Files Background: context for all scenarios in the feature file Scenario: Description of acceptance criterion Angenommen/Wenn/Dann: Automatisierte Szenario Schritte Given/When/Then: automated scenario steps Scenario Outline: Description of acceptance criterion Angenommen/Wenn/Dann: Automatisierte Szenario Schritte Given/When/Then: automated scenario steps with <place holder> Examples: table with examples for <place holders>Examples: table with examples for <place holders> Given: automated scenario steps @tagname @tagname @tagname @tagname
  22. 22. 22 Feature Feature: Description of feature or user story #language:en-en @done Feature: Shopping cart As visitor of the web shop I want to collect books in a shopping cart so that I can - prepare a short list of books I want to order - combine multiple books in one shipment @Tagname
  23. 23. 23 Scenario Scenario: Description of acceptance criterion Angenommen/Wenn/Dann: Automatisierte Szenario Schritte Given/When/Then: automated scenario steps @Tagname @inprogress Scenario: The same book can be added multiple times to shopping cart Given my shopping cart contains 1 copy of "Harry Potter" When I add the book "Harry Potter" to the shopping cart Then my shopping cart contains 2 copies of "Harry Potter" And a warning is displayed: "You have added the same book again"
  24. 24. 24 Scenario Outline Scenario Outline: Description of acceptance criterion Angenommen/Wenn/Dann: Automatisierte Szenario Schritte Examples: table with examples for <place holders> Given/When/Then: automated scenario steps with <place holder> Examples: table with examples for <place holders> @Tagname @Tagname Scenario Outline: Simple search When I search for books by the phrase '<search phrase>' Then the list of found books should contain only: <books> Examples: | search phrase | books | Explanation | | Domain | 'Domain …' | whole words are matched | | Analysis Communication | 'Bridging…', 'Analysis …' | multiple words are matched with OR | Scenario Outline: Candidate profile can only be saved with <mandatory field> When I enter a valid candidate profile, without filling in '<mandatory field>' Then I see the following validation error: 'field <mandatory field> missing' Examples: Main fields | mandatory field | | email | | name | | first name | Examples: Extended fields | mandatory field | | birthdate |
  25. 25. 25 Background Background: context for all scenarios in the feature file Given: automated scenario steps Background: Given the catalog contains the following books | Title | Author | | Specification-By-Example | Gojko Adzic | | Agile Testing | Lisa Crispin, Janet Gregory | | Scrum Field Guide | Mitch Lacey |
  26. 26. 26 Scenario step arguments Given the catalog contains the following books | Title | Author | | Specification-By-Example | Gojko Adzic | | Agile Testing | Lisa Crispin, Janet Gregory | | Scrum Field Guide | Mitch Lacey | When I add the book "Harry Potter" to the shopping cart Scalar arguments: Table arguments:
  27. 27. 27 Exercise • Work in same previous groups of 2-3 • Pick an acceptance criteria from the previously compiled list • Outline usage examples with the following structure • Given: Context, initial state • When: utilization of the feature • Then: assertions of observable behaviour • Validate that scenario title (acceptance criterion) still describes intention of formalized scenario
  28. 28. 28 Life time of examples
  29. 29. 29 Purpose of the examples • Shared understanding: acceptance criteria • Documentation: system details • Regression-tests: violated assumptions
  30. 30. 30 Continuous validation with automation Given my shopping basket contains 1 copy of “Harry Potter” When I add the book “Harry Potter” to my shopping basket Then my shopping basket should contain 2 copies of “Harry Potter” System „Step Definitions“ are binding individual steps to an automatable interface of the application. Automatable interface UI Automation Automation does not necessarily have to bind to the UI. Automatability of system is supported/evolving with development.
  31. 31. 31 Demo http://www.specflow.org Gherkin automation for .NET • Visual Studio plugin (VS-Gallery) • NuGet Package
  32. 32. 32 Some more complex exercises Choose between • Best-fare option ticket • Smart Alarm
  33. 33. 33 Best-fare option ticket • Assumptions • Ticket price is always adjusted during validation to the best option (considering all previous trips) • Feedback about cost of travel and ticket option (type) after each validation • Valid in the core zone 100 • Scope of the story • Different travel scenarios and how the price adjusts • Billings resulting from the user’s travel • Feedback to traveler (cost of travel, status of ticket) when validating ticket • Out of scope for this story • Credit limit of traveler • Authentication of ticket (validity) As an ad-hoc traveler I want my ticket to adjust to the best available fare whenever I validate So that I don’t need to study upfront which ticket I need Ticket Type Price Single trip 2,10 24 hours 7,10 48 hours 12,40 72 hours 15,40 8 days 35,80 Week 15,00 Month 45,00 Year 365,00
  34. 34. 34 Smart Alarm • Assumptions • User can specify route to monitor for disruptions • User can specify when disruptions should trigger the alarm earlier (minutes before alarm) • Scope of the story • Different possibilities of disruptions provided by GTFS and their impact on activating the alarm • Out of scope for this story • Configuring routes (the user is selecting an already defined route) • Setting up alarms (the user has already setup an alarm) As a regular traveler I want to have a smart alarm that activates earlier when there are traffic disruptions on my route So that I can travel earlier to be still on time for my appointment
  35. 35. 35 Exercise Choose between • Best-fare option ticket • Smart Alarm Collect and discuss scenarios • What would you try out? • Collect examples • Limit scope of the story
  36. 36. 36 Exercise Refine scenarios and formalize into Gherkin • Start with a “warm-up” scenario • Increase complexity with additional scenarios • Reconsider structure of scenarios as you refine them • Validate scenario title/description after refinement
  37. 37. 37 Impact on testing
  38. 38. 38 Test automation becomes expensive when … • trying to automate manual tests • making tests unreadable when automating them • automating after completing development structure readability point in time
  39. 39. 39 Structure Manual tests Asserts Multiple combined features Structure ACT-ASSERT- ACT-ASSERT- ACT-ASSERT- … Dependent features Long test path with high chance to break Cause and impact of error hard to trace Automated Check Single aspect of a single feature ARRANGE – ACT – ASSERT Independent features Short test path with lower chance to break Cause and impact of error easy to relate
  40. 40. 40 Test automation pyramid User journeys Acceptance- criteria Units exploratory testing Source: Mike Cohn many few hard easy Automatability
  41. 41. 41 // Go to web page 'http://localhost:40001/' using new browser instance BrowserWindow localhostBrowser = BrowserWindow.Launch( new System.Uri(this.RecordedMethod1Params.Url)); // Click 'Register found item' link Mouse.Click(uIFundstückerfassenHyperlink, new Point(56, 9)); // Click 'Save' button Mouse.Click(uISpeichernButton, new Point(44, 14)); int fundNr1 = int.Parse(uIFundNr127Pane.InnerText.Substring(9)); // Click 'Register found item' link Mouse.Click(uIFundstückerfassenHyperlink, new Point(63, 7)); // Click 'Save' button Mouse.Click(uISpeichernButton, new Point(34, 11)); int fundNr2 = int.Parse(uIFundNr128Pane.InnerText.Substring(9)); Assert.IsTrue(fundNr1 + 1 == fundNr2); // Click 'Close' button Mouse.Click(uICloseButton, new Point(26, 11)); Readability
  42. 42. 42 A readable test case Scenario: New found items should receive a consecutive number for the current year Given the previous found item of the current year had the number 145 When I register a new found item Then the last found item of the current year should have the number 146
  43. 43. 43 When to test (point in time) Acceptance criteria (ATDD, BDD) Unit Tests (TDD) business view technical view Exploratory tests Workflow tests Performance, Scalability, Usability,Security, … definingtheproduct criticizingtheproduct New dimension: defining the product Synergy: Specification of requirements and tests Agile Testing Quadrants: Brian Marick
  44. 44. 44 Manual Testing is always necessary! User journeys Acceptance- criteria Units exploratory testing Source: Mike Cohn many few harder easier Automatability Manual Check after Story Done Main success pathes Undiscovered acceptance criteria No/(few) manual regression checks Few pathes are enough More time for exploration
  45. 45. 47 Impact on development
  46. 46. 48 Workflow - TDD
  47. 47. 49 Extending TDD for business
  48. 48. 50 Transparency for stakeholders In Progress
  49. 49. 51 Current sprint report: all sprint scenarios
  50. 50. 52 Starting with first scenario (AC)
  51. 51. 53 Finishing the first scenario (AC)
  52. 52. 54 Progressing scenario after scenario
  53. 53. 55 Progressing scenario after scenario
  54. 54. 56 Progressing scenario after scenario
  55. 55. 57 Implementing user stories in parallel
  56. 56. 58 First user story ready for testing
  57. 57. 59 Manual testing can start even earlier
  58. 58. 60 Work done already can break again
  59. 59. 61 Work done already can break again
  60. 60. 62 See what is temporarily not working
  61. 61. 64 Living documentation
  62. 62. 65 Living documentation Drill into Details (Gherkin scenarios) Overview (Story Map)
  63. 63. 66 User Stories vs. Features Product/Sprint Backlog User Story 1 AccCrit 1 AccCrit 2 User Story 2 AccCrit 3 AccCrit 4 Living Documentation Feature 1 AccCrit 1 AccCrit 2 Feature n AccCrit 4 AccCrit m User Story n AccCrit 5 AccCrit m AccCrit 3 AccCrit 5 „Done“ • Future options of the system • Organized/refined according to priority, value, effort, risk, ... • Next possible increments of the product (units of work) • Current state of the system • Organized/refined for functional overview • Versioned and maintained together with source code
  64. 64. 67 Lessons learned
  65. 65. 68 Collaboration over specification Documentation after conversation • Collaborative discovery • Should trigger new questions • Infinite number of examples • Readability and ability to automate
  66. 66. 69 Collaboration smells • Silos and too formal hand-overs • Formalization cannot follow implementation • Scenarios replace communication • Scenarios inhibit consideration of solutions • Scenarios inhibit discovery of new things • Duty after development is done • Unreadable scenarios without value • High cost for creation and maintenance • Technical problems with automation
  67. 67. 70 Level of automation Controller Business Layer Data Layer Model View Browser automation Trigger behaviour through controller Assert behaviour on model, db, .. Setup pre-conditions through service interfaces Out-of-process In-process
  68. 68. 71 Test execution performance • Grouping tests • Current WIP • Completed work • Database • In-memory • Templates for setup • Parallel execution • Smart execution order
  69. 69. 72 Internal vs. external DSL Example Source: Liz Keogh https://github.com/lunivore/tictactoe-java/blob/master/ scenarios/com/lunivore/tictactoe/scenarios/ Three_in_a_row_wins.java
  70. 70. 73 Non-functional acceptance criteria Given there are 100,000 users registered on the system When I create a new account Then I should be taken to my dashboard within 5ms Given 1000 users are hitting the homepage simultaneously Then each user should get a response within 2ms Matt Wynne http://blog.mattwynne.net/2012/03/13/using-cucumber-for-load-testing
  71. 71. 74 Tools
  72. 72. 75 Gherkin automation tools www.cukes.info www.behat.org Ruby, Java, JavaScript, C++ www.specflow.org .NET, Mono, Silverlight, WP7 PHP
  73. 73. 76
  74. 74. 77 Books Gojko Adzic Bridiging the Communication Gap Gojko Adzic Specification by Example Elisabeth Hendrickson Explore IT!
  75. 75. 78 Christian Hassa: ch@techtalk.at - @chrishassa

×