Tutorial: Implementing Specification-By-Example with Gherkin

1. CHRISTIAN HASSA (CH@TECHTALK.AT) TWITTER: @CHRISHASSA XP 2013 Vienna, June 3rd 2013 Specification-By-Example with Gherkin

2. About me • Managing Partner at TechTalk • Working as/coaching Product Owners TechTalk office Vienna/Austria

3. • Agile consulting and delivery • Offices: Vienna, Budapest, Zurich TechTalk Team About my Team

4. 4 Refining User Stories

5. 5 Impact Mapping Story Mapping Specification-By-Example Establishing a shared understanding Why? How? Code Acceptance Criterion Epic Capability Impact, Goal Easier to define upfront Harder to define upfront Bug report Isolated, formalized example Feature User Story Example Reminder for a discussion

6. 6 Collecting Acceptance Criteria “I would try to put a book into the shopping cart …” “I would try to remove a book from the shopping cart…” “I’d check whether the shopping cart is empty, when I enter the shop …” “I would try to add the same book again to the shopping cart …” Books can be placed into shopping cart. Books can be removed from shopping cart. Shopping cart should be empty when entering the shop. Adding same book again to shopping cart should increase quantity. As a potential customer I want to collect books in a shopping cart So that I can order several books at once. “Imagine this story is already implemented: how would you verify it?”

7. 7 Example: Mobile Ticket for Wiener Linien

8. 8 Exercise: Smart Alarm • What would you try out when the following user story was done? • Collect and write down the list of discussed acceptance criteria • Work in groups of 2-3 As a regular traveler I want to setup a smart alarm so that I can be warned about service disruptions I want to setup multiple smart alarms so that I can plan ahead for different travel plans I have

9. 9 Using examples

10. 10 UI wire frames, existing UI rules, key examples existing artifacts, samples Examples for user stories

11. 11 Specification-by-Example Examples … • make abstract descriptions better understandable However … • examples are usually not formally exchanged or documented Brian Marick Examples Tests Requirements consist of describe verify fulfillment of

12. 12 Discussion of acceptance criteria public void TestInitialOrderDiscount() { Customer newCustomer = new Customer(); Order newOrder = new Order(newCustomer); newOrder.AddBook( Catalog.Find(“ISBN-0955683610”) ); Assert.Equals(33.75, newOrder.Subtotal); } Register as “bart_bookworm” Go to “/catalog/search” Enter “ISBN-0955683610” Click “Search” Click “Add to Cart” Click “View Cart” Verify “Subtotal” is “$33.75” We would like to encourage new users to buy in our shop. Therefore we offer 10% discount for their first order. Original idea for the illustration: George Dinwiddie http://blog.gdinwidiee.com

13. 13 … illustrated with formalized examples Given the user has not ordered yet When the user adds a book with the price of EUR 37.5 into the shopping cart Then the shopping cart sub-total is EUR 33.75. Original idea for the illustration: George Dinwiddie http://blog.gdinwidiee.com

14. 14 Discover hidden assumptions Actually, this is not quite right: Books on sale should be excluded. Original idea for the illustration: George Dinwiddie http://blog.gdinwidiee.com

15. 15 Collaboration: 3 amigos “Happy Path” Technical feasability Exceptions, border cases Original idea for the illustration: George Dinwiddie http://blog.gdinwidiee.com

16. 16 Abstract acceptance criteria As a shop visitor I want to collect books in my shopping basket so that I can purchase multiple books at once. Books can be added to the shopping basket Books can be removed from the shopping basket Shopping basket is initially empty The same book can be added multiple times to the shopping basket

17. 17 Examples for acceptance criteria As a shop visitor I want to collect books in my shopping basket so that I can purchase multiple books at once. Books can be added to the shopping basket Given my shopping basket is empty When I add the book “Harry Potter” to my shopping basket Then my shopping basket should contain 1 copy of “Harry Potter”

18. 18 As a shop visitor I want to collect books in my shopping basket so that I can purchase multiple books at once. Books can be added to the shopping basket Examples for acceptance criteria Given my shopping basket contains 1 copy of “Harry Potter” When I add the book “Harry Potter” to my shopping basket Then my shopping basket should contain 2 copies of “Harry Potter” The same book can be added multiple times to the shopping basket

19. 19 The same book can be added multiple times to the shopping basket Structure of examples Given my shopping basket contains 1 copy of “Harry Potter” When I add the book “Harry Potter” to my shopping basket Then my shopping basket should contain 2 copies of “Harry Potter” Title: Describes intention/abstract acceptance criterion Arrange: Context, initial state of the system Act: Execution of the feature Assert: Assertion of observable behaviour And I should see the warning: “Book already existed in basket” Triple-A constraint “Checks” Chaining up steps

20. 20 Gherkin

21. 21 Feature: Description of feature or user story Szenariogrundriss: Beschreibung des Akzeptanzkriteriums Szenario: Beschreibung des Akzeptanzkriteriums Gherkin Feature Files Background: context for all scenarios in the feature file Scenario: Description of acceptance criterion Angenommen/Wenn/Dann: Automatisierte Szenario Schritte Given/When/Then: automated scenario steps Scenario Outline: Description of acceptance criterion Angenommen/Wenn/Dann: Automatisierte Szenario Schritte Given/When/Then: automated scenario steps with <place holder> Examples: table with examples for <place holders>Examples: table with examples for <place holders> Given: automated scenario steps @tagname @tagname @tagname @tagname

22. 22 Feature Feature: Description of feature or user story #language:en-en @done Feature: Shopping cart As visitor of the web shop I want to collect books in a shopping cart so that I can - prepare a short list of books I want to order - combine multiple books in one shipment @Tagname

23. 23 Scenario Scenario: Description of acceptance criterion Angenommen/Wenn/Dann: Automatisierte Szenario Schritte Given/When/Then: automated scenario steps @Tagname @inprogress Scenario: The same book can be added multiple times to shopping cart Given my shopping cart contains 1 copy of "Harry Potter" When I add the book "Harry Potter" to the shopping cart Then my shopping cart contains 2 copies of "Harry Potter" And a warning is displayed: "You have added the same book again"

24. 24 Scenario Outline Scenario Outline: Description of acceptance criterion Angenommen/Wenn/Dann: Automatisierte Szenario Schritte Examples: table with examples for <place holders> Given/When/Then: automated scenario steps with <place holder> Examples: table with examples for <place holders> @Tagname @Tagname Scenario Outline: Simple search When I search for books by the phrase '<search phrase>' Then the list of found books should contain only: <books> Examples: | search phrase | books | Explanation | | Domain | 'Domain …' | whole words are matched | | Analysis Communication | 'Bridging…', 'Analysis …' | multiple words are matched with OR | Scenario Outline: Candidate profile can only be saved with <mandatory field> When I enter a valid candidate profile, without filling in '<mandatory field>' Then I see the following validation error: 'field <mandatory field> missing' Examples: Main fields | mandatory field | | email | | name | | first name | Examples: Extended fields | mandatory field | | birthdate |

27. 27 Exercise • Work in same previous groups of 2-3 • Pick an acceptance criteria from the previously compiled list • Outline usage examples with the following structure • Given: Context, initial state • When: utilization of the feature • Then: assertions of observable behaviour • Validate that scenario title (acceptance criterion) still describes intention of formalized scenario

28. 28 Life time of examples

29. 29 Purpose of the examples • Shared understanding: acceptance criteria • Documentation: system details • Regression-tests: violated assumptions

30. 30 Continuous validation with automation Given my shopping basket contains 1 copy of “Harry Potter” When I add the book “Harry Potter” to my shopping basket Then my shopping basket should contain 2 copies of “Harry Potter” System „Step Definitions“ are binding individual steps to an automatable interface of the application. Automatable interface UI Automation Automation does not necessarily have to bind to the UI. Automatability of system is supported/evolving with development.

31. 31 Demo http://www.specflow.org Gherkin automation for .NET • Visual Studio plugin (VS-Gallery) • NuGet Package

32. 32 Some more complex exercises Choose between • Best-fare option ticket • Smart Alarm

33. 33 Best-fare option ticket • Assumptions • Ticket price is always adjusted during validation to the best option (considering all previous trips) • Feedback about cost of travel and ticket option (type) after each validation • Valid in the core zone 100 • Scope of the story • Different travel scenarios and how the price adjusts • Billings resulting from the user’s travel • Feedback to traveler (cost of travel, status of ticket) when validating ticket • Out of scope for this story • Credit limit of traveler • Authentication of ticket (validity) As an ad-hoc traveler I want my ticket to adjust to the best available fare whenever I validate So that I don’t need to study upfront which ticket I need Ticket Type Price Single trip 2,10 24 hours 7,10 48 hours 12,40 72 hours 15,40 8 days 35,80 Week 15,00 Month 45,00 Year 365,00

34. 34 Smart Alarm • Assumptions • User can specify route to monitor for disruptions • User can specify when disruptions should trigger the alarm earlier (minutes before alarm) • Scope of the story • Different possibilities of disruptions provided by GTFS and their impact on activating the alarm • Out of scope for this story • Configuring routes (the user is selecting an already defined route) • Setting up alarms (the user has already setup an alarm) As a regular traveler I want to have a smart alarm that activates earlier when there are traffic disruptions on my route So that I can travel earlier to be still on time for my appointment

35. 35 Exercise Choose between • Best-fare option ticket • Smart Alarm Collect and discuss scenarios • What would you try out? • Collect examples • Limit scope of the story

36. 36 Exercise Refine scenarios and formalize into Gherkin • Start with a “warm-up” scenario • Increase complexity with additional scenarios • Reconsider structure of scenarios as you refine them • Validate scenario title/description after refinement

37. 37 Impact on testing

38. 38 Test automation becomes expensive when … • trying to automate manual tests • making tests unreadable when automating them • automating after completing development structure readability point in time

39. 39 Structure Manual tests Asserts Multiple combined features Structure ACT-ASSERT- ACT-ASSERT- ACT-ASSERT- … Dependent features Long test path with high chance to break Cause and impact of error hard to trace Automated Check Single aspect of a single feature ARRANGE – ACT – ASSERT Independent features Short test path with lower chance to break Cause and impact of error easy to relate

40. 40 Test automation pyramid User journeys Acceptance- criteria Units exploratory testing Source: Mike Cohn many few hard easy Automatability

41. 41 // Go to web page 'http://localhost:40001/' using new browser instance BrowserWindow localhostBrowser = BrowserWindow.Launch( new System.Uri(this.RecordedMethod1Params.Url)); // Click 'Register found item' link Mouse.Click(uIFundstückerfassenHyperlink, new Point(56, 9)); // Click 'Save' button Mouse.Click(uISpeichernButton, new Point(44, 14)); int fundNr1 = int.Parse(uIFundNr127Pane.InnerText.Substring(9)); // Click 'Register found item' link Mouse.Click(uIFundstückerfassenHyperlink, new Point(63, 7)); // Click 'Save' button Mouse.Click(uISpeichernButton, new Point(34, 11)); int fundNr2 = int.Parse(uIFundNr128Pane.InnerText.Substring(9)); Assert.IsTrue(fundNr1 + 1 == fundNr2); // Click 'Close' button Mouse.Click(uICloseButton, new Point(26, 11)); Readability

42. 42 A readable test case Scenario: New found items should receive a consecutive number for the current year Given the previous found item of the current year had the number 145 When I register a new found item Then the last found item of the current year should have the number 146

43. 43 When to test (point in time) Acceptance criteria (ATDD, BDD) Unit Tests (TDD) business view technical view Exploratory tests Workflow tests Performance, Scalability, Usability,Security, … definingtheproduct criticizingtheproduct New dimension: defining the product Synergy: Specification of requirements and tests Agile Testing Quadrants: Brian Marick

44. 44 Manual Testing is always necessary! User journeys Acceptance- criteria Units exploratory testing Source: Mike Cohn many few harder easier Automatability Manual Check after Story Done Main success pathes Undiscovered acceptance criteria No/(few) manual regression checks Few pathes are enough More time for exploration

45. 47 Impact on development

46. 48 Workflow - TDD

47. 49 Extending TDD for business

48. 50 Transparency for stakeholders In Progress

49. 51 Current sprint report: all sprint scenarios

50. 52 Starting with first scenario (AC)

51. 53 Finishing the first scenario (AC)

52. 54 Progressing scenario after scenario

55. 57 Implementing user stories in parallel

56. 58 First user story ready for testing

57. 59 Manual testing can start even earlier

58. 60 Work done already can break again

59. 61 Work done already can break again

60. 62 See what is temporarily not working

61. 64 Living documentation

62. 65 Living documentation Drill into Details (Gherkin scenarios) Overview (Story Map)

63. 66 User Stories vs. Features Product/Sprint Backlog User Story 1 AccCrit 1 AccCrit 2 User Story 2 AccCrit 3 AccCrit 4 Living Documentation Feature 1 AccCrit 1 AccCrit 2 Feature n AccCrit 4 AccCrit m User Story n AccCrit 5 AccCrit m AccCrit 3 AccCrit 5 „Done“ • Future options of the system • Organized/refined according to priority, value, effort, risk, ... • Next possible increments of the product (units of work) • Current state of the system • Organized/refined for functional overview • Versioned and maintained together with source code

64. 67 Lessons learned

65. 68 Collaboration over specification Documentation after conversation • Collaborative discovery • Should trigger new questions • Infinite number of examples • Readability and ability to automate

66. 69 Collaboration smells • Silos and too formal hand-overs • Formalization cannot follow implementation • Scenarios replace communication • Scenarios inhibit consideration of solutions • Scenarios inhibit discovery of new things • Duty after development is done • Unreadable scenarios without value • High cost for creation and maintenance • Technical problems with automation

67. 70 Level of automation Controller Business Layer Data Layer Model View Browser automation Trigger behaviour through controller Assert behaviour on model, db, .. Setup pre-conditions through service interfaces Out-of-process In-process

68. 71 Test execution performance • Grouping tests • Current WIP • Completed work • Database • In-memory • Templates for setup • Parallel execution • Smart execution order

69. 72 Internal vs. external DSL Example Source: Liz Keogh https://github.com/lunivore/tictactoe-java/blob/master/ scenarios/com/lunivore/tictactoe/scenarios/ Three_in_a_row_wins.java

70. 73 Non-functional acceptance criteria Given there are 100,000 users registered on the system When I create a new account Then I should be taken to my dashboard within 5ms Given 1000 users are hitting the homepage simultaneously Then each user should get a response within 2ms Matt Wynne http://blog.mattwynne.net/2012/03/13/using-cucumber-for-load-testing

71. 74 Tools

72. 75 Gherkin automation tools www.cukes.info www.behat.org Ruby, Java, JavaScript, C++ www.specflow.org .NET, Mono, Silverlight, WP7 PHP

73. 76

74. 77 Books Gojko Adzic Bridiging the Communication Gap Gojko Adzic Specification by Example Elisabeth Hendrickson Explore IT!

75. 78 Christian Hassa: ch@techtalk.at - @chrishassa