Specification-By-Example with Gherkin

7,874 views

Published on

Published in: Technology

Specification-By-Example with Gherkin

  1. 1. Specification-By-Examplewith GherkinFROM STAKEHOLDER EXAMPLES TO LIVING DOCUMENTATION Christian Hassa - ch@techtalk.at - @chr99ha .NET User Group Bern, 5th of December 2012
  2. 2. TechTalk at a glance• Agile consulting and delivery• Offices in: Vienna, Budapest, Zurich• Established in 1993 Vienna/Austria
  3. 3. How do we specify requirements?3
  4. 4. The purpose of user stories …• Describe user needs• Describe system features• Unit of planning• Reminder for a discussion• Mechanism to defer a discussion4
  5. 5. Establishing a shared understanding Why? Impact Mapping Impact, Goal Isolated, Capability Story Mapping formalized example User Activity Epic User Story Reminder Acceptance for a Bug Criterion discussion report Example Specification-By-Example Code How? Easier to define upfront Harder to define upfront5
  6. 6. Collecting Acceptance Criteria As a potential customer “Imagine this story is I want to collect books in a shopping cart already implemented: So that I can order several books at once. how would you verify it?” “I would try to put a book into the shopping cart …” Books can be placed into shopping cart. “I would try to remove a book from the shopping cart…” Books can be removed from shopping cart. “I’d check whether the shopping cart is empty, when I enter the shop …” Shopping cart should be empty when entering the shop. Adding same book again to “I would try to add the same book shopping cart should increase again to the shopping cart …” quantity.6
  7. 7. Why do we need examples?7
  8. 8. Examples for user stories UI wire frames, existing UI rules, key examples existing artifacts, samples8
  9. 9. Specification-by-Example Examples … • make abstract descriptions better understandable However … • examples are usually not formally Brian Marick exchanged or documented consist of Examples Tests describe verify fulfillment of Requirements10
  10. 10. Discussion of acceptance criteria We would like to encourage new users to buy in our shop. Therefore we offer 10% discount for their first order. public void TestInitialOrderDiscount() Register as “bart_bookworm” { Go to “/catalog/search” Customer newCustomer = new Customer(); Enter “ISBN-0955683610” Order newOrder = new Order(newCustomer); Click “Search” newOrder.AddBook( Catalog.Find(“ISBN-0955683610”) Click “Add to Cart” ); Click “View Cart” Assert.Equals(33.75, Verify “Subtotal” is “$33.75” newOrder.Subtotal); } Original idea for the illustration: George Dinwiddie http://blog.gdinwidiee.com11
  11. 11. … illustrated with formalized examples Given the user has not ordered yet When the user adds a book with the price of EUR 37.5 into the shopping cart Then the shopping cart sub-total is EUR 33.75. Original idea for the illustration: George Dinwiddie http://blog.gdinwidiee.com12
  12. 12. Discover hidden assumptions Actually, this not quite right: Books on sale should be excluded. Original idea for the illustration: George Dinwiddie http://blog.gdinwidiee.com13
  13. 13. Collaboration: 3 amigos Exceptions, “Happy Technical border cases Path” feasability Original idea for the illustration: George Dinwiddie http://blog.gdinwidiee.com14
  14. 14. Abstract acceptance criteria As a shop visitor I want to collect books in my shopping basket so that I can purchase multiple books at once. Books can be added to the shopping basket Books can be removed from the shopping basket Shopping basket is initially empty The same book can be added multiple times to the shopping basket15
  15. 15. Examples for acceptance criteria As a shop visitor I want to collect books in my shopping basket so that I can purchase multiple books at once. Books can be added to the shopping basket Given my shopping basket is empty When I add the book “Harry Potter” to my shopping basket Then my shopping basket should contain 1 copy of “Harry Potter”16
  16. 16. Examples for acceptance criteria As a shop visitor I want to collect books in my shopping basket so that I can purchase multiple books at once. Books can be added to the shopping basket The same book can be added multiple times to the shopping basket Given my shopping basket contains 1 copy of “Harry Potter” When I add the book “Harry Potter” to my shopping basket Then my shopping basket should contain 2 copies of “Harry Potter”17
  17. 17. Structure of examples Title: Describes intention/abstract acceptance criterion Arrange: Context, initial state of the system Triple-A Act: Execution of the feature constraint “Checks” Assert: Assertion of observable behaviour The same book can be added multiple times to the shopping basket Given my shopping basket contains 1 copy of “Harry Potter” When I add the book “Harry Potter” to my shopping basket Then my shopping basket should contain 2 copies of “Harry Potter” And I should see the warning: “Book already existed in basket” Chaining up steps18
  18. 18. How long do we care about the examples?20
  19. 19. Purpose of the examples• Shared understanding: acceptance criteria• Documentation: Look-up detail aspects of the system• Regression-tests: Understand what assumptions have been violated21
  20. 20. Continuous validation with automation „Step Definitions“ are binding individual steps to an automatable interface of the application. Automation does not necessarily have to bind to the UI. Automatability of system is supported/evolving with development. Given my shopping basket contains 1 copy of “Harry Potter” When I add the book “Harry Potter” to my shopping basket Then my shopping basket should contain 2 copies of “Harry Potter” UI System Automation Automatable interface22
  21. 21. Demohttp://www.specflow.orgGherkin automation for .NET• Visual Studio plugin (VS-Gallery)• NuGet Package23
  22. 22. What does that mean for testing?24
  23. 23. UI automation becomes expensivewhen … • trying to automate structure manual tests • making tests unreadable when readability automating them • automating after completing point in time development25
  24. 24. Structure Manual tests Automated Check Asserts Multiple combined Single aspect of a features single feature Structure ACT-ASSERT- ARRANGE – ACT-ASSERT- ACT – ACT-ASSERT- ASSERT … Dependent on other Independent of features other features Long test path with Short test path with high chance to break lower chance to break26
  25. 25. Test automation pyramid few exploratory hard testing User Automatability journeys Acceptance- criteria Units easy many27 Source: Mike Cohn
  26. 26. Readability // Go to web page http://localhost:40001/ using new browser instance BrowserWindow localhostBrowser = BrowserWindow.Launch( new System.Uri(this.RecordedMethod1Params.Url)); // Click Register found item link Mouse.Click(uIFundstückerfassenHyperlink, new Point(56, 9)); // Click Save button Mouse.Click(uISpeichernButton, new Point(44, 14)); int fundNr1 = int.Parse(uIFundNr127Pane.InnerText.Substring(9)); // Click Register found item link Mouse.Click(uIFundstückerfassenHyperlink, new Point(63, 7)); // Click Save button Mouse.Click(uISpeichernButton, new Point(34, 11)); int fundNr2 = int.Parse(uIFundNr128Pane.InnerText.Substring(9)); Assert.IsTrue(fundNr1 + 1 == fundNr2); // Click Close button Mouse.Click(uICloseButton, new Point(26, 11));28
  27. 27. A readable test caseScenario: New found items should receive aconsecutive number for the current yearGiven the previous found item of thecurrent year had the number 145When I register a new found itemThen the last found item of thecurrent year should have the number 14629
  28. 28. When to test (point in time) business view criticizing the product defining the product Acceptance criteria Exploratory tests (ATDD, BDD) Workflow tests Unit Tests Performance, Scalability, (TDD) Usability,Security, … technical view Agile Testing Quadrants: Brian Marick New dimension: defining the product Synergy: Specification of requirements and tests30
  29. 29. Cross-functional teamsCo-creationFast feedback31
  30. 30. Cross-functional work Limit Collaboration for defining Extension of Collaboration for WIP acceptance criteria “Test Cases” automation US1 Plan Implement & US4 US7 Exploratory Tests autom. test US5 US8 Plan Plan US6 Plan US9 PlanCollaboration US2 Implement & Plan Implement & Plan in manual Implement & Implement & US3 autom. test autom. test testing Plan autom. test & Implement autom. test & Implement Plan autom. test Implement & autom. test autom. test & Implement autom. test Specification and test Sprint 1 Sprint 2 Sprint 3 Preventing bugs short iteration instead of finding them! 32
  31. 31. Manual Testing is always necessary! few Undiscovered acceptance More time exploratory for exploration harder criteria testing Main success Few pathes pathes User are enough Automatability journeys Manual No/(few) Check manual after Acceptance- regression Story Done criteria checks Units easier many33 Source: Mike Cohn
  32. 32. What does that mean for development?34
  33. 33. Workflow - TDD35
  34. 34. Extending TDD for business36
  35. 35. Transparency for stakeholdersIn Progress37
  36. 36. Current sprint report: all sprint scenarios38
  37. 37. Starting with first scenario (AC)39
  38. 38. Finishing the first scenario (AC)40
  39. 39. Progressing scenario after scenario41
  40. 40. Progressing scenario after scenario42
  41. 41. Progressing scenario after scenario43
  42. 42. Implementing user stories in parallel44
  43. 43. First user story ready for testing45
  44. 44. Manual testing can start even earlier46
  45. 45. Already done work can break again47
  46. 46. Already done work can break again48
  47. 47. See what is temporarily not working49
  48. 48. Living documentation51
  49. 49. Optimizing and refining scope Why? Impact, Goal Capability Story Mapping User Activity Epic User Story Acceptance Criterion Example Code How? Easier to define upfront Harder to define upfront52
  50. 50. Story Maps• Original concept by Jeff Patton• Supports iterative product design• Optimized for desired outcome or deliverable the system should support53
  51. 51. Building story maps Get mainstream book fast and conveniently desired outcome delivered or deliverable Find book Collect Complete Wait for Receive user I want books order book book activities time browse receive receive put into enter system best delivery delivery basket address featuresnecessity sellers notificat. slip search pay with inquiry create book by credit order wish list title card status 56
  52. 52. Enabling build – measure - learn Get mainstream book fast and conveniently delivered Find book Collect Complete Wait for Receive I want books order book book time browse omitted receive manual enter Walking best workaround steps delivery address skeletonnecessity sellers slip search pay with inquiry create receive book by put into credit order wish list delivery title basket card status notificat. 57
  53. 53. Story Maps in ALM with TFS http://speclog.net TFS work items in story maps60
  54. 54. Story Map in SpecLog61
  55. 55. Sprint 162
  56. 56. Sprint 263
  57. 57. Sprint 364
  58. 58. Sprint 465
  59. 59. Dropped Scope66
  60. 60. Added Scope67
  61. 61. User Stories vs. Features Product/Sprint Backlog Living Documentation AccCrit 1 AccCrit 1 User Story 1 AccCrit 2 „Done“ Feature 1 AccCrit 2 AccCrit 3 AccCrit 3 User Story 2 AccCrit 4 AccCrit 5 AccCrit 5 AccCrit 4 User Story n Feature n AccCrit m AccCrit m • Future options of the system • Current state of the system • Organized/refined according to • Organized/refined for priority, value, effort, risk, ... functional overview • Next possible increments of • Versioned and maintained the product (units of work) together with source code69
  62. 62. Living documentation• Link business readable automation from source code to story maps• Feed execution results into story maps71
  63. 63. Additional thoughts72
  64. 64. Level of automation Out-of-process Browser automation In-process Assert behaviour Trigger behaviour View on model, db, .. through controller Model Controller Setup pre-conditions Business Layer through service interfaces Data Layer73
  65. 65. Performance• Grouping tests • Current WIP • Completed work• Database • In-memory • Templates for setup• Parallel execution• Smart execution order74
  66. 66. Internal vs. external DSL Example Source: Liz Keogh https://github.com/lunivore/tictactoe-java/blob/master/ scenarios/com/lunivore/tictactoe/scenarios/ Three_in_a_row_wins.java75
  67. 67. Non-functional acceptance criteriaGiven there are 100,000 users registered on the systemWhen I create a new accountThen I should be taken to my dashboard within 5msGiven 1000 users are hitting the homepagesimultaneouslyThen each user should get a response within 2ms Matt Wynne http://blog.mattwynne.net/2012/03/13/using-cucumber-for-load-testing76
  68. 68. Tools77
  69. 69. Gherkin automation tools Ruby, Java, JavaScript, C++ www.cukes.info .NET, Mono, Silverlight, WP7 www.specflow.org PHP www.behat.org78
  70. 70. 79
  71. 71. Books Gojko Adzic Gojko Adzic Elisabeth Hendrickson Bridiging the Specification by Explore It! Communication Gap Example80
  72. 72. 81 Christian Hassa: ch@techtalk.at - @chr99ha

×