Specification-By-Example with Gherkin

16,529 views

Published on

Many teams struggle with the implementation of user story acceptance criteria and having a shared understanding about the expected story outcomes. This often results in missed stakeholder expectations, ad-hoc assumptions made by the team during implementation and conflict between team members and the product owner around testing.

This session shows how specification-by-example and acceptance test driven development will address team conflict, missed stakeholder expectations and overall increasing the level of clarity on the project end-to-end. The presentation will cover the theory behind ATDD and case-studies and practical experience from real projects.

The talk was held at the ALM summit 3 in Redmond, January 2013. Recording of the talk can be found here: http://channel9.msdn.com/Events/ALM-Summit/ALM-Summit-3/Implementing-ATDD-and-Specification-By-Example

4 Comments
32 Likes
Statistics
Notes
  • Nice presentation, thanks.
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Thank you! Your presentation helps a lot.
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • The video is posted here: http://channel9.msdn.com/Events/ALM-Summit/ALM-Summit-3/Implementing-ATDD-and-Specification-By-Example
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • I wonder if there is a presentation video of this slides. I want to know more in detail.
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Views
Total views
16,529
On SlideShare
0
From Embeds
0
Number of Embeds
717
Actions
Shares
0
Downloads
429
Comments
4
Likes
32
Embeds 0
No embeds

No notes for slide

Specification-By-Example with Gherkin

  1. 1. Specification-By-Examplewith Gherkin Christian Hassa, TechTalk (ch@techtalk.at, @chr99ha)
  2. 2. Almost 4 years later …#135 in NuGet (30.1.2013) ~ 4000 active users > 30 contributors #106 in Visual Studio Gallery (30.1.2013) ~800 visits daily
  3. 3. GoalsMake you curious about Specification-By-Example New perspective for SpecFlow users
  4. 4. About me• Managing Partner at TechTalk• Agile consulting and delivery• Based in Europe: Vienna, Budapest, Zurich
  5. 5. Terminology• Specification-By-Example• Acceptance Test Driven Development (ATDD)• Behaviour Driven Development (BDD)
  6. 6. Agile requirements
  7. 7. Purpose of user stories?• Describe user needs• Unit of prioritization• Unit of planning• Reminder for a discussion• Mechanism to defer detail
  8. 8. Why? Requirement levels Impact Mapping Isolated, Impact, Goal Story Mapping formalized Deliverable, Outcome example User Activity Epic User Story Reminder Acceptance for a Bug Criterion discussion report Specification-By-Example Example CodeHow? Easier to define upfront Harder to define upfront
  9. 9. Collecting acceptance criteriaAs a potential customer “Imagine this story is already implemented:I want to collect books in a shopping cart What would you try out?”So that I can order several books at once. “I would try to put a book into Books can be placed into shopping cart. the shopping cart …” Books can be removed from shopping cart. “I would try to remove a book from the shopping cart…” Shopping cart should be empty when entering the shop. “I’d check whether the shopping cart is empty, when I enter the shop …” Adding same book again to shopping cart should increase quantity. “I would try to add the same book again to the shopping cart …”
  10. 10. Using examples
  11. 11. Examples for user stories UI wire frames, existing UI existing artifacts, mock-ups rules, key examples
  12. 12. Discussion of acceptance criteriaWe would like to encourage new users to buy in our shop.Therefore we offer 10% discount for their first order. public void TestInitialOrderDiscount() Register as “bart_bookworm” { Go to “/catalog/search” Customer newCustomer = new Customer(); Enter “ISBN-0955683610” Order newOrder = new Order(newCustomer); newOrder.AddBook( Click “Search” Catalog.Find(“ISBN-0955683610”) Click “Add to Cart” ); Click “View Cart” Assert.Equals(33.75, Verify “Subtotal” is “$33.75” newOrder.Subtotal); } Original idea for the illustration: George Dinwiddie http://blog.gdinwidiee.com
  13. 13. Specification-By-ExampleExamples … – make abstract descriptions better understandable – are usually not formally exchanged or documented Brian Marick consist of Examples Tests describe verify fulfillment of Requirements
  14. 14. Discussion of acceptance criteriaWe would like to encourage new users to buy in our shop.Therefore we offer 10% discount for their first order. public void TestInitialOrderDiscount() Register as “bart_bookworm” { Go to “/catalog/search” Customer newCustomer = new Customer(); Enter “ISBN-0955683610” Order newOrder = new Order(newCustomer); newOrder.AddBook( Click “Search” Catalog.Find(“ISBN-0955683610”) Click “Add to Cart” ); Click “View Cart” Assert.Equals(33.75, Verify “Subtotal” is “$33.75” newOrder.Subtotal); } Original idea for the illustration: George Dinwiddie http://blog.gdinwidiee.com
  15. 15. Illustrated with shared examplesGiven the user has not ordered yetWhen the user adds a book with the price of USD 37.50 into the shopping cartThen the shopping cart sub-total should be USD 33.75. Original idea for the illustration: George Dinwiddie http://blog.gdinwidiee.com
  16. 16. Discover new aspectsActually, this not quite right:Books on sale should be excluded. Original idea for the illustration: George Dinwiddie http://blog.gdinwidiee.com
  17. 17. Collaboration: 3 amigos Technical“Happy Exceptions, feasability Path” border cases
  18. 18. Abstract acceptance criteriaAs a shop visitorI want to collect books in my shopping basketso that I can purchase multiple books at once. Books can be added to the shopping basket Books can be removed from the shopping basket Shopping basket is initially empty The same book can be added multiple times to the shopping basket
  19. 19. Formalizing examplesAs a shop visitorI want to collect books in my shopping basketso that I can purchase multiple books at once. Books can be added to the shopping basketGiven my shopping basket is emptyWhen I add the book “Harry Potter” to my shopping basketThen my shopping basket should contain 1 copy of “Harry Potter”
  20. 20. Formalizing examplesAs a shop visitorI want to collect books in my shopping basketso that I can purchase multiple books at once. Books can be added to the shopping basket to the shopping basket The same book can be added multiple times Given my shopping basket contains 1 copy of “Harry Potter”When I add the book “Harry Potter” to my shopping basket Then my shopping basket should contain 2 copies of “Harry Potter”
  21. 21. Example structure Title: Describes isolated, focused intention=abstract acceptance criterion Arrange: Context, initial state of the system Triple-A Act: Execution of the feature constraint Assert: Assertion of observable behavior “Checks” The same book can be added multiple times to the shopping basketGiven my shopping basket contains 1 copy of “Harry Potter” Evolution of a domain specificWhen I add the book “Harry Potter” to my shopping basketThen my shopping basket specification copies of “Harry Potter” test and should contain 2 language. And I should see the warning: “Book already existed in basket” Steps can be chained up
  22. 22. Life time of examples
  23. 23. Purpose of examples• Shared understanding: acceptance criteria• Documentation: system details• Regression-tests: violated assumptions
  24. 24. Continuous validation„Step Definitions“ are binding individual stepsto an automatable interface of the application.Automation does not necessarily have to bind to the UI.Automation of system is supported/evolving with development.Given my shopping basket contains 1 copy of “Harry Potter”When I add the book “Harry Potter” to my shopping basketThen my shopping basket should contain 2 copies of “Harry Potter” UI Automation System
  25. 25. Demohttp://www.specflow.orgGherkin automation for .NET• Visual Studio plugin (VS-Gallery)• NuGet Package
  26. 26. Impact on testing
  27. 27. Test automation becomes expensive, when …• trying to automate manual tests structure• making tests unreadable when automating them readability• automating only after completing implementation when to test
  28. 28. Structure Manual tests Automated CheckAsserts Multiple combined features Single aspect of a single featureStructure ACT-ASSERT- ARRANGE – ACT-ASSERT- ACT – ACT-ASSERT- ASSERT … Dependent on existence of other features Independent of other features Long test path with high chance to break Short test path with lower chance to break Cause and impact of an error hard to Cause and impact of an error easier to trace. trace.
  29. 29. Test automation pyramid few harder exploratory testing User Automation journeys Acceptance- criteria Units many easier Source: Mike Cohn
  30. 30. Readability// Go to web page http://localhost:40001/ using new browser instanceBrowserWindow localhostBrowser = BrowserWindow.Launch( new System.Uri(this.RecordedMethod1Params.Url));// Click Register found item linkMouse.Click(uIRegisterFoundItemHyperlink, new Point(56, 9));// Click Save buttonMouse.Click(uISaveButton, new Point(44, 14));int foundItemNo1 = int.Parse(uIFundNr127Pane.InnerText.Substring(9));// Click Register found item linkMouse.Click(uIRegisterFoundItemHyperlink, new Point(63, 7));// Click Save buttonMouse.Click(uISaveButton, new Point(34, 11));int foundItemNo2 = int.Parse(uIFundNr128Pane.InnerText.Substring(9));Assert.IsTrue(foundItemNo1 + 1 == foundItemNo2);// Click Close buttonMouse.Click(uICloseButton, new Point(26, 11));
  31. 31. A readable test caseScenario: New found items should receive a consecutive number for thecurrent yearGiven the previous found item of thecurrent year had the number 145When I register a new found itemThen the last found item of thecurrent year should have the number 146
  32. 32. When to testNew dimension: defining the productSynergy: Specification of requirements and tests business view criticizing the product defining the product Acceptance criteria Exploratory tests (ATDD, BDD) Workflow tests Unit Tests Performance, Scalability, (TDD) Usability,Security, … technical view Agile Testing Quadrants: Brian Marick
  33. 33. Manual testing is still necessary! Undiscovered exploratory More timeacceptance criteria testing for exploration Main User Few paths success paths journeys are enough Manual Check Acceptance- No/(few) manual after story done criteria regression checks Units
  34. 34. Cross-functional work Limit Collaboration for defining Extension of Collaboration for WIP acceptance criteria “Test Cases” automation US1 Plan Exploratory Tests Implement & US4 US7 autom. test US5 Plan US8 Plan US6 Plan US9 PlanCollaboration Implement & Implement & US2 Plan Plan in manual US3 Implement & autom. test Implement & autom. test testing Plan autom. test & Implement autom. test & Implement Plan autom. test Implement & autom. test autom. test & Implement autom. test Specification and test Sprint 1 Sprint 2 Sprint 3 short iteration Preventing bugs instead of finding them!
  35. 35. Impact on development
  36. 36. TDD workflow
  37. 37. Extending TDD for business
  38. 38. Transparency for stakeholders In Progress
  39. 39. Current sprint report: all scenarios
  40. 40. Starting with first scenario
  41. 41. Finishing first scenario
  42. 42. Progressing on scenarios
  43. 43. Progressing on scenarios
  44. 44. Progressing on scenarios
  45. 45. Progressing on scenarios
  46. 46. First user story ready for testing
  47. 47. Earlier start of manual testing
  48. 48. Done work can break again
  49. 49. Done work can break again
  50. 50. Business readable error reports
  51. 51. Living documentation
  52. 52. Why? Higher level scenario structure Impact, Goal Story Mapping Deliverable, Outcome User Activity Epic User Story Acceptance Criterion Example CodeHow? Easier to define upfront Harder to define upfront
  53. 53. Story Maps• Original concept by Jeff Patton• Support iterative product design• Optimized for desired outcome or deliverable the system should support
  54. 54. Building story maps Get popular books desired outcome delivered fast and or deliverable conveniently Find Collect Wait for Receive user Order book books delivery delivery activities time browse shopping enter delivery delivery systemnecessity best basket address notification slip features sellers search pay with inquiry book by wish list credit order title card status
  55. 55. Enabling build–measure-learn Get popular books desired outcome delivered fast and or deliverable conveniently Find Collect Wait for Receive user Order book books delivery delivery activities time browse manual enter omitted delivery Walking systemnecessity best work- address steps slip skeleton features sellers around search pay with inquiry book by wish list shopping credit order delivery title basket card status notification
  56. 56. Story Map example
  57. 57. Sprint 1
  58. 58. Sprint 2
  59. 59. Sprint 3
  60. 60. Sprint 4
  61. 61. Dropped Scope
  62. 62. Added scope
  63. 63. User Stories vs. Features Product/Sprint Backlog Living Documentation AccCrit 1 AccCrit 1 User Story 1 AccCrit 2 „Done“ Feature 1 AccCrit 2 AccCrit 3 AccCrit 3 User Story 2 AccCrit 4 AccCrit 5 AccCrit 5 AccCrit 4 User Story n Feature n AccCrit m AccCrit m• Future options of the system • Current state of the system• Organized/refined according to • Organized/refined for priority, value, effort, risk, ... functional overview• Next possible increments of • Versioned and maintained the product (units of work) together with source code
  64. 64. Living documentation • Link business readable automation from source code to story maps • Feed execution results into story maps • Sync Story Story Map requirements with TFS work items
  65. 65. Lessons learned
  66. 66. Collaboration over specification Documentation after conversation • Collaborative discovery • Should trigger new questions • Infinite number of examples • Readability and ability to automate
  67. 67. Level of automationBrowser automation Out-of-process In-process View Assert behavior Trigger behavior on model, db, .. through controller Model Controller Business Layer Setup pre-conditions through service Data Layer interfaces
  68. 68. Test execution performance• Grouping tests – Current WIP – Completed work• Database – In-memory – Templates for setup• Parallel execution• Smart execution order• Level of automation
  69. 69. Internal vs. external DSL Example Source: Liz Keogh https://github.com/lunivore/tictactoe-java/blob/master/ scenarios/com/lunivore/tictactoe/scenarios/Three_in_a_row_wins.java
  70. 70. Non-functional acceptance criteriaGiven there are 100,000 users registered on the systemWhen I create a new accountThen I should be taken to my dashboard within 5msWhen 1000 users are hitting the homepage simultaneouslyThen each user should get a response within 2msMatt Wynnehttp://blog.mattwynne.net/2012/03/13/using-cucumber-for-load-testing
  71. 71. Tools
  72. 72. Gherkin automation tools Ruby, Java, JavaScript, C++ www.cukes.info .NET, Mono, Silverlight, WP7www.specflow.org PHP www.behat.org
  73. 73. Books Explore It! Elisabeth Hendrickson Exploratory Software TestingBridiging the Specification by JamesCommunication Gap Example WhittakerGojko Adzic Gojko Adzic
  74. 74. SummarySpecification-by-example• Collaborative discovery and specification• Impact on development and test• Combine with other agile reqmt methods• Can be supported by tools
  75. 75. Join the BoF Session„Specification-By-Example (ATDD, BDD) in practice“Today at 15h15 (right after this talk) Questions Got curious about Specification-By-Example? Learned something new about using SpecFlow? Christian Hassa, TechTalk (ch@techtalk.at, @chr99ha)

×