Specification-By-Example with Gherkin
Upcoming SlideShare
Loading in...5
×
 

Specification-By-Example with Gherkin

on

  • 4,439 views

 

Statistics

Views

Total Views
4,439
Views on SlideShare
4,390
Embed Views
49

Actions

Likes
11
Downloads
103
Comments
0

3 Embeds 49

http://xwiki.ar.oracle.com 27
http://wiki.bruce.kr 18
https://twitter.com 4

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Specification-By-Example with Gherkin Specification-By-Example with Gherkin Presentation Transcript

  • Specification-By-Examplewith GherkinFROM STAKEHOLDER EXAMPLES TO LIVING DOCUMENTATION Christian Hassa - ch@techtalk.at - @chr99ha .NET User Group Bern, 5th of December 2012
  • TechTalk at a glance• Agile consulting and delivery• Offices in: Vienna, Budapest, Zurich• Established in 1993 Vienna/Austria
  • How do we specify requirements?3
  • The purpose of user stories …• Describe user needs• Describe system features• Unit of planning• Reminder for a discussion• Mechanism to defer a discussion4
  • Establishing a shared understanding Why? Impact Mapping Impact, Goal Isolated, Capability Story Mapping formalized example User Activity Epic User Story Reminder Acceptance for a Bug Criterion discussion report Example Specification-By-Example Code How? Easier to define upfront Harder to define upfront5
  • Collecting Acceptance Criteria As a potential customer “Imagine this story is I want to collect books in a shopping cart already implemented: So that I can order several books at once. how would you verify it?” “I would try to put a book into the shopping cart …” Books can be placed into shopping cart. “I would try to remove a book from the shopping cart…” Books can be removed from shopping cart. “I’d check whether the shopping cart is empty, when I enter the shop …” Shopping cart should be empty when entering the shop. Adding same book again to “I would try to add the same book shopping cart should increase again to the shopping cart …” quantity.6
  • Why do we need examples?7
  • Examples for user stories UI wire frames, existing UI rules, key examples existing artifacts, samples8
  • Specification-by-Example Examples … • make abstract descriptions better understandable However … • examples are usually not formally Brian Marick exchanged or documented consist of Examples Tests describe verify fulfillment of Requirements10
  • Discussion of acceptance criteria We would like to encourage new users to buy in our shop. Therefore we offer 10% discount for their first order. public void TestInitialOrderDiscount() Register as “bart_bookworm” { Go to “/catalog/search” Customer newCustomer = new Customer(); Enter “ISBN-0955683610” Order newOrder = new Order(newCustomer); Click “Search” newOrder.AddBook( Catalog.Find(“ISBN-0955683610”) Click “Add to Cart” ); Click “View Cart” Assert.Equals(33.75, Verify “Subtotal” is “$33.75” newOrder.Subtotal); } Original idea for the illustration: George Dinwiddie http://blog.gdinwidiee.com11
  • … illustrated with formalized examples Given the user has not ordered yet When the user adds a book with the price of EUR 37.5 into the shopping cart Then the shopping cart sub-total is EUR 33.75. Original idea for the illustration: George Dinwiddie http://blog.gdinwidiee.com12
  • Discover hidden assumptions Actually, this not quite right: Books on sale should be excluded. Original idea for the illustration: George Dinwiddie http://blog.gdinwidiee.com13
  • Collaboration: 3 amigos Exceptions, “Happy Technical border cases Path” feasability Original idea for the illustration: George Dinwiddie http://blog.gdinwidiee.com14
  • Abstract acceptance criteria As a shop visitor I want to collect books in my shopping basket so that I can purchase multiple books at once. Books can be added to the shopping basket Books can be removed from the shopping basket Shopping basket is initially empty The same book can be added multiple times to the shopping basket15
  • Examples for acceptance criteria As a shop visitor I want to collect books in my shopping basket so that I can purchase multiple books at once. Books can be added to the shopping basket Given my shopping basket is empty When I add the book “Harry Potter” to my shopping basket Then my shopping basket should contain 1 copy of “Harry Potter”16
  • Examples for acceptance criteria As a shop visitor I want to collect books in my shopping basket so that I can purchase multiple books at once. Books can be added to the shopping basket The same book can be added multiple times to the shopping basket Given my shopping basket contains 1 copy of “Harry Potter” When I add the book “Harry Potter” to my shopping basket Then my shopping basket should contain 2 copies of “Harry Potter”17
  • Structure of examples Title: Describes intention/abstract acceptance criterion Arrange: Context, initial state of the system Triple-A Act: Execution of the feature constraint “Checks” Assert: Assertion of observable behaviour The same book can be added multiple times to the shopping basket Given my shopping basket contains 1 copy of “Harry Potter” When I add the book “Harry Potter” to my shopping basket Then my shopping basket should contain 2 copies of “Harry Potter” And I should see the warning: “Book already existed in basket” Chaining up steps18
  • How long do we care about the examples?20
  • Purpose of the examples• Shared understanding: acceptance criteria• Documentation: Look-up detail aspects of the system• Regression-tests: Understand what assumptions have been violated21
  • Continuous validation with automation „Step Definitions“ are binding individual steps to an automatable interface of the application. Automation does not necessarily have to bind to the UI. Automatability of system is supported/evolving with development. Given my shopping basket contains 1 copy of “Harry Potter” When I add the book “Harry Potter” to my shopping basket Then my shopping basket should contain 2 copies of “Harry Potter” UI System Automation Automatable interface22
  • Demohttp://www.specflow.orgGherkin automation for .NET• Visual Studio plugin (VS-Gallery)• NuGet Package23
  • What does that mean for testing?24
  • UI automation becomes expensivewhen … • trying to automate structure manual tests • making tests unreadable when readability automating them • automating after completing point in time development25
  • Structure Manual tests Automated Check Asserts Multiple combined Single aspect of a features single feature Structure ACT-ASSERT- ARRANGE – ACT-ASSERT- ACT – ACT-ASSERT- ASSERT … Dependent on other Independent of features other features Long test path with Short test path with high chance to break lower chance to break26
  • Test automation pyramid few exploratory hard testing User Automatability journeys Acceptance- criteria Units easy many27 Source: Mike Cohn
  • Readability // Go to web page http://localhost:40001/ using new browser instance BrowserWindow localhostBrowser = BrowserWindow.Launch( new System.Uri(this.RecordedMethod1Params.Url)); // Click Register found item link Mouse.Click(uIFundstückerfassenHyperlink, new Point(56, 9)); // Click Save button Mouse.Click(uISpeichernButton, new Point(44, 14)); int fundNr1 = int.Parse(uIFundNr127Pane.InnerText.Substring(9)); // Click Register found item link Mouse.Click(uIFundstückerfassenHyperlink, new Point(63, 7)); // Click Save button Mouse.Click(uISpeichernButton, new Point(34, 11)); int fundNr2 = int.Parse(uIFundNr128Pane.InnerText.Substring(9)); Assert.IsTrue(fundNr1 + 1 == fundNr2); // Click Close button Mouse.Click(uICloseButton, new Point(26, 11));28
  • A readable test caseScenario: New found items should receive aconsecutive number for the current yearGiven the previous found item of thecurrent year had the number 145When I register a new found itemThen the last found item of thecurrent year should have the number 14629
  • When to test (point in time) business view criticizing the product defining the product Acceptance criteria Exploratory tests (ATDD, BDD) Workflow tests Unit Tests Performance, Scalability, (TDD) Usability,Security, … technical view Agile Testing Quadrants: Brian Marick New dimension: defining the product Synergy: Specification of requirements and tests30
  • Cross-functional teamsCo-creationFast feedback31
  • Cross-functional work Limit Collaboration for defining Extension of Collaboration for WIP acceptance criteria “Test Cases” automation US1 Plan Implement & US4 US7 Exploratory Tests autom. test US5 US8 Plan Plan US6 Plan US9 PlanCollaboration US2 Implement & Plan Implement & Plan in manual Implement & Implement & US3 autom. test autom. test testing Plan autom. test & Implement autom. test & Implement Plan autom. test Implement & autom. test autom. test & Implement autom. test Specification and test Sprint 1 Sprint 2 Sprint 3 Preventing bugs short iteration instead of finding them! 32
  • Manual Testing is always necessary! few Undiscovered acceptance More time exploratory for exploration harder criteria testing Main success Few pathes pathes User are enough Automatability journeys Manual No/(few) Check manual after Acceptance- regression Story Done criteria checks Units easier many33 Source: Mike Cohn
  • What does that mean for development?34
  • Workflow - TDD35
  • Extending TDD for business36
  • Transparency for stakeholdersIn Progress37
  • Current sprint report: all sprint scenarios38
  • Starting with first scenario (AC)39
  • Finishing the first scenario (AC)40
  • Progressing scenario after scenario41
  • Progressing scenario after scenario42
  • Progressing scenario after scenario43
  • Implementing user stories in parallel44
  • First user story ready for testing45
  • Manual testing can start even earlier46
  • Already done work can break again47
  • Already done work can break again48
  • See what is temporarily not working49
  • Living documentation51
  • Optimizing and refining scope Why? Impact, Goal Capability Story Mapping User Activity Epic User Story Acceptance Criterion Example Code How? Easier to define upfront Harder to define upfront52
  • Story Maps• Original concept by Jeff Patton• Supports iterative product design• Optimized for desired outcome or deliverable the system should support53
  • Building story maps Get mainstream book fast and conveniently desired outcome delivered or deliverable Find book Collect Complete Wait for Receive user I want books order book book activities time browse receive receive put into enter system best delivery delivery basket address featuresnecessity sellers notificat. slip search pay with inquiry create book by credit order wish list title card status 56
  • Enabling build – measure - learn Get mainstream book fast and conveniently delivered Find book Collect Complete Wait for Receive I want books order book book time browse omitted receive manual enter Walking best workaround steps delivery address skeletonnecessity sellers slip search pay with inquiry create receive book by put into credit order wish list delivery title basket card status notificat. 57
  • Story Maps in ALM with TFS http://speclog.net TFS work items in story maps60
  • Story Map in SpecLog61
  • Sprint 162
  • Sprint 263
  • Sprint 364
  • Sprint 465
  • Dropped Scope66
  • Added Scope67
  • User Stories vs. Features Product/Sprint Backlog Living Documentation AccCrit 1 AccCrit 1 User Story 1 AccCrit 2 „Done“ Feature 1 AccCrit 2 AccCrit 3 AccCrit 3 User Story 2 AccCrit 4 AccCrit 5 AccCrit 5 AccCrit 4 User Story n Feature n AccCrit m AccCrit m • Future options of the system • Current state of the system • Organized/refined according to • Organized/refined for priority, value, effort, risk, ... functional overview • Next possible increments of • Versioned and maintained the product (units of work) together with source code69
  • Living documentation• Link business readable automation from source code to story maps• Feed execution results into story maps71
  • Additional thoughts72
  • Level of automation Out-of-process Browser automation In-process Assert behaviour Trigger behaviour View on model, db, .. through controller Model Controller Setup pre-conditions Business Layer through service interfaces Data Layer73
  • Performance• Grouping tests • Current WIP • Completed work• Database • In-memory • Templates for setup• Parallel execution• Smart execution order74
  • Internal vs. external DSL Example Source: Liz Keogh https://github.com/lunivore/tictactoe-java/blob/master/ scenarios/com/lunivore/tictactoe/scenarios/ Three_in_a_row_wins.java75
  • Non-functional acceptance criteriaGiven there are 100,000 users registered on the systemWhen I create a new accountThen I should be taken to my dashboard within 5msGiven 1000 users are hitting the homepagesimultaneouslyThen each user should get a response within 2ms Matt Wynne http://blog.mattwynne.net/2012/03/13/using-cucumber-for-load-testing76
  • Tools77
  • Gherkin automation tools Ruby, Java, JavaScript, C++ www.cukes.info .NET, Mono, Silverlight, WP7 www.specflow.org PHP www.behat.org78
  • 79
  • Books Gojko Adzic Gojko Adzic Elisabeth Hendrickson Bridiging the Specification by Explore It! Communication Gap Example80
  • 81 Christian Hassa: ch@techtalk.at - @chr99ha