SlideShare a Scribd company logo
1 of 77
Download to read offline
Specification-By-Example
with Gherkin
Christian Hassa,TechTalk
(ch@techtalk.at, @chr99ha)
Almost 4 years later …
#106 inVisual StudioGallery (30.1.2013)
#135 in NuGet (30.1.2013)
~800 visits daily
~ 4000 active users
> 30 contributors
Goals
Make you curious about Specification-By-Example
New perspective for SpecFlow users
About me
• Managing Partner atTechTalk
• Agile consulting and delivery
• Based in Europe:
Vienna, Budapest, Zurich
Terminology
• Specification-By-Example
• AcceptanceTest Driven Development (ATDD)
• Behaviour Driven Development (BDD)
Agile requirements
Purpose of user stories?
• Describe user needs
• Unit of prioritization
• Unit of planning
• Reminder for a discussion
• Mechanism to defer detail
Requirement levels
Impact Mapping
Story Mapping
Specification-By-Example
Why?
How?
Acceptance
Criterion
Epic
Deliverable, Outcome
Impact, Goal
Easier to define upfront Harder to define upfront
Reminder
for a
discussion
Bug
report
Isolated,
formalized
example
User Activity
User Story
Example
Code
Collecting acceptance criteria
“I would try to put a book into
the shopping cart …”
“I would try to remove a book
from the shopping cart…”
“I’d check whether the shopping cart
is empty, when I enter the shop …”
“I would try to add the same book
again to the shopping cart …”
Books can be placed into shopping cart.
Books can be removed from shopping cart.
Shopping cart should be empty when
entering the shop.
Adding same book again to shopping cart
should increase quantity.
As a potential customer
I want to collect books in a shopping cart
So that I can order several books at once.
“Imagine this story is already implemented:
What would you try out?”
Using examples
Examples for user stories
UI wire frames,
existing UI
rules, key examples
existing artifacts,
mock-ups
Discussion of acceptance criteria
Original idea for the illustration:George Dinwiddie
http://blog.gdinwidiee.com
public void TestInitialOrderDiscount()
{
Customer newCustomer = new Customer();
Order newOrder = new Order(newCustomer);
newOrder.AddBook(
Catalog.Find(“ISBN-0955683610”)
);
Assert.Equals(33.75,
newOrder.Subtotal);
}
Register as “bart_bookworm”
Go to “/catalog/search”
Enter “ISBN-0955683610”
Click “Search”
Click “Add to Cart”
Click “View Cart”
Verify “Subtotal” is “$33.75”
We would like to encourage new users to buy in our shop.
Therefore we offer 10% discount for their first order.
Specification-By-Example
Examples …
– make abstract descriptions
better understandable
– are usually not formally
exchanged or documented
Brian Marick
Examples Tests
Requirements
consist of
describe verify
fulfillment of
Discussion of acceptance criteria
Original idea for the illustration:George Dinwiddie
http://blog.gdinwidiee.com
public void TestInitialOrderDiscount()
{
Customer newCustomer = new Customer();
Order newOrder = new Order(newCustomer);
newOrder.AddBook(
Catalog.Find(“ISBN-0955683610”)
);
Assert.Equals(33.75,
newOrder.Subtotal);
}
Register as “bart_bookworm”
Go to “/catalog/search”
Enter “ISBN-0955683610”
Click “Search”
Click “Add to Cart”
Click “View Cart”
Verify “Subtotal” is “$33.75”
We would like to encourage new users to buy in our shop.
Therefore we offer 10% discount for their first order.
Illustrated with shared examples
Original idea for the illustration:George Dinwiddie
http://blog.gdinwidiee.com
Given the user has not ordered yet
When the user adds a book with the price of USD 37.50 into the shopping cart
Then the shopping cart sub-total should be USD 33.75.
Discover new aspects
Original idea for the illustration:George Dinwiddie
http://blog.gdinwidiee.com
Actually, this not quite right:
Books on sale should be excluded.
Collaboration: 3 amigos
“Happy
Path”
Technical
feasability
Exceptions,
border cases
Abstract acceptance criteria
As a shop visitor
I want to collect books in my shopping basket
so that I can purchase multiple books at once.
Books can be added to the shopping basket
Books can be removed from the shopping basket
Shopping basket is initially empty
The same book can be added multiple times to the shopping basket
Formalizing examples
As a shop visitor
I want to collect books in my shopping basket
so that I can purchase multiple books at once.
Books can be added to the shopping basket
Given my shopping basket is empty
When I add the book “Harry Potter” to my shopping basket
Then my shopping basket should contain 1 copy of “Harry Potter”
Formalizing examples
As a shop visitor
I want to collect books in my shopping basket
so that I can purchase multiple books at once.
Books can be added to the shopping basketThe same book can be added multiple times to the shopping basket
When I add the book “Harry Potter” to my shopping basket
Then my shopping basket should contain 2 copies of “Harry Potter”
Given my shopping basket contains 1 copy of “Harry Potter”
Example structure
The same book can be added multiple times to the shopping basket
When I add the book “Harry Potter” to my shopping basket
Then my shopping basket should contain 2 copies of “Harry Potter”
Given my shopping basket contains 1 copy of “Harry Potter”
Title: Describes isolated, focused intention=abstract acceptance criterion
Arrange: Context, initial state of the system
Act: Execution of the feature
Assert: Assertion of observable behavior
And I should see the warning: “Book already existed in basket”
Triple-A
constraint
“Checks”
Steps can be chained up
Evolution of a domain specific
test and specification language.
Life time of examples
Purpose of examples
• Shared understanding:
acceptance criteria
• Documentation:
system details
• Regression-tests:
violated assumptions
Continuous validation
When I add the book “Harry Potter” to my shopping basket
Then my shopping basket should contain 2 copies of “Harry Potter”
Given my shopping basket contains 1 copy of “Harry Potter”
„Step Definitions“ are binding individual steps
to an automatable interface of the application.
Automation does not necessarily have to bind to the UI.
Automation of system is supported/evolving with development.
SystemUI Automation
Demo
Gherkin automation for .NET
• Visual Studio plugin (VS-Gallery)
• NuGet Package
http://www.specflow.org
Impact on testing
Test automation becomes expensive, when …
• trying to automate manual tests
• making tests unreadable when
automating them
• automating only after
completing implementation
structure
readability
when to test
Structure
Manual tests
Asserts Multiple combined features
Structure ACT-ASSERT-
ACT-ASSERT-
ACT-ASSERT-
…
Dependent on existence of other features
Long test path with high chance to break
Cause and impact of an error hard to
trace.
Automated Check
Single aspect of a single feature
ARRANGE –
ACT –
ASSERT
Independent of other features
Short test path with lower chance to break
Cause and impact of an error easier to
trace.
Test automation pyramid
User
journeys
Acceptance-
criteria
Units
many
harder
easier
Automation
few
exploratory
testing
Source: Mike Cohn
Readability
// Go to web page 'http://localhost:40001/' using new browser instance
BrowserWindow localhostBrowser = BrowserWindow.Launch(
new System.Uri(this.RecordedMethod1Params.Url));
// Click 'Register found item' link
Mouse.Click(uIRegisterFoundItemHyperlink, new Point(56, 9));
// Click 'Save' button
Mouse.Click(uISaveButton, new Point(44, 14));
int foundItemNo1 = int.Parse(uIFundNr127Pane.InnerText.Substring(9));
// Click 'Register found item' link
Mouse.Click(uIRegisterFoundItemHyperlink, new Point(63, 7));
// Click 'Save' button
Mouse.Click(uISaveButton, new Point(34, 11));
int foundItemNo2 = int.Parse(uIFundNr128Pane.InnerText.Substring(9));
Assert.IsTrue(foundItemNo1 + 1 == foundItemNo2);
// Click 'Close' button
Mouse.Click(uICloseButton, new Point(26, 11));
A readable test case
Scenario: New found items should receive a consecutive number for the
current year
Given the previous found item of the
current year had the number 145
When I register a new found item
Then the last found item of the
current year should have the number 146
When to test
Acceptance criteria
(ATDD, BDD)
UnitTests
(TDD)
business view
technical view
Exploratory tests
Workflow tests
Performance, Scalability,
Usability,Security, …
definingtheproduct
criticizingtheproduct
New dimension: defining the product
Synergy: Specification of requirements and tests
AgileTesting Quadrants: Brian Marick
Manual testing is still necessary!
User
journeys
Acceptance-
criteria
Units
exploratory
testing
Manual Check
after story done
Main
success paths
Undiscovered
acceptance criteria
No/(few) manual
regression checks
Few paths
are enough
More time
for exploration
Cross-functional work
Sprint 1 Sprint 2 Sprint 3
short iteration
US4
Plan
Implement &
autom. test
US5
Plan
Implement &
autom. test
US2
Plan
Implement &
autom. test
US3
Plan
Implement &
autom. test
US6
Plan
Implement &
autom. test
US1
Plan
Implement &
autom. test
US7
Plan
Implement &
autom. test
US8
Plan
Implement &
autom. test
US9
Plan
Implement &
autom. test
ExploratoryTests
Specification and test
Collaboration for defining
acceptance criteria
Collaboration for
automation
Preventing bugs
instead of finding them!
Extension of
“Test Cases”
Limit
WIP
Collaboration
in manual
testing
Impact on development
TDD workflow
Extending TDD for business
Transparency for stakeholders
In Progress
Current sprint report: all scenarios
Starting with first scenario
Finishing first scenario
Progressing on scenarios
Progressing on scenarios
Progressing on scenarios
Progressing on scenarios
First user story ready for testing
Earlier start of manual testing
Done work can break again
Done work can break again
Business readable error reports
Living documentation
Higher level scenario structure
Story Mapping
Why?
How?
Acceptance
Criterion
Epic
Deliverable, Outcome
Impact, Goal
User Activity
User Story
Example
Code
Easier to define upfront Harder to define upfront
Story Maps
• Original concept by Jeff Patton
• Support iterative product design
• Optimized for desired outcome or
deliverable the system should support
Building story maps
Get popular books
delivered fast and
conveniently
Find
book
Collect
books
Order
Wait for
delivery
Receive
delivery
time
browse
best
sellers
shopping
basket
enter
address
delivery
slip
delivery
notification
pay with
credit
card
search
book by
title
wish list
inquiry
order
status
desired outcome
or deliverable
user
activities
system
features
necessity
Walking
skeleton
Enabling build–measure-learn
Get popular books
delivered fast and
conveniently
Find
book
Collect
books
Order
Wait for
delivery
Receive
delivery
time
browse
best
sellers
enter
address
delivery
slip
pay with
credit
card
search
book by
title
wish list
inquiry
order
status
necessity
shopping
basket
delivery
notification
manual
work-
around
omitted
steps
desired outcome
or deliverable
user
activities
system
features
Story Map example
Sprint 1
Sprint 2
Sprint 3
Sprint 4
Dropped Scope
Added scope
User Stories vs. Features
Product/Sprint Backlog
User Story 1
AccCrit 1
AccCrit 2
User Story 2
AccCrit 3
AccCrit 4
Living Documentation
Feature 1
AccCrit 1
AccCrit 2
Feature n
AccCrit 4
AccCrit m
User Story n
AccCrit 5
AccCrit m
AccCrit 3
AccCrit 5
„Done“
• Future options of the system
• Organized/refined according to
priority, value, effort, risk, ...
• Next possible increments of
the product (units of work)
• Current state of the system
• Organized/refined for
functional overview
• Versioned and maintained
together with source code
Living documentation
• Link business
readable
automation
from source
code to story
maps
• Feed execution
results into
story maps
• Sync Story
Story Map
requirements
withTFS
work items
Lessons learned
Collaboration over specification
Documentation after conversation
• Collaborative discovery
• Should trigger new questions
• Infinite number of examples
• Readability and ability to automate
Level of automation
Controller
Business Layer
Data Layer
Model
View
Browser automation
Trigger behavior
through controller
Assert behavior
on model, db, ..
Setup pre-conditions
through service
interfaces
Out-of-process
In-process
Test execution performance
• Grouping tests
– Current WIP
– Completed work
• Database
– In-memory
– Templates for setup
• Parallel execution
• Smart execution order
• Level of automation
Internal vs. external DSL
Example Source: Liz Keogh
https://github.com/lunivore/tictactoe-java/blob/master/
scenarios/com/lunivore/tictactoe/scenarios/Three_in_a_row_wins.java
Non-functional acceptance criteria
Given there are 100,000 users registered on the system
When I create a new account
Then I should be taken to my dashboard within 5ms
When 1000 users are hitting the homepage simultaneously
Then each user should get a response within 2ms
Matt Wynne
http://blog.mattwynne.net/2012/03/13/using-cucumber-for-load-testing
Tools
Gherkin automation tools
www.cukes.info
www.behat.org
www.specflow.org
Ruby, Java, JavaScript, C++
.NET, Mono, Silverlight,WP7
PHP
Books
Bridiging the
Communication Gap
Gojko Adzic
Specification by
Example
Gojko Adzic
Explore It!
Elisabeth
Hendrickson
Exploratory
Software
Testing
James
Whittaker
Summary
Specification-by-example
• Collaborative discovery and specification
• Impact on development and test
• Combine with other agile reqmt methods
• Can be supported by tools
Questions
Got curious about Specification-By-Example?
Learned something new about using SpecFlow?
Christian Hassa,TechTalk
(ch@techtalk.at, @chr99ha)
Join the BoF Session
„Specification-By-Example (ATDD, BDD) in practice“
Today at 15h15 (right after this talk)

More Related Content

What's hot

Techniques for Effectively Slicing User Stories by Naresh Jain
Techniques for Effectively Slicing User Stories by Naresh JainTechniques for Effectively Slicing User Stories by Naresh Jain
Techniques for Effectively Slicing User Stories by Naresh Jain
Naresh Jain
 
Acceptance criteria
Acceptance criteriaAcceptance criteria
Acceptance criteria
Softheme
 
How to Break the Requirements into User Stories
How to Break the Requirements into User StoriesHow to Break the Requirements into User Stories
How to Break the Requirements into User Stories
ShriKant Vashishtha
 
BDD in Action – principles, practices and real-world application
BDD in Action – principles, practices and real-world applicationBDD in Action – principles, practices and real-world application
BDD in Action – principles, practices and real-world application
John Ferguson Smart Limited
 

What's hot (20)

BDD & Cucumber
BDD & CucumberBDD & Cucumber
BDD & Cucumber
 
Specification-By-Example with Gherkin
Specification-By-Example with GherkinSpecification-By-Example with Gherkin
Specification-By-Example with Gherkin
 
Behavior-Driven Development and Automation Testing Using Cucumber Framework W...
Behavior-Driven Development and Automation Testing Using Cucumber Framework W...Behavior-Driven Development and Automation Testing Using Cucumber Framework W...
Behavior-Driven Development and Automation Testing Using Cucumber Framework W...
 
Specification by Example
Specification by ExampleSpecification by Example
Specification by Example
 
BDD presentation
BDD presentationBDD presentation
BDD presentation
 
Gherkin /BDD intro
Gherkin /BDD introGherkin /BDD intro
Gherkin /BDD intro
 
Tutorial: Implementing Specification-By-Example with Gherkin
Tutorial: Implementing Specification-By-Example with GherkinTutorial: Implementing Specification-By-Example with Gherkin
Tutorial: Implementing Specification-By-Example with Gherkin
 
Bdd – with cucumber and gherkin
Bdd – with cucumber and gherkinBdd – with cucumber and gherkin
Bdd – with cucumber and gherkin
 
Test Automation Framework using Cucumber BDD overview (part 1)
Test Automation Framework using Cucumber BDD overview (part 1)Test Automation Framework using Cucumber BDD overview (part 1)
Test Automation Framework using Cucumber BDD overview (part 1)
 
Techniques for Effectively Slicing User Stories by Naresh Jain
Techniques for Effectively Slicing User Stories by Naresh JainTechniques for Effectively Slicing User Stories by Naresh Jain
Techniques for Effectively Slicing User Stories by Naresh Jain
 
Acceptance criteria
Acceptance criteriaAcceptance criteria
Acceptance criteria
 
BDD with Cucumber
BDD with CucumberBDD with Cucumber
BDD with Cucumber
 
BDD testing with cucumber
BDD testing with cucumberBDD testing with cucumber
BDD testing with cucumber
 
Acceptance Test Driven Development
Acceptance Test Driven DevelopmentAcceptance Test Driven Development
Acceptance Test Driven Development
 
Automated Test Framework with Cucumber
Automated Test Framework with CucumberAutomated Test Framework with Cucumber
Automated Test Framework with Cucumber
 
"How to write better User Stories" por @jrhuerta
"How to write better User Stories" por @jrhuerta"How to write better User Stories" por @jrhuerta
"How to write better User Stories" por @jrhuerta
 
User Story Mapping
User Story MappingUser Story Mapping
User Story Mapping
 
How to Break the Requirements into User Stories
How to Break the Requirements into User StoriesHow to Break the Requirements into User Stories
How to Break the Requirements into User Stories
 
Splitting Stories with the Hamburger Method - A Simple 5 Step Process
Splitting Stories with the Hamburger Method - A Simple 5 Step ProcessSplitting Stories with the Hamburger Method - A Simple 5 Step Process
Splitting Stories with the Hamburger Method - A Simple 5 Step Process
 
BDD in Action – principles, practices and real-world application
BDD in Action – principles, practices and real-world applicationBDD in Action – principles, practices and real-world application
BDD in Action – principles, practices and real-world application
 

Viewers also liked

Как быть заказчиком продукта?
Как быть заказчиком продукта?Как быть заказчиком продукта?
Как быть заказчиком продукта?
Denis Beskov
 
Practical way to experience of Specification by Example
Practical way to experience of Specification by ExamplePractical way to experience of Specification by Example
Practical way to experience of Specification by Example
Larry Cai
 
Обзор рынка проектирования интерфейсов 2010
Обзор рынка проектирования интерфейсов 2010Обзор рынка проектирования интерфейсов 2010
Обзор рынка проектирования интерфейсов 2010
Denis Beskov
 
Use cases на практике
Use cases на практикеUse cases на практике
Use cases на практике
Softline
 
Разработка сценариев использования (use cases)
Разработка сценариев использования (use cases)Разработка сценариев использования (use cases)
Разработка сценариев использования (use cases)
Dmitry Strunkin
 

Viewers also liked (20)

Варианты использования. Введение
Варианты использования. ВведениеВарианты использования. Введение
Варианты использования. Введение
 
Specification by example and agile acceptance testing
Specification by example and agile acceptance testingSpecification by example and agile acceptance testing
Specification by example and agile acceptance testing
 
24 Typical Mistakes In Documents
24 Typical Mistakes In Documents24 Typical Mistakes In Documents
24 Typical Mistakes In Documents
 
Specification by example - course summary
Specification by example - course summarySpecification by example - course summary
Specification by example - course summary
 
Практики продуктовой разработки, которые я хотел бы видеть в заказной
Практики продуктовой разработки, которые я хотел бы видеть в заказнойПрактики продуктовой разработки, которые я хотел бы видеть в заказной
Практики продуктовой разработки, которые я хотел бы видеть в заказной
 
Полнота ролей и целей пользователей
Полнота ролей и целей пользователейПолнота ролей и целей пользователей
Полнота ролей и целей пользователей
 
Как быть заказчиком продукта?
Как быть заказчиком продукта?Как быть заказчиком продукта?
Как быть заказчиком продукта?
 
An Overview of User Acceptance Testing (UAT)
An Overview of User Acceptance Testing (UAT)An Overview of User Acceptance Testing (UAT)
An Overview of User Acceptance Testing (UAT)
 
BDD - beyond: Given, When and Then
BDD - beyond: Given, When and ThenBDD - beyond: Given, When and Then
BDD - beyond: Given, When and Then
 
Practical way to experience of Specification by Example
Practical way to experience of Specification by ExamplePractical way to experience of Specification by Example
Practical way to experience of Specification by Example
 
Acceptance Test Driven Development and Robot Framework
Acceptance Test Driven Development and Robot FrameworkAcceptance Test Driven Development and Robot Framework
Acceptance Test Driven Development and Robot Framework
 
Обзор рынка проектирования интерфейсов 2010
Обзор рынка проектирования интерфейсов 2010Обзор рынка проектирования интерфейсов 2010
Обзор рынка проектирования интерфейсов 2010
 
Death By PowerPoint Rus
Death By PowerPoint RusDeath By PowerPoint Rus
Death By PowerPoint Rus
 
Gherkin - crash course
Gherkin - crash courseGherkin - crash course
Gherkin - crash course
 
A New Dojo: The Art of Story Splitting
A New Dojo: The Art of Story SplittingA New Dojo: The Art of Story Splitting
A New Dojo: The Art of Story Splitting
 
Scrum Bangalore 13th meet up 13 june 2015 - behaviour driven development - vi...
Scrum Bangalore 13th meet up 13 june 2015 - behaviour driven development - vi...Scrum Bangalore 13th meet up 13 june 2015 - behaviour driven development - vi...
Scrum Bangalore 13th meet up 13 june 2015 - behaviour driven development - vi...
 
Specification by Example - Agile India 2015
Specification by Example - Agile India 2015Specification by Example - Agile India 2015
Specification by Example - Agile India 2015
 
Use cases на практике
Use cases на практикеUse cases на практике
Use cases на практике
 
Разработка сценариев использования (use cases)
Разработка сценариев использования (use cases)Разработка сценариев использования (use cases)
Разработка сценариев использования (use cases)
 
The Gherkin: Case Study
The Gherkin: Case StudyThe Gherkin: Case Study
The Gherkin: Case Study
 

Similar to Specification-By-Example with Gherkin

Specification-By-Example with Gherkin
Specification-By-Example with GherkinSpecification-By-Example with Gherkin
Specification-By-Example with Gherkin
Christian Hassa
 
The Growth Hacker's Playbook
The Growth Hacker's PlaybookThe Growth Hacker's Playbook
The Growth Hacker's Playbook
Danny Beck
 
Omni Channel Marketing Conference - Tim Fung
Omni Channel Marketing Conference - Tim FungOmni Channel Marketing Conference - Tim Fung
Omni Channel Marketing Conference - Tim Fung
Tony Booth
 

Similar to Specification-By-Example with Gherkin (20)

Specification-By-Example with Gherkin
Specification-By-Example with GherkinSpecification-By-Example with Gherkin
Specification-By-Example with Gherkin
 
Agile requirements management
Agile requirements managementAgile requirements management
Agile requirements management
 
Functional testing the_good_the_bad_and_the_ugly
Functional testing the_good_the_bad_and_the_uglyFunctional testing the_good_the_bad_and_the_ugly
Functional testing the_good_the_bad_and_the_ugly
 
Specification-By-Example with Gherkin
Specification-By-Example with GherkinSpecification-By-Example with Gherkin
Specification-By-Example with Gherkin
 
BDD, Gherkin, Cucumber and why we need it.
BDD, Gherkin, Cucumber and why we need it.BDD, Gherkin, Cucumber and why we need it.
BDD, Gherkin, Cucumber and why we need it.
 
«BDD, Gherkin, Cucumber and why we need it for successful product development»
«BDD, Gherkin, Cucumber and why we need it for successful product development»«BDD, Gherkin, Cucumber and why we need it for successful product development»
«BDD, Gherkin, Cucumber and why we need it for successful product development»
 
Cross mobile testautomation mit Xamarin & SpecFlow
Cross mobile testautomation mit Xamarin & SpecFlowCross mobile testautomation mit Xamarin & SpecFlow
Cross mobile testautomation mit Xamarin & SpecFlow
 
From impact to stakeholder examples: Three techniques for end-to-end requirem...
From impact to stakeholder examples: Three techniques for end-to-end requirem...From impact to stakeholder examples: Three techniques for end-to-end requirem...
From impact to stakeholder examples: Three techniques for end-to-end requirem...
 
To atdd-and-beyond
To atdd-and-beyondTo atdd-and-beyond
To atdd-and-beyond
 
Teaching Machines to Fish -- How eBay Improves Itself
Teaching Machines to Fish -- How eBay Improves ItselfTeaching Machines to Fish -- How eBay Improves Itself
Teaching Machines to Fish -- How eBay Improves Itself
 
Pragmatic Guide to Enhanced Data Capabilities
Pragmatic Guide to Enhanced Data CapabilitiesPragmatic Guide to Enhanced Data Capabilities
Pragmatic Guide to Enhanced Data Capabilities
 
Prepare for Peak Holiday Season with MongoDB
Prepare for Peak Holiday Season with MongoDBPrepare for Peak Holiday Season with MongoDB
Prepare for Peak Holiday Season with MongoDB
 
The Growth Hacker's Playbook
The Growth Hacker's PlaybookThe Growth Hacker's Playbook
The Growth Hacker's Playbook
 
Jbehave + serenity
Jbehave + serenityJbehave + serenity
Jbehave + serenity
 
apriori.pptx
apriori.pptxapriori.pptx
apriori.pptx
 
Building a Pyramid: Symfony Testing Strategies
Building a Pyramid: Symfony Testing StrategiesBuilding a Pyramid: Symfony Testing Strategies
Building a Pyramid: Symfony Testing Strategies
 
CS 101- Content Inventory and Audit Process Overview
CS 101- Content Inventory and Audit Process OverviewCS 101- Content Inventory and Audit Process Overview
CS 101- Content Inventory and Audit Process Overview
 
The Art and Science of Selling and Growth Hacking
The Art and Science of Selling and  Growth HackingThe Art and Science of Selling and  Growth Hacking
The Art and Science of Selling and Growth Hacking
 
Building Autonomous Services
Building Autonomous ServicesBuilding Autonomous Services
Building Autonomous Services
 
Omni Channel Marketing Conference - Tim Fung
Omni Channel Marketing Conference - Tim FungOmni Channel Marketing Conference - Tim Fung
Omni Channel Marketing Conference - Tim Fung
 

More from Christian Hassa

Impact Mapping - strategische Steuerung agiler Entwicklung
Impact Mapping - strategische Steuerung agiler EntwicklungImpact Mapping - strategische Steuerung agiler Entwicklung
Impact Mapping - strategische Steuerung agiler Entwicklung
Christian Hassa
 
Impact Maps and Story Maps: delivering what really matters
Impact Maps and Story Maps: delivering what really mattersImpact Maps and Story Maps: delivering what really matters
Impact Maps and Story Maps: delivering what really matters
Christian Hassa
 
Live it - or leave it! Returning your investment into Agile
Live it - or leave it! Returning your investment into AgileLive it - or leave it! Returning your investment into Agile
Live it - or leave it! Returning your investment into Agile
Christian Hassa
 

More from Christian Hassa (20)

Impact Mapping: Guiding Agile Teams with Customer Obsession (workshop)
Impact Mapping: Guiding Agile Teams with Customer Obsession (workshop)Impact Mapping: Guiding Agile Teams with Customer Obsession (workshop)
Impact Mapping: Guiding Agile Teams with Customer Obsession (workshop)
 
Agile Software Development - Cargo Cult or Competitive Advantage?
Agile Software Development - Cargo Cult or Competitive Advantage?Agile Software Development - Cargo Cult or Competitive Advantage?
Agile Software Development - Cargo Cult or Competitive Advantage?
 
Impact Mapping - strategische Steuerung agiler Entwicklung
Impact Mapping - strategische Steuerung agiler EntwicklungImpact Mapping - strategische Steuerung agiler Entwicklung
Impact Mapping - strategische Steuerung agiler Entwicklung
 
Impact Mapping - strategische Steuerung agiler Entwicklung
Impact Mapping - strategische Steuerung agiler EntwicklungImpact Mapping - strategische Steuerung agiler Entwicklung
Impact Mapping - strategische Steuerung agiler Entwicklung
 
Scrum Alliance Webinar: Impact Mapping
Scrum Alliance Webinar: Impact MappingScrum Alliance Webinar: Impact Mapping
Scrum Alliance Webinar: Impact Mapping
 
Impact Mapping - delivering what really matters!
Impact Mapping - delivering what really matters!Impact Mapping - delivering what really matters!
Impact Mapping - delivering what really matters!
 
Impact Mapping - strategische Steuerung für agile Entwicklung
Impact Mapping - strategische Steuerung für agile EntwicklungImpact Mapping - strategische Steuerung für agile Entwicklung
Impact Mapping - strategische Steuerung für agile Entwicklung
 
Upcoming events 2017
Upcoming events 2017Upcoming events 2017
Upcoming events 2017
 
Impact Mapping with Innovation Games (TM)
Impact Mapping with Innovation Games (TM)Impact Mapping with Innovation Games (TM)
Impact Mapping with Innovation Games (TM)
 
Impact Mapping with Innovation Games (R)
Impact Mapping with Innovation Games (R)Impact Mapping with Innovation Games (R)
Impact Mapping with Innovation Games (R)
 
Impact Map Your Project
Impact Map Your ProjectImpact Map Your Project
Impact Map Your Project
 
Impact Maps/Story Maps - liefern was wirklich zählt
Impact Maps/Story Maps - liefern was wirklich zähltImpact Maps/Story Maps - liefern was wirklich zählt
Impact Maps/Story Maps - liefern was wirklich zählt
 
Agile Requirements
Agile RequirementsAgile Requirements
Agile Requirements
 
Impact Maps and Story Maps: delivering what really matters
Impact Maps and Story Maps: delivering what really mattersImpact Maps and Story Maps: delivering what really matters
Impact Maps and Story Maps: delivering what really matters
 
Impact Maps und Story Maps - liefern was wirklich zählt
Impact Maps und Story Maps - liefern was wirklich zähltImpact Maps und Story Maps - liefern was wirklich zählt
Impact Maps und Story Maps - liefern was wirklich zählt
 
How I learned to stop worrying and love flexible scope.
How I learned to stop worrying and love flexible scope.How I learned to stop worrying and love flexible scope.
How I learned to stop worrying and love flexible scope.
 
Story Maps Workshop (German) - DNUG Bern
Story Maps Workshop (German) - DNUG BernStory Maps Workshop (German) - DNUG Bern
Story Maps Workshop (German) - DNUG Bern
 
Live it - or leave it! Returning your investment into Agile
Live it - or leave it! Returning your investment into AgileLive it - or leave it! Returning your investment into Agile
Live it - or leave it! Returning your investment into Agile
 
How I learned stop worrying and how to love flexible scope.
How I learned stop worrying and how to love flexible scope.How I learned stop worrying and how to love flexible scope.
How I learned stop worrying and how to love flexible scope.
 
Story Maps - Liefern was wirklich zählt
Story Maps - Liefern was wirklich zähltStory Maps - Liefern was wirklich zählt
Story Maps - Liefern was wirklich zählt
 

Specification-By-Example with Gherkin

  • 2. Almost 4 years later … #106 inVisual StudioGallery (30.1.2013) #135 in NuGet (30.1.2013) ~800 visits daily ~ 4000 active users > 30 contributors
  • 3. Goals Make you curious about Specification-By-Example New perspective for SpecFlow users
  • 4. About me • Managing Partner atTechTalk • Agile consulting and delivery • Based in Europe: Vienna, Budapest, Zurich
  • 5. Terminology • Specification-By-Example • AcceptanceTest Driven Development (ATDD) • Behaviour Driven Development (BDD)
  • 7. Purpose of user stories? • Describe user needs • Unit of prioritization • Unit of planning • Reminder for a discussion • Mechanism to defer detail
  • 8. Requirement levels Impact Mapping Story Mapping Specification-By-Example Why? How? Acceptance Criterion Epic Deliverable, Outcome Impact, Goal Easier to define upfront Harder to define upfront Reminder for a discussion Bug report Isolated, formalized example User Activity User Story Example Code
  • 9. Collecting acceptance criteria “I would try to put a book into the shopping cart …” “I would try to remove a book from the shopping cart…” “I’d check whether the shopping cart is empty, when I enter the shop …” “I would try to add the same book again to the shopping cart …” Books can be placed into shopping cart. Books can be removed from shopping cart. Shopping cart should be empty when entering the shop. Adding same book again to shopping cart should increase quantity. As a potential customer I want to collect books in a shopping cart So that I can order several books at once. “Imagine this story is already implemented: What would you try out?”
  • 11. Examples for user stories UI wire frames, existing UI rules, key examples existing artifacts, mock-ups
  • 12. Discussion of acceptance criteria Original idea for the illustration:George Dinwiddie http://blog.gdinwidiee.com public void TestInitialOrderDiscount() { Customer newCustomer = new Customer(); Order newOrder = new Order(newCustomer); newOrder.AddBook( Catalog.Find(“ISBN-0955683610”) ); Assert.Equals(33.75, newOrder.Subtotal); } Register as “bart_bookworm” Go to “/catalog/search” Enter “ISBN-0955683610” Click “Search” Click “Add to Cart” Click “View Cart” Verify “Subtotal” is “$33.75” We would like to encourage new users to buy in our shop. Therefore we offer 10% discount for their first order.
  • 13. Specification-By-Example Examples … – make abstract descriptions better understandable – are usually not formally exchanged or documented Brian Marick Examples Tests Requirements consist of describe verify fulfillment of
  • 14. Discussion of acceptance criteria Original idea for the illustration:George Dinwiddie http://blog.gdinwidiee.com public void TestInitialOrderDiscount() { Customer newCustomer = new Customer(); Order newOrder = new Order(newCustomer); newOrder.AddBook( Catalog.Find(“ISBN-0955683610”) ); Assert.Equals(33.75, newOrder.Subtotal); } Register as “bart_bookworm” Go to “/catalog/search” Enter “ISBN-0955683610” Click “Search” Click “Add to Cart” Click “View Cart” Verify “Subtotal” is “$33.75” We would like to encourage new users to buy in our shop. Therefore we offer 10% discount for their first order.
  • 15. Illustrated with shared examples Original idea for the illustration:George Dinwiddie http://blog.gdinwidiee.com Given the user has not ordered yet When the user adds a book with the price of USD 37.50 into the shopping cart Then the shopping cart sub-total should be USD 33.75.
  • 16. Discover new aspects Original idea for the illustration:George Dinwiddie http://blog.gdinwidiee.com Actually, this not quite right: Books on sale should be excluded.
  • 18. Abstract acceptance criteria As a shop visitor I want to collect books in my shopping basket so that I can purchase multiple books at once. Books can be added to the shopping basket Books can be removed from the shopping basket Shopping basket is initially empty The same book can be added multiple times to the shopping basket
  • 19. Formalizing examples As a shop visitor I want to collect books in my shopping basket so that I can purchase multiple books at once. Books can be added to the shopping basket Given my shopping basket is empty When I add the book “Harry Potter” to my shopping basket Then my shopping basket should contain 1 copy of “Harry Potter”
  • 20. Formalizing examples As a shop visitor I want to collect books in my shopping basket so that I can purchase multiple books at once. Books can be added to the shopping basketThe same book can be added multiple times to the shopping basket When I add the book “Harry Potter” to my shopping basket Then my shopping basket should contain 2 copies of “Harry Potter” Given my shopping basket contains 1 copy of “Harry Potter”
  • 21. Example structure The same book can be added multiple times to the shopping basket When I add the book “Harry Potter” to my shopping basket Then my shopping basket should contain 2 copies of “Harry Potter” Given my shopping basket contains 1 copy of “Harry Potter” Title: Describes isolated, focused intention=abstract acceptance criterion Arrange: Context, initial state of the system Act: Execution of the feature Assert: Assertion of observable behavior And I should see the warning: “Book already existed in basket” Triple-A constraint “Checks” Steps can be chained up Evolution of a domain specific test and specification language.
  • 22. Life time of examples
  • 23. Purpose of examples • Shared understanding: acceptance criteria • Documentation: system details • Regression-tests: violated assumptions
  • 24. Continuous validation When I add the book “Harry Potter” to my shopping basket Then my shopping basket should contain 2 copies of “Harry Potter” Given my shopping basket contains 1 copy of “Harry Potter” „Step Definitions“ are binding individual steps to an automatable interface of the application. Automation does not necessarily have to bind to the UI. Automation of system is supported/evolving with development. SystemUI Automation
  • 25. Demo Gherkin automation for .NET • Visual Studio plugin (VS-Gallery) • NuGet Package http://www.specflow.org
  • 27. Test automation becomes expensive, when … • trying to automate manual tests • making tests unreadable when automating them • automating only after completing implementation structure readability when to test
  • 28. Structure Manual tests Asserts Multiple combined features Structure ACT-ASSERT- ACT-ASSERT- ACT-ASSERT- … Dependent on existence of other features Long test path with high chance to break Cause and impact of an error hard to trace. Automated Check Single aspect of a single feature ARRANGE – ACT – ASSERT Independent of other features Short test path with lower chance to break Cause and impact of an error easier to trace.
  • 30. Readability // Go to web page 'http://localhost:40001/' using new browser instance BrowserWindow localhostBrowser = BrowserWindow.Launch( new System.Uri(this.RecordedMethod1Params.Url)); // Click 'Register found item' link Mouse.Click(uIRegisterFoundItemHyperlink, new Point(56, 9)); // Click 'Save' button Mouse.Click(uISaveButton, new Point(44, 14)); int foundItemNo1 = int.Parse(uIFundNr127Pane.InnerText.Substring(9)); // Click 'Register found item' link Mouse.Click(uIRegisterFoundItemHyperlink, new Point(63, 7)); // Click 'Save' button Mouse.Click(uISaveButton, new Point(34, 11)); int foundItemNo2 = int.Parse(uIFundNr128Pane.InnerText.Substring(9)); Assert.IsTrue(foundItemNo1 + 1 == foundItemNo2); // Click 'Close' button Mouse.Click(uICloseButton, new Point(26, 11));
  • 31. A readable test case Scenario: New found items should receive a consecutive number for the current year Given the previous found item of the current year had the number 145 When I register a new found item Then the last found item of the current year should have the number 146
  • 32. When to test Acceptance criteria (ATDD, BDD) UnitTests (TDD) business view technical view Exploratory tests Workflow tests Performance, Scalability, Usability,Security, … definingtheproduct criticizingtheproduct New dimension: defining the product Synergy: Specification of requirements and tests AgileTesting Quadrants: Brian Marick
  • 33. Manual testing is still necessary! User journeys Acceptance- criteria Units exploratory testing Manual Check after story done Main success paths Undiscovered acceptance criteria No/(few) manual regression checks Few paths are enough More time for exploration
  • 34. Cross-functional work Sprint 1 Sprint 2 Sprint 3 short iteration US4 Plan Implement & autom. test US5 Plan Implement & autom. test US2 Plan Implement & autom. test US3 Plan Implement & autom. test US6 Plan Implement & autom. test US1 Plan Implement & autom. test US7 Plan Implement & autom. test US8 Plan Implement & autom. test US9 Plan Implement & autom. test ExploratoryTests Specification and test Collaboration for defining acceptance criteria Collaboration for automation Preventing bugs instead of finding them! Extension of “Test Cases” Limit WIP Collaboration in manual testing
  • 37. Extending TDD for business
  • 39. Current sprint report: all scenarios
  • 46. First user story ready for testing
  • 47. Earlier start of manual testing
  • 48. Done work can break again
  • 49. Done work can break again
  • 52. Higher level scenario structure Story Mapping Why? How? Acceptance Criterion Epic Deliverable, Outcome Impact, Goal User Activity User Story Example Code Easier to define upfront Harder to define upfront
  • 53. Story Maps • Original concept by Jeff Patton • Support iterative product design • Optimized for desired outcome or deliverable the system should support
  • 54. Building story maps Get popular books delivered fast and conveniently Find book Collect books Order Wait for delivery Receive delivery time browse best sellers shopping basket enter address delivery slip delivery notification pay with credit card search book by title wish list inquiry order status desired outcome or deliverable user activities system features necessity
  • 55. Walking skeleton Enabling build–measure-learn Get popular books delivered fast and conveniently Find book Collect books Order Wait for delivery Receive delivery time browse best sellers enter address delivery slip pay with credit card search book by title wish list inquiry order status necessity shopping basket delivery notification manual work- around omitted steps desired outcome or deliverable user activities system features
  • 63. User Stories vs. Features Product/Sprint Backlog User Story 1 AccCrit 1 AccCrit 2 User Story 2 AccCrit 3 AccCrit 4 Living Documentation Feature 1 AccCrit 1 AccCrit 2 Feature n AccCrit 4 AccCrit m User Story n AccCrit 5 AccCrit m AccCrit 3 AccCrit 5 „Done“ • Future options of the system • Organized/refined according to priority, value, effort, risk, ... • Next possible increments of the product (units of work) • Current state of the system • Organized/refined for functional overview • Versioned and maintained together with source code
  • 64. Living documentation • Link business readable automation from source code to story maps • Feed execution results into story maps • Sync Story Story Map requirements withTFS work items
  • 66. Collaboration over specification Documentation after conversation • Collaborative discovery • Should trigger new questions • Infinite number of examples • Readability and ability to automate
  • 67. Level of automation Controller Business Layer Data Layer Model View Browser automation Trigger behavior through controller Assert behavior on model, db, .. Setup pre-conditions through service interfaces Out-of-process In-process
  • 68. Test execution performance • Grouping tests – Current WIP – Completed work • Database – In-memory – Templates for setup • Parallel execution • Smart execution order • Level of automation
  • 69. Internal vs. external DSL Example Source: Liz Keogh https://github.com/lunivore/tictactoe-java/blob/master/ scenarios/com/lunivore/tictactoe/scenarios/Three_in_a_row_wins.java
  • 70. Non-functional acceptance criteria Given there are 100,000 users registered on the system When I create a new account Then I should be taken to my dashboard within 5ms When 1000 users are hitting the homepage simultaneously Then each user should get a response within 2ms Matt Wynne http://blog.mattwynne.net/2012/03/13/using-cucumber-for-load-testing
  • 71. Tools
  • 72. Gherkin automation tools www.cukes.info www.behat.org www.specflow.org Ruby, Java, JavaScript, C++ .NET, Mono, Silverlight,WP7 PHP
  • 73.
  • 74.
  • 75. Books Bridiging the Communication Gap Gojko Adzic Specification by Example Gojko Adzic Explore It! Elisabeth Hendrickson Exploratory Software Testing James Whittaker
  • 76. Summary Specification-by-example • Collaborative discovery and specification • Impact on development and test • Combine with other agile reqmt methods • Can be supported by tools
  • 77. Questions Got curious about Specification-By-Example? Learned something new about using SpecFlow? Christian Hassa,TechTalk (ch@techtalk.at, @chr99ha) Join the BoF Session „Specification-By-Example (ATDD, BDD) in practice“ Today at 15h15 (right after this talk)