Specification By
   Example
   Declan Whelan
    @dwhelan
What’s the Problem?
Waterfall




Source: http://www.cs.umd.edu/class/spring2003/cmsc838p/Process/waterfall.pdf
Waterfall



                              X
Source: http://www.cs.umd.edu/class/spring2003/cmsc838p/Process/waterfall.pdf
Cost Of Change


 Cost%
  Of%
Change%




                 Time%
Agile Specification
                  Agile&Approach&
           Itera/on&1&          Itera/on&2&           Itera/on&3&          Itera/on&4&
           Requirements&        Requirements&         Requirements&        Requirements&

          Analysis&&&Design&   Analysis&&&Design&    Analysis&&&Design&   Analysis&&&Design&
 Cost%
  of%           Code&                Code&                 Code&                Code&
Change%
                Test&                Test&                 Test&                Test&

                                                                               Deploy&




                                             Time%
Traditional Flow
Traditional Flow



Specification
Traditional Flow

               Build


Specification
Traditional Flow

                 Build


Specification


               Tests Cases
Traditional Flow

                 Build


Specification                 Test

               Tests Cases
Traditional Flow

                 Build


Specification                 Test
                                    Deploy

               Tests Cases
We Found a Bug!
                                                How did QA miss this?



  That was obvious! I shouldn't                                             Developers don't test before
    have to spell out every                                                  they throw it over the wall!
            detail!


                                            Exec


                                                                          Tester
Product
Manager


                                                                                     We did what was in
                                                                                      functional spec!
                 Business requirements
                     weren't clear!

                                                  BA missed it in the
                                                technical requirements!
                                                                               Dev


            BA


                                         Architect
We Found a Bug!
                                                How did QA miss this?



  That was obvious! I shouldn't                                             Developers don't test before
    have to spell out every                                                  they throw it over the wall!
            detail!


                                            Exec


                                                                          Tester
Product
Manager


                                                                                     We did what was in
                                                                                      functional spec!
                 Business requirements
                     weren't clear!

                                                  BA missed it in the
                                                technical requirements!
                                                                               Dev


            BA


                                         Architect
We Found a Bug!
                                                How did QA miss this?



  That was obvious! I shouldn't                                             Developers don't test before
    have to spell out every                                                  they throw it over the wall!
            detail!


                                            Exec


                                                                          Tester
Product
Manager


                                                                                     We did what was in
                                                                                      functional spec!
                 Business requirements
                     weren't clear!

                                                  BA missed it in the
                                                technical requirements!
                                                                               Dev


            BA


                                         Architect
We Found a Bug!
                                                How did QA miss this?



  That was obvious! I shouldn't                                             Developers don't test before
    have to spell out every                                                  they throw it over the wall!
            detail!


                                            Exec


                                                                          Tester
Product
Manager


                                                                                     We did what was in
                                                                                      functional spec!
                 Business requirements
                     weren't clear!

                                                  BA missed it in the
                                                technical requirements!
                                                                               Dev


            BA


                                         Architect
We Found a Bug!
                                                How did QA miss this?



  That was obvious! I shouldn't                                             Developers don't test before
    have to spell out every                                                  they throw it over the wall!
            detail!


                                            Exec


                                                                          Tester
Product
Manager


                                                                                     We did what was in
                                                                                      functional spec!
                 Business requirements
                     weren't clear!

                                                  BA missed it in the
                                                technical requirements!
                                                                               Dev


            BA


                                         Architect
We Found a Bug!
                                                How did QA miss this?



  That was obvious! I shouldn't                                             Developers don't test before
    have to spell out every                                                  they throw it over the wall!
            detail!


                                            Exec


                                                                          Tester
Product
Manager


                                                                                     We did what was in
                                                                                      functional spec!
                 Business requirements
                     weren't clear!

                                                  BA missed it in the
                                                technical requirements!
                                                                               Dev


            BA


                                         Architect
We Found a Bug!
                                                How did QA miss this?



  That was obvious! I shouldn't                                             Developers don't test before
    have to spell out every                                                  they throw it over the wall!
            detail!


                                            Exec


                                                                          Tester
Product
Manager


                                                                                     We did what was in
                                                                                      functional spec!
                 Business requirements
                     weren't clear!

                                                  BA missed it in the
                                                technical requirements!
                                                                               Dev


            BA


                                         Architect
Accidental Adversaries
                               +
                                       Testing




                                                 +
                                                          New Bugs

                                                                                -      +
              -                                                             +
Testing           Additional                                                    Development
                                                         Software Fix
Success             Tests                                                         Success

          +                                                             -
+     -

                  New Build
                                   +



                                   Development

                                                     +
Functional Silos




Source: http://www.danpontefract.com/images/silo.jpg
Functional Silos



                    X
Source: http://www.danpontefract.com/images/silo.jpg
Build it Right




                                            Build the Right Thing




Specification By Example
Gojko Adzic, 2011 page 4
Build it Right




                                            Build the Right Thing



    Useless Crap
Specification By Example
Gojko Adzic, 2011 page 4
Build it Right
  Business Failure




                                            Build the Right Thing



    Useless Crap
Specification By Example
Gojko Adzic, 2011 page 4
Build it Right
  Business Failure




                                            Build the Right Thing



    Useless Crap                      Maintenance Nightmare
Specification By Example
Gojko Adzic, 2011 page 4
Build it Right
  Business Failure                             Business Success




                                            Build the Right Thing



    Useless Crap                      Maintenance Nightmare
Specification By Example
Gojko Adzic, 2011 page 4
Build it Right
  Business Failure                             Business Success


                                               Specification By
                                                  Example

                                            Build the Right Thing



    Useless Crap                      Maintenance Nightmare
Specification By Example
Gojko Adzic, 2011 page 4
What are Specifications
    By Example?
What are Specifications
    By Example?
• Thin slices of system behaviour
• that deliver business value
• described as concrete examples
• that are potentially automatable
• without translation
• to create executable specifications
• captured in live documentation.
Agile Testing Quadrants
Specification By Example
Specification By Example
        Business Goal
Specification By Example
        Business Goal
                Derive the scope
           Scope
Specification By Example
        Business Goal
                Derive the scope
           Scope
                Specify collaboratively
        Key Examples
Specification By Example
           Business Goal
                   Derive the scope
               Scope
                   Specify collaboratively
           Key Examples
                   Refine the specification
     Specification With Examples
Specification By Example
           Business Goal
                   Derive the scope
               Scope
                   Specify collaboratively
           Key Examples
                   Refine the specification
     Specification With Examples
                   Automate literally
      Executable Specification
Specification By Example
           Business Goal
                   Derive the scope
               Scope
                   Specify collaboratively
           Key Examples
                   Refine the specification
     Specification With Examples
                   Automate literally
      Executable Specification
                   Validate frequently
       Living Documentation
Specification By Example
                             Business Goal
                                     Derive the scope
                                 Scope




                                                               Shared Understanding
 Ubiquitous Language



                                     Specify collaboratively
                             Key Examples
                                     Refine the specification
                       Specification With Examples
                                     Automate literally
                        Executable Specification
                                     Validate frequently
                         Living Documentation
Source: https://docs.google.com/drawings/d/1cbfKq-KazcbMVCnRfih6zMSDBdtf90KviV7l2oxGyWM/edit
                                                                                              Specification By Example
                                                                                                                           Business Goal
                                                                                                                                   Derive the scope
                                                                                                                               Scope




                                                                                                                                                             Shared Understanding
                                                                                               Ubiquitous Language



                                                                                                                                   Specify collaboratively
                                                                                                                           Key Examples
                                                                                                                                   Refine the specification
                                                                                                                     Specification With Examples
                                                                                                                                   Automate literally
                                                                                                                      Executable Specification
                                                                                                                                   Validate frequently
                                                                                                                       Living Documentation
Derive the Scope: Story Mapping




Source: http://availagility.co.uk/wp-content/uploads/2008/10/user-story-mapping.png
Derive the Scope: User Stories
Derive the Scope: User Stories

            As a _______
         I want to _______
          So that _______
Derive the Scope: User Stories

               As a _______
            I want to _______
             So that _______


                As a student
    I want to purchase used books online
          So that I can save money
Specify Collaboratively: Workshops
Specify Collaboratively: Workshops
Specify Collaboratively: Workshops


 • Hold regular product backlog workshops
Specify Collaboratively: Workshops


 • Hold regular product backlog workshops
 • Full team workshops - when starting
Specify Collaboratively: Workshops


 • Hold regular product backlog workshops
 • Full team workshops - when starting
 • Three amigo workshops:
  • One developer
  • One tester
  • One analyst
Specify Collaboratively: Key Examples
Specify Collaboratively: Key Examples

             Given _______
             When _______
             Then _______
Specify Collaboratively: Key Examples

                     Given _______
                     When _______
                     Then _______


Given “War and Peace” is available as a used book for $2.99
        When Susan selects book“War and Peace”
          Then “Buy used for $2.99” is displayed
Refining the Specification
Refining the Specification
“Specifications with examples are acceptance tests”
                                     Gojko Adzic
Refining the Specification
“Specifications with examples are acceptance tests”
                                     Gojko Adzic


• Be precise and make sure spec is testable
Refining the Specification
“Specifications with examples are acceptance tests”
                                     Gojko Adzic


• Be precise and make sure spec is testable
• Avoid “scripts” and “flows”
Refining the Specification
“Specifications with examples are acceptance tests”
                                     Gojko Adzic


• Be precise and make sure spec is testable
• Avoid “scripts” and “flows”
• Focus on business functionality not design
Refining the Specification
“Specifications with examples are acceptance tests”
                                     Gojko Adzic


• Be precise and make sure spec is testable
• Avoid “scripts” and “flows”
• Focus on business functionality not design
• Avoid UI details
Refining the Specification
“Specifications with examples are acceptance tests”
                                     Gojko Adzic


• Be precise and make sure spec is testable
• Avoid “scripts” and “flows”
• Focus on business functionality not design
• Avoid UI details
• Avoid covering every possible combination
Refining the Specification: An Example
   Free Delivery
   Free delivery is offered to VIP customers once they purchase a certain number of books.
   Free delivery is not offered to regular customers or VIP customers buying anything other than
   books.

      Customer Type                      Cart Contents                        Delivery
                VIP                            5 books                    Free, Standard
                VIP                            4 books                        Standard
             Regular                          10 books                        Standard
                VIP                        5 dishwashers                      Standard
                VIP                  5 books, 1 dishwasher                    Standard

Source: Specification by Example: How successful teams deliver the right software, Gojko Adzic, pg. 116
Automating Examples
Automating Examples

• Start small
Automating Examples

• Start small
• Select important examples for automation
Automating Examples

• Start small
• Select important examples for automation
• Plan up-front to automate
Automating Examples

• Start small
• Select important examples for automation
• Plan up-front to automate
• Be prepared to go slower at the start
Automating Examples

• Start small
• Select important examples for automation
• Plan up-front to automate
• Be prepared to go slower at the start
• Treat automation code as a first class citizen
Automating Examples

• Start small
• Select important examples for automation
• Plan up-front to automate
• Be prepared to go slower at the start
• Treat automation code as a first class citizen
• Avoid record and playback
Automating Examples

• Start small
• Select important examples for automation
• Plan up-front to automate
• Be prepared to go slower at the start
• Treat automation code as a first class citizen
• Avoid record and playback
• Avoid using pre-populated data
Minimize Tests Through the UI
Automation Tools

FitNesse    Cucumber    Concordian

Green
             JBehave      Twist
Pepper
              Robot
Text Test                SpecFlow
            Framework
Validate Frequently
Validate Frequently
• Start with a Continuous Integration system
Validate Frequently
• Start with a Continuous Integration system
• Set up a Continuous Deployment system
Validate Frequently
• Start with a Continuous Integration system
• Set up a Continuous Deployment system
• Specify and test business logic separately
  from end-to-end flows
Validate Frequently
• Start with a Continuous Integration system
• Set up a Continuous Deployment system
• Specify and test business logic separately
  from end-to-end flows
• Organize tests along functional lines
Validate Frequently
• Start with a Continuous Integration system
• Set up a Continuous Deployment system
• Specify and test business logic separately
  from end-to-end flows
• Organize tests along functional lines
• Run all test nightly
Validate Frequently
• Start with a Continuous Integration system
• Set up a Continuous Deployment system
• Specify and test business logic separately
  from end-to-end flows
• Organize tests along functional lines
• Run all test nightly
• Consider an iteration “test pack”
Living Documentation
• Keep specifications short
• Evolve a specification language and leverage
  in with “common fixtures”
• Make documentation accessible - consider
  a wiki
• Organize the documentation
• Put specifications under version control
A Worked Example
Executable Specification
Cucumber Organization
                  fixtures are common




   Features     Step



               System
Cucumber Organization
                  fixtures are common




         ad
        Lo


   Features     Step



               System
Cucumber Organization
                     fixtures are common




              M
         ad




              at
        Lo




                ch
   Features       Step



               System
Cucumber Organization
                            fixtures are common




              M
         ad




              at
        Lo




                ch
   Features       Step



                     Call
               System
Feature File
  Feature: Turn cucumber into beer
   As a cucumber presenter
   I want beer after my presentation
   So I can enjoy the rest of DemoCampGuelph

   Scenario: Brydon buys Declan beer
    Given Brydon hosts DemoCampGuelph
    When Declan demos Cucumber
    Then Brydon should buy Declan 1 beer

   Scenario: Ali buys Declan beer
    Given Ali hosts DemoCampGuelph
    When Declan demos Cucumber
    Then Ali should buy Declan 1 beer
Step Definitions
Given /^(.+) hosts/ do |host|
 @event = Event.new(host)
end

When /^(.+) demos/ do |presenter|
 @event.add(presenter)
end

Then /^(.+) should buy (.+) (d+) (.*)$/ do |buyer, drinker,
qty, item|
 perk = @event.perks[0];
 perk.buyer.should == buyer; perk.receiver.should ==
drinker
 perk.quantity.should == quantity.to_i; perk.item.should
== item
end
System Under Test
 class Event
  attr_reader :perks

  def initialize(host) @host = host; @perks = [] end

  def add(presenter)
   @perks.push Perk.new(@host, presenter, 1, "beer")
  end
 end

 class Perk
  attr_reader :buyer, :receiver, :quantity, :item

  def initialize(buyer, receiver, quantity, item)
   @buyer = buyer; @receiver = receiver
   @quantity = quantity; @item = item
  end
 end
Execution
Scenario: Brydon buys Declan beer
 Given Brydon hosts DemoCampGuelph
 When Declan demos Cucumber
 Then Brydon should buy Declan 1 beer
Execution
Scenario: Brydon buys Declan beer
 Given Brydon hosts DemoCampGuelph
 When Declan demos Cucumber
 Then Brydon should buy Declan 1 beer
Given /^(.+) hosts/ do | When /^(.+) demos/ do |
host|                     presenter|
 @event =                   @event.add(presenter)
Event.new(host)           end
End
Then /^(.+) should buy (.+) (d+) (.*)$/ do |buyer,
drinker, qty, item|
 perk = @event.perks[0];
 perk.buyer.should == buyer; perk.receiver.should ==
drinker
Execution
Scenario: Brydon buys Declan beer
 Given Brydon hosts DemoCampGuelph
 When Declan demos Cucumber
 Then Brydon should buy Declan 1 beer
Given /^(.+) hosts/ do |
host|
 @event =
Event.new(host)
End
Execution
Scenario: Brydon buys Declan beer
 Given Brydon hosts DemoCampGuelph
 When Declan demos Cucumber
 Then Brydon should buy Declan 1 beer
Given /^(.+) hosts/ do |
host|                      “Brydon”
 @event =
Event.new(host)
End
Execution
Scenario: Brydon buys Declan beer
 Given Brydon hosts DemoCampGuelph
 When Declan demos Cucumber
 Then Brydon should buy Declan 1 beer
Execution
Scenario: Brydon buys Declan beer
 Given Brydon hosts DemoCampGuelph
 When Declan demos Cucumber
 Then Brydon should buy Declan 1 beer
Given /^(.+) hosts/ do | When /^(.+) demos/ do |
host|                     presenter|
 @event =                   @event.add(presenter)
Event.new(host)           end
End
Then /^(.+) should buy (.+) (d+) (.*)$/ do |buyer,
drinker, qty, item|
 perk = @event.perks[0];
 perk.buyer.should == buyer; perk.receiver.should ==
drinker
Execution
Scenario: Brydon buys Declan beer
 Given Brydon hosts DemoCampGuelph
 When Declan demos Cucumber
 Then Brydon should buy Declan 1 beer
                  When /^(.+) demos/ do |
                  presenter|
                   @event.add(presenter)
                  end
Execution
Scenario: Brydon buys Declan beer
 Given Brydon hosts DemoCampGuelph
 When Declan demos Cucumber
 Then Brydon should buy Declan 1 beer
                   When /^(.+) demos/ do |
                   presenter|
                    @event.add(presenter)
                   end
        “Declan”
Execution
Scenario: Brydon buys Declan beer
 Given Brydon hosts DemoCampGuelph
 When Declan demos Cucumber
 Then Brydon should buy Declan 1 beer
Execution
Scenario: Brydon buys Declan beer
 Given Brydon hosts DemoCampGuelph
 When Declan demos Cucumber
 Then Brydon should buy Declan 1 beer
Given /^(.+) hosts/ do | When /^(.+) demos/ do |
host|                     presenter|
 @event =                   @event.add(presenter)
Event.new(host)           end
End
Then /^(.+) should buy (.+) (d+) (.*)$/ do |buyer,
drinker, qty, item|
 perk = @event.perks[0];
 perk.buyer.should == buyer; perk.receiver.should ==
drinker
Execution
Scenario: Brydon buys Declan beer
 Given Brydon hosts DemoCampGuelph
 When Declan demos Cucumber
 Then Brydon should buy Declan 1 beer




Then /^(.+) should buy (.+) (d+) (.*)$/ do |buyer,
drinker, qty, item|
 perk = @event.perks[0];
 perk.buyer.should == buyer; perk.receiver.should ==
drinker
Execution
Scenario: Brydon buys Declan beer
 Given Brydon hosts DemoCampGuelph
 When Declan demos Cucumber
 Then Brydon should buy Declan 1 beer




Then /^(.+) should buy (.+) (d+) (.*)$/ do |buyer,
drinker, qty, item|
 perk = @event.perks[0];
 perk.buyer.should == buyer; perk.receiver.should ==
drinker
Source: https://docs.google.com/drawings/d/1cbfKq-KazcbMVCnRfih6zMSDBdtf90KviV7l2oxGyWM/edit
                                                                                              Specification By Example
                                                                                                                           Business Goal
                                                                                                                                   Derive the scope
                                                                                                                               Scope




                                                                                                                                                             Shared Understanding
                                                                                               Ubiquitous Language



                                                                                                                                   Specify collaboratively
                                                                                                                           Key Examples
                                                                                                                                   Refine the specification
                                                                                                                     Specification With Examples
                                                                                                                                   Automate literally
                                                                                                                      Executable Specification
                                                                                                                                   Validate frequently
                                                                                                                       Living Documentation
Build it Right




                                            Build the Right Thing




Specification By Example
Gojko Adzic, 2011 page 4
Build it Right




                                            Build the Right Thing



    Useless Crap
Specification By Example
Gojko Adzic, 2011 page 4
Build it Right
  Business Failure




                                            Build the Right Thing



    Useless Crap
Specification By Example
Gojko Adzic, 2011 page 4
Build it Right
  Business Failure




                                            Build the Right Thing



    Useless Crap                      Maintenance Nightmare
Specification By Example
Gojko Adzic, 2011 page 4
Build it Right
  Business Failure                             Business Success




                                            Build the Right Thing



    Useless Crap                      Maintenance Nightmare
Specification By Example
Gojko Adzic, 2011 page 4
Build it Right
  Business Failure                             Business Success


                                               Specification By
                                                  Example

                                            Build the Right Thing



    Useless Crap                      Maintenance Nightmare
Specification By Example
Gojko Adzic, 2011 page 4
Reading
Specification By Example
Gojko Adzic

The RSpec Book: Behaviour Driven Development with
RSpec, Cucumber and Friends
David Cheliminksy et al

Agile Testing: A Practical Guide for Testers and Agile Teams
Lisa Crispin, Janet Gregory
Diagram Credits

   Lisa Crispin and Janet Gregory
   Agile Testing: A Practical Guide for Testers
   and Agile Teams
   Addison-Wesley Professional; January 9, 2009.




   Mike Cohn
   Succeeding with Agile: Software Development
   Using Scrum
   Addison-Wesley Professional; November 5, 2009.

Specification by Example

  • 1.
    Specification By Example Declan Whelan @dwhelan
  • 2.
  • 3.
  • 4.
    Waterfall X Source: http://www.cs.umd.edu/class/spring2003/cmsc838p/Process/waterfall.pdf
  • 5.
    Cost Of Change Cost% Of% Change% Time%
  • 6.
    Agile Specification Agile&Approach& Itera/on&1& Itera/on&2& Itera/on&3& Itera/on&4& Requirements& Requirements& Requirements& Requirements& Analysis&&&Design& Analysis&&&Design& Analysis&&&Design& Analysis&&&Design& Cost% of% Code& Code& Code& Code& Change% Test& Test& Test& Test& Deploy& Time%
  • 7.
  • 8.
  • 9.
    Traditional Flow Build Specification
  • 10.
    Traditional Flow Build Specification Tests Cases
  • 11.
    Traditional Flow Build Specification Test Tests Cases
  • 12.
    Traditional Flow Build Specification Test Deploy Tests Cases
  • 13.
    We Found aBug! How did QA miss this? That was obvious! I shouldn't Developers don't test before have to spell out every they throw it over the wall! detail! Exec Tester Product Manager We did what was in functional spec! Business requirements weren't clear! BA missed it in the technical requirements! Dev BA Architect
  • 14.
    We Found aBug! How did QA miss this? That was obvious! I shouldn't Developers don't test before have to spell out every they throw it over the wall! detail! Exec Tester Product Manager We did what was in functional spec! Business requirements weren't clear! BA missed it in the technical requirements! Dev BA Architect
  • 15.
    We Found aBug! How did QA miss this? That was obvious! I shouldn't Developers don't test before have to spell out every they throw it over the wall! detail! Exec Tester Product Manager We did what was in functional spec! Business requirements weren't clear! BA missed it in the technical requirements! Dev BA Architect
  • 16.
    We Found aBug! How did QA miss this? That was obvious! I shouldn't Developers don't test before have to spell out every they throw it over the wall! detail! Exec Tester Product Manager We did what was in functional spec! Business requirements weren't clear! BA missed it in the technical requirements! Dev BA Architect
  • 17.
    We Found aBug! How did QA miss this? That was obvious! I shouldn't Developers don't test before have to spell out every they throw it over the wall! detail! Exec Tester Product Manager We did what was in functional spec! Business requirements weren't clear! BA missed it in the technical requirements! Dev BA Architect
  • 18.
    We Found aBug! How did QA miss this? That was obvious! I shouldn't Developers don't test before have to spell out every they throw it over the wall! detail! Exec Tester Product Manager We did what was in functional spec! Business requirements weren't clear! BA missed it in the technical requirements! Dev BA Architect
  • 19.
    We Found aBug! How did QA miss this? That was obvious! I shouldn't Developers don't test before have to spell out every they throw it over the wall! detail! Exec Tester Product Manager We did what was in functional spec! Business requirements weren't clear! BA missed it in the technical requirements! Dev BA Architect
  • 20.
    Accidental Adversaries + Testing + New Bugs - + - + Testing Additional Development Software Fix Success Tests Success + - + - New Build + Development +
  • 21.
  • 22.
    Functional Silos X Source: http://www.danpontefract.com/images/silo.jpg
  • 23.
    Build it Right Build the Right Thing Specification By Example Gojko Adzic, 2011 page 4
  • 24.
    Build it Right Build the Right Thing Useless Crap Specification By Example Gojko Adzic, 2011 page 4
  • 25.
    Build it Right Business Failure Build the Right Thing Useless Crap Specification By Example Gojko Adzic, 2011 page 4
  • 26.
    Build it Right Business Failure Build the Right Thing Useless Crap Maintenance Nightmare Specification By Example Gojko Adzic, 2011 page 4
  • 27.
    Build it Right Business Failure Business Success Build the Right Thing Useless Crap Maintenance Nightmare Specification By Example Gojko Adzic, 2011 page 4
  • 28.
    Build it Right Business Failure Business Success Specification By Example Build the Right Thing Useless Crap Maintenance Nightmare Specification By Example Gojko Adzic, 2011 page 4
  • 29.
  • 30.
    What are Specifications By Example? • Thin slices of system behaviour • that deliver business value • described as concrete examples • that are potentially automatable • without translation • to create executable specifications • captured in live documentation.
  • 31.
  • 32.
  • 33.
  • 34.
    Specification By Example Business Goal Derive the scope Scope
  • 35.
    Specification By Example Business Goal Derive the scope Scope Specify collaboratively Key Examples
  • 36.
    Specification By Example Business Goal Derive the scope Scope Specify collaboratively Key Examples Refine the specification Specification With Examples
  • 37.
    Specification By Example Business Goal Derive the scope Scope Specify collaboratively Key Examples Refine the specification Specification With Examples Automate literally Executable Specification
  • 38.
    Specification By Example Business Goal Derive the scope Scope Specify collaboratively Key Examples Refine the specification Specification With Examples Automate literally Executable Specification Validate frequently Living Documentation
  • 39.
    Specification By Example Business Goal Derive the scope Scope Shared Understanding Ubiquitous Language Specify collaboratively Key Examples Refine the specification Specification With Examples Automate literally Executable Specification Validate frequently Living Documentation
  • 40.
    Source: https://docs.google.com/drawings/d/1cbfKq-KazcbMVCnRfih6zMSDBdtf90KviV7l2oxGyWM/edit Specification By Example Business Goal Derive the scope Scope Shared Understanding Ubiquitous Language Specify collaboratively Key Examples Refine the specification Specification With Examples Automate literally Executable Specification Validate frequently Living Documentation
  • 41.
    Derive the Scope:Story Mapping Source: http://availagility.co.uk/wp-content/uploads/2008/10/user-story-mapping.png
  • 42.
    Derive the Scope:User Stories
  • 43.
    Derive the Scope:User Stories As a _______ I want to _______ So that _______
  • 44.
    Derive the Scope:User Stories As a _______ I want to _______ So that _______ As a student I want to purchase used books online So that I can save money
  • 45.
  • 46.
  • 47.
    Specify Collaboratively: Workshops • Hold regular product backlog workshops
  • 48.
    Specify Collaboratively: Workshops • Hold regular product backlog workshops • Full team workshops - when starting
  • 49.
    Specify Collaboratively: Workshops • Hold regular product backlog workshops • Full team workshops - when starting • Three amigo workshops: • One developer • One tester • One analyst
  • 50.
  • 51.
    Specify Collaboratively: KeyExamples Given _______ When _______ Then _______
  • 52.
    Specify Collaboratively: KeyExamples Given _______ When _______ Then _______ Given “War and Peace” is available as a used book for $2.99 When Susan selects book“War and Peace” Then “Buy used for $2.99” is displayed
  • 53.
  • 54.
    Refining the Specification “Specificationswith examples are acceptance tests” Gojko Adzic
  • 55.
    Refining the Specification “Specificationswith examples are acceptance tests” Gojko Adzic • Be precise and make sure spec is testable
  • 56.
    Refining the Specification “Specificationswith examples are acceptance tests” Gojko Adzic • Be precise and make sure spec is testable • Avoid “scripts” and “flows”
  • 57.
    Refining the Specification “Specificationswith examples are acceptance tests” Gojko Adzic • Be precise and make sure spec is testable • Avoid “scripts” and “flows” • Focus on business functionality not design
  • 58.
    Refining the Specification “Specificationswith examples are acceptance tests” Gojko Adzic • Be precise and make sure spec is testable • Avoid “scripts” and “flows” • Focus on business functionality not design • Avoid UI details
  • 59.
    Refining the Specification “Specificationswith examples are acceptance tests” Gojko Adzic • Be precise and make sure spec is testable • Avoid “scripts” and “flows” • Focus on business functionality not design • Avoid UI details • Avoid covering every possible combination
  • 60.
    Refining the Specification:An Example Free Delivery Free delivery is offered to VIP customers once they purchase a certain number of books. Free delivery is not offered to regular customers or VIP customers buying anything other than books. Customer Type Cart Contents Delivery VIP 5 books Free, Standard VIP 4 books Standard Regular 10 books Standard VIP 5 dishwashers Standard VIP 5 books, 1 dishwasher Standard Source: Specification by Example: How successful teams deliver the right software, Gojko Adzic, pg. 116
  • 61.
  • 62.
  • 63.
    Automating Examples • Startsmall • Select important examples for automation
  • 64.
    Automating Examples • Startsmall • Select important examples for automation • Plan up-front to automate
  • 65.
    Automating Examples • Startsmall • Select important examples for automation • Plan up-front to automate • Be prepared to go slower at the start
  • 66.
    Automating Examples • Startsmall • Select important examples for automation • Plan up-front to automate • Be prepared to go slower at the start • Treat automation code as a first class citizen
  • 67.
    Automating Examples • Startsmall • Select important examples for automation • Plan up-front to automate • Be prepared to go slower at the start • Treat automation code as a first class citizen • Avoid record and playback
  • 68.
    Automating Examples • Startsmall • Select important examples for automation • Plan up-front to automate • Be prepared to go slower at the start • Treat automation code as a first class citizen • Avoid record and playback • Avoid using pre-populated data
  • 69.
  • 70.
    Automation Tools FitNesse Cucumber Concordian Green JBehave Twist Pepper Robot Text Test SpecFlow Framework
  • 71.
  • 72.
    Validate Frequently • Startwith a Continuous Integration system
  • 73.
    Validate Frequently • Startwith a Continuous Integration system • Set up a Continuous Deployment system
  • 74.
    Validate Frequently • Startwith a Continuous Integration system • Set up a Continuous Deployment system • Specify and test business logic separately from end-to-end flows
  • 75.
    Validate Frequently • Startwith a Continuous Integration system • Set up a Continuous Deployment system • Specify and test business logic separately from end-to-end flows • Organize tests along functional lines
  • 76.
    Validate Frequently • Startwith a Continuous Integration system • Set up a Continuous Deployment system • Specify and test business logic separately from end-to-end flows • Organize tests along functional lines • Run all test nightly
  • 77.
    Validate Frequently • Startwith a Continuous Integration system • Set up a Continuous Deployment system • Specify and test business logic separately from end-to-end flows • Organize tests along functional lines • Run all test nightly • Consider an iteration “test pack”
  • 78.
    Living Documentation • Keepspecifications short • Evolve a specification language and leverage in with “common fixtures” • Make documentation accessible - consider a wiki • Organize the documentation • Put specifications under version control
  • 79.
  • 80.
  • 81.
    Cucumber Organization fixtures are common Features Step System
  • 82.
    Cucumber Organization fixtures are common ad Lo Features Step System
  • 83.
    Cucumber Organization fixtures are common M ad at Lo ch Features Step System
  • 84.
    Cucumber Organization fixtures are common M ad at Lo ch Features Step Call System
  • 85.
    Feature File Feature: Turn cucumber into beer As a cucumber presenter I want beer after my presentation So I can enjoy the rest of DemoCampGuelph Scenario: Brydon buys Declan beer Given Brydon hosts DemoCampGuelph When Declan demos Cucumber Then Brydon should buy Declan 1 beer Scenario: Ali buys Declan beer Given Ali hosts DemoCampGuelph When Declan demos Cucumber Then Ali should buy Declan 1 beer
  • 86.
    Step Definitions Given /^(.+)hosts/ do |host| @event = Event.new(host) end When /^(.+) demos/ do |presenter| @event.add(presenter) end Then /^(.+) should buy (.+) (d+) (.*)$/ do |buyer, drinker, qty, item| perk = @event.perks[0]; perk.buyer.should == buyer; perk.receiver.should == drinker perk.quantity.should == quantity.to_i; perk.item.should == item end
  • 87.
    System Under Test class Event attr_reader :perks def initialize(host) @host = host; @perks = [] end def add(presenter) @perks.push Perk.new(@host, presenter, 1, "beer") end end class Perk attr_reader :buyer, :receiver, :quantity, :item def initialize(buyer, receiver, quantity, item) @buyer = buyer; @receiver = receiver @quantity = quantity; @item = item end end
  • 88.
    Execution Scenario: Brydon buysDeclan beer Given Brydon hosts DemoCampGuelph When Declan demos Cucumber Then Brydon should buy Declan 1 beer
  • 89.
    Execution Scenario: Brydon buysDeclan beer Given Brydon hosts DemoCampGuelph When Declan demos Cucumber Then Brydon should buy Declan 1 beer Given /^(.+) hosts/ do | When /^(.+) demos/ do | host| presenter| @event = @event.add(presenter) Event.new(host) end End Then /^(.+) should buy (.+) (d+) (.*)$/ do |buyer, drinker, qty, item| perk = @event.perks[0]; perk.buyer.should == buyer; perk.receiver.should == drinker
  • 90.
    Execution Scenario: Brydon buysDeclan beer Given Brydon hosts DemoCampGuelph When Declan demos Cucumber Then Brydon should buy Declan 1 beer Given /^(.+) hosts/ do | host| @event = Event.new(host) End
  • 91.
    Execution Scenario: Brydon buysDeclan beer Given Brydon hosts DemoCampGuelph When Declan demos Cucumber Then Brydon should buy Declan 1 beer Given /^(.+) hosts/ do | host| “Brydon” @event = Event.new(host) End
  • 92.
    Execution Scenario: Brydon buysDeclan beer Given Brydon hosts DemoCampGuelph When Declan demos Cucumber Then Brydon should buy Declan 1 beer
  • 93.
    Execution Scenario: Brydon buysDeclan beer Given Brydon hosts DemoCampGuelph When Declan demos Cucumber Then Brydon should buy Declan 1 beer Given /^(.+) hosts/ do | When /^(.+) demos/ do | host| presenter| @event = @event.add(presenter) Event.new(host) end End Then /^(.+) should buy (.+) (d+) (.*)$/ do |buyer, drinker, qty, item| perk = @event.perks[0]; perk.buyer.should == buyer; perk.receiver.should == drinker
  • 94.
    Execution Scenario: Brydon buysDeclan beer Given Brydon hosts DemoCampGuelph When Declan demos Cucumber Then Brydon should buy Declan 1 beer When /^(.+) demos/ do | presenter| @event.add(presenter) end
  • 95.
    Execution Scenario: Brydon buysDeclan beer Given Brydon hosts DemoCampGuelph When Declan demos Cucumber Then Brydon should buy Declan 1 beer When /^(.+) demos/ do | presenter| @event.add(presenter) end “Declan”
  • 96.
    Execution Scenario: Brydon buysDeclan beer Given Brydon hosts DemoCampGuelph When Declan demos Cucumber Then Brydon should buy Declan 1 beer
  • 97.
    Execution Scenario: Brydon buysDeclan beer Given Brydon hosts DemoCampGuelph When Declan demos Cucumber Then Brydon should buy Declan 1 beer Given /^(.+) hosts/ do | When /^(.+) demos/ do | host| presenter| @event = @event.add(presenter) Event.new(host) end End Then /^(.+) should buy (.+) (d+) (.*)$/ do |buyer, drinker, qty, item| perk = @event.perks[0]; perk.buyer.should == buyer; perk.receiver.should == drinker
  • 98.
    Execution Scenario: Brydon buysDeclan beer Given Brydon hosts DemoCampGuelph When Declan demos Cucumber Then Brydon should buy Declan 1 beer Then /^(.+) should buy (.+) (d+) (.*)$/ do |buyer, drinker, qty, item| perk = @event.perks[0]; perk.buyer.should == buyer; perk.receiver.should == drinker
  • 99.
    Execution Scenario: Brydon buysDeclan beer Given Brydon hosts DemoCampGuelph When Declan demos Cucumber Then Brydon should buy Declan 1 beer Then /^(.+) should buy (.+) (d+) (.*)$/ do |buyer, drinker, qty, item| perk = @event.perks[0]; perk.buyer.should == buyer; perk.receiver.should == drinker
  • 100.
    Source: https://docs.google.com/drawings/d/1cbfKq-KazcbMVCnRfih6zMSDBdtf90KviV7l2oxGyWM/edit Specification By Example Business Goal Derive the scope Scope Shared Understanding Ubiquitous Language Specify collaboratively Key Examples Refine the specification Specification With Examples Automate literally Executable Specification Validate frequently Living Documentation
  • 101.
    Build it Right Build the Right Thing Specification By Example Gojko Adzic, 2011 page 4
  • 102.
    Build it Right Build the Right Thing Useless Crap Specification By Example Gojko Adzic, 2011 page 4
  • 103.
    Build it Right Business Failure Build the Right Thing Useless Crap Specification By Example Gojko Adzic, 2011 page 4
  • 104.
    Build it Right Business Failure Build the Right Thing Useless Crap Maintenance Nightmare Specification By Example Gojko Adzic, 2011 page 4
  • 105.
    Build it Right Business Failure Business Success Build the Right Thing Useless Crap Maintenance Nightmare Specification By Example Gojko Adzic, 2011 page 4
  • 106.
    Build it Right Business Failure Business Success Specification By Example Build the Right Thing Useless Crap Maintenance Nightmare Specification By Example Gojko Adzic, 2011 page 4
  • 107.
    Reading Specification By Example GojkoAdzic The RSpec Book: Behaviour Driven Development with RSpec, Cucumber and Friends David Cheliminksy et al Agile Testing: A Practical Guide for Testers and Agile Teams Lisa Crispin, Janet Gregory
  • 108.
    Diagram Credits Lisa Crispin and Janet Gregory Agile Testing: A Practical Guide for Testers and Agile Teams Addison-Wesley Professional; January 9, 2009. Mike Cohn Succeeding with Agile: Software Development Using Scrum Addison-Wesley Professional; November 5, 2009.