Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Specification by Example

58,777 views

Published on

An introduction to Specification by Example.

Published in: Technology, Business

Specification by Example

  1. 1. Specification By Example My role in ATDD BDD, ATDD Declan Whelan @dwhelan
  2. 2. What’s the Problem?
  3. 3. Waterfall Winston Royce, 1970Source: http://www.cs.umd.edu/class/spring2003/cmsc838p/Process/waterfall.pdf
  4. 4. Cost Of Change We add formal handoffs. Documents for handoffs. Documents reviews. Review committees. Document templates. Cost% Standards. Of%Change% Time%
  5. 5. Traditional Flow
  6. 6. Traditional FlowSpecification
  7. 7. Traditional Flow BuildSpecification
  8. 8. Traditional Flow BuildSpecification Tests Cases
  9. 9. Traditional Flow BuildSpecification Test Tests Cases
  10. 10. Traditional Flow BuildSpecification Test Deploy Tests Cases
  11. 11. We Found a Bug! How did QA miss this? That was obvious! I shouldnt Developers dont test before have to spell out every they throw it over the wall! detail! Exec TesterProductManager We did what was in functional spec! Business requirements werent clear! BA missed it in the technical requirements! Dev BA Architect
  12. 12. We Found a Bug! How did QA miss this? That was obvious! I shouldnt Developers dont test before have to spell out every they throw it over the wall! detail! Exec TesterProductManager We did what was in functional spec! Business requirements werent clear! BA missed it in the technical requirements! Dev BA Architect
  13. 13. We Found a Bug! How did QA miss this? That was obvious! I shouldnt Developers dont test before have to spell out every they throw it over the wall! detail! Exec TesterProductManager We did what was in functional spec! Business requirements werent clear! BA missed it in the technical requirements! Dev BA Architect
  14. 14. We Found a Bug! How did QA miss this? That was obvious! I shouldnt Developers dont test before have to spell out every they throw it over the wall! detail! Exec TesterProductManager We did what was in functional spec! Business requirements werent clear! BA missed it in the technical requirements! Dev BA Architect
  15. 15. We Found a Bug! How did QA miss this? That was obvious! I shouldnt Developers dont test before have to spell out every they throw it over the wall! detail! Exec TesterProductManager We did what was in functional spec! Business requirements werent clear! BA missed it in the technical requirements! Dev BA Architect
  16. 16. We Found a Bug! How did QA miss this? That was obvious! I shouldnt Developers dont test before have to spell out every they throw it over the wall! detail! Exec TesterProductManager We did what was in functional spec! Business requirements werent clear! BA missed it in the technical requirements! Dev BA Architect
  17. 17. We Found a Bug! How did QA miss this? That was obvious! I shouldnt Developers dont test before have to spell out every they throw it over the wall! detail! Exec TesterProductManager We did what was in functional spec! Business requirements werent clear! BA missed it in the technical requirements! Dev BA Architect
  18. 18. Accidental Adversaries + Testing + New Bugs - + - +Testing Additional Development Software FixSuccess Tests Success + -+ - New Build + Development +
  19. 19. Functional SilosSource: http://www.danpontefract.com/images/silo.jpg
  20. 20. Build it Right Build the Right ThingSpecification By ExampleGojko Adzic, 2011 page 4
  21. 21. Build it Right Build the Right Thing Useless CrapSpecification By ExampleGojko Adzic, 2011 page 4
  22. 22. Build it Right Business Failure Build the Right Thing Useless CrapSpecification By ExampleGojko Adzic, 2011 page 4
  23. 23. Build it Right Business Failure Build the Right Thing Useless Crap Maintenance NightmareSpecification By ExampleGojko Adzic, 2011 page 4
  24. 24. Build it Right Business Failure Business Success Build the Right Thing Useless Crap Maintenance NightmareSpecification By ExampleGojko Adzic, 2011 page 4
  25. 25. Build it Right Business Failure Business Success Specification By Example Build the Right Thing Useless Crap Maintenance NightmareSpecification By ExampleGojko Adzic, 2011 page 4
  26. 26. What are Specifications By Example?
  27. 27. What are Specifications By Example?• Thin slices of system behaviour• that deliver business value• described as concrete examples• that are potentially automatable• without translation• to create executable specifications• captured in live documentation.
  28. 28. Agile Testing Quadrants
  29. 29. Specification By Example
  30. 30. Specification By Example Business Goal
  31. 31. Specification By Example Business Goal Derive the scope Scope
  32. 32. Specification By Example Business Goal Derive the scope Scope Specify collaboratively Key Examples
  33. 33. Specification By Example Business Goal Derive the scope Scope Specify collaboratively Key Examples Refine the specification Specification With Examples
  34. 34. Specification By Example Business Goal Derive the scope Scope Specify collaboratively Key Examples Refine the specification Specification With Examples Automate literally Executable Specification
  35. 35. Specification By Example Business Goal Derive the scope Scope Specify collaboratively Key Examples Refine the specification Specification With Examples Automate literally Executable Specification Validate frequently Living Documentation
  36. 36. Specification By Example Business Goal Derive the scope Scope Shared Understanding Ubiquitous Language Specify collaboratively Key Examples Refine the specification Specification With Examples Automate literally Executable Specification Validate frequently Living Documentation
  37. 37. Source: https://docs.google.com/drawings/d/1cbfKq-KazcbMVCnRfih6zMSDBdtf90KviV7l2oxGyWM/edit Specification By Example Business Goal Derive the scope Scope Shared Understanding Ubiquitous Language Specify collaboratively Key Examples Refine the specification Specification With Examples Automate literally Executable Specification Validate frequently Living Documentation
  38. 38. Derive the Scope: Story MappingSource: http://availagility.co.uk/wp-content/uploads/2008/10/user-story-mapping.png
  39. 39. Derive the Scope: User Stories
  40. 40. Derive the Scope: User Stories As a _______ I want to _______ So that _______
  41. 41. Derive the Scope: User Stories As a _______ I want to _______ So that _______ As a student I want to purchase used books online So that I can save money
  42. 42. Specify Collaboratively: Workshops
  43. 43. Specify Collaboratively: Workshops
  44. 44. Specify Collaboratively: Workshops • Hold regular product backlog workshops
  45. 45. Specify Collaboratively: Workshops • Hold regular product backlog workshops • Full team workshops - when starting
  46. 46. Specify Collaboratively: Workshops • Hold regular product backlog workshops • Full team workshops - when starting • Three amigo workshops: • One developer • One tester • One analyst
  47. 47. Specify Collaboratively: Key Examples
  48. 48. Specify Collaboratively: Key Examples Given _______ When _______ Then _______
  49. 49. Specify Collaboratively: Key Examples Given _______ When _______ Then _______Given “War and Peace” is available as a used book for $2.99 When Susan selects book“War and Peace” Then “Buy used for $2.99” is displayed
  50. 50. Refining the Specification
  51. 51. Refining the Specification“Specifications with examples are acceptance tests” Gojko Adzic
  52. 52. Refining the Specification“Specifications with examples are acceptance tests” Gojko Adzic• Be precise and make sure spec is testable
  53. 53. Refining the Specification“Specifications with examples are acceptance tests” Gojko Adzic• Be precise and make sure spec is testable• Avoid “scripts” and “flows”
  54. 54. Refining the Specification“Specifications with examples are acceptance tests” Gojko Adzic• Be precise and make sure spec is testable• Avoid “scripts” and “flows”• Focus on business functionality not design
  55. 55. Refining the Specification“Specifications with examples are acceptance tests” Gojko Adzic• Be precise and make sure spec is testable• Avoid “scripts” and “flows”• Focus on business functionality not design• Avoid UI details
  56. 56. Refining the Specification“Specifications with examples are acceptance tests” Gojko Adzic• Be precise and make sure spec is testable• Avoid “scripts” and “flows”• Focus on business functionality not design• Avoid UI details• Avoid covering every possible combination
  57. 57. Refining the Specification: An Example Free Delivery Free delivery is offered to VIP customers once they purchase a certain number of books. Free delivery is not offered to regular customers or VIP customers buying anything other than books. Customer Type Cart Contents Delivery VIP 5 books Free, Standard VIP 4 books Standard Regular 10 books Standard VIP 5 dishwashers Standard VIP 5 books, 1 dishwasher StandardSource: Specification by Example: How successful teams deliver the right software, Gojko Adzic, pg. 116
  58. 58. Automating Examples
  59. 59. Automating Examples• Start small
  60. 60. Automating Examples• Start small• Select important examples for automation
  61. 61. Automating Examples• Start small• Select important examples for automation• Plan up-front to automate
  62. 62. Automating Examples• Start small• Select important examples for automation• Plan up-front to automate• Be prepared to go slower at the start
  63. 63. Automating Examples• Start small• Select important examples for automation• Plan up-front to automate• Be prepared to go slower at the start• Treat automation code as a first class citizen
  64. 64. Automating Examples• Start small• Select important examples for automation• Plan up-front to automate• Be prepared to go slower at the start• Treat automation code as a first class citizen• Avoid record and playback
  65. 65. Automating Examples• Start small• Select important examples for automation• Plan up-front to automate• Be prepared to go slower at the start• Treat automation code as a first class citizen• Avoid record and playback• Avoid using pre-populated data
  66. 66. Minimize Tests Through the UI build trust via UI tests automate below the UI
  67. 67. Automation ToolsFitNesse Cucumber ConcordianGreen JBehave TwistPepper RobotText Test SpecFlow Framework
  68. 68. Validate Frequently
  69. 69. Validate Frequently• Start with a Continuous Integration system
  70. 70. Validate Frequently• Start with a Continuous Integration system• Set up a Continuous Deployment system
  71. 71. Validate Frequently• Start with a Continuous Integration system• Set up a Continuous Deployment system• Specify and test business logic separately from end-to-end flows
  72. 72. Validate Frequently• Start with a Continuous Integration system• Set up a Continuous Deployment system• Specify and test business logic separately from end-to-end flows• Organize tests along functional lines
  73. 73. Validate Frequently• Start with a Continuous Integration system• Set up a Continuous Deployment system• Specify and test business logic separately from end-to-end flows• Organize tests along functional lines• Run all test nightly
  74. 74. Validate Frequently• Start with a Continuous Integration system• Set up a Continuous Deployment system• Specify and test business logic separately from end-to-end flows• Organize tests along functional lines• Run all test nightly• Consider an iteration “test pack”
  75. 75. Living Documentation• Keep specifications short• Evolve a specification language and leverage in with “common fixtures”• Make documentation accessible - consider a wiki• Organize the documentation• Put specifications under version control
  76. 76. A Worked Example
  77. 77. Executable Specification
  78. 78. Cucumber Organization fixtures are common Features Step System
  79. 79. Cucumber Organization fixtures are common ad Lo Features Step System
  80. 80. Cucumber Organization fixtures are common M ad at Lo ch Features Step System
  81. 81. Cucumber Organization fixtures are common M ad at Lo ch Features Step Call System
  82. 82. Feature File Feature: Turn cucumber into beer As a cucumber presenter I want beer after my presentation So I can enjoy the rest of DemoCampGuelph Scenario: Brydon buys Declan beer Given Brydon hosts DemoCampGuelph When Declan demos Cucumber Then Brydon should buy Declan 1 beer Scenario: Ali buys Declan beer Given Ali hosts DemoCampGuelph When Declan demos Cucumber Then Ali should buy Declan 1 beer
  83. 83. Step DefinitionsGiven /^(.+) hosts/ do |host| @event = Event.new(host)endWhen /^(.+) demos/ do |presenter| @event.add(presenter)endThen /^(.+) should buy (.+) (d+) (.*)$/ do |buyer, drinker,qty, item| perk = @event.perks[0]; perk.buyer.should == buyer; perk.receiver.should ==drinker perk.quantity.should == quantity.to_i; perk.item.should== itemend
  84. 84. System Under Test class Event attr_reader :perks def initialize(host) @host = host; @perks = [] end def add(presenter) @perks.push Perk.new(@host, presenter, 1, "beer") end end class Perk attr_reader :buyer, :receiver, :quantity, :item def initialize(buyer, receiver, quantity, item) @buyer = buyer; @receiver = receiver @quantity = quantity; @item = item end end
  85. 85. ExecutionScenario: Brydon buys Declan beer Given Brydon hosts DemoCampGuelph When Declan demos Cucumber Then Brydon should buy Declan 1 beer
  86. 86. ExecutionScenario: Brydon buys Declan beer Given Brydon hosts DemoCampGuelph When Declan demos Cucumber Then Brydon should buy Declan 1 beerGiven /^(.+) hosts/ do | When /^(.+) demos/ do |host| presenter| @event = @event.add(presenter)Event.new(host) endEndThen /^(.+) should buy (.+) (d+) (.*)$/ do |buyer,drinker, qty, item| perk = @event.perks[0]; perk.buyer.should == buyer; perk.receiver.should ==drinker
  87. 87. ExecutionScenario: Brydon buys Declan beer Given Brydon hosts DemoCampGuelph When Declan demos Cucumber Then Brydon should buy Declan 1 beerGiven /^(.+) hosts/ do |host| @event =Event.new(host)End
  88. 88. ExecutionScenario: Brydon buys Declan beer Given Brydon hosts DemoCampGuelph When Declan demos Cucumber Then Brydon should buy Declan 1 beerGiven /^(.+) hosts/ do |host| “Brydon” @event =Event.new(host)End
  89. 89. ExecutionScenario: Brydon buys Declan beer Given Brydon hosts DemoCampGuelph When Declan demos Cucumber Then Brydon should buy Declan 1 beer
  90. 90. ExecutionScenario: Brydon buys Declan beer Given Brydon hosts DemoCampGuelph When Declan demos Cucumber Then Brydon should buy Declan 1 beerGiven /^(.+) hosts/ do | When /^(.+) demos/ do |host| presenter| @event = @event.add(presenter)Event.new(host) endEndThen /^(.+) should buy (.+) (d+) (.*)$/ do |buyer,drinker, qty, item| perk = @event.perks[0]; perk.buyer.should == buyer; perk.receiver.should ==drinker
  91. 91. ExecutionScenario: Brydon buys Declan beer Given Brydon hosts DemoCampGuelph When Declan demos Cucumber Then Brydon should buy Declan 1 beer When /^(.+) demos/ do | presenter| @event.add(presenter) end
  92. 92. ExecutionScenario: Brydon buys Declan beer Given Brydon hosts DemoCampGuelph When Declan demos Cucumber Then Brydon should buy Declan 1 beer When /^(.+) demos/ do | presenter| @event.add(presenter) end “Declan”
  93. 93. ExecutionScenario: Brydon buys Declan beer Given Brydon hosts DemoCampGuelph When Declan demos Cucumber Then Brydon should buy Declan 1 beer
  94. 94. ExecutionScenario: Brydon buys Declan beer Given Brydon hosts DemoCampGuelph When Declan demos Cucumber Then Brydon should buy Declan 1 beerGiven /^(.+) hosts/ do | When /^(.+) demos/ do |host| presenter| @event = @event.add(presenter)Event.new(host) endEndThen /^(.+) should buy (.+) (d+) (.*)$/ do |buyer,drinker, qty, item| perk = @event.perks[0]; perk.buyer.should == buyer; perk.receiver.should ==drinker
  95. 95. ExecutionScenario: Brydon buys Declan beer Given Brydon hosts DemoCampGuelph When Declan demos Cucumber Then Brydon should buy Declan 1 beerThen /^(.+) should buy (.+) (d+) (.*)$/ do |buyer,drinker, qty, item| perk = @event.perks[0]; perk.buyer.should == buyer; perk.receiver.should ==drinker
  96. 96. ExecutionScenario: Brydon buys Declan beer Given Brydon hosts DemoCampGuelph When Declan demos Cucumber Then Brydon should buy Declan 1 beerThen /^(.+) should buy (.+) (d+) (.*)$/ do |buyer,drinker, qty, item| perk = @event.perks[0]; perk.buyer.should == buyer; perk.receiver.should ==drinker
  97. 97. Specification By Example
  98. 98. Specification By Example Business Goal
  99. 99. Specification By Example Business Goal Derive the scope Scope
  100. 100. Specification By Example Business Goal Derive the scope Scope Specify collaboratively Key Examples
  101. 101. Specification By Example Business Goal Derive the scope Scope Specify collaboratively Key Examples Refine the specification Specification With Examples
  102. 102. Specification By Example Business Goal Derive the scope Scope Specify collaboratively Key Examples Refine the specification Specification With Examples Automate literally Executable Specification
  103. 103. Specification By Example Business Goal Derive the scope Scope Specify collaboratively Key Examples Refine the specification Specification With Examples Automate literally Executable Specification Validate frequently Living Documentation
  104. 104. Specification By Example Business Goal Derive the scope Scope Shared Understanding Ubiquitous Language Specify collaboratively Key Examples Refine the specification Specification With Examples Automate literally Executable Specification Validate frequently Living Documentation
  105. 105. Source: https://docs.google.com/drawings/d/1cbfKq-KazcbMVCnRfih6zMSDBdtf90KviV7l2oxGyWM/edit Specification By Example Business Goal Derive the scope Scope Shared Understanding Ubiquitous Language Specify collaboratively Key Examples Refine the specification Specification With Examples Automate literally Executable Specification Validate frequently Living Documentation
  106. 106. WaterfallSource: http://www.cs.umd.edu/class/spring2003/cmsc838p/Process/waterfall.pdf
  107. 107. Waterfall XSource: http://www.cs.umd.edu/class/spring2003/cmsc838p/Process/waterfall.pdf
  108. 108. Agile Specification Agile&Approach& Itera/on&1& Itera/on&2& Itera/on&3& Itera/on&4& Requirements& Requirements& Requirements& Requirements& Analysis&&&Design& Analysis&&&Design& Analysis&&&Design& Analysis&&&Design& Cost% of% Code& Code& Code& Code&Change% Test& Test& Test& Test& Deploy& Time%
  109. 109. Functional SilosSource: http://www.danpontefract.com/images/silo.jpg
  110. 110. Functional Silos XSource: http://www.danpontefract.com/images/silo.jpg
  111. 111. Collaborative Specification
  112. 112. Build it Right Build the Right ThingSpecification By ExampleGojko Adzic, 2011 page 4
  113. 113. Build it Right Build the Right Thing Useless CrapSpecification By ExampleGojko Adzic, 2011 page 4
  114. 114. Build it Right Business Failure Build the Right Thing Useless CrapSpecification By ExampleGojko Adzic, 2011 page 4
  115. 115. Build it Right Business Failure Build the Right Thing Useless Crap Maintenance NightmareSpecification By ExampleGojko Adzic, 2011 page 4
  116. 116. Build it Right Business Failure Business Success Build the Right Thing Useless Crap Maintenance NightmareSpecification By ExampleGojko Adzic, 2011 page 4
  117. 117. Build it Right Business Failure Business Success Specification By Example Build the Right Thing Useless Crap Maintenance NightmareSpecification By ExampleGojko Adzic, 2011 page 4
  118. 118. ReadingSpecification By ExampleGojko AdzicThe RSpec Book: Behaviour Driven Development withRSpec, Cucumber and FriendsDavid Cheliminksy et alAgile Testing: A Practical Guide for Testers and Agile TeamsLisa Crispin, Janet Gregory
  119. 119. Diagram Credits Lisa Crispin and Janet Gregory Agile Testing: A Practical Guide for Testers and Agile Teams Addison-Wesley Professional; January 9, 2009. Mike Cohn Succeeding with Agile: Software Development Using Scrum Addison-Wesley Professional; November 5, 2009.

×