Cucumber: Automating the Requirements Language You Already Speak

6,112 views

Published on

Cucumber: Automating the Requirements Language You Already Speak

  1. 1. Automating the Requirements Language You Already Speak Ben Mabey @bmabey
  2. 2. Developer Tester Product Manager Designer/UX
  3. 3. Requirements
  4. 4. 56% of all bugs are introduced in requirements. (CHAOS Report)
  5. 5. 45% of functionality is never used
  6. 6. Only 20% makes up core functionality that is “Always” or “Often” used.
  7. 7. Feature Devotion Text Placing emphasis on features instead of overall outcome http://martinfowler.com/bliki/FeatureDevotion.html
  8. 8. t bu yer imp atien A s an y se arch t to refi ne m I wan dw hat that I ca n fin So ant qui ckly Iw
  9. 9. As an impatient buyer I want to refine my search So that I can find what I want quickly
  10. 10. Feature: Advanced Search As an impatient buyer I want to refine my search So that I can find what I want quickly
  11. 11. token for conversation
  12. 12. Gherkin
  13. 13. Scenario: title Given [Context] When [Action] Then [Expected Outcome]
  14. 14. Scenario: title Given [Context] And [More Context] When [Action] And [Other Action] Then [Expected Outcome] But [Unexpected Outcome]
  15. 15. Scenario: search by director Given the store has movies directed by “Steven Spielberg” When I search for “Steven Spielberg” Then I should see all of the movies directed by “Steven Spielberg”
  16. 16. Scenario: no results Given the store has no movies directed by “Steven Spielberg” When I search for “Steven Spielberg” Then I should see “Sorry, but no movies were found”
  17. 17. Advanced Gherkin
  18. 18. Scenario: search by director Given movies directed by "Steven Spielberg" are in stock When I search for "Spielberg" under "Director" Then I the search results should be "E.T., and Jaws"
  19. 19. Step Tables Scenario: search by director Given the following movies are in stock: | Title | Director | Year | | Jaws | Steven Spielberg | 1975 | | Star Wars | George Lucas | 1975 | | Dawn of the Dead | George Romero | 1978 | | E.T. | Steven Spielberg | 1982 | When I search for "Spielberg" under "Director" Then I the search results should be "E.T., and Jaws"
  20. 20. Step Tables Scenario: search by director Given the following movies are in stock: | Title | Director | Year | | Jaws | Steven Spielberg | 1975 | | Star Wars | George Lucas | 1975 | | Dawn of the Dead | George Romero | 1978 | | E.T. | Steven Spielberg | 1982 | When I search for "Spielberg" under "Director" Then I should see the following table: | Title | Director | Year | | Jaws | Steven Spielberg | 1975 | | E.T. | Steven Spielberg | 1982 |
  21. 21. Scenario: search by director Given the following movies are in stock: | Title | Director | Year | | Jaws | Steven Spielberg | 1975 | | Star Wars | George Lucas | 1975 | | Dawn of the Dead | George Romero | 1978 | | E.T. | Steven Spielberg | 1982 | When I search for "Spielberg" under "Director" Then I the search results should be "E.T., and Jaws"
  22. 22. Scenario: search by director Given the following movies are in stock: | Title | Director | Year | | Jaws | Steven Spielberg | 1975 | | Star Wars | George Lucas | 1975 | | Dawn of the Dead | George Romero | 1978 | | E.T. | Steven Spielberg | 1982 | When I search for "Spielberg" under "Director" Then I the search results should be "E.T., and Jaws" I only want to change the search query and results part...
  23. 23. Scenario Outlines Scenario Outline: search by director Given the following movies are in stock: | Title | Director | Year | | Jaws | Steven Spielberg | 1975 | | Star Wars | George Lucas | 1975 | | Dawn of the Dead | George Romero | 1978 | | E.T. | Steven Spielberg | 1982 | When I search for "<Director Query>" under "Director" Then I the search results should be "<Search Results>" Examples: | Director Query | Search Results | | Steve | E.T., Jaws | | George | Dawn of the Dead, Star Wars | | Lucas | Star Wars |
  24. 24. Scenario Outlines Scenario Outline: search by director Given the following movies are in stock: | Title | Director | Year | | Jaws | Steven Spielberg | 1975 | | Star Wars | George Lucas | 1975 | | Dawn of the Dead | George Romero | 1978 | | E.T. | Steven Spielberg | 1982 | When I search for "<Director Query>" under "Director" Then I the search results should be "<Search Results>" Examples: | Director Query | Search Results | | Steve | E.T., Jaws | | George | Dawn of the Dead, Star Wars | | Lucas | Star Wars |
  25. 25. Background Feature: Account Profile Scenario: change password success Given I'm logged in ... Scenario: update contact info Given I'm logged in ...
  26. 26. Background Feature: Account Profile Background: Given I'm logged in Scenario: change password success ... Scenario: update contact info ...
  27. 27. Multi-Line String Steps Scenario: register successfully Given I am on on the registration page When I sign up as "Jojo Binks" Then I should receive the following email: """ Thanks for signing up Jojo! Important information about here. """
  28. 28. # language: ja : 2    : 2 < 1>      < 2>      < >      < >    :     | 1 | 2 | | |     | 20 | 30 | add | 50 |     | 2 | 5 | add | 7 |     | 0 | 40 | add | 40 |
  29. 29. So now what?
  30. 30. uyer patient b As an im ine my search ref I want to find wha t can So that I kly Iw ant quic
  31. 31. uyer patient b As an im ine my search ref I want to find wha t can So that I kly Iw ant quic
  32. 32. patient b uyer As an im ine my search Given... I want to ref find wha t When... So that I Iw can ant quic kly Then...
  33. 33. patient b uyer As an im ine my search Given... I want to ref find wha t When... So that I Iw can ant quic kly Then...
  34. 34. patient b uyer As an im ine my search Given... I want to ref find wha t When... So that I Iw can ant quic kly Then...
  35. 35. Automating Gherkin
  36. 36. project_root/ | `-- features
  37. 37. project_root/ | `-- features |-- awesomeness.feature |-- greatest_ever.feature
  38. 38. project_root/ | `-- features |-- awesomeness.feature |-- greatest_ever.feature `-- support |-- env.rb `-- other_helpers.rb
  39. 39. project_root/ | `-- features |-- awesomeness.feature |-- greatest_ever.feature `-- support |-- env.rb `-- other_helpers.rb |-- step_definitions | |-- domain_concept_A.rb | `-- domain_concept_B.rb
  40. 40. Step Given a widget
  41. 41. Step Definition Given /^a widget$/ do Given a widget #codes go here end
  42. 42. When /^I search by director for "([^"]*)"$/ do |director| end
  43. 43. Regexp Capture -> Yielded Variable When /^I search by director for "([^"]*)"$/ do |director| end
  44. 44. When /^I search by director for "([^"]*)"$/ do |director|   visit advanced_search_path   fill_in "By Director", :with => director   click_button "Search" end Webrat, Webdriver, Watir, etc..
  45. 45. Not Just for Ruby
  46. 46. When /^I search by director for "([^"]*)"$/ do |director| end [Given(@"^I search by director for "([^"]*)"$")] public void searchDirector(String director) { } http://wiki.github.com/richardlawrence/Cuke4Nuke/
  47. 47. Gherkin
  48. 48. Gherkin Ragel
  49. 49. Gherkin Ragel C, C++, Ruby, Java, .net, etc,
  50. 50. Gherkin Ragel C, C++, Ruby, Java, .net, etc, http://specflow.org/
  51. 51. http://wiki.github.com/aslakhellesoy/cuke4duke/
  52. 52. When /^I search by director for "([^"]*)"$/ do |director| end @When("^I search by director for "([^"]*)"$") public void searchDirector(String director) { } When("^I search by director for "([^"]*)"$"{ String d -> } (When #"^I search by director for "([^"]*)"$"   (fn [director] )) When("^I search by director for "([^"]*)"$"{ d: String => }
  53. 53. Uhh... this just seems like more work for me.
  54. 54. Scenario Outline: search by director Given the following movies are in stock: | Title | Director | Year | | Jaws | Steven Spielberg | 1975 | | Star Wars | George Lucas | 1975 | | Dawn of the Dead | George Romero | 1978 | | E.T. | Steven Spielberg | 1982 | When I search for "<Director Query>" under "Director" Then I the search results should be "<Search Results>" Examples: | Director Query | Search Results | | Steve | E.T., Jaws | | George | Dawn of the Dead, Star Wars | | Zombie guy | Dawn of the Dead | | Lucas | Star Wars | Scenario: search by director on a full moon Given that it is a full moon ...
  55. 55. Scenario Outline: search by director Given the following movies are in stock: | Title | Director | Year | | Jaws | Steven Spielberg | 1975 | | Star Wars | George Lucas | 1975 | | Dawn of the Dead | George Romero | 1978 | | E.T. | Steven Spielberg | 1982 | When I search for "<Director Query>" under "Director" Then I the search results should be "<Search Results>" Examples: | Director Query | Search Results | | Steve | E.T., Jaws | | George | Dawn of the Dead, Star Wars | | Zombie guy | Dawn of the Dead | | Lucas | Star Wars | Scenario: search by director on a full moon Given that it is a full moon ...
  56. 56. You know where to begin and end. Yo
  57. 57. You know when you broke something.
  58. 58. Outside-In
  59. 59. Model
  60. 60. Controller Model
  61. 61. View Controller Model
  62. 62. UI/UX View Controller Model
  63. 63. UI/UX View Controller Model
  64. 64. Write Scenarios
  65. 65. Steps are pending
  66. 66. Write Step Definition
  67. 67. Go Down A Gear
  68. 68. RSpec, xUnit, etc
  69. 69. Write Unit Test
  70. 70. Make Unit Test Pass
  71. 71. REFACTOR!!
  72. 72. Where Are we?
  73. 73. Continue until...
  74. 74. REFACTOR and REPEAT
  75. 75. Next set of slides stolen from Aslak Hellesøy, creator of http://www.slideshare.net/aslak.hellesoy/cuke4duke-javazone-2009
  76. 76. Feature: code-breaker submits guess In order to make time pass when I'm alone As a player I want to play the against a machine Scenario: all correct Given the secret code is "r g y c" When I guess "r g y c" Then the mark should be "bbbb"
  77. 77. $ gem install cucumber $ cucumber features
  78. 78. <repositories> <repository> <id>cukes</id> <url>http://cukes.info/maven</url> </repository> </repositories> <pluginRepositories> <pluginRepository> <id>cukes</id> <url>http://cukes.info/maven</url> </pluginRepository> </pluginRepositories>
  79. 79. <dependencies> <dependency> <groupId>cuke4duke</groupId> <artifactId>cuke4duke</artifactId> <version>0.3.0</version> </dependency> <dependency> <groupId>org.picocontainer</groupId> <artifactId>picocontainer</artifactId> <version>2.8.3</version> </dependency> </dependencies>
  80. 80. <plugin> <groupId>cuke4duke</groupId> <artifactId>cuke4duke-maven-plugin</artifactId> <configuration> <jvmArgs> <jvmArg> -Dcuke4duke.objectFactory= cuke4duke.internal.java.PicoFactory </jvmArg> </jvmArgs> <cucumberArgs> <cucumberArg>${basedir}/src/test/java</cucumberArg> </cucumberArgs> <gems> <gem>install cuke4duke --version x.y.x</gem> </gems> </configuration> </plugin>
  81. 81. $ mvn integration-test -Dcucumber.installGems=true
  82. 82. Feature: code-breaker submits guess In order to make time pass when I'm alone As a player I want to play the against a machine Scenario: all correct # features/c..s.feature:6 Given the secret code is "r g y c" # features/c..s.feature:7 When I guess "r g y c" # features/c..s.feature:8 Then the mark should be "bbbb" # features/c..s.feature:9 1 scenario (1 undefined) 3 steps (3 undefined) 0m0.076s
  83. 83. You can implement step definitions for undefined steps with these snippets: @Given("^the secret code is "([^"]*)"$") @Pending public void theSecretCodeIsRGYC_(String arg1) { } @When("^I guess "([^"]*)"$") @Pending public void iGuessRGYC_(String arg1) { } @Then("^the mark should be "([^"]*)"$") @Pending public void theMarkShouldBeBbbb_(String arg1) { }
  84. 84. package codebreaker; import cuke4duke.*; public class CodeBreakerSteps { @Given("^the secret code is "([^"]*)"$") @Pending public void theSecretCodeIs(String code) { } @When("^I guess "([^"]*)"$") @Pending public void iGuess(String guess) { } @Then("^the mark should be "([^"]*)"$") @Pending public void theMarkShouldBe(String mark) { } }
  85. 85. $ mvn integration-test Feature: code-breaker submits guess In order to make time pass when I'm alone As a player I want to play the against a machine Scenario: all correct # features/c..s.feature:6 Given the secret code is "r g y c" # public void theS..(..) TODO (Cucumber::Pending) f../c..s.feature:7:in `Given the secret code is "r g y c"' When I guess "r g y c" # public void iGue..(..) Then the mark should be "bbbb" # public void theM..(..) 1 scenario (1 pending) 3 steps (2 skipped, 1 pending) 0m0.079s
  86. 86. Given the secret code is "r g y c" # public void codebreaker.GameSteps .theSecretCodeIs(java.lang.String)
  87. 87. Given the secret code is "r g y c"
  88. 88. Given the secret code is "r g y c" @Given("^the secret code is "([^"]*)"$") public void theSecretCodeIs(String code) { }
  89. 89. public class GameSteps { private Game game; @Given("^the secret code is "([^"]*)"$") public void theSecretCodeIs(String code) { game = new Game(code); } }
  90. 90. Compilation failure src/test/java/codebreaker/ CodeBreakerSteps.java:[6,12] cannot find symbol symbol : class Game location: class codebreaker.CodeBreakerSteps
  91. 91. package codebreaker; public class Game { public Game(String code) { } }
  92. 92. $ mvn integration-test Feature: code-breaker submits guess In order to make time pass when I'm alone As a player I want to play the against a machine Scenario: all correct # features/c..s.feature:6 Given the secret code is "r g y c" # public void theS..(..) When I guess "r g y c" # public void iGue..(..) TODO (Cucumber::Pending) f../c..s.feature:8:in `When I guess "r g y c"' Then the mark should be "bbbb" # public void theM..(..) 1 scenario (1 pending) 3 steps (1 skipped, 1 pending, 1 passed) 0m0.112s
  93. 93. $ mvn integration-test Feature: code-breaker submits guess In order to make time pass when I'm alone As a player I want to play the against a machine Scenario: all correct # features/c..s.feature:6 Given the secret code is "r g y c" # public void theS..(..) When I guess "r g y c" # public void iGue..(..) Then the mark should be "bbbb" # public void theM..(..) 1 scenario (1 passed) 3 steps (3 passed) 0m0.121s
  94. 94. Scenario: all correct Given the secret code is "r g y c" When I guess "r g y c" Then the mark should be "bbbb" Scenario: 2 wrong pos, 2 correct Given the secret code is "r g y c" When I guess "r g c y" Then the mark should be "bbww"
  95. 95. Scenario: all correct # features/c..s.feature:6 Given the secret code is "r g y c" # public void theS..(..) When I guess "r g y c" # public void iGue..(..) Then the mark should be "bbbb" # public void theM..(..) Scenario: all correct # features/c..s.feature:6 Given the secret code is "r g y c" # public void theS..(..) When I guess "r g y c" # public void iGue..(..) Then the mark should be "bbbb" # public void theM..(..) org.junit.ComparisonFailure: expected:<bb[bb]> but was:<bb[ww]> (NativeException) codebreaker/CodeBreakerSteps.java:20:in `theMarkShouldBe' features/codebreaker_submits_guess.feature:14:in `Then the mark should be "bbww"' 1 scenario (1 failed, 1 passed) 3 steps (1 failed, 3 passed) 0m0.121s
  96. 96. Scenario: all correct Given the secret code is "r g y c" When I guess "r g y c" Then the mark should be "bbbb" Scenario: 2 wrong pos, 2 correct Given the secret code is "r g y c" When I guess "r g c y" Then the mark should be "bbww"
  97. 97. Scenario Outline: submit guess Given the secret code is "<code>" When I guess "<guess>" Then the mark should be "<mark>" Examples: | code | guess | mark | | r g y c | r g y c | bbbb | | r g y c | r g c y | bbww | | r g y c | y r g c | bwww | | r g y c | c r g y | wwww |
  98. 98. More Tricks
  99. 99. Tags
  100. 100. @store @lucene Feature: Advanced Search ... @wip Scenario: search by director ... @proposed @pending_ui Scenario: items not carried ... @nightly Scenario: long running ... @third_party Scenario: search Amazon ...
  101. 101. @store @lucene Feature: Advanced Search ... @wip Scenario: search by director ... @proposed @pending_ui Scenario: items not carried ... @nightly Scenario: long running ... @third_party Scenario: search Amazon ...
  102. 102. @store @lucene Feature: Advanced Search ... @wip Tag Exclusion Scenario: search by director ... @proposed @pending_ui cucumber --tags ~@proposed Scenario: items not carried ... @nightly Scenario: long running ... @third_party Scenario: search Amazon ...
  103. 103. @store @lucene Feature: Advanced Search ... @wip Scenario: search by director ... @proposed @pending_ui Scenario: items not carried ... @nightly Scenario: long running ... @third_party Scenario: search Amazon ...
  104. 104. @store @lucene Feature: Advanced Search ... @wip Scenario: search by director ... @proposed @pending_ui Scenario: items --tags @wip:2 cucumber not carried --wip ... @nightly Scenario: long running ... @third_party Scenario: search Amazon ...
  105. 105. @store @lucene Feature: Advanced Search ... @wip Limit scenarios in flow Scenario: search by director ... @proposed @pending_ui Scenario: items --tags @wip:2 cucumber not carried --wip ... @nightly Scenario: long running ... @third_party Scenario: search Amazon ...
  106. 106. @store @lucene Feature: Advanced Search ... @wip Limit scenarios in flow Scenario: search by director ... @proposed @pending_ui Scenario: items --tags @wip:2 cucumber not carried --wip ... @nightly Expect failure - Success == Failure Scenario: long running ... @third_party Scenario: search Amazon ...
  107. 107. Hooks Before do end After do |scenario| end World do end World(MyModule) World(HerModule)
  108. 108. Tagged Hooks Before('@im_special', '@me_too') do @icecream = true end @me_too Feature: Sit Feature: Lorem @im_special Scenario: Ipsum Scenario: Amet Scenario: Dolor Scenario: Consec
  109. 109. Acceptance Tests == End-to-End?
  110. 110. Slow Fast
  111. 111. Slow Integrated Fast Isolated
  112. 112. Slow Integrated Fast Isolated
  113. 113. Slow Integrated HtmlUnit V8 Fast Isolated
  114. 114. Slow Integrated HtmlUnit V8 Fast Isolated
  115. 115. What about me?
  116. 116. patient b uyer As an im ine my search Given... I want to ref find wha t When... So that I Iw can ant quic kly Then...
  117. 117. patient b uyer As an im ine my search Given... I want to ref find wha t When... So that I Iw can ant quic kly Then...
  118. 118. patient b uyer As an im ine my search Given... I want to ref find wha t When... So that I Iw can ant quic kly Then...
  119. 119. patient b uyer As an im ine my search Given... I want to ref find wha t When... So that I Iw can ant quic kly Then...
  120. 120. patient b uyer As an im ine my search Given... I want to ref find wha t When... So that I Iw can ant quic kly Then...
  121. 121. Dead er t ient buy As an impa m y search I want to refine t find wha So that I can y I wan t quickl D PR
  122. 122. Dead Living Feature: Advanced Search As an impatient buyer I want to refine my search So that I can find what I want quickly Scenario Outline: search by director er Given the following movies are in stock: t ient buy an impa | Title | Director | Year | As m y search | Jaws | Steven Spielberg | 1975 | I want to refine t | Star Wars | George Lucas | 1975 | find wha | Dawn of the Dead | George Romero | 1978 | that I can | E.T. | Steven Spielberg | 1982 | So y When I search for "<Director Query>" under "Director" I wan t quickl Then I the search results should be "<Search Results>" Examples: | Director Query | Search Results | | Steve | E.T., Jaws | | George | Dawn of the Dead, Star Wars | D | Lucas | Star Wars | PR
  123. 123. Gherkin Subtleties
  124. 124. Feature: Advanced Search As an impatient buyer I want to refine my search So that I can find what I want quickly
  125. 125. Feature: Advanced Search As an impatient buyer I want to refine my search So that I can find what I want quickly Scenario: search by director Given movies directed by "Steven Spielberg" are in stock When I search by director for "Spielberg" Then I should see all of the movies directed by "Steven Spielberg"
  126. 126. Feature: Advanced Search As an impatient buyer I want to refine my search So that I can find what I want quickly Declarative Scenario: search by director Given movies directed by "Steven Spielberg" are in stock When I search by director for "Spielberg" Then I should see all of the movies directed by "Steven Spielberg"
  127. 127. Feature: Advanced Search As an impatient buyer I want to refine my search So that I can find what I want quickly Imperative Scenario: search by director Given movies directed by "Steven Spielberg" are in stock And I am on the "Advanced Search" page When I fill in "Spielberg" for "Director" And press "Submit" Then I should see all of the movies directed by "Steven Spielberg"
  128. 128. Feature: Advanced Search As an impatient buyer I want to refine my search So that I can find what I want quickly Imperative Scenario: search by director Given movies directed by "Steven Spielberg" are in stock And I am on the "Advanced Search" page When I fill in "Spielberg" for "Director" And press "Submit" Then I should see all of the movies directed by "Steven Spielberg" I like it! I actually know how a user can use it!
  129. 129. Balance Abstraction
  130. 130. Scenario: successful login Given I'm on the login page When I fill in "jimmy" for "Login" And fill in "password" for "Password" And click "Login" Then I should see "Welcome back jimmy!"
  131. 131. Scenario: change password success
  132. 132. Scenario: change password success Given I'm on the login page When I fill in "jimmy" for "Login" And fill in "password" for "Password" And click "Login" Then I should see "Welcome back jimmy!"
  133. 133. Scenario: change password success Given I'm on the login page When I fill in "jimmy" for "Login" And fill in "password" for "Password" And click "Login" Then I should see "Welcome back jimmy!" When I click "Change Password" And fill in the following | Old Password | password | | New Password | brand-new | | Confirmation | brand-new | And click "Change Password" Then I should see "Your password has been changed."
  134. 134. Scenario: change password success Given I'm on the login page When I fill in "jimmy" for "Login" And fill in "password" for "Password" And click "Login" Then I should see "Welcome back jimmy!" When I click "Change Password" Incidental Details And fill in the following | Old Password | password | | New Password | brand-new | | Confirmation | brand-new | And click "Change Password" Then I should see "Your password has been changed."
  135. 135. Scenario: change password success Given I'm logged in When I click "Change Password" Hide the noise! And fill in the following | Old Password | password | | New Password | brand-new | | Confirmation | brand-new | And click "Change Password" Then I should see "Your password has been changed."
  136. 136. Resources http://cukes.info http://wiki.github.com/aslakhellesoy/cuke4duke/ http://blog.dannorth.net/whats-in-a-story/ http://blog.josephwilk.net/ruby/rocket-fuelled-cucumbers.html http://benmabey.com/2008/05/19/imperative-vs- declarative-scenarios-in-user-stories.html
  137. 137. Thanks! BenMabey.com github.com/bmabey Twitter: bmabey

×