Keeping your cucumber suite maintainable

3,145 views

Published on

Cucumber is a very good tool for achieving functional testing. However your cuke suite can become unwieldy very soon. Even a little bit of indiscipline can lead to huge build times, with random failures. Over time, maintaining and writing the cukes starts costing more than the benefit derived out of it. On the other hand, you can start ignoring your functional build only to discover some nasty bug later which your poor cuke suite was telling you.

In this presentation we try and look at what can be done to avoid this and derive complete value from cukes. We share experiences and learnings about cuking the right way from real projects
Learnings
1) best practices of writing and maintaining cukes
2) some new things that could be tried out with cukes to make them more reliable and valuable.

Published in: Technology, Business
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
3,145
On SlideShare
0
From Embeds
0
Number of Embeds
16
Actions
Shares
0
Downloads
41
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide
  • Introduce yourselves
  • Some knowledge of cucumber
  • When I started the Cucumber project in 2008 I had three major goals:Help stakeholders express what they want via executable specificationsHelp programmers write better software by making TDD/BDD easyReduce misinterpretations through a ubiquitous language
  • When I started the Cucumber project in 2008 I had three major goals:Help stakeholders express what they want via executable specificationsHelp programmers write better software by making TDD/BDD easyReduce misinterpretations through a ubiquitous language
  • small changes causing lot of cukes to failfailures causing long build timesSo much so that you seem to land up spending more time on the cukes rather than on the implementation
  • small changes causing lot of cukes to failfailures causing long build timesSo much so that you seem to land up spending more time on the cukes rather than on the implementation
  • Steps in different filesRedundant codeLess predictability
  • Steps in wrong filesOrganized readableFlexible
  • Dependencyoverriding
  • Dependencyoverriding
  • you can then specify the output location for your failures. And then run the failures in rerun profile. “A test is non-deterministic when it passes sometimes and fails sometimes, without any noticeable change in the code, tests, or environment. Such tests fail, then you re-run them and they pass. Test failures for such tests are seemingly random.” - Martin Fowler
  • you can then specify the output location for your failures. And then run the failures in rerun profile. “A test is non-deterministic when it passes sometimes and fails sometimes, without any noticeable change in the code, tests, or environment. Such tests fail, then you re-run them and they pass. Test failures for such tests are seemingly random.” - Martin Fowler
  • 21 mins to 17 mins
  • 21 mins to 17 mins
  • Reduced time for one feature from 7.5 minutes to 2.4 minutes
  • Reduced time for one feature from 7.5 minutes to 2.4 minutes
  • Parallel_tests, hydra, tlb
  • Use more of Capybara DSL than web_stepsUse of CSS locator strategy than xpathConsistency set of guidelines for teamUse proper taggingStub out third party services calls and have seperate integration testsTreat cucumber scenarios like your unit tests code.
  • Keeping your cucumber suite maintainable

    1. 1. Ketan Soni@ketan_soniAtreyee Maiti@AtreyeeMaiti
    2. 2. Cucumber is meant to provide stakeholders a wayto express specifications in plain english. Allteam members can be on the same page about thespecifications. Also since this is automated, youeffectively have executable specifications.Cucumber consists of feature files which hasscenarios in given, when then statements andcorresponding implementation of those instep_definition files.
    3. 3. Scenario: Purchasing books online Given I have selected following books in myshopping cart: |title | price| |Sherlock Homes |12.35 | |Little Women |5.00 | When I proceed to checkout And I fill out the billing and shipping details And I give my credit card details Then I should see the order confirmation message
    4. 4. We will look at a sample scenario andtry and analyse what’s wrong in theway it is written
    5. 5. Scenario: User should be able to verify items in his shopping cartand proceed to checkout by filling in billing and shipping details.Given I have a user "john“ with last name “doe” and email“jdoe@gmail.com”And I have a product “Kindle” in products listAnd product “ipad" exists with color “white”And I have added 2 “Kindle” and 2 “ipad” with color “white” in myshopping cart.When I click on view shopping cart within “//div[@id=header]”And I wait “10"Then I should see "Kindle" and “ipad" in the view shopping cartoverlayWhen I press "//span[@id=proceed_to_checkout]" buttonThen I see the page title as "Billing Page“When I fill in “First Name” with “John”And I fill in “Last Name” with “Doe”And I fill in “Email” with “jdoe@gmail.com”And I fill in “Address” with “Chicago, IL”And I select “US” from “Country”Then the countries list should show all countriesAnd I fill in “Zipcode” with “60601”When I check “shipping_same_as_billing”And I press “Next”Then I should see my shipping details on the order confirmationpage
    6. 6. Scenario: User should be able to verify items in his shopping cart andproceed to checkout by filling in billing and shipping details.Given I have a user "john“ with last name “doe” and email “jdoe@gmail.com”And I have a product “Kindle” with price “50$” inproducts listAnd product “ipad" exists with color “white”And I have added 2 “Kindle” and 2 “ipad” with color “white” in my shoppingcart.When I click on view shopping cart within “//div[@id=header]”And I wait “10"Then I should see "Kindle" and “ipad" in the view shopping cart overlayWhen I press "//span[@id=proceed_to_checkout]" buttonThen I see the page title as "Billing Page"When I fill in “First Name” with “John”And I fill in “Last Name” with “Doe”And I fill in “Email” with “jdoe@gmail.com”And I fill in “Address” with “Chicago, IL”And I select “US” from “Country”Then the countries list should show all countriesAnd I fill in “Zipcode” with “60601”When I check “shipping_same_as_billing”And I press “Next”Then I should see my shipping details on the order confirmation page
    7. 7. Step rewritten:Given following products are available: |product title| price | |Kindle | 50.00 | |ipad | 50.00 |And product “ipad” is available with followingoptions: |product option|product option value| | colour | white |And I wait “10"Then I should see "Kindle" and “ipad" in the view shopping cart overlayWhen I press "//span[@id=proceed_to_checkout]" buttonThen I see the page title as "Billing Page"When I fill in “First Name” with “John”And I fill in “Last Name” with “Doe”And I fill in “Email” with “jdoe@gmail.com”And I fill in “Address” with “Chicago, IL”And I select “US” from “Country”Then the countries list should show all countriesAnd I fill in “Zipcode” with “60601”When I check “shipping_same_as_billing”And I press “Next”Then I should see my shipping details on the order confirmation page
    8. 8. Scenario: User should be able to verify items in his shopping cart andproceed to checkout by filling in billing and shipping details.Given I have a user "john“ with last name “doe” and email “jdoe@gmail.com”And I have a product “Kindle” with price “50$” in products listAnd product “ipad" exists with color “white”And I have added 2 “Kindle” and 2 “ipad” with color“white” in my shopping cart.When I click on view shopping cart within “//div[@id=header]”And I wait “10"Then I should see "Kindle" and “ipad" in the view shopping cart overlayWhen I press "//span[@id=proceed_to_checkout]" buttonThen I see the page title as "Billing Page"When I fill in “First Name” with “John”And I fill in “Last Name” with “Doe”And I fill in “Email” with “jdoe@gmail.com”And I fill in “Address” with “Chicago, IL”And I select “US” from “Country”Then the countries list should show all countriesAnd I fill in “Zipcode” with “60601”When I check “shipping_same_as_billing”And I press “Next”Then I should see my shipping details on the order confirmation page
    9. 9. Scenario: User should be able to verify items in his shopping cart and proceed to checkout byfilling in billing and shipping details.Step rewritten:And I add the following to my shopping cart:|product name|product option|option value|quantity|| kindle | color | white | 2 || ipad | | | 2 |And I have a product “Kindle” in products listAnd product “ipad" exists with color “white”And I have added 2 “Kindle” and 2 “ipad” with color “white” in my shopping cart.When I click on view shopping cart within “//div[@id=header]”And I wait “10"Then I should see "Kindle" and “ipad" in the view shopping cart overlayWhen I press "//span[@id=proceed_to_checkout]" buttonThen I see the page title as "Billing Page"When I fill in “First Name” with “John”And I fill in “Last Name” with “Doe”And I fill in “Email” with “jdoe@gmail.com”And I fill in “Address” with “Chicago, IL”And I select “US” from “Country”Then the countries list should show all countriesAnd I fill in “Zipcode” with “60601”When I check “shipping_same_as_billing”And I press “Next”Then I should see my shipping details on the order confirmation page
    10. 10. Scenario: User should be able to verify items in his shopping cart andproceed to checkout by filling in billing and shipping details.Given I have a user "john“ with last name “doe” and email “jdoe@gmail.com”And I have a product “Kindle” with price “50$” in products listAnd product “ipad" exists with color “white”And I have added 2 “Kindle” and 2 “ipad” with color “white” in my shoppingcart. When I click on view shopping cart within“//div[@id=header]”And I wait “10"Then I should see "Kindle" and “ipad" in the view shopping cart overlayWhen I press "//span[@id=proceed_to_checkout]" buttonThen I see the page title as "Billing Page"When I fill in “First Name” with “John”And I fill in “Last Name” with “Doe”And I fill in “Email” with “jdoe@gmail.com”And I fill in “Address” with “Chicago, IL”And I select “US” from “Country”Then the countries list should show all countriesAnd I fill in “Zipcode” with “60601”When I check “shipping_same_as_billing”And I press “Next”Then I should see my shipping details on the order confirmation page
    11. 11. filling in billing and shipping details.Step rewritten:When I view the shopping cartWhen /^I view the shopping cart$/ do with_scope(Sections::MAP[“header”]) do click_link(“Shopping Cart”) endendAnd I have a product “Kindle” in products listAnd product “ipad" exists with color “white” e.g. sections. rbAnd I have added 2 “Kindle” and 2 “ipad” with color “white” in my shopping cart. module SectionsWhen I click on view shopping cart within “//div[@id=header]”And I wait “10" MAP =Then I should see "Kindle" and “ipad" in the view shopping cart overlayWhen I press "//span[@id=proceed_to_checkout]" buttonThen I see the page title as "Billing Page" {When I fill in “First Name” with “John” „header =>And I fill in “Last Name” with “Doe”And I fill in “Email” with “jdoe@gmail.com” //div[@id=„header‟]„And I fill in “Address” with “Chicago, IL”And I select “US” from “Country”Then the countries list should show all countries }And I fill in “Zipcode” with “60601”When I check “shipping_same_as_billing” endAnd I press “Next”Then I should see my shipping details on the order confirmation page
    12. 12. Scenario: User should be able to verify items in his shopping cart andproceed to checkout by filling in billing and shipping details.Given I have a user "john“ with last name “doe” and email “jdoe@gmail.com”And I have a product “Kindle” with price “50$” in products listAnd product “ipad" exists with color “white”And I have added 2 “Kindle” and 2 “ipad” with color “white” in my shoppingcart.When I click on view shopping cart within “//div[@id=header]”And I wait “10"Then I should see "Kindle" and “ipad" in the view shopping cart overlayWhen I press "//span[@id=proceed_to_checkout]" buttonThen I see the page title as "Billing Page"When I fill in “First Name” with “John”And I fill in “Last Name” with “Doe”And I fill in “Email” with “jdoe@gmail.com”And I fill in “Address” with “Chicago, IL”And I select “US” from “Country”Then the countries list should show all countriesAnd I fill in “Zipcode” with “60601”When I check “shipping_same_as_billing”And I press “Next”Then I should see my shipping details on the order confirmation page
    13. 13. Scenario: User should be able to verify items in his shopping cart and proceed to checkout byWhen I view the shopping cartWhen /^I view the shopping cart$/ do with_scope(sections[“header”]) do click_link(“Shopping Cart”) end And %{wait for element to appearsections[„cart‟]”}EndWhen /^wait for element to appear "([^"]*)"$/ do|selector| loop_until { find(:xpath, selector).visible? }endAnd I fill in “Email” with “jdoe@gmail.com”And I fill in “Address” with “Chicago, IL”And I select “US” from “Country”Then the countries list should show all countriesAnd I fill in “Zipcode” with “60601”When I check “shipping_same_as_billing”And I press “Next”Then I should see my shipping details on the order confirmation page
    14. 14. Scenario: User should be able to verify items in his shopping cart andproceed to checkout by filling in billing and shipping details.And product “ipad" exists with color “white”And I have added 2 “Kindle” and 2 “ipad” with color “white” in my shoppingcart.When I click on view shopping cart within “//div[@id=header]”And I wait “10"Then I should see "Kindle" and “ipad" in the view shopping cart overlayWhen I press "//span[@id=proceed_to_checkout]" buttonThen I see the page title as "Billing Page" When I fill in “First Name” with “John” And I fill in “Last Name” with “Doe” And I fill in “Email” with “jdoe@gmail.com” And I fill in “Address” with “Chicago, IL” And I select “US” from “Country” Then the countries list should show allcountries And I fill in “Zipcode” with “60601” When I check “shipping_same_as_billing”And I press “Next”Then I should see my shipping details on the order confirmation page
    15. 15. Scenario: User should be able to verify items in his shopping cart andproceed to checkout by filling in billing and shipping details.And product “ipad" exists with color “white”And I have added 2 “Kindle” and 2 “ipad” with color “white” in my shoppingcart.When I click on view shopping cart within “//div[@id=header]”And I wait “10"Then I should see "Kindle" and “ipad" in the view shopping cart overlayWhen I press "//span[@id=proceed_to_checkout]" buttonThen I see the page title as "Billing Page" When I fill in “First Name” with “John” And I fill in “Last Name” with “Doe” And I fill in “Email” with “jdoe@gmail.com” And I fill in “Address” with “Chicago, IL” And I select “US” from “Country” Then the countries list should show allcountries And I fill in “Zipcode” with “60601” When I check “shipping_same_as_billing”And I press “Next”Then I should see my shipping details on the order confirmation page
    16. 16. Step rewritten:When I fill in my billing detailsAnd I choose to keep shipping address same asbillingWhen /^I fill in my billing details$/ do fill_in(“first_name”, :with => “John”) fill_in(“last_name”, :with => “Doe”) fill_in(“email”, :with => “jdoe@gmail.com”) fill_in(“address”, :with => “Chicago, IL”) fill_in(“zipcode”, :with => “John”) select(“US”, :from => “Country”)EndWhen /^I choose to keep shipping address same asbilling$/ do check(“shipping_same_as_billing”)EndAnd I fill in “Address” with “Chicago, IL”And I press “Next”
    17. 17. Step rewritten:When I fill in my billing detailsAnd I choose to keep shipping address same asbillingWhen /^I fill in my billing details$/ do fill_in(“first_name”, :with => “John”) fill_in(“last_name”, :with => “Doe”) fill_in(“email”, :with => “jdoe@gmail.com”) fill_in(“address”, :with => “Chicago, IL”) fill_in(“zipcode”, :with => “John”) select(“US”, :from => “Country”)EndWhen /^I choose to keep shipping address same asbilling$/ do check(“shipping_same_as_billing”)EndAnd I fill in “Address” with “Chicago, IL”And I press “Next”
    18. 18. Scenario: User should be able to verify items in his shopping cart andproceed to checkout by filling in billing and shipping details.Given I have a user "john“ with last name “doe” and email “jdoe@gmail.com”And I have a product “Kindle” with price “50$” in products listAnd product “ipad" exists with color “white”And I have added 2 “Kindle” and 2 “ipad” with color “white” in my shoppingcart.When I press "//span[@id=proceed_to_checkout]" buttonThen I see the page title as "Billing Page" When I fill in “First Name” with “John” And I fill in “Last Name” with “Doe” And I fill in “Email” with “jdoe@gmail.com” And I fill in “Address” with “Chicago, IL” And I select “US” from “Country” Then the countries list should show allcountries And I fill in “Zipcode” with “60601” When I check “shipping_same_as_billing”And I press “Next”Then I should see my shipping details on the order confirmation page
    19. 19. Then I should see my shipping details on the orderconfirmation pageWhen I press "//span[@id=proceed_to_checkout]" buttonThen /^I should see my shipping details on theorder confirmation page$/ do shipping_details = @user.shipping_address [:fist_name, :email, :zipcode].each do |field| Then %{I should see “#{shipping_details[field]}”}endom “Country” Then the countries list should show all countries This step depends on And I fill in “Zipcode” with “60601” When I check “shipping_same_as_billing” an instance variableAnd I press “Next” being populated from some previous step
    20. 20. Scenario: User should be able to verify items in his shopping cart and proceed to checkout by filling in billingand shipping details.Given I have a user "john“ with last name “doe” and email jdoe@gmail.comWhen I click on view shopping cart within “//div[@id=header]”And I wait “10"Then I should see "Kindle" and “ipad" in the view shopping cart overlayThen I should see my shipping details on the orderconfirmation pageWhen I press "//span[@id=proceed_to_checkout]" buttonThen I see the page title as "Billing Page" When I fill in “First Name” with “John” And I fill in “Last Name” with “Doe” And I fill in “Email” with “jdoe@gmail.comThen /^I should see my shipping details on theorder confirmation page$/ do shipping_details = @user.shipping_address [:fist_name, :email, :zipcode].each do |field| Then %{I should see “#{shipping_details[field]}”}endom “Country” Then the countries list should show all countries And I fill in “Zipcode” with “60601” When I check “shipping_same_as_billing”And I press “Next”
    21. 21. Scenario: User should be able to verify items in his shopping cart and proceed to checkout byfilling in billing and shipping details.Then I should see shipping details for user “john”on the order confirmation pageThen /^I should see shipping details for user“(.*)” on the order confirmation page$/ do|identifier| user = User.find_by_identifier(identifier) shipping_details = user.shipping_address [:fist_name, :email, :zipcode].each do |field| Then %{I should see “#{shipping_details[field]}”}end Send in uniqueThen the countries list should show socountries the identifiers all thatAnd I select “US” from “Country”And I fill in “Zipcode” with “60601” specific record can beWhen I check “shipping_same_as_billing”And I press “Next” retrieved and testing canThen I should see my shipping details on the order confirmation page be done
    22. 22. Given I have a user with details as: |first name| John | |last name | Doe | |email | jdoe@gmail.com|Given following products are available: |product title| price | |Kindle | 50.00 | |ipad | 50.00 |And product “ipad” is available with following options: |product option|product option value| | colour | white |And I add the following to my shopping cart:|product title|product option|option value| quantity || kindle | color | white | 2 || ipad | | | 2 |When I view the shopping cartThen I should see the following items in “Cart” section: | product |quantity| |kindle |2 | | ipad |1 |When I proceed to checkoutWhen I fill in my billing detailsAnd I choose to keep shipping address same as billingAnd I submit the billing formThen I should see shipping details for user “john” on the orderconfirmation page
    23. 23. Use the rerun profile feature (with caution!) rerun = File.file?(rerun.txt) ? IO.read(rerun.txt) : "" rerun_opts = rerun.to_s.strip.empty? ? "--format #{ENV[CUCUMBER_FORMAT] || progress} features" : "--format #{ENV[CUCUMBER_FORMAT] || pretty} #{rerun}" %> rerun: <%= rerun_opts %> --format rerun --out rerun.txt --strict --tags ~@wip
    24. 24. Backdoor entry for login: Quite often end to end scenarios consist of a user getting logged in and then doing things on the app. You need not repeat the complicated login process (via the ui). Simply get the user logged in the fastest way via a backdoor entry. Saves a lot of time !! Cut down 5 mins in a feature file.
    25. 25. Backdoor entry for login:Without backdoor With backdoorWhen /^I log in as "([^"]*)" Given /^I am logged in aswith password "([^"]*)" without "(.*)"$/ do |screen_name|overlay$/ do |email, password| url =When %{I go to the login page} "/backdoor/#{screen_name}“When %{authenticate user call to visit urlcis is stubbed to return end"<user><id>1</id></user>"}When %{search user call to cis isstubbed to return"<user><id>1</id></user>"}When %{I fill in "email" with"#{email}"}When %{I fill in "password" with"#{password}"}When %{I press "login"}And %{I wait "1"} @logged_in_user =User.find_by_email(email)end
    26. 26. if Rails.env.test? get "/backdoor/:screen_name", :to => "sessions#backdoor", :as => "backdoor“enddef backdoor user = User.find_by_screen_name(params[:screen_na me]) set_current_user(user) head :okend
    27. 27.  Do not use page.has_xpath? should be false. Go for page.should_not have_xpath.Has_xpath? would try repeatedly (till capybara timeout)to find the element. When it doesn’t find it, the valuereturned would be false which would match with theassert. Rather than that use has_no_xpath? So thencapybara exits the first time itself when it does not findthe element. Will save a lot of time!!! Cut down buildtime by 7 mins for us!
    28. 28.  Use scenario outlines with caution Scenario outlines should be used only if you actually need the whole scenario to be repeated for your examples not otherwise.
    29. 29. When /^I fill in my billing details$/ do And %{I fill in “first_name” with “John”} And %{I fill in “zipcode” with “John”} And %{I select “US” from “Country”}EndWhen /^I fill in my billing details$/ do fill_in(“first_name”, :with => “John”) fill_in(“zipcode”, :with => “John”) select(“US”, :from => “Country”)EndRather than dealing with regex matches deal withcapybara methods which are equally readable.Introduce methods, modules, and keep your code dryand clean.
    30. 30. Stop doing Start doing
    31. 31. Happy cuking…
    32. 32.  http://benmabey.com/2008/05/19/imperative-vs-declarative- scenarios-in-user-stories.html http://elabs.se/blog/15-you-re-cuking-it-wrong mislav.uniqpath.com/2010/09/cuking-it-right/ http://aslakhellesoy.com/post/11055981222/the-training-wheels- came-off http://skillsmatter.com/podcast/home/refuctoring-your-cukes

    ×