Keeping your cucumber suite maintainable

  • 2,537 views
Uploaded on

Cucumber is a very good tool for achieving functional testing. However your cuke suite can become unwieldy very soon. Even a little bit of indiscipline can lead to huge build times, with random …

Cucumber is a very good tool for achieving functional testing. However your cuke suite can become unwieldy very soon. Even a little bit of indiscipline can lead to huge build times, with random failures. Over time, maintaining and writing the cukes starts costing more than the benefit derived out of it. On the other hand, you can start ignoring your functional build only to discover some nasty bug later which your poor cuke suite was telling you.

In this presentation we try and look at what can be done to avoid this and derive complete value from cukes. We share experiences and learnings about cuking the right way from real projects
Learnings
1) best practices of writing and maintaining cukes
2) some new things that could be tried out with cukes to make them more reliable and valuable.

More in: Technology , Business
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
2,537
On Slideshare
0
From Embeds
0
Number of Embeds
3

Actions

Shares
Downloads
33
Comments
0
Likes
2

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide
  • Introduce yourselves
  • Some knowledge of cucumber
  • When I started the Cucumber project in 2008 I had three major goals:Help stakeholders express what they want via executable specificationsHelp programmers write better software by making TDD/BDD easyReduce misinterpretations through a ubiquitous language
  • When I started the Cucumber project in 2008 I had three major goals:Help stakeholders express what they want via executable specificationsHelp programmers write better software by making TDD/BDD easyReduce misinterpretations through a ubiquitous language
  • small changes causing lot of cukes to failfailures causing long build timesSo much so that you seem to land up spending more time on the cukes rather than on the implementation
  • small changes causing lot of cukes to failfailures causing long build timesSo much so that you seem to land up spending more time on the cukes rather than on the implementation
  • Steps in different filesRedundant codeLess predictability
  • Steps in wrong filesOrganized readableFlexible
  • Dependencyoverriding
  • Dependencyoverriding
  • you can then specify the output location for your failures. And then run the failures in rerun profile. “A test is non-deterministic when it passes sometimes and fails sometimes, without any noticeable change in the code, tests, or environment. Such tests fail, then you re-run them and they pass. Test failures for such tests are seemingly random.” - Martin Fowler
  • you can then specify the output location for your failures. And then run the failures in rerun profile. “A test is non-deterministic when it passes sometimes and fails sometimes, without any noticeable change in the code, tests, or environment. Such tests fail, then you re-run them and they pass. Test failures for such tests are seemingly random.” - Martin Fowler
  • 21 mins to 17 mins
  • 21 mins to 17 mins
  • Reduced time for one feature from 7.5 minutes to 2.4 minutes
  • Reduced time for one feature from 7.5 minutes to 2.4 minutes
  • Parallel_tests, hydra, tlb
  • Use more of Capybara DSL than web_stepsUse of CSS locator strategy than xpathConsistency set of guidelines for teamUse proper taggingStub out third party services calls and have seperate integration testsTreat cucumber scenarios like your unit tests code.

Transcript

  • 1. Ketan Soni@ketan_soniAtreyee Maiti@AtreyeeMaiti
  • 2. Cucumber is meant to provide stakeholders a wayto express specifications in plain english. Allteam members can be on the same page about thespecifications. Also since this is automated, youeffectively have executable specifications.Cucumber consists of feature files which hasscenarios in given, when then statements andcorresponding implementation of those instep_definition files.
  • 3. Scenario: Purchasing books online Given I have selected following books in myshopping cart: |title | price| |Sherlock Homes |12.35 | |Little Women |5.00 | When I proceed to checkout And I fill out the billing and shipping details And I give my credit card details Then I should see the order confirmation message
  • 4. We will look at a sample scenario andtry and analyse what’s wrong in theway it is written
  • 5. Scenario: User should be able to verify items in his shopping cartand proceed to checkout by filling in billing and shipping details.Given I have a user "john“ with last name “doe” and email“jdoe@gmail.com”And I have a product “Kindle” in products listAnd product “ipad" exists with color “white”And I have added 2 “Kindle” and 2 “ipad” with color “white” in myshopping cart.When I click on view shopping cart within “//div[@id=header]”And I wait “10"Then I should see "Kindle" and “ipad" in the view shopping cartoverlayWhen I press "//span[@id=proceed_to_checkout]" buttonThen I see the page title as "Billing Page“When I fill in “First Name” with “John”And I fill in “Last Name” with “Doe”And I fill in “Email” with “jdoe@gmail.com”And I fill in “Address” with “Chicago, IL”And I select “US” from “Country”Then the countries list should show all countriesAnd I fill in “Zipcode” with “60601”When I check “shipping_same_as_billing”And I press “Next”Then I should see my shipping details on the order confirmationpage
  • 6. Scenario: User should be able to verify items in his shopping cart andproceed to checkout by filling in billing and shipping details.Given I have a user "john“ with last name “doe” and email “jdoe@gmail.com”And I have a product “Kindle” with price “50$” inproducts listAnd product “ipad" exists with color “white”And I have added 2 “Kindle” and 2 “ipad” with color “white” in my shoppingcart.When I click on view shopping cart within “//div[@id=header]”And I wait “10"Then I should see "Kindle" and “ipad" in the view shopping cart overlayWhen I press "//span[@id=proceed_to_checkout]" buttonThen I see the page title as "Billing Page"When I fill in “First Name” with “John”And I fill in “Last Name” with “Doe”And I fill in “Email” with “jdoe@gmail.com”And I fill in “Address” with “Chicago, IL”And I select “US” from “Country”Then the countries list should show all countriesAnd I fill in “Zipcode” with “60601”When I check “shipping_same_as_billing”And I press “Next”Then I should see my shipping details on the order confirmation page
  • 7. Step rewritten:Given following products are available: |product title| price | |Kindle | 50.00 | |ipad | 50.00 |And product “ipad” is available with followingoptions: |product option|product option value| | colour | white |And I wait “10"Then I should see "Kindle" and “ipad" in the view shopping cart overlayWhen I press "//span[@id=proceed_to_checkout]" buttonThen I see the page title as "Billing Page"When I fill in “First Name” with “John”And I fill in “Last Name” with “Doe”And I fill in “Email” with “jdoe@gmail.com”And I fill in “Address” with “Chicago, IL”And I select “US” from “Country”Then the countries list should show all countriesAnd I fill in “Zipcode” with “60601”When I check “shipping_same_as_billing”And I press “Next”Then I should see my shipping details on the order confirmation page
  • 8. Scenario: User should be able to verify items in his shopping cart andproceed to checkout by filling in billing and shipping details.Given I have a user "john“ with last name “doe” and email “jdoe@gmail.com”And I have a product “Kindle” with price “50$” in products listAnd product “ipad" exists with color “white”And I have added 2 “Kindle” and 2 “ipad” with color“white” in my shopping cart.When I click on view shopping cart within “//div[@id=header]”And I wait “10"Then I should see "Kindle" and “ipad" in the view shopping cart overlayWhen I press "//span[@id=proceed_to_checkout]" buttonThen I see the page title as "Billing Page"When I fill in “First Name” with “John”And I fill in “Last Name” with “Doe”And I fill in “Email” with “jdoe@gmail.com”And I fill in “Address” with “Chicago, IL”And I select “US” from “Country”Then the countries list should show all countriesAnd I fill in “Zipcode” with “60601”When I check “shipping_same_as_billing”And I press “Next”Then I should see my shipping details on the order confirmation page
  • 9. Scenario: User should be able to verify items in his shopping cart and proceed to checkout byfilling in billing and shipping details.Step rewritten:And I add the following to my shopping cart:|product name|product option|option value|quantity|| kindle | color | white | 2 || ipad | | | 2 |And I have a product “Kindle” in products listAnd product “ipad" exists with color “white”And I have added 2 “Kindle” and 2 “ipad” with color “white” in my shopping cart.When I click on view shopping cart within “//div[@id=header]”And I wait “10"Then I should see "Kindle" and “ipad" in the view shopping cart overlayWhen I press "//span[@id=proceed_to_checkout]" buttonThen I see the page title as "Billing Page"When I fill in “First Name” with “John”And I fill in “Last Name” with “Doe”And I fill in “Email” with “jdoe@gmail.com”And I fill in “Address” with “Chicago, IL”And I select “US” from “Country”Then the countries list should show all countriesAnd I fill in “Zipcode” with “60601”When I check “shipping_same_as_billing”And I press “Next”Then I should see my shipping details on the order confirmation page
  • 10. Scenario: User should be able to verify items in his shopping cart andproceed to checkout by filling in billing and shipping details.Given I have a user "john“ with last name “doe” and email “jdoe@gmail.com”And I have a product “Kindle” with price “50$” in products listAnd product “ipad" exists with color “white”And I have added 2 “Kindle” and 2 “ipad” with color “white” in my shoppingcart. When I click on view shopping cart within“//div[@id=header]”And I wait “10"Then I should see "Kindle" and “ipad" in the view shopping cart overlayWhen I press "//span[@id=proceed_to_checkout]" buttonThen I see the page title as "Billing Page"When I fill in “First Name” with “John”And I fill in “Last Name” with “Doe”And I fill in “Email” with “jdoe@gmail.com”And I fill in “Address” with “Chicago, IL”And I select “US” from “Country”Then the countries list should show all countriesAnd I fill in “Zipcode” with “60601”When I check “shipping_same_as_billing”And I press “Next”Then I should see my shipping details on the order confirmation page
  • 11. filling in billing and shipping details.Step rewritten:When I view the shopping cartWhen /^I view the shopping cart$/ do with_scope(Sections::MAP[“header”]) do click_link(“Shopping Cart”) endendAnd I have a product “Kindle” in products listAnd product “ipad" exists with color “white” e.g. sections. rbAnd I have added 2 “Kindle” and 2 “ipad” with color “white” in my shopping cart. module SectionsWhen I click on view shopping cart within “//div[@id=header]”And I wait “10" MAP =Then I should see "Kindle" and “ipad" in the view shopping cart overlayWhen I press "//span[@id=proceed_to_checkout]" buttonThen I see the page title as "Billing Page" {When I fill in “First Name” with “John” „header =>And I fill in “Last Name” with “Doe”And I fill in “Email” with “jdoe@gmail.com” //div[@id=„header‟]„And I fill in “Address” with “Chicago, IL”And I select “US” from “Country”Then the countries list should show all countries }And I fill in “Zipcode” with “60601”When I check “shipping_same_as_billing” endAnd I press “Next”Then I should see my shipping details on the order confirmation page
  • 12. Scenario: User should be able to verify items in his shopping cart andproceed to checkout by filling in billing and shipping details.Given I have a user "john“ with last name “doe” and email “jdoe@gmail.com”And I have a product “Kindle” with price “50$” in products listAnd product “ipad" exists with color “white”And I have added 2 “Kindle” and 2 “ipad” with color “white” in my shoppingcart.When I click on view shopping cart within “//div[@id=header]”And I wait “10"Then I should see "Kindle" and “ipad" in the view shopping cart overlayWhen I press "//span[@id=proceed_to_checkout]" buttonThen I see the page title as "Billing Page"When I fill in “First Name” with “John”And I fill in “Last Name” with “Doe”And I fill in “Email” with “jdoe@gmail.com”And I fill in “Address” with “Chicago, IL”And I select “US” from “Country”Then the countries list should show all countriesAnd I fill in “Zipcode” with “60601”When I check “shipping_same_as_billing”And I press “Next”Then I should see my shipping details on the order confirmation page
  • 13. Scenario: User should be able to verify items in his shopping cart and proceed to checkout byWhen I view the shopping cartWhen /^I view the shopping cart$/ do with_scope(sections[“header”]) do click_link(“Shopping Cart”) end And %{wait for element to appearsections[„cart‟]”}EndWhen /^wait for element to appear "([^"]*)"$/ do|selector| loop_until { find(:xpath, selector).visible? }endAnd I fill in “Email” with “jdoe@gmail.com”And I fill in “Address” with “Chicago, IL”And I select “US” from “Country”Then the countries list should show all countriesAnd I fill in “Zipcode” with “60601”When I check “shipping_same_as_billing”And I press “Next”Then I should see my shipping details on the order confirmation page
  • 14. Scenario: User should be able to verify items in his shopping cart andproceed to checkout by filling in billing and shipping details.And product “ipad" exists with color “white”And I have added 2 “Kindle” and 2 “ipad” with color “white” in my shoppingcart.When I click on view shopping cart within “//div[@id=header]”And I wait “10"Then I should see "Kindle" and “ipad" in the view shopping cart overlayWhen I press "//span[@id=proceed_to_checkout]" buttonThen I see the page title as "Billing Page" When I fill in “First Name” with “John” And I fill in “Last Name” with “Doe” And I fill in “Email” with “jdoe@gmail.com” And I fill in “Address” with “Chicago, IL” And I select “US” from “Country” Then the countries list should show allcountries And I fill in “Zipcode” with “60601” When I check “shipping_same_as_billing”And I press “Next”Then I should see my shipping details on the order confirmation page
  • 15. Scenario: User should be able to verify items in his shopping cart andproceed to checkout by filling in billing and shipping details.And product “ipad" exists with color “white”And I have added 2 “Kindle” and 2 “ipad” with color “white” in my shoppingcart.When I click on view shopping cart within “//div[@id=header]”And I wait “10"Then I should see "Kindle" and “ipad" in the view shopping cart overlayWhen I press "//span[@id=proceed_to_checkout]" buttonThen I see the page title as "Billing Page" When I fill in “First Name” with “John” And I fill in “Last Name” with “Doe” And I fill in “Email” with “jdoe@gmail.com” And I fill in “Address” with “Chicago, IL” And I select “US” from “Country” Then the countries list should show allcountries And I fill in “Zipcode” with “60601” When I check “shipping_same_as_billing”And I press “Next”Then I should see my shipping details on the order confirmation page
  • 16. Step rewritten:When I fill in my billing detailsAnd I choose to keep shipping address same asbillingWhen /^I fill in my billing details$/ do fill_in(“first_name”, :with => “John”) fill_in(“last_name”, :with => “Doe”) fill_in(“email”, :with => “jdoe@gmail.com”) fill_in(“address”, :with => “Chicago, IL”) fill_in(“zipcode”, :with => “John”) select(“US”, :from => “Country”)EndWhen /^I choose to keep shipping address same asbilling$/ do check(“shipping_same_as_billing”)EndAnd I fill in “Address” with “Chicago, IL”And I press “Next”
  • 17. Step rewritten:When I fill in my billing detailsAnd I choose to keep shipping address same asbillingWhen /^I fill in my billing details$/ do fill_in(“first_name”, :with => “John”) fill_in(“last_name”, :with => “Doe”) fill_in(“email”, :with => “jdoe@gmail.com”) fill_in(“address”, :with => “Chicago, IL”) fill_in(“zipcode”, :with => “John”) select(“US”, :from => “Country”)EndWhen /^I choose to keep shipping address same asbilling$/ do check(“shipping_same_as_billing”)EndAnd I fill in “Address” with “Chicago, IL”And I press “Next”
  • 18. Scenario: User should be able to verify items in his shopping cart andproceed to checkout by filling in billing and shipping details.Given I have a user "john“ with last name “doe” and email “jdoe@gmail.com”And I have a product “Kindle” with price “50$” in products listAnd product “ipad" exists with color “white”And I have added 2 “Kindle” and 2 “ipad” with color “white” in my shoppingcart.When I press "//span[@id=proceed_to_checkout]" buttonThen I see the page title as "Billing Page" When I fill in “First Name” with “John” And I fill in “Last Name” with “Doe” And I fill in “Email” with “jdoe@gmail.com” And I fill in “Address” with “Chicago, IL” And I select “US” from “Country” Then the countries list should show allcountries And I fill in “Zipcode” with “60601” When I check “shipping_same_as_billing”And I press “Next”Then I should see my shipping details on the order confirmation page
  • 19. Then I should see my shipping details on the orderconfirmation pageWhen I press "//span[@id=proceed_to_checkout]" buttonThen /^I should see my shipping details on theorder confirmation page$/ do shipping_details = @user.shipping_address [:fist_name, :email, :zipcode].each do |field| Then %{I should see “#{shipping_details[field]}”}endom “Country” Then the countries list should show all countries This step depends on And I fill in “Zipcode” with “60601” When I check “shipping_same_as_billing” an instance variableAnd I press “Next” being populated from some previous step
  • 20. Scenario: User should be able to verify items in his shopping cart and proceed to checkout by filling in billingand shipping details.Given I have a user "john“ with last name “doe” and email jdoe@gmail.comWhen I click on view shopping cart within “//div[@id=header]”And I wait “10"Then I should see "Kindle" and “ipad" in the view shopping cart overlayThen I should see my shipping details on the orderconfirmation pageWhen I press "//span[@id=proceed_to_checkout]" buttonThen I see the page title as "Billing Page" When I fill in “First Name” with “John” And I fill in “Last Name” with “Doe” And I fill in “Email” with “jdoe@gmail.comThen /^I should see my shipping details on theorder confirmation page$/ do shipping_details = @user.shipping_address [:fist_name, :email, :zipcode].each do |field| Then %{I should see “#{shipping_details[field]}”}endom “Country” Then the countries list should show all countries And I fill in “Zipcode” with “60601” When I check “shipping_same_as_billing”And I press “Next”
  • 21. Scenario: User should be able to verify items in his shopping cart and proceed to checkout byfilling in billing and shipping details.Then I should see shipping details for user “john”on the order confirmation pageThen /^I should see shipping details for user“(.*)” on the order confirmation page$/ do|identifier| user = User.find_by_identifier(identifier) shipping_details = user.shipping_address [:fist_name, :email, :zipcode].each do |field| Then %{I should see “#{shipping_details[field]}”}end Send in uniqueThen the countries list should show socountries the identifiers all thatAnd I select “US” from “Country”And I fill in “Zipcode” with “60601” specific record can beWhen I check “shipping_same_as_billing”And I press “Next” retrieved and testing canThen I should see my shipping details on the order confirmation page be done
  • 22. Given I have a user with details as: |first name| John | |last name | Doe | |email | jdoe@gmail.com|Given following products are available: |product title| price | |Kindle | 50.00 | |ipad | 50.00 |And product “ipad” is available with following options: |product option|product option value| | colour | white |And I add the following to my shopping cart:|product title|product option|option value| quantity || kindle | color | white | 2 || ipad | | | 2 |When I view the shopping cartThen I should see the following items in “Cart” section: | product |quantity| |kindle |2 | | ipad |1 |When I proceed to checkoutWhen I fill in my billing detailsAnd I choose to keep shipping address same as billingAnd I submit the billing formThen I should see shipping details for user “john” on the orderconfirmation page
  • 23. Use the rerun profile feature (with caution!) rerun = File.file?(rerun.txt) ? IO.read(rerun.txt) : "" rerun_opts = rerun.to_s.strip.empty? ? "--format #{ENV[CUCUMBER_FORMAT] || progress} features" : "--format #{ENV[CUCUMBER_FORMAT] || pretty} #{rerun}" %> rerun: <%= rerun_opts %> --format rerun --out rerun.txt --strict --tags ~@wip
  • 24. Backdoor entry for login: Quite often end to end scenarios consist of a user getting logged in and then doing things on the app. You need not repeat the complicated login process (via the ui). Simply get the user logged in the fastest way via a backdoor entry. Saves a lot of time !! Cut down 5 mins in a feature file.
  • 25. Backdoor entry for login:Without backdoor With backdoorWhen /^I log in as "([^"]*)" Given /^I am logged in aswith password "([^"]*)" without "(.*)"$/ do |screen_name|overlay$/ do |email, password| url =When %{I go to the login page} "/backdoor/#{screen_name}“When %{authenticate user call to visit urlcis is stubbed to return end"<user><id>1</id></user>"}When %{search user call to cis isstubbed to return"<user><id>1</id></user>"}When %{I fill in "email" with"#{email}"}When %{I fill in "password" with"#{password}"}When %{I press "login"}And %{I wait "1"} @logged_in_user =User.find_by_email(email)end
  • 26. if Rails.env.test? get "/backdoor/:screen_name", :to => "sessions#backdoor", :as => "backdoor“enddef backdoor user = User.find_by_screen_name(params[:screen_na me]) set_current_user(user) head :okend
  • 27.  Do not use page.has_xpath? should be false. Go for page.should_not have_xpath.Has_xpath? would try repeatedly (till capybara timeout)to find the element. When it doesn’t find it, the valuereturned would be false which would match with theassert. Rather than that use has_no_xpath? So thencapybara exits the first time itself when it does not findthe element. Will save a lot of time!!! Cut down buildtime by 7 mins for us!
  • 28.  Use scenario outlines with caution Scenario outlines should be used only if you actually need the whole scenario to be repeated for your examples not otherwise.
  • 29. When /^I fill in my billing details$/ do And %{I fill in “first_name” with “John”} And %{I fill in “zipcode” with “John”} And %{I select “US” from “Country”}EndWhen /^I fill in my billing details$/ do fill_in(“first_name”, :with => “John”) fill_in(“zipcode”, :with => “John”) select(“US”, :from => “Country”)EndRather than dealing with regex matches deal withcapybara methods which are equally readable.Introduce methods, modules, and keep your code dryand clean.
  • 30. Stop doing Start doing
  • 31. Happy cuking…
  • 32.  http://benmabey.com/2008/05/19/imperative-vs-declarative- scenarios-in-user-stories.html http://elabs.se/blog/15-you-re-cuking-it-wrong mislav.uniqpath.com/2010/09/cuking-it-right/ http://aslakhellesoy.com/post/11055981222/the-training-wheels- came-off http://skillsmatter.com/podcast/home/refuctoring-your-cukes