Advanced Testing on RubyEnRails '09

1,867 views
1,787 views

Published on

Presentation during RubyEnRails2009 in Amsterdam about Advanced Testing. Goal of the presentation was to share the experiences we have at MoneyBird about testing and to make people start thinking about testing themselves. Unit testing with RSpec and integration testing with Cucumber are discussed.

Published in: Technology
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,867
On SlideShare
0
From Embeds
0
Number of Embeds
16
Actions
Shares
0
Downloads
0
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

Advanced Testing on RubyEnRails '09

  1. 1. Advanced Testing RubyEnRails Conference ’09
  2. 2. Goals • Share our experiences about testing • Make you start thinking critical about testing like an advanced tester • Unit testing with RSpec • Integration testing with Cucumber
  3. 3. Student Master of Computer Science @edwin_v Edwin Vlieg Course: Testing Techniques
  4. 4. Student Master of Computer Science @edwin_v Edwin Vlieg Course: Testing 5 Techniques
  5. 5. • Web application written in Ruby on Rails • Create and send invoices online • More than 16.000 invoices sent • Short development cycle: 2 months • API for web developers • More info: www.moneybird.com
  6. 6. “A donkey does not knock himself to the same stone twice”
  7. 7. It is okay to write code that doesn’t work... But never ‘cap deploy’ it!
  8. 8. So we started testing
  9. 9. Unit testing
  10. 10. Software testing ... is a technical process performed by executing a product in a controlled environment, following a specified procedure, with the intent of measuring the quality of the software product by demonstrating deviations of the requirements.
  11. 11. Unit testing • Test individual units of code • A unit is the smallest testable piece of code: methods of classes • E.g. methods of controllers and models
  12. 12. TDD / BDD Test before you write a single piece of code
  13. 13. TDD / BDD Test before you write a single piece of code Can be a nice approach, but requires a lot of discipline and might not fit your needs, but know how it can help you!
  14. 14. Example Recurring invoices Calculate the next occurence of a recurring invoice. Start date Current occurence Next occurence Jan Febr Mar Apr May
  15. 15. First version def next_occurence(today) today + 1.month end
  16. 16. RSpec-t describe "Date calculation" do it "should return the next month" do next_occurence(Date.parse('2009-01-01')).should == Date.parse('2009-02-01') end end You ain’t testing this, because it looks too easy
  17. 17. Done! It is green.
  18. 18. Customer: “I want to bill every last day of the month” ‘2009-01-31’ + 1.month = ‘2009-02-28’ ‘2009-02-28’ + 1.month = ‘2009-03-28’ ‘2009-01-31’ + 2.months = ‘2009-03-31’
  19. 19. Boundary Value Analysis Find the boundaries of the input and test them
  20. 20. Boundary Value Analysis Find the boundaries of the input and test them Start Current Next 2009-01-31 2009-01-31 2009-02-28 2009-01-31 2009-02-28 2009-03-31 2009-01-31 2009-03-31 2009-04-30 2009-01-30 2009-01-30 2009-02-28 2009-01-30 2009-02-28 2009-03-30 2009-01-30 2009-03-30 2009-04-30 2009-01-29 2009-01-29 2009-02-28 2009-01-29 2009-02-28 2009-03-29 2009-01-29 2009-03-29 2009-04-29 2009-01-28 2009-01-28 2009-02-28 2009-01-28 2009-02-28 2009-03-28 2009-01-28 2009-03-28 2009-04-28
  21. 21. Boundary Value Analysis Find the boundaries of the input and test them Start Current Next 2009-01-31 2009-01-31 2009-02-28 2009-01-31 2009-02-28 2009-03-31 2009-01-31 2009-03-31 2009-04-30 2009-01-30 2009-01-30 2009-02-28 2009-01-30 2009-02-28 2009-03-30 2009-01-30 2009-03-30 2009-04-30 2009-01-29 2009-01-29 2009-02-28 2009-01-29 2009-02-28 2009-03-29 2009-01-29 2009-03-29 2009-04-29 2009-01-28 2009-01-28 2009-02-28 2009-01-28 2009-02-28 2009-03-28 2009-01-28 2009-03-28 2009-04-28
  22. 22. Boundary Value Analysis Find the boundaries of the input and test them Start Current Next 2009-01-31 2009-01-31 2009-02-28 2009-01-31 2009-02-28 2009-03-31 2009-01-31 2009-03-31 2009-04-30 2009-01-30 2009-01-30 2009-02-28 2009-01-30 2009-02-28 2009-03-30 2009-01-30 2009-03-30 2009-04-30 2009-01-29 2009-01-29 2009-02-28 2009-01-29 2009-02-28 2009-03-29 2009-01-29 2009-03-29 2009-04-29 2009-01-28 2009-01-28 2009-02-28 2009-01-28 2009-02-28 2009-03-28 2009-01-28 2009-03-28 2009-04-28
  23. 23. Boundary Value Analysis Find the boundaries of the input and test them Start Current Next 2009-01-31 2009-01-31 2009-02-28 2009-01-31 2009-02-28 2009-03-31 2009-01-31 2009-03-31 2009-04-30 2009-01-30 2009-01-30 2009-02-28 2009-01-30 2009-02-28 2009-03-30 2009-01-30 2009-03-30 2009-04-30 2009-01-29 2009-01-29 2009-02-28 2009-01-29 2009-02-28 2009-03-29 2009-01-29 2009-03-29 2009-04-29 2009-01-28 2009-01-28 2009-02-28 2009-01-28 2009-02-28 2009-03-28 2009-01-28 2009-03-28 2009-04-28
  24. 24. Create tests for all boundary values... ...and improve your code!
  25. 25. The solution def self.calculate_next_occurence(start_date, current_date) next_date = current_date + 1.month while start_date.day != next_date.day and next_date != next_date.end_of_month next_date = next_date.tomorrow end next_date end
  26. 26. “Testing shows the presence, not the absence of bugs” - Dijkstra
  27. 27. So, why test?
  28. 28. So, why test? When working with dates, it takes a while before the Boundary Values occure.
  29. 29. Isolation
  30. 30. Isolate for productivity • Test isolated units of code, assuring they are working • No need to bother about it later • No need to find all edge cases in the user interface: F5 syndrom
  31. 31. Isolate your tests • Don’t test code that is tested elsewhere • Only make sure it is used well in the code you’re testing • The solution: mocks and stubs
  32. 32. Mock objects are simulated objects that mimic the behavior of real objects in controlled ways A stub is a piece of code used to stand in for some other programming functionality
  33. 33. Very hard Method A Method B work
  34. 34. Method B Very hard work Method A
  35. 35. Method B Very hard work Method A Mock
  36. 36. Method B Very hard work Method A Stub Mock
  37. 37. Method B Very hard work Method A Mock Stub
  38. 38. Mock Stub
  39. 39. How to isolate? def create_invoice(contact) invoice = Invoice.new invoice.contact_id = contact.id invoice.details_attributes = [{ :description => "RER09", :price => 79 }] invoice.save end class Invoice < ActiveResource::Base self.site = "https://account.moneybird.com" end
  40. 40. How to isolate? def create_invoice(contact) invoice = Invoice.new invoice.contact_id = contact.id invoice.details_attributes = [{ :description => "RER09", :price => 79 }] invoice.save end class Invoice < ActiveResource::Base self.site = "https://account.moneybird.com" end ActiveResource is tested elsewhere + very slow to test
  41. 41. describe "MoneyBird API" do it "should create the invoice" do create_invoice(contact_mock).should be_true end end
  42. 42. describe "MoneyBird API" do it "should create the invoice" do contact_mock = mock(:contact) contact_mock.should_receive(:id).and_return(3) create_invoice(contact_mock).should be_true end end
  43. 43. describe "MoneyBird API" do it "should create the invoice" do Invoice.should_receive(:new).and_return(invoice_mock) contact_mock = mock(:contact) contact_mock.should_receive(:id).and_return(3) create_invoice(contact_mock).should be_true end end
  44. 44. describe "MoneyBird API" do it "should create the invoice" do invoice_mock = mock(:invoice) invoice_mock.stub!(:contact_id=) invoice_mock.stub!(:details_attributes=) invoice_mock.should_receive(:save).and_return(true) Invoice.should_receive(:new).and_return(invoice_mock) contact_mock = mock(:contact) contact_mock.should_receive(:id).and_return(3) create_invoice(contact_mock).should be_true end end
  45. 45. Isolation Isolated A B
  46. 46. Isolation Isolated A B Not isolated A B
  47. 47. Isolation Isolated A B Not isolated A B Double = Extra test work + maintenance work!
  48. 48. What to test? Everything! But never too much!! Be critical
  49. 49. Downside of testing • Every test gives overhead • Testing the obvious takes time • Ruby library, CRUD controllers
  50. 50. Downside of testing • Every test gives overhead • Testing the obvious takes time • Ruby library, CRUD controllers But what if suddenly: BigDecimal("10.03").to_f != 10.03
  51. 51. We use (very) fat models So we unit test them
  52. 52. RSpec • All business logic in models • Models tested with RSpec • Basic data model in fixtures
  53. 53. Fixtures? • Models make use of the database • Sometimes data in de database is needed for the model to work • Fixtures are easy to manage on the filesystem (+ version control) • Fixtures can be used for bootstrapping application: rake db:fixtures:load
  54. 54. first_contact: company: bluetools name: Edwins company contact_name: Edwin Vlieg address1: Street 82 zipcode: 7541 XA city: Enschede country: NL send_method: mail created_at: <%= 60.days.ago.to_s :db %> contact_hash: 1 second_contact: company: bluetools name: BlueTools B.V. address1: Postbus 123 zipcode: 7500EA city: Enschede country: NL send_method: post contact_hash: 2
  55. 55. first_contact: company: bluetools Unique identifier for name: Edwins company ‘row’ in database. contact_name: Edwin Vlieg Don’t set the id column!! address1: Street 82 zipcode: 7541 XA city: Enschede country: NL send_method: mail created_at: <%= 60.days.ago.to_s :db %> contact_hash: 1 second_contact: company: bluetools name: BlueTools B.V. address1: Postbus 123 zipcode: 7500EA city: Enschede country: NL send_method: post contact_hash: 2
  56. 56. first_contact: company: bluetools Unique identifier for name: Edwins company ‘row’ in database. contact_name: Edwin Vlieg Don’t set the id column!! address1: Street 82 zipcode: 7541 XA city: Enschede country: NL send_method: mail created_at: <%= 60.days.ago.to_s :db %> contact_hash: 1 second_contact: company: bluetools name: BlueTools B.V. Yes, you can use Ruby! address1: Postbus 123 zipcode: 7500EA city: Enschede country: NL send_method: post contact_hash: 2
  57. 57. first_contact: company: bluetools Unique identifier for name: Edwins company ‘row’ in database. contact_name: Edwin Vlieg Don’t set the id column!! address1: Street 82 zipcode: 7541 XA city: Enschede country: NL send_method: mail created_at: <%= 60.days.ago.to_s :db %> contact_hash: 1 second_contact: company: bluetools name: BlueTools B.V. Yes, you can use Ruby! address1: Postbus 123 zipcode: 7500EA city: Enschede country: NL send_method: post contact_hash: 2 Reference to unique identifier in companies table
  58. 58. We don’t test controllers and views.
  59. 59. Because, • Controllers are just plain CRUD actions • Views just contain HTML • And both are tested with integration testing in Cucumber
  60. 60. Integration testing
  61. 61. The goal • Test the full stack: models, views and controllers • Behaviour driven approach
  62. 62. Our story
  63. 63. So we started testing
  64. 64. Feature: Homepage Visitors at the homepage should get clear information about our product and be able to create an account Scenario: Create a new free account When I visit the homepage And I follow "pricing" And I follow "Signup" And I fill in the following: | Company name | Test company | | Your fullname | Edwin Vlieg | | E-mail | test@test.com | | company_domain | testcompany | | Username | Edwin | | Password | testtest | | Password confirmation | testtest | And I press "Create your account" Then a company with name "Test company" should exist And an e-mail with subject "Welcome to MoneyBird" should have been sent And I should see "Thanks for signing up!"
  65. 65. Cucumber stories • Each line is parsed and matched on a step • Step contains actual execution of code • 3 types of steps: Given, When and Then.
  66. 66. homepage.feature When I follow "Signup" webrat_steps.rb When /^I follow "([^"]*)"$/ do |link| click_link(link) end
  67. 67. Reuse step definitions In MoneyBird: 40 step definitions 695 steps executed
  68. 68. 40 steps • 21 Webrat steps (Cucumber default) •4 Common steps (DB & Mailer) • 15 Domain specific steps
  69. 69. Webrat • Webrat gives you control over the user interface (almost) like an end user has • But doesn’t emulate a browser like Selenium or Watir (= no JavaScript)
  70. 70. Web-what? Web browser Apache Mongrel / Passenger Webrat ActionController
  71. 71. When I follow "Signup" 1. Locate the link on the page <a href="/signup">Signup</a> <a href="/signup" title="Signup"><img src="..." /></a> <a href="/signup" id="Signup" title="Click to signup"><img src="..."></a>
  72. 72. When I follow "Signup" 1. Locate the link on the page <a href="/signup">Signup</a> <a href="/signup" title="Signup"><img src="..." /></a> <a href="/signup" id="Signup" title="Click to signup"><img src="..."></a>
  73. 73. When I follow "Signup" 1. Locate the link on the page <a href="/signup">Signup</a> <a href="/signup" title="Signup"><img src="..." /></a> <a href="/signup" id="Signup" title="Click to signup"><img src="..."></a> 2. ‘Click’ the link Grab the ‘href’ attribute from the element and feed it to the Rails application. HTML is replaced: next search of element will be in new page.
  74. 74. Cucumber & Webrat Cucumber comes with default Webrat steps to: • Visit pages • Click links • Fill in forms • Submit forms • Assert the content of the page
  75. 75. Testability Webrat doesn’t execute JavaScript, so write unobtrusive JavaScript to keep it testable.
  76. 76. Testability Webrat doesn’t execute JavaScript, so write unobtrusive JavaScript to keep it testable. HTML <a href="/popup.html" id="open">Open popup</a> Link opens popup, even without JavaScript
  77. 77. Testability Webrat doesn’t execute JavaScript, so write unobtrusive JavaScript to keep it testable. HTML <a href="/popup.html" id="open">Open popup</a> Link opens popup, even without JavaScript JavaScript $('open').click(...); JavaScript adds extra behaviour to link
  78. 78. Testability Webrat doesn’t execute JavaScript, so write unobtrusive JavaScript to keep it testable. HTML <a href="/popup.html" id="open">Open popup</a> Link opens popup, even without JavaScript JavaScript $('open').click(...); Rails 3 JavaScript adds extra behaviour to link
  79. 79. Common steps Database steps: Given I've created an invoice with invoice id "2008-0100" Then a contact with name "Test company" should exist Mailer steps: Then an e-mail with subject "Invoice 2008-0100 from Bluetools" should have been sent Full source: http://pastie.org/667777
  80. 80. Database steps Given I've created an invoice with invoice id "2008-0100" Should create a new invoice with invoice id “2008-0100”, but what about the rest of the attributes?
  81. 81. FactoryGirl • Makes it easy to create records in the database • Define default attributes for a record: Factory.define :contact do |c| c.company_id { |c| Company.find_by_domain($subdomain).id } c.name "Test company" c.contact_name "Edwin Vlieg" c.address1 "Hengelosestraat 538" c.zipcode "7500AG" c.city "Enschede" c.country "NLD" c.email "test@moneybird.nl" end
  82. 82. FactoryGirl • Easy instantiation of records in database: Factory(:contact, :name => "Foobar") Factory name Override default attributes More information: http://github.com/thoughtbot/factory_girl
  83. 83. Downside When I follow "Signup" What if we want to change “Signup” to “Create account”? Test breaks, but application is functionally correct.
  84. 84. Conclusions
  85. 85. Testing takes time, so be critical about what to test
  86. 86. Don’t stare blindly at TDD or BDD Creating mesmerizing products needs creativity, this doesn’t always fit into the ‘test-first’ approach, but know the ideas behind the approaches, so you can use them when needed!
  87. 87. Creating a good test environment takes time: it just doesn’t end with RSpec or Cucumber Mocks, stubs, fixtures and factories are your friends!
  88. 88. Write testable code Keep the units small for easy unit testing Keep the HTML clean for webrat
  89. 89. Thanks!

×