Advanced Testing
  RubyEnRails Conference ’09
Goals

• Share our experiences about testing
• Make you start thinking critical about testing
  like an advanced tester
• ...
Student Master of
               Computer Science



@edwin_v   Edwin Vlieg
                            Course: Testing
  ...
Student Master of
               Computer Science



@edwin_v   Edwin Vlieg
                            Course: Testing
  ...
• Web application written in Ruby on Rails
• Create and send invoices online
• More than 16.000 invoices sent
• Short deve...
“A donkey does not knock
himself to the same stone twice”
It is okay to write code that doesn’t work...

         But never ‘cap deploy’ it!
So we started testing
Unit testing
Software testing
... is a technical process performed by executing
a product in a controlled environment, following
  a sp...
Unit testing

• Test individual units of code
• A unit is the smallest testable piece of
  code: methods of classes
• E.g....
TDD / BDD

Test before you write a single piece of code
TDD / BDD

Test before you write a single piece of code

Can be a nice approach, but requires a lot of
  discipline and mi...
Example
                 Recurring invoices


Calculate the next occurence of a recurring invoice.



  Start date        ...
First version


def next_occurence(today)
  today + 1.month
end
RSpec-t

describe "Date calculation" do
  it "should return the next month" do
    next_occurence(Date.parse('2009-01-01')...
Done! It is green.
Customer: “I want to bill every
         last day of the month”



‘2009-01-31’ + 1.month = ‘2009-02-28’
‘2009-02-28’ + 1....
Boundary Value Analysis
Find the boundaries of the input and test them
Boundary Value Analysis
Find the boundaries of the input and test them

        Start       Current       Next
       2009...
Boundary Value Analysis
Find the boundaries of the input and test them

        Start       Current       Next
       2009...
Boundary Value Analysis
Find the boundaries of the input and test them

        Start       Current       Next
       2009...
Boundary Value Analysis
Find the boundaries of the input and test them

        Start       Current       Next
       2009...
Create tests for all boundary values...

      ...and improve your code!
The solution

def self.calculate_next_occurence(start_date, current_date)
  next_date = current_date + 1.month
  while sta...
“Testing shows the presence,
  not the absence of bugs”
                        - Dijkstra
So, why test?
So, why test?



     When working with dates, it takes a while
      before the Boundary Values occure.
Isolation
Isolate for productivity

• Test isolated units of code, assuring they are working
• No need to bother about it later
• No...
Isolate your tests

• Don’t test code that is tested elsewhere
• Only make sure it is used well in the code you’re testing...
Mock objects are simulated objects that mimic
the behavior of real objects in controlled ways




  A stub is a piece of c...
Very hard
Method A   Method B
                        work
Method B   Very hard
                        work




Method A
Method B   Very hard
                        work




Method A    Mock
Method B   Very hard
                           work




Method A        Stub




       Mock
Method B          Very hard
                                  work




Method A




       Mock              Stub
Mock   Stub
How to isolate?
def create_invoice(contact)
  invoice = Invoice.new
  invoice.contact_id = contact.id
  invoice.details_at...
How to isolate?
def create_invoice(contact)
  invoice = Invoice.new
  invoice.contact_id = contact.id
  invoice.details_at...
describe "MoneyBird API" do
  it "should create the invoice" do




    create_invoice(contact_mock).should be_true
  end
...
describe "MoneyBird API" do
  it "should create the invoice" do




   contact_mock = mock(:contact)
   contact_mock.shoul...
describe "MoneyBird API" do
  it "should create the invoice" do




   Invoice.should_receive(:new).and_return(invoice_moc...
describe "MoneyBird API" do
  it "should create the invoice" do
   invoice_mock = mock(:invoice)
   invoice_mock.stub!(:co...
Isolation

Isolated    A    B
Isolation

Isolated        A       B




Not isolated    A   B
Isolation

Isolated        A       B




Not isolated    A   B
                                 Double
                   ...
What to test?
          Everything!


But never too much!! Be critical
Downside of testing

• Every test gives overhead
• Testing the obvious takes time
• Ruby library, CRUD controllers
Downside of testing

 • Every test gives overhead
 • Testing the obvious takes time
 • Ruby library, CRUD controllers
But ...
We use (very) fat models




                           So we unit test them
RSpec

• All business logic in models
• Models tested with RSpec
• Basic data model in fixtures
Fixtures?
• Models make use of the database
• Sometimes data in de database is needed
  for the model to work
• Fixtures a...
first_contact:
  company: bluetools
  name: Edwins company
  contact_name: Edwin Vlieg
  address1: Street 82
  zipcode: 75...
first_contact:
  company: bluetools
                                Unique identifier for
  name: Edwins company          ‘...
first_contact:
  company: bluetools
                                Unique identifier for
  name: Edwins company          ‘...
first_contact:
  company: bluetools
                                Unique identifier for
  name: Edwins company          ‘...
We don’t test controllers and views.
Because,

• Controllers are just plain CRUD actions
• Views just contain HTML
• And both are tested with integration
  tes...
Integration testing
The goal
• Test the full stack: models, views and
  controllers
• Behaviour driven approach
Our story
So we started testing
Feature: Homepage
    Visitors at the homepage should get clear information about
    our product and be able to create an...
Cucumber stories

• Each line is parsed and matched on a step
• Step contains actual execution of code
• 3 types of steps:...
homepage.feature

        When I follow "Signup"




             webrat_steps.rb

When /^I follow "([^"]*)"$/ do |link|
 ...
Reuse step definitions




                       In MoneyBird:

              40 step definitions

      695 steps executed
40 steps

• 21   Webrat steps (Cucumber default)
•4     Common steps (DB & Mailer)
• 15   Domain specific steps
Webrat

• Webrat gives you control over the user
  interface (almost) like an end user has
• But doesn’t emulate a browser...
Web-what?
  Web browser



     Apache



Mongrel / Passenger              Webrat



              ActionController
When I follow "Signup"

1. Locate the link on the page
<a href="/signup">Signup</a>

<a href="/signup" title="Signup"><img...
When I follow "Signup"

1. Locate the link on the page
<a href="/signup">Signup</a>

<a href="/signup" title="Signup"><img...
When I follow "Signup"

1. Locate the link on the page
<a href="/signup">Signup</a>

<a href="/signup" title="Signup"><img...
Cucumber & Webrat
Cucumber comes with default Webrat steps to:
• Visit pages
• Click links
• Fill in forms
• Submit forms
...
Testability
Webrat doesn’t execute JavaScript, so write
unobtrusive JavaScript to keep it testable.
Testability
  Webrat doesn’t execute JavaScript, so write
  unobtrusive JavaScript to keep it testable.

                 ...
Testability
  Webrat doesn’t execute JavaScript, so write
  unobtrusive JavaScript to keep it testable.

                 ...
Testability
          Webrat doesn’t execute JavaScript, so write
          unobtrusive JavaScript to keep it testable.

 ...
Common steps
Database steps:
Given I've created an invoice with invoice id "2008-0100"

Then a contact with name "Test com...
Database steps

 Given I've created an invoice with invoice id "2008-0100"




     Should create a new invoice with invoi...
FactoryGirl
     • Makes it easy to create records in the
         database
     • Define default attributes for a record:
...
FactoryGirl
• Easy instantiation of records in database:
     Factory(:contact, :name => "Foobar")



    Factory name    ...
Downside
           When I follow "Signup"



       What if we want to change “Signup”
              to “Create account”?...
Conclusions
Testing takes time,
so be critical about what to test
Don’t stare blindly at TDD or BDD


 Creating mesmerizing products needs creativity,
this doesn’t always fit into the ‘test...
Creating a good test environment takes time:
   it just doesn’t end with RSpec or Cucumber



Mocks, stubs, fixtures and fa...
Write testable code



Keep the units small for easy unit testing



   Keep the HTML clean for webrat
Thanks!
Advanced Testing on RubyEnRails '09
Advanced Testing on RubyEnRails '09
Advanced Testing on RubyEnRails '09
Upcoming SlideShare
Loading in...5
×

Advanced Testing on RubyEnRails '09

1,665

Published on

Presentation during RubyEnRails2009 in Amsterdam about Advanced Testing. Goal of the presentation was to share the experiences we have at MoneyBird about testing and to make people start thinking about testing themselves. Unit testing with RSpec and integration testing with Cucumber are discussed.

Published in: Technology
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
1,665
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
0
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

Transcript of "Advanced Testing on RubyEnRails '09"

  1. 1. Advanced Testing RubyEnRails Conference ’09
  2. 2. Goals • Share our experiences about testing • Make you start thinking critical about testing like an advanced tester • Unit testing with RSpec • Integration testing with Cucumber
  3. 3. Student Master of Computer Science @edwin_v Edwin Vlieg Course: Testing Techniques
  4. 4. Student Master of Computer Science @edwin_v Edwin Vlieg Course: Testing 5 Techniques
  5. 5. • Web application written in Ruby on Rails • Create and send invoices online • More than 16.000 invoices sent • Short development cycle: 2 months • API for web developers • More info: www.moneybird.com
  6. 6. “A donkey does not knock himself to the same stone twice”
  7. 7. It is okay to write code that doesn’t work... But never ‘cap deploy’ it!
  8. 8. So we started testing
  9. 9. Unit testing
  10. 10. Software testing ... is a technical process performed by executing a product in a controlled environment, following a specified procedure, with the intent of measuring the quality of the software product by demonstrating deviations of the requirements.
  11. 11. Unit testing • Test individual units of code • A unit is the smallest testable piece of code: methods of classes • E.g. methods of controllers and models
  12. 12. TDD / BDD Test before you write a single piece of code
  13. 13. TDD / BDD Test before you write a single piece of code Can be a nice approach, but requires a lot of discipline and might not fit your needs, but know how it can help you!
  14. 14. Example Recurring invoices Calculate the next occurence of a recurring invoice. Start date Current occurence Next occurence Jan Febr Mar Apr May
  15. 15. First version def next_occurence(today) today + 1.month end
  16. 16. RSpec-t describe "Date calculation" do it "should return the next month" do next_occurence(Date.parse('2009-01-01')).should == Date.parse('2009-02-01') end end You ain’t testing this, because it looks too easy
  17. 17. Done! It is green.
  18. 18. Customer: “I want to bill every last day of the month” ‘2009-01-31’ + 1.month = ‘2009-02-28’ ‘2009-02-28’ + 1.month = ‘2009-03-28’ ‘2009-01-31’ + 2.months = ‘2009-03-31’
  19. 19. Boundary Value Analysis Find the boundaries of the input and test them
  20. 20. Boundary Value Analysis Find the boundaries of the input and test them Start Current Next 2009-01-31 2009-01-31 2009-02-28 2009-01-31 2009-02-28 2009-03-31 2009-01-31 2009-03-31 2009-04-30 2009-01-30 2009-01-30 2009-02-28 2009-01-30 2009-02-28 2009-03-30 2009-01-30 2009-03-30 2009-04-30 2009-01-29 2009-01-29 2009-02-28 2009-01-29 2009-02-28 2009-03-29 2009-01-29 2009-03-29 2009-04-29 2009-01-28 2009-01-28 2009-02-28 2009-01-28 2009-02-28 2009-03-28 2009-01-28 2009-03-28 2009-04-28
  21. 21. Boundary Value Analysis Find the boundaries of the input and test them Start Current Next 2009-01-31 2009-01-31 2009-02-28 2009-01-31 2009-02-28 2009-03-31 2009-01-31 2009-03-31 2009-04-30 2009-01-30 2009-01-30 2009-02-28 2009-01-30 2009-02-28 2009-03-30 2009-01-30 2009-03-30 2009-04-30 2009-01-29 2009-01-29 2009-02-28 2009-01-29 2009-02-28 2009-03-29 2009-01-29 2009-03-29 2009-04-29 2009-01-28 2009-01-28 2009-02-28 2009-01-28 2009-02-28 2009-03-28 2009-01-28 2009-03-28 2009-04-28
  22. 22. Boundary Value Analysis Find the boundaries of the input and test them Start Current Next 2009-01-31 2009-01-31 2009-02-28 2009-01-31 2009-02-28 2009-03-31 2009-01-31 2009-03-31 2009-04-30 2009-01-30 2009-01-30 2009-02-28 2009-01-30 2009-02-28 2009-03-30 2009-01-30 2009-03-30 2009-04-30 2009-01-29 2009-01-29 2009-02-28 2009-01-29 2009-02-28 2009-03-29 2009-01-29 2009-03-29 2009-04-29 2009-01-28 2009-01-28 2009-02-28 2009-01-28 2009-02-28 2009-03-28 2009-01-28 2009-03-28 2009-04-28
  23. 23. Boundary Value Analysis Find the boundaries of the input and test them Start Current Next 2009-01-31 2009-01-31 2009-02-28 2009-01-31 2009-02-28 2009-03-31 2009-01-31 2009-03-31 2009-04-30 2009-01-30 2009-01-30 2009-02-28 2009-01-30 2009-02-28 2009-03-30 2009-01-30 2009-03-30 2009-04-30 2009-01-29 2009-01-29 2009-02-28 2009-01-29 2009-02-28 2009-03-29 2009-01-29 2009-03-29 2009-04-29 2009-01-28 2009-01-28 2009-02-28 2009-01-28 2009-02-28 2009-03-28 2009-01-28 2009-03-28 2009-04-28
  24. 24. Create tests for all boundary values... ...and improve your code!
  25. 25. The solution def self.calculate_next_occurence(start_date, current_date) next_date = current_date + 1.month while start_date.day != next_date.day and next_date != next_date.end_of_month next_date = next_date.tomorrow end next_date end
  26. 26. “Testing shows the presence, not the absence of bugs” - Dijkstra
  27. 27. So, why test?
  28. 28. So, why test? When working with dates, it takes a while before the Boundary Values occure.
  29. 29. Isolation
  30. 30. Isolate for productivity • Test isolated units of code, assuring they are working • No need to bother about it later • No need to find all edge cases in the user interface: F5 syndrom
  31. 31. Isolate your tests • Don’t test code that is tested elsewhere • Only make sure it is used well in the code you’re testing • The solution: mocks and stubs
  32. 32. Mock objects are simulated objects that mimic the behavior of real objects in controlled ways A stub is a piece of code used to stand in for some other programming functionality
  33. 33. Very hard Method A Method B work
  34. 34. Method B Very hard work Method A
  35. 35. Method B Very hard work Method A Mock
  36. 36. Method B Very hard work Method A Stub Mock
  37. 37. Method B Very hard work Method A Mock Stub
  38. 38. Mock Stub
  39. 39. How to isolate? def create_invoice(contact) invoice = Invoice.new invoice.contact_id = contact.id invoice.details_attributes = [{ :description => "RER09", :price => 79 }] invoice.save end class Invoice < ActiveResource::Base self.site = "https://account.moneybird.com" end
  40. 40. How to isolate? def create_invoice(contact) invoice = Invoice.new invoice.contact_id = contact.id invoice.details_attributes = [{ :description => "RER09", :price => 79 }] invoice.save end class Invoice < ActiveResource::Base self.site = "https://account.moneybird.com" end ActiveResource is tested elsewhere + very slow to test
  41. 41. describe "MoneyBird API" do it "should create the invoice" do create_invoice(contact_mock).should be_true end end
  42. 42. describe "MoneyBird API" do it "should create the invoice" do contact_mock = mock(:contact) contact_mock.should_receive(:id).and_return(3) create_invoice(contact_mock).should be_true end end
  43. 43. describe "MoneyBird API" do it "should create the invoice" do Invoice.should_receive(:new).and_return(invoice_mock) contact_mock = mock(:contact) contact_mock.should_receive(:id).and_return(3) create_invoice(contact_mock).should be_true end end
  44. 44. describe "MoneyBird API" do it "should create the invoice" do invoice_mock = mock(:invoice) invoice_mock.stub!(:contact_id=) invoice_mock.stub!(:details_attributes=) invoice_mock.should_receive(:save).and_return(true) Invoice.should_receive(:new).and_return(invoice_mock) contact_mock = mock(:contact) contact_mock.should_receive(:id).and_return(3) create_invoice(contact_mock).should be_true end end
  45. 45. Isolation Isolated A B
  46. 46. Isolation Isolated A B Not isolated A B
  47. 47. Isolation Isolated A B Not isolated A B Double = Extra test work + maintenance work!
  48. 48. What to test? Everything! But never too much!! Be critical
  49. 49. Downside of testing • Every test gives overhead • Testing the obvious takes time • Ruby library, CRUD controllers
  50. 50. Downside of testing • Every test gives overhead • Testing the obvious takes time • Ruby library, CRUD controllers But what if suddenly: BigDecimal("10.03").to_f != 10.03
  51. 51. We use (very) fat models So we unit test them
  52. 52. RSpec • All business logic in models • Models tested with RSpec • Basic data model in fixtures
  53. 53. Fixtures? • Models make use of the database • Sometimes data in de database is needed for the model to work • Fixtures are easy to manage on the filesystem (+ version control) • Fixtures can be used for bootstrapping application: rake db:fixtures:load
  54. 54. first_contact: company: bluetools name: Edwins company contact_name: Edwin Vlieg address1: Street 82 zipcode: 7541 XA city: Enschede country: NL send_method: mail created_at: <%= 60.days.ago.to_s :db %> contact_hash: 1 second_contact: company: bluetools name: BlueTools B.V. address1: Postbus 123 zipcode: 7500EA city: Enschede country: NL send_method: post contact_hash: 2
  55. 55. first_contact: company: bluetools Unique identifier for name: Edwins company ‘row’ in database. contact_name: Edwin Vlieg Don’t set the id column!! address1: Street 82 zipcode: 7541 XA city: Enschede country: NL send_method: mail created_at: <%= 60.days.ago.to_s :db %> contact_hash: 1 second_contact: company: bluetools name: BlueTools B.V. address1: Postbus 123 zipcode: 7500EA city: Enschede country: NL send_method: post contact_hash: 2
  56. 56. first_contact: company: bluetools Unique identifier for name: Edwins company ‘row’ in database. contact_name: Edwin Vlieg Don’t set the id column!! address1: Street 82 zipcode: 7541 XA city: Enschede country: NL send_method: mail created_at: <%= 60.days.ago.to_s :db %> contact_hash: 1 second_contact: company: bluetools name: BlueTools B.V. Yes, you can use Ruby! address1: Postbus 123 zipcode: 7500EA city: Enschede country: NL send_method: post contact_hash: 2
  57. 57. first_contact: company: bluetools Unique identifier for name: Edwins company ‘row’ in database. contact_name: Edwin Vlieg Don’t set the id column!! address1: Street 82 zipcode: 7541 XA city: Enschede country: NL send_method: mail created_at: <%= 60.days.ago.to_s :db %> contact_hash: 1 second_contact: company: bluetools name: BlueTools B.V. Yes, you can use Ruby! address1: Postbus 123 zipcode: 7500EA city: Enschede country: NL send_method: post contact_hash: 2 Reference to unique identifier in companies table
  58. 58. We don’t test controllers and views.
  59. 59. Because, • Controllers are just plain CRUD actions • Views just contain HTML • And both are tested with integration testing in Cucumber
  60. 60. Integration testing
  61. 61. The goal • Test the full stack: models, views and controllers • Behaviour driven approach
  62. 62. Our story
  63. 63. So we started testing
  64. 64. Feature: Homepage Visitors at the homepage should get clear information about our product and be able to create an account Scenario: Create a new free account When I visit the homepage And I follow "pricing" And I follow "Signup" And I fill in the following: | Company name | Test company | | Your fullname | Edwin Vlieg | | E-mail | test@test.com | | company_domain | testcompany | | Username | Edwin | | Password | testtest | | Password confirmation | testtest | And I press "Create your account" Then a company with name "Test company" should exist And an e-mail with subject "Welcome to MoneyBird" should have been sent And I should see "Thanks for signing up!"
  65. 65. Cucumber stories • Each line is parsed and matched on a step • Step contains actual execution of code • 3 types of steps: Given, When and Then.
  66. 66. homepage.feature When I follow "Signup" webrat_steps.rb When /^I follow "([^"]*)"$/ do |link| click_link(link) end
  67. 67. Reuse step definitions In MoneyBird: 40 step definitions 695 steps executed
  68. 68. 40 steps • 21 Webrat steps (Cucumber default) •4 Common steps (DB & Mailer) • 15 Domain specific steps
  69. 69. Webrat • Webrat gives you control over the user interface (almost) like an end user has • But doesn’t emulate a browser like Selenium or Watir (= no JavaScript)
  70. 70. Web-what? Web browser Apache Mongrel / Passenger Webrat ActionController
  71. 71. When I follow "Signup" 1. Locate the link on the page <a href="/signup">Signup</a> <a href="/signup" title="Signup"><img src="..." /></a> <a href="/signup" id="Signup" title="Click to signup"><img src="..."></a>
  72. 72. When I follow "Signup" 1. Locate the link on the page <a href="/signup">Signup</a> <a href="/signup" title="Signup"><img src="..." /></a> <a href="/signup" id="Signup" title="Click to signup"><img src="..."></a>
  73. 73. When I follow "Signup" 1. Locate the link on the page <a href="/signup">Signup</a> <a href="/signup" title="Signup"><img src="..." /></a> <a href="/signup" id="Signup" title="Click to signup"><img src="..."></a> 2. ‘Click’ the link Grab the ‘href’ attribute from the element and feed it to the Rails application. HTML is replaced: next search of element will be in new page.
  74. 74. Cucumber & Webrat Cucumber comes with default Webrat steps to: • Visit pages • Click links • Fill in forms • Submit forms • Assert the content of the page
  75. 75. Testability Webrat doesn’t execute JavaScript, so write unobtrusive JavaScript to keep it testable.
  76. 76. Testability Webrat doesn’t execute JavaScript, so write unobtrusive JavaScript to keep it testable. HTML <a href="/popup.html" id="open">Open popup</a> Link opens popup, even without JavaScript
  77. 77. Testability Webrat doesn’t execute JavaScript, so write unobtrusive JavaScript to keep it testable. HTML <a href="/popup.html" id="open">Open popup</a> Link opens popup, even without JavaScript JavaScript $('open').click(...); JavaScript adds extra behaviour to link
  78. 78. Testability Webrat doesn’t execute JavaScript, so write unobtrusive JavaScript to keep it testable. HTML <a href="/popup.html" id="open">Open popup</a> Link opens popup, even without JavaScript JavaScript $('open').click(...); Rails 3 JavaScript adds extra behaviour to link
  79. 79. Common steps Database steps: Given I've created an invoice with invoice id "2008-0100" Then a contact with name "Test company" should exist Mailer steps: Then an e-mail with subject "Invoice 2008-0100 from Bluetools" should have been sent Full source: http://pastie.org/667777
  80. 80. Database steps Given I've created an invoice with invoice id "2008-0100" Should create a new invoice with invoice id “2008-0100”, but what about the rest of the attributes?
  81. 81. FactoryGirl • Makes it easy to create records in the database • Define default attributes for a record: Factory.define :contact do |c| c.company_id { |c| Company.find_by_domain($subdomain).id } c.name "Test company" c.contact_name "Edwin Vlieg" c.address1 "Hengelosestraat 538" c.zipcode "7500AG" c.city "Enschede" c.country "NLD" c.email "test@moneybird.nl" end
  82. 82. FactoryGirl • Easy instantiation of records in database: Factory(:contact, :name => "Foobar") Factory name Override default attributes More information: http://github.com/thoughtbot/factory_girl
  83. 83. Downside When I follow "Signup" What if we want to change “Signup” to “Create account”? Test breaks, but application is functionally correct.
  84. 84. Conclusions
  85. 85. Testing takes time, so be critical about what to test
  86. 86. Don’t stare blindly at TDD or BDD Creating mesmerizing products needs creativity, this doesn’t always fit into the ‘test-first’ approach, but know the ideas behind the approaches, so you can use them when needed!
  87. 87. Creating a good test environment takes time: it just doesn’t end with RSpec or Cucumber Mocks, stubs, fixtures and factories are your friends!
  88. 88. Write testable code Keep the units small for easy unit testing Keep the HTML clean for webrat
  89. 89. Thanks!

×