Advanced Testing on RubyEnRails '09

Advanced Testing RubyEnRails Conference ’09

Goals • Share our experiences about testing • Make you start thinking critical about testing like an advanced tester • Unit testing with RSpec • Integration testing with Cucumber

Student Master of Computer Science @edwin_v Edwin Vlieg Course: Testing Techniques

Student Master of Computer Science @edwin_v Edwin Vlieg Course: Testing 5 Techniques

• Web application written in Ruby on Rails • Create and send invoices online • More than 16.000 invoices sent • Short development cycle: 2 months • API for web developers • More info: www.moneybird.com

“A donkey does not knock himself to the same stone twice”

It is okay to write code that doesn’t work... But never ‘cap deploy’ it!

So we started testing

Unit testing

Software testing ... is a technical process performed by executing a product in a controlled environment, following a speciﬁed procedure, with the intent of measuring the quality of the software product by demonstrating deviations of the requirements.

Unit testing • Test individual units of code • A unit is the smallest testable piece of code: methods of classes • E.g. methods of controllers and models

TDD / BDD Test before you write a single piece of code

TDD / BDD Test before you write a single piece of code Can be a nice approach, but requires a lot of discipline and might not ﬁt your needs, but know how it can help you!

Example Recurring invoices Calculate the next occurence of a recurring invoice. Start date Current occurence Next occurence Jan Febr Mar Apr May

First version def next_occurence(today) today + 1.month end

RSpec-t describe "Date calculation" do it "should return the next month" do next_occurence(Date.parse('2009-01-01')).should == Date.parse('2009-02-01') end end You ain’t testing this, because it looks too easy

Done! It is green.

Customer: “I want to bill every last day of the month” ‘2009-01-31’ + 1.month = ‘2009-02-28’ ‘2009-02-28’ + 1.month = ‘2009-03-28’ ‘2009-01-31’ + 2.months = ‘2009-03-31’

Boundary Value Analysis Find the boundaries of the input and test them

Boundary Value Analysis Find the boundaries of the input and test them Start Current Next 2009-01-31 2009-01-31 2009-02-28 2009-01-31 2009-02-28 2009-03-31 2009-01-31 2009-03-31 2009-04-30 2009-01-30 2009-01-30 2009-02-28 2009-01-30 2009-02-28 2009-03-30 2009-01-30 2009-03-30 2009-04-30 2009-01-29 2009-01-29 2009-02-28 2009-01-29 2009-02-28 2009-03-29 2009-01-29 2009-03-29 2009-04-29 2009-01-28 2009-01-28 2009-02-28 2009-01-28 2009-02-28 2009-03-28 2009-01-28 2009-03-28 2009-04-28

Create tests for all boundary values... ...and improve your code!

The solution def self.calculate_next_occurence(start_date, current_date) next_date = current_date + 1.month while start_date.day != next_date.day and next_date != next_date.end_of_month next_date = next_date.tomorrow end next_date end

“Testing shows the presence, not the absence of bugs” - Dijkstra

So, why test?

So, why test? When working with dates, it takes a while before the Boundary Values occure.

Isolation

Isolate for productivity • Test isolated units of code, assuring they are working • No need to bother about it later • No need to ﬁnd all edge cases in the user interface: F5 syndrom

Isolate your tests • Don’t test code that is tested elsewhere • Only make sure it is used well in the code you’re testing • The solution: mocks and stubs

Mock objects are simulated objects that mimic the behavior of real objects in controlled ways A stub is a piece of code used to stand in for some other programming functionality

Very hard Method A Method B work

Method B Very hard work Method A

Method B Very hard work Method A Mock

Method B Very hard work Method A Stub Mock

Method B Very hard work Method A Mock Stub

Mock Stub

How to isolate? def create_invoice(contact) invoice = Invoice.new invoice.contact_id = contact.id invoice.details_attributes = [{ :description => "RER09", :price => 79 }] invoice.save end class Invoice < ActiveResource::Base self.site = "https://account.moneybird.com" end

How to isolate? def create_invoice(contact) invoice = Invoice.new invoice.contact_id = contact.id invoice.details_attributes = [{ :description => "RER09", :price => 79 }] invoice.save end class Invoice < ActiveResource::Base self.site = "https://account.moneybird.com" end ActiveResource is tested elsewhere + very slow to test

describe "MoneyBird API" do it "should create the invoice" do create_invoice(contact_mock).should be_true end end

describe "MoneyBird API" do it "should create the invoice" do contact_mock = mock(:contact) contact_mock.should_receive(:id).and_return(3) create_invoice(contact_mock).should be_true end end

describe "MoneyBird API" do it "should create the invoice" do Invoice.should_receive(:new).and_return(invoice_mock) contact_mock = mock(:contact) contact_mock.should_receive(:id).and_return(3) create_invoice(contact_mock).should be_true end end

describe "MoneyBird API" do it "should create the invoice" do invoice_mock = mock(:invoice) invoice_mock.stub!(:contact_id=) invoice_mock.stub!(:details_attributes=) invoice_mock.should_receive(:save).and_return(true) Invoice.should_receive(:new).and_return(invoice_mock) contact_mock = mock(:contact) contact_mock.should_receive(:id).and_return(3) create_invoice(contact_mock).should be_true end end

Isolation Isolated A B

Isolation Isolated A B Not isolated A B

Isolation Isolated A B Not isolated A B Double = Extra test work + maintenance work!

What to test? Everything! But never too much!! Be critical

Downside of testing • Every test gives overhead • Testing the obvious takes time • Ruby library, CRUD controllers

Downside of testing • Every test gives overhead • Testing the obvious takes time • Ruby library, CRUD controllers But what if suddenly: BigDecimal("10.03").to_f != 10.03

We use (very) fat models So we unit test them

RSpec • All business logic in models • Models tested with RSpec • Basic data model in ﬁxtures

Fixtures? • Models make use of the database • Sometimes data in de database is needed for the model to work • Fixtures are easy to manage on the ﬁlesystem (+ version control) • Fixtures can be used for bootstrapping application: rake db:ﬁxtures:load

first_contact: company: bluetools name: Edwins company contact_name: Edwin Vlieg address1: Street 82 zipcode: 7541 XA city: Enschede country: NL send_method: mail created_at: <%= 60.days.ago.to_s :db %> contact_hash: 1 second_contact: company: bluetools name: BlueTools B.V. address1: Postbus 123 zipcode: 7500EA city: Enschede country: NL send_method: post contact_hash: 2

first_contact: company: bluetools Unique identiﬁer for name: Edwins company ‘row’ in database. contact_name: Edwin Vlieg Don’t set the id column!! address1: Street 82 zipcode: 7541 XA city: Enschede country: NL send_method: mail created_at: <%= 60.days.ago.to_s :db %> contact_hash: 1 second_contact: company: bluetools name: BlueTools B.V. address1: Postbus 123 zipcode: 7500EA city: Enschede country: NL send_method: post contact_hash: 2

first_contact: company: bluetools Unique identiﬁer for name: Edwins company ‘row’ in database. contact_name: Edwin Vlieg Don’t set the id column!! address1: Street 82 zipcode: 7541 XA city: Enschede country: NL send_method: mail created_at: <%= 60.days.ago.to_s :db %> contact_hash: 1 second_contact: company: bluetools name: BlueTools B.V. Yes, you can use Ruby! address1: Postbus 123 zipcode: 7500EA city: Enschede country: NL send_method: post contact_hash: 2

first_contact: company: bluetools Unique identiﬁer for name: Edwins company ‘row’ in database. contact_name: Edwin Vlieg Don’t set the id column!! address1: Street 82 zipcode: 7541 XA city: Enschede country: NL send_method: mail created_at: <%= 60.days.ago.to_s :db %> contact_hash: 1 second_contact: company: bluetools name: BlueTools B.V. Yes, you can use Ruby! address1: Postbus 123 zipcode: 7500EA city: Enschede country: NL send_method: post contact_hash: 2 Reference to unique identiﬁer in companies table

We don’t test controllers and views.

Because, • Controllers are just plain CRUD actions • Views just contain HTML • And both are tested with integration testing in Cucumber

Integration testing

The goal • Test the full stack: models, views and controllers • Behaviour driven approach

Our story

So we started testing

Cucumber stories • Each line is parsed and matched on a step • Step contains actual execution of code • 3 types of steps: Given, When and Then.

homepage.feature When I follow "Signup" webrat_steps.rb When /^I follow "([^"]*)"$/ do |link| click_link(link) end

Reuse step deﬁnitions In MoneyBird: 40 step deﬁnitions 695 steps executed

40 steps • 21 Webrat steps (Cucumber default) •4 Common steps (DB & Mailer) • 15 Domain speciﬁc steps

Webrat • Webrat gives you control over the user interface (almost) like an end user has • But doesn’t emulate a browser like Selenium or Watir (= no JavaScript)

Web-what? Web browser Apache Mongrel / Passenger Webrat ActionController

When I follow "Signup" 1. Locate the link on the page <a href="/signup">Signup</a> <a href="/signup" title="Signup"><img src="..." /></a> <a href="/signup" id="Signup" title="Click to signup"><img src="..."></a>

When I follow "Signup" 1. Locate the link on the page <a href="/signup">Signup</a> <a href="/signup" title="Signup"><img src="..." /></a> <a href="/signup" id="Signup" title="Click to signup"><img src="..."></a> 2. ‘Click’ the link Grab the ‘href’ attribute from the element and feed it to the Rails application. HTML is replaced: next search of element will be in new page.

Cucumber & Webrat Cucumber comes with default Webrat steps to: • Visit pages • Click links • Fill in forms • Submit forms • Assert the content of the page

Testability Webrat doesn’t execute JavaScript, so write unobtrusive JavaScript to keep it testable.

Testability Webrat doesn’t execute JavaScript, so write unobtrusive JavaScript to keep it testable. HTML <a href="/popup.html" id="open">Open popup</a> Link opens popup, even without JavaScript

Testability Webrat doesn’t execute JavaScript, so write unobtrusive JavaScript to keep it testable. HTML <a href="/popup.html" id="open">Open popup</a> Link opens popup, even without JavaScript JavaScript $('open').click(...); JavaScript adds extra behaviour to link

Testability Webrat doesn’t execute JavaScript, so write unobtrusive JavaScript to keep it testable. HTML <a href="/popup.html" id="open">Open popup</a> Link opens popup, even without JavaScript JavaScript $('open').click(...); Rails 3 JavaScript adds extra behaviour to link

Common steps Database steps: Given I've created an invoice with invoice id "2008-0100" Then a contact with name "Test company" should exist Mailer steps: Then an e-mail with subject "Invoice 2008-0100 from Bluetools" should have been sent Full source: http://pastie.org/667777

Database steps Given I've created an invoice with invoice id "2008-0100" Should create a new invoice with invoice id “2008-0100”, but what about the rest of the attributes?

FactoryGirl • Makes it easy to create records in the database • Deﬁne default attributes for a record: Factory.define :contact do |c| c.company_id { |c| Company.find_by_domain($subdomain).id } c.name "Test company" c.contact_name "Edwin Vlieg" c.address1 "Hengelosestraat 538" c.zipcode "7500AG" c.city "Enschede" c.country "NLD" c.email "test@moneybird.nl" end

FactoryGirl • Easy instantiation of records in database: Factory(:contact, :name => "Foobar") Factory name Override default attributes More information: http://github.com/thoughtbot/factory_girl

Downside When I follow "Signup" What if we want to change “Signup” to “Create account”? Test breaks, but application is functionally correct.

Conclusions

Testing takes time, so be critical about what to test

Don’t stare blindly at TDD or BDD Creating mesmerizing products needs creativity, this doesn’t always ﬁt into the ‘test-ﬁrst’ approach, but know the ideas behind the approaches, so you can use them when needed!

Creating a good test environment takes time: it just doesn’t end with RSpec or Cucumber Mocks, stubs, ﬁxtures and factories are your friends!

Write testable code Keep the units small for easy unit testing Keep the HTML clean for webrat

Thanks!

Advanced Testing on RubyEnRails '09

0 comments

Notes on slide 1

2 Favorites