Automated Testing
with Ruby
Keith Pitty
Open Source Developers’ Conference, Sydney 2008
4th December

Once upon a time...

way back in 1983...

a young programmer made his way
from the world
of Unix at university...

to the world of
IBM mainframe programming.

He learnt to use such
weird and wonderful
technologies as...

PL/I, IMS, CICS, TSO and ISPF/PDF.

Fortunately he had a good mentor
who taught him about
modular programming...

and how to write unit tests with...

Job Control Language

“JCL is the stuff of nightmares for
many programmers and operators.”
From editorial review for “MVS JCL in Plain English”

fast forward to 2001

Our programmer was
not so young any more...

but he had moved on
from mainframe progamming...

to the exciting new world of Java...

and...

as a byproduct of
eXtreme Programming...

he had fallen in love with...

JUnit

He loved using JUnit to do
test-driven development
and refactoring...

and encouraged others to use it
whenever he had the opportunity.

fast forward to 2005

By now our hero had
begun to explore...

and...

He liked the way
tests were “baked in”
and how you could
run them with rake.

Then, one day in 2007,
a fellow Rails programmer
enquired...

“Are you using autotest?”

Sheepishly he admitted
he hadn’t even heard of it...

We’ll leave our story there for now.

Autotest

“a continuous testing facility
meant to be used during development.”

sudo gem install ZenTest
cd path/to/railsapp
autotest

“As soon as you save a file,
autotest will run the
corresponding dependent tests.”

Test::Unit

an implementation of xUnit

the default Ruby testing tool

distributed with Ruby since 2001

also the default testing tool for Rails

assert_kind_of BowlingAnalysis, @bowling_analysis
assert_equal \"Brett Lee\", @bowling_analysis.player.full_name
assert_equal \"9.3\", @bowling_analysis.overs.to_s
assert_equal 1, @bowling_analysis.maidens
assert_equal 2, @bowling_analysis.wickets
assert_equal 50, @bowling_analysis.runs

Rails by default provides
unit, functional and integration tests

RSpec

RSpec allows you to specify behaviour

Specifying a model

describe Player do
before(:each) do
@valid_attributes = {
:first_name => \"value for first_name\",
:last_name => \"value for last_name\",
:active => true
}
end
it \"should create a new instance given valid attributes\" do
Player.create!(@valid_attributes)
end
end

Specifying a controller

describe PlayersController do
# missing code here will be revealed soon
it \"should respond successfully to get 'index' with list of
players\" do
Player.should_receive(:find).and_return(@players)
get 'index'
response.should be_success
assigns[:players].should == @players
end
end

What makes a
good test?

1. Isolated
2. Automated
Kent Beck suggests: 3. Quick to write
4. Quick to run
5. Unique

describe PlayersController do
before(:each) do
@player_1 = mock(Player, {:first_name => \"Greg\",
:last_name => \"Norman\",
:active => true})
@player_2 = mock(Player, {:first_name => \"Adam\",
:last_name => \"Scott\",
:active => true})
@players = [@player_1, @player_2]
end
it \"should respond successfully to get 'index' with list of
players\" do
Player.should_receive(:find).and_return(@players)
get 'index'
response.should be_success
assigns[:players].should == @players
end
end

Mocks

Mocks mimic real objects

Mocks allow tests to be isolated

For more see “Mock Dialogue”
by Jim Weirich and Joe O’Brien

Other Mock
Frameworks

mocha
flexmock
rr

Autospec

Autospec rather than autotest
since RSpec 1.1.5

More on RSpec

Can also specify views
or
integrate views with controller specs

Integration
Testing

Can use Test::Unit
but I prefer...

Cucumber

A replacement for RSpec Story Runner

based on definition of
scenarios within features

Feature: Name of feature
In order to derive some business value
As a user
I want to take certain actions
Scenario: Name of scenario
Given a prerequisite
When I take some specific action
Then I should notice the expected outcome

cucumber
or
AUTOFEATURE=true autospec

can be used in conjunction with tools like
Webrat and Selenium

Webrat

simulates in-browser testing
by leveraging the DOM
without performance hit

Feature: Player belongs to group
In order for a player to be included
As a recorder
I want to record when a player joins or leaves
Scenario: Record when a new player joins
Given a player is not in the system
When I request a list of active players
And I follow \"Add Player\"
And I fill in \"player_first_name\" with \"Greg\"
And I fill in \"player_last_name\" with \"Norman\"
And I press \"Save Player\"
Then I should see \"Active Players\"
And I should see \"Greg Norman\"

makes use of
Cucumber step definitions
for Webrat, for example...

When /^I follow \"(.*)\"$/ do |link|
clicks_link(link)
end
When /^I fill in \"(.*)\" with \"(.*)\"$/ do |field, value|
fills_in(field, :with => value)
end
When /^I press \"(.*)\"$/ do |button|
clicks_button(button)
end
Then /^I should see \"(.*)\"$/ do |text|
response.body.should =~ /#{text}/m
end

Selenium

• runs tests via a browser
• handles JavaScript

a little more effort than
just installing the Selenium gem

Feature: Check OSDC Site
In order to keep track of conference details
A participant
Should be able to browse the OSDC site
Scenario: Check the home page
Given I am on the OSDC home page
Then I should see the text \"Welcome to the
Open Source Developers' Conference\"

require 'spec'
require 'selenium'
Before do
@browser = Selenium::SeleniumDriver.new(\"localhost\",
4444, \"*chrome\", \"http://localhost\", 15000)
@browser.start
end
After do
@browser.stop
end
Given /I am on the OSDC home page/ do
@browser.open 'http://www.osdc.com.au/'
end
Then /I should see the text \"(.*)\"/ do |text|
@browser.is_text_present(text).should == true
end

Fixtures

allow data for automated tests
to be defined using yaml

can be a pain to maintain

Machinist

Blueprints to generate
ActiveRecord objects

Hole.blueprint do
number 1
end
(1..18).each {|n| Hole.make(:number => n) }

Machinist with Sham

Sham.name { Faker::Name.name }
Sham.date do
Date.civil((2005..2009).to_a.rand,
(1..12).to_a.rand, (1..28).to_a.rand)
end
Player.blueprint do
first_name { Sham.name }
last_name { Sham.name }
active true
end
Round.blueprint do
date_played { Sham.date }
end

use blueprints in cucumber steps

now available as a gem on github

Other Tools

JavaScript
Unit Testing

Rails plugins:
javascript_test
javascript_test_autotest

js_autotest

see Dr Nic’s blog post

Shoulda

Extends Test::Unit
with the idea of contexts

Testing
Philosophy

How much effort
is warranted?

Is it taking
too much effort
to learn this
new testing tool?

My Opinion

Automated testing
can be viewed as a trade-off.

E <= ΔQ
Effort invested in automated tests
should result in at least
an equivalent improvement
in software quality.

I think it is worth striving
to continually improve my command
of automated testing tools...

always remembering
the business context.

Quotes

“Producing successful, reliable software
involves mixing and matching
an all-too-variable number
of error removal approaches,
typically the more of them the better.”
Robert L. Glass
Facts and Fallacies of Software Engineering

“Whether you are a hard-core
‘test first’ person is not the point.
The point is that everyone
can benefit from tests that are
automated, easy to write,
and easy to run.”
Hal Fulton
The Ruby Way

References

• Hal Fulton. The Ruby Way. Addison-Wesley, 2007
• Obie Fernandez. The Rails Way. Addison-Wesley,
2008
• David Chelimsky et al. The RSpec Book: Behaviour
Driven Development with Ruby. The Pragmatic
Programmers, 2009
• Robert L. Glass. Facts and Fallacies of Software
Engineering. Addison-Wesley, 2003

• http://zentest.rubyforge.org/ZenTest/
• http://rspec.info
• http://rubyhoedown2008.confreaks.com/02-joe-
obrien-and-jim-weirich-mock-dialogue.html
• http://www.infoq.com/news/2008/10/
qualities_good_test
• http://github.com/aslakhellesoy/cucumber

• http://github.com/brynary/webrat
• http://github.com/aslakhellesoy/cucumber/wikis/
setting-up-selenium
• http://github.com/notahat/machinist
• http://toolmantim.com/article/2008/10/27/
fixtureless_datas_with_machinist_and_sham

• http://drnicwilliams.com/2008/01/04/autotesting-
javascript-in-rails/
• http://www.thoughtbot.com/projects/shoulda
• http://pragdave.blogs.pragprog.com/pragdave/
2008/04/shoulda-used-th.html

Thanks for listening!
http://keithpitty.com