Cross-browser testing in the real world

Cross-‐browser

tes*ng in
the

real
world
Mar*n
Kleppmann rapportive

http://www.ﬂickr.com/photos/stuart_spivack/2322070560/

Cross-‐browser
urgh

tes*ng in
the

real
world
Mar*n

Cross-‐browser

tes*ng in
the
mess
y
real
world
Mar*n

Cross-‐browser t?
a ?
h
w ow
h
tes*ng in
the

real
world
Mar*n

Tes*ng
what?
Unit
tes(ng
Applica(on
End-‐to-‐end
(incl.
external
services)

Tes*ng
why?
Func(onal
Visual
Performance
Load/Scalability
Security
Usability

Tes*ng
how?
Automated
(TDD,
BDD,
regression
tests,
smoke
tests,
...)

Manual
(exploratory,
scripted,
user-‐centered,
...)

Tes*ng
dimensions
Unit Applica(on End-‐to-‐end

Func(onal Visual Performance
Load Security Usability

Automated Manual

Tes*ng
dimensions
Unit Applica(on End-‐to-‐end
RSpec/
Func(onal Visual Performance
Shoulda/
Load Security whatever
Usability

Automated Manual

(Side-‐note)

wri te
a n ’t
Yo u
c cs
R S p e
b i li ty
r
u s a
fo

http://www.ﬂickr.com/photos/71263221@N00/4230334515/

P.S.
We’re
hiring!
http://www.ﬂickr.com/photos/snake-eyes/410092369/

Cucumber/RSpec/whatever

Selenium
2
WebDriver


Capybara

Selenium
2
WebDriver


Capybara

Selenium
2
*Wa(r
WebDriver


Capybara
Culerity
Selenium
2
*Wa(r
WebDriver Celerity
HTML
Unit


Capybara
Culerity
OMGWTFBBQ?!
Selenium
2
*Wa(r
WebDriver Celerity
HTML
Unit

http://www.ﬂickr.com/photos/snake-eyes/449442699/

require 'watir'
Watir::Browser.default = 'firefox'

describe 'Google' do
before(:each) { @browser = Watir::Browser.new }
after(:each) { @browser.close }

it 'should return search results for "hello world"' do
@browser.goto "http://www.google.co.uk"
@browser.text_field(:name, "q").set("hello world")
@browser.button(:name, "btnG").click
@browser.contains_text(
"Hello world program - Wikipedia").should be_true
end
end

require 'selenium-webdriver'

before(:each){ @browser = Selenium::WebDriver.for :firefox }
after(:each) { @browser.quit }

@browser.navigate.to "http://www.google.co.uk"
@browser.find_element(:name, "q").send_keys("hello world")
@browser.find_element(:name, "btnG").submit
@browser.find_element(:partial_link_text,
"Hello world program - Wikipedia")
end
end

require 'capybara'; require 'capybara/dsl'
Capybara.default_driver = :selenium

Spec::Runner.configure do |config|
config.include(Capybara, :type => :integration)
config.before(:each) { Capybara.reset_sessions! }
end

visit "http://www.google.co.uk"
fill_in "q", :with => "hello world"
click "Google Search"
page.should have_content(
"Hello world program - Wikipedia")
end
end

# API beauty contest!

goto "http://www.google.co.uk" # Watir
navigate.to "http://www.google.co.uk" # WebDriver
visit "http://www.google.co.uk" # Capybara

text_field(:name, "q").set("hello world") # Watir
find_element(:name, "q").send_keys("hello world") # WD
fill_in "q", :with => "hello world" # Capybara

button(:name, "btnG").click # Watir
find_element(:name, "btnG").submit # WebDriver
click "Google Search" # Capybara

# You choose...

Cucumber

Capybara
Culerity
Selenium
2
*Wa(r
WebDriver Celerity
HTML
Unit

Feature: Looking up a contact
In order to understand my contacts better
As a Gmail user
I want to see social information next to my email thread

Scenario: Load an email conversation
Given I am logged into Gmail
And Rapportive is loaded
When I search for "martin@rapportive.com"
And I click on the conversation with subject "Testing 1 2 3"
Then I should be on a conversation view
And Rapportive should show "Co-founder at Rapportive"

Given /^I am logged into Gmail$/ do
visit 'https://mail.google.com/a/rapportive.com'
fill_in 'Email', :with => 'test'
fill_in 'Passwd', :with => GMAIL_TEST_ACCOUNT_PASSWORD
click 'Sign in'
end

Then /^Rapportive should show "([^"]*)"$/ do |text_to_show|
page.within_frame('canvas_frame') do
page.within(:css, '#rapportive-sidebar') do
page.should have_content(text_to_show)
end
end
end

FEI
(Frequently

Encountered
Issues)

10.
Time
dependence
‣ Modify
date(mes
in
DB

using
models
‣ Create
wrapper
around

Time.now

9.
Real-‐*me
comms

8.
Managing
test
VMs

8.
Managing
test
VMs
‣ Very
tedious
(updates
etc.)
‣ Test
management

infrastructure
‣ Commercial
services
(e.g.

Sauce
Labs,
BrowserMob)

7.
Random
DOM
IDs
‣ Ext.js,
GWT,
Cappuccino
‣ Use
toolkit
APIs
// Selecting an item in a Ext.js Combo Box
var combo = Ext.getCmp('countryComboBox');
combo.setValue('Germany');
// setValue() doesn't trigger the event
combo.fireEvent('select');

6.
Tes*ng
layout
‣ Automa(cally
spoWng

broken
layout?
srsly?
‣ Interes(ng
experiment
h"p://code.google.com/p/ﬁgh/ng-‐layout-‐bugs/

$(‘*’).css(‘color’,
‘white’);

$(‘*’).css(‘color’,
‘black’);

image1
–
image2
=
...

5.
Unit
tes*ng
JS
‣ Command
line
&&
browser
‣ DOM
manipula(on
→
HTML

ﬁxtures

(...and
rollback?)

5.
Unit
tes*ng
JS
‣ JSpec
h"p://visionmedia.github.com/jspec/

‣ Blue
Ridge/Screw.Unit
h"p://github.com/relevance/blue-‐ridge

‣ JsTestDriver
h"p://code.google.com/p/js-‐test-‐driver/

4.
Model
layer
access

4.
Model
layer
access
Tests
HTTP/Framework
Views
Controllers
Models

4.
Model
layer
access
Tests
HTTP/Framework
Views
Controllers
Factory/
Fixtures? Models

4.
Model
layer
access
‣ Violates
abstrac(on
‣ BUT:
factories
=
only
sane

way
of
managing
DB
state

Factory.define :user do |u|
u.first_name 'John'
u.last_name 'Doe'
u.admin false
end

Given /^I am on John's profile page$/ do
user = Factory.create(:user)
visit user_url(user)
end

3.
Star*ng
state
‣ Previous,
failed
tests
‣ Cookies,
browser
cache
‣ Unexpected
persistent
state

3.
Star*ng
state
‣ Long-‐running
transac(on
&

rollback
(e.g.
Cucumber
per-‐scenario
transac/on
rollback)

‣ Clean
up
database
h"p://github.com/bmabey/database_cleaner

2.
Parallel
test
runs

2.
Parallel
test
runs
‣ End-‐to-‐end
tests
are
SLOW
‣ Start
up
several
browsers
(TestSwarm,
Selenium
Grid,
...)

‣ BUT:
concurrent

modiﬁca(on
of
DB
state!

2.
Parallel
test
runs
One
DB
per
test
process
‣ h"p://github.com/grosser/parallel_tests
‣ h"p://github.com/qxjit/deep-‐test

2.
Parallel
test
runs
Caveat:
state
outside
of
your

control
‣ e.g.
OAuth
status
‣ use
mocks,
or
work
around
it

Fragility
(tests
break
too
easily,

even
if
the
app
is
ok)

2
schools
of
thought
1. “Disposable”
test
scripts
(invest
licle
eﬀort,
use

recording
tools)

2. “Engineered”
test
scripts
(use
carefully
designed

abstrac(ons)

Fragility
No
silver
bullet,
obvious
stuﬀ:
‣ Consistent
naming
of
IDs
&

CSS
classes
‣ Maintain
&
refactor
your

tests

kthxbai

Mar*n
Kleppmann
mar*n@rappor*ve.com
@mar*nkl rapportive

Cross-browser testing in the real world

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (20)

Similar to Cross-browser testing in the real world

Similar to Cross-browser testing in the real world (20)

Recently uploaded

Recently uploaded (20)

Cross-browser testing in the real world