Browser-level testing

Browser-level testing Martin Kleppmann Go Test It Email: martin@go-test.it Blog: yes-no-cancel.co.uk Twitter: @martinkl

“But I have unit tests!”

Array literals? var x = [1,]; x.length == 1 // Firefox 3.5 x.length == 2 // IE 7

Uninitialised vars? <div id="hello"></div> typeof(hello) == 'undefined' // Firefox 3.5 typeof(hello) == 'object' // IE 7

Attempt #1: Simulate events in JavaScript. (Selenium, Windmill, Sahi)

Attempt #1: Simulate events in JavaScript. Selenium example.com

Attempt #1: Simulate events in JavaScript. click Selenium example.com

Attempt #1: Simulate events in JavaScript. click Selenium example.com http://localhost:8080 http://test.example.com

Attempt #1: Simulate events in JavaScript. Cross-domain security policy click Selenium example.com http://localhost:8080 http://test.example.com

Attempt #1: Simulate events in JavaScript. Selenium example.com

Attempt #1: Simulate events in JavaScript. Selenium example.com test.example.com proxy

Attempt #1: Simulate events in JavaScript. Selenium example.com test.example.com proxy /selenium-server/ /*

Attempt #1: Simulate events in JavaScript. click Selenium example.com test.example.com proxy /selenium-server/ /*

Limitations • Redirect to other domain or https • Outside of DOM: pop-ups, Flash, ... • Slow

Attempt #2: Browser-speciﬁc automation APIs. (WebDriver, Watir)

IE Firefox Safari Chrome Opera HTMLUnit

IE Firefox Safari WebDriver Chrome Opera HTMLUnit

IE Firefox Safari WebDriver Chrome Opera Watir HTMLUnit

IE Selenium Firefox Safari WebDriver Chrome Opera Watir HTMLUnit

Selenium WebDriver Watir IE Firefox Safari Chrome Opera HTMLUnit

Selenium WebDriver Watir IE HTA/Proxy ✔ COM ✔ COM Firefox Ext/Proxy ✔ Extension ✔ Extension Safari Proxy ✘ ✔ AppleScript Chrome Proxy ✔ Extension ✔ V8 Debug Opera Proxy Dragonﬂy? ✘ HTMLUnit ✘ ✔ ✔ Celerity

Watir require 'watir' Watir::Browser.default = 'firefox' describe 'Google' do before(:each) { @browser = Watir::Browser.new } after(:each) { @browser.close } it 'should return search results for "hello world"' do @browser.goto "http://www.google.co.uk" @browser.text_field(:name, "q").set("hello world") @browser.button(:name, "btnG").click @browser.contains_text( "Hello world program - Wikipedia").should be_true end end

WebDriver require 'selenium-webdriver' describe 'Google' do before(:each) { @browser = Selenium::WebDriver.for :firefox } after(:each) { @browser.quit } it 'should return search results for "hello world"' do @browser.navigate.to "http://www.google.co.uk" @browser.find_element(:name, "q").send_keys("hello world") @browser.find_element(:name, "btnG").submit @browser.find_element(:partial_link_text, "Hello world program - Wikipedia") end end

Selenium gem 'selenium-client'; require 'selenium/client' describe 'Google' do before(:each) { @browser = Selenium::Client::Driver.new :browser => "*firefox", :url => "http://www.google.co.uk" @browser.start } after(:each) { @browser.stop } it 'should return search results for "hello world"' do @browser.open "/" @browser.type "q", "hello world" @browser.click "btnG", :wait_for => :text, :text => 'Results' @browser.is_text_present("Hello world program - Wikipedia"). should be_true end end

Go Test It (≈Selenium) require 'gotest' gotest "http://www.google.co.uk" do open "/" type "q", "hello world" click "btnG" verify_text_present "Hello world program - Wikipedia" end

Cucumber Feature: Google search In order to learn something new As an enthusiast for obscure programming languages I want to find "Hello world" programs on Google Scenario: Search for "hello world" Given that I am on google.co.uk When I enter "hello world" into the search box And I submit the form Then I should be shown a list of results And the page should contain "Hello world program - Wikipedia"

IE Selenium Firefox Safari WebDriver Chrome Opera Watir HTMLUnit

The future? IE Selenium Selenium API Firefox Safari WebDriver Chrome Opera Watir Watir API HTMLUnit

Fundamental goal: Robust tests

Choose your locators carefully

Choose your locators carefully id = searchInput name = search xpath = //form[@id='searchform']/input[2]

link = EU makes 7bn euro climate pledge

link = EU makes 7bn euro climate pledge xpath = //td[@class='text']/div[1]/a

link = EU makes 7bn euro climate pledge xpath = //td[@class='text']/div[1]/a css = a.tsh

People are complicated Frontend tests are messy

2 approaches: Disposable tests (record/playback, quick&dirty) Domain-speciﬁc abstractions (carefully crafted)

Which approach will you take?

Thank you! Martin Kleppmann Go Test It Email: martin@go-test.it Blog: yes-no-cancel.co.uk Twitter: @martinkl

Fundamental goal: Parallel tests

Don’t assume a clean database

Don’t assume a clean database Unit tests: Load ﬁxtures Given... When... Then... Reset database

Don’t assume a clean database Unit tests: Functional tests: Load ﬁxtures Set up example data Given... Do stuff Do stuff When... Check Check Then... outcome outcome Reset database

Tolerate partially run tests

Tolerate partially run tests (Assume there is no frob) Create frob Check frob was created Delete frob

Tolerate partially run tests Bad: Boom! (Assume there is no frob) Create frob Check frob was created Delete frob

Tolerate partially run tests Bad: Better: Boom! (Assume there is no frob) Delete frob if it exists Create frob Create frob Check frob was created Check frob was created Delete frob Delete frob (optional)

Avoid race conditions: isolation

Avoid race conditions: isolation Delete frob if it exists Delete frob if it exists Create frob Create frob Check frob was created Check frob was created

Avoid race conditions: isolation Log in as User A Log in as User B Delete frob if it exists Delete frob if it exists Create frob Create frob Check frob was created Check frob was created

Avoid race conditions: isolation Log in as User A Log in as User B Delete frob if it exists Delete frob if it exists Create frob Create frob Check frob was created Check frob was created User A and User B: independent & not used concurrently

Make test runs unique

Take care with external services

Thank you! Martin Kleppmann Go Test It Email: martin@go-test.it Blog: yes-no-cancel.co.uk Twitter: @martinkl

http://www.slideshare.net	21
http://speakerrate.com	13
http://lanyrd.com	2
http://67-207-142-125.slicehost.net	1

Browser-level testing

by Martin Kleppmann on Dec 12, 2009

Accessibility

Categories

Tags

Upload Details

Usage Rights

4 Embeds 37

Statistics

Browser-level testing — Presentation Transcript