Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Browser-level testing

7,256 views

Published on

Overview of Selenium, WebDriver, Watir and related open source cross-browser testing technologies. Presentation given by Martin Kleppmann, founder of browser testing service Go Test It, at Ruby Manor 2009.

Published in: Technology
  • Login to see the comments

Browser-level testing

  1. 1. Browser-level testing Martin Kleppmann Go Test It Email: martin@go-test.it Blog: yes-no-cancel.co.uk Twitter: @martinkl
  2. 2. “But I have unit tests!”
  3. 3. ?
  4. 4. Array literals? var x = [1,]; x.length == 1 // Firefox 3.5 x.length == 2 // IE 7
  5. 5. Uninitialised vars? <div id="hello"></div> typeof(hello) == 'undefined' // Firefox 3.5 typeof(hello) == 'object' // IE 7
  6. 6. Attempt #1: Simulate events in JavaScript. (Selenium, Windmill, Sahi)
  7. 7. Attempt #1: Simulate events in JavaScript. Selenium example.com
  8. 8. Attempt #1: Simulate events in JavaScript. click Selenium example.com
  9. 9. Attempt #1: Simulate events in JavaScript. click Selenium example.com http://localhost:8080 http://test.example.com
  10. 10. Attempt #1: Simulate events in JavaScript. Cross-domain security policy click Selenium example.com http://localhost:8080 http://test.example.com
  11. 11. Attempt #1: Simulate events in JavaScript. Selenium example.com
  12. 12. Attempt #1: Simulate events in JavaScript. Selenium example.com test.example.com proxy
  13. 13. Attempt #1: Simulate events in JavaScript. Selenium example.com test.example.com proxy /selenium-server/ /*
  14. 14. Attempt #1: Simulate events in JavaScript. click Selenium example.com test.example.com proxy /selenium-server/ /*
  15. 15. Limitations • Redirect to other domain or https • Outside of DOM: pop-ups, Flash, ... • Slow
  16. 16. Attempt #2: Browser-specific automation APIs. (WebDriver, Watir)
  17. 17. IE Firefox Safari Chrome Opera HTMLUnit
  18. 18. IE Firefox Safari WebDriver Chrome Opera HTMLUnit
  19. 19. IE Firefox Safari WebDriver Chrome Opera HTMLUnit
  20. 20. IE Firefox Safari WebDriver Chrome Opera Watir HTMLUnit
  21. 21. IE Firefox Safari WebDriver Chrome Opera Watir HTMLUnit
  22. 22. IE Selenium Firefox Safari WebDriver Chrome Opera Watir HTMLUnit
  23. 23. IE Selenium Firefox Safari WebDriver Chrome Opera Watir HTMLUnit
  24. 24. IE Selenium Firefox Safari WebDriver Chrome Opera Watir HTMLUnit
  25. 25. Selenium WebDriver Watir IE Firefox Safari Chrome Opera HTMLUnit
  26. 26. Selenium WebDriver Watir IE HTA/Proxy ✔ COM ✔ COM Firefox Ext/Proxy ✔ Extension ✔ Extension Safari Proxy ✘ ✔ AppleScript Chrome Proxy ✔ Extension ✔ V8 Debug Opera Proxy Dragonfly? ✘ HTMLUnit ✘ ✔ ✔ Celerity
  27. 27. Watir require 'watir' Watir::Browser.default = 'firefox' describe 'Google' do before(:each) { @browser = Watir::Browser.new } after(:each) { @browser.close } it 'should return search results for "hello world"' do @browser.goto "http://www.google.co.uk" @browser.text_field(:name, "q").set("hello world") @browser.button(:name, "btnG").click @browser.contains_text( "Hello world program - Wikipedia").should be_true end end
  28. 28. WebDriver require 'selenium-webdriver' describe 'Google' do before(:each) { @browser = Selenium::WebDriver.for :firefox } after(:each) { @browser.quit } it 'should return search results for "hello world"' do @browser.navigate.to "http://www.google.co.uk" @browser.find_element(:name, "q").send_keys("hello world") @browser.find_element(:name, "btnG").submit @browser.find_element(:partial_link_text, "Hello world program - Wikipedia") end end
  29. 29. Selenium gem 'selenium-client'; require 'selenium/client' describe 'Google' do before(:each) { @browser = Selenium::Client::Driver.new :browser => "*firefox", :url => "http://www.google.co.uk" @browser.start } after(:each) { @browser.stop } it 'should return search results for "hello world"' do @browser.open "/" @browser.type "q", "hello world" @browser.click "btnG", :wait_for => :text, :text => 'Results' @browser.is_text_present("Hello world program - Wikipedia"). should be_true end end
  30. 30. Go Test It (≈Selenium) require 'gotest' gotest "http://www.google.co.uk" do open "/" type "q", "hello world" click "btnG" verify_text_present "Hello world program - Wikipedia" end
  31. 31. Cucumber Feature: Google search In order to learn something new As an enthusiast for obscure programming languages I want to find "Hello world" programs on Google Scenario: Search for "hello world" Given that I am on google.co.uk When I enter "hello world" into the search box And I submit the form Then I should be shown a list of results And the page should contain "Hello world program - Wikipedia"
  32. 32. IE Selenium Firefox Safari WebDriver Chrome Opera Watir HTMLUnit
  33. 33. IE Selenium Firefox Safari WebDriver Chrome Opera Watir HTMLUnit
  34. 34. IE Selenium Firefox Safari WebDriver Chrome Opera Watir HTMLUnit
  35. 35. The future? IE Selenium Selenium API Firefox Safari WebDriver Chrome Opera Watir Watir API HTMLUnit
  36. 36. Ok.
  37. 37. Fundamental goal: Robust tests
  38. 38. Choose your locators carefully
  39. 39. Choose your locators carefully id = searchInput name = search xpath = //form[@id='searchform']/input[2]
  40. 40. link = EU makes 7bn euro climate pledge
  41. 41. link = EU makes 7bn euro climate pledge xpath = //td[@class='text']/div[1]/a
  42. 42. link = EU makes 7bn euro climate pledge xpath = //td[@class='text']/div[1]/a css = a.tsh
  43. 43. People are complicated Frontend tests are messy
  44. 44. 2 approaches: Disposable tests (record/playback, quick&dirty) Domain-specific abstractions (carefully crafted)
  45. 45. Which approach will you take?
  46. 46. Thank you! Martin Kleppmann Go Test It Email: martin@go-test.it Blog: yes-no-cancel.co.uk Twitter: @martinkl
  47. 47. Fundamental goal: Parallel tests
  48. 48. Don’t assume a clean database
  49. 49. Don’t assume a clean database Unit tests: Load fixtures Given... When... Then... Reset database
  50. 50. Don’t assume a clean database Unit tests: Functional tests: Load fixtures Set up example data Given... Do stuff Do stuff When... Check Check Then... outcome outcome Reset database
  51. 51. Tolerate partially run tests
  52. 52. Tolerate partially run tests (Assume there is no frob) Create frob Check frob was created Delete frob
  53. 53. Tolerate partially run tests (Assume there is no frob) Create frob Check frob was created Delete frob
  54. 54. Tolerate partially run tests Bad: Boom! (Assume there is no frob) Create frob Check frob was created Delete frob
  55. 55. Tolerate partially run tests Bad: Better: Boom! (Assume there is no frob) Delete frob if it exists Create frob Create frob Check frob was created Check frob was created Delete frob Delete frob (optional)
  56. 56. Avoid race conditions: isolation
  57. 57. Avoid race conditions: isolation Delete frob if it exists Delete frob if it exists Create frob Create frob Check frob was created Check frob was created
  58. 58. Avoid race conditions: isolation Log in as User A Log in as User B Delete frob if it exists Delete frob if it exists Create frob Create frob Check frob was created Check frob was created
  59. 59. Avoid race conditions: isolation Log in as User A Log in as User B Delete frob if it exists Delete frob if it exists Create frob Create frob Check frob was created Check frob was created User A and User B: independent & not used concurrently
  60. 60. Make test runs unique
  61. 61. Make test runs unique
  62. 62. Make test runs unique
  63. 63. Take care with external services
  64. 64. Thank you! Martin Kleppmann Go Test It Email: martin@go-test.it Blog: yes-no-cancel.co.uk Twitter: @martinkl

×