Your SlideShare is downloading. ×
0
Full-stack webapp testing with Selenium and Rails

• Alex Chaffee

• Brian Takita
Abstract

• Want to test your entire Web 2.0 app, from AJAX and DHTML through browser-rendered
  HTML into a live instance...
Selenium Features

• Selenium runs in the browser

• Executes your app in a frame

• Simulates user actions via JavaScript...
First Example

• Basic example (login_test.rb)

   •   def test_login
          open "/"
          assert_text_present("Pl...
Accessing Your World

• Can access database and/or domain objects

  • def test_signup
      open "/signup"
      assert_t...
Or even email!

 • def test_signup_via_email
     open "/signup"
     assert_text_present("Please sign up")
     type "ema...
Or even email!

 • def test_signup_via_email
     open "/signup"
     assert_text_present("Please sign up")
     type "ema...
Or even email!

 • def test_signup_via_email
     open "/signup"
     assert_text_present("Please sign up")
     type "ema...
Or even email!

 • def test_signup_via_email
     open "/signup"
     assert_text_present("Please sign up")
     type "ema...
Or even email!

 • def test_signup_via_email
     open "/signup"
     assert_text_present("Please sign up")
     type "ema...
And don’t forget to extract a DSL

    def login(user_name,password)
      open('/login')
      type "id=login", user_name...
Selenium RC Architecture: launch

       rake selenium:test                 rake selenium:server




               Launch...
Selenium RC Architecture: run test

        rake selenium:test                         rake selenium:server




          ...
Multiple Platform Support

• OS (Unix, Windows, Mac)

• Browser (IE, Firefox, Safari)

• The Selenium Farm (Continuous Int...
Good Things Come To Those Who Wait

• Web 2.0 is Asynchronous and Dynamic

  • The “A” in "AJAX"

  • The "D" in "DHTML"

...
Good Things Come To Those Who Wait

• wait_for function

   • executes in Ruby

• wait_for_condition action

   • takes a ...
More examples

• Tracker

  • project_page_editing_test



• Peer2Patent

  • create_prior_art_test

  • registration_mode...
What kind of tests?

• Integration Tests

   • Test multiple layers of your app working together

• Acceptance Tests

   •...
Wait, do you mean Integration Tests or
Integration Tests?

• Selenium tests are integration tests in the abstract

• Not d...
When to use Selenium?

• Test full stack

• Test JS->server interaction

• Test emails

• Smoke testing

• Testing in mult...
Particular Strengths of Selenium Tests

• AJAX/RJS tests

  • Test what state the browser is in after the RJS/AJAX call

•...
When not to use Selenium?

• For every edge case

   • Want to test happy path, and one or two error paths

• When speed i...
Slow-lenium

• Selenium is much faster than a human, but still much slower than unit tests.
  How do we deal with that?

•...
Locators

• Specify how to locate an HTML element

  • id

     • "id" attribute

     • e.g. id=userName

  • name

     ...
Locators (cont.)

• identifier

   • first id, then name

• dom

   • JavaScript expression

   • e.g. dom=document.forms['i...
Locators (cont.)

• link

   • selects the <A>` element whose Text content matches the given text

   • e.g. link=Click He...
Tips

• use ids when possible

  • XPath makes your tests brittle and dependent on page structure

• use slow mode for deb...
Gotchas

• Disable Transactional Fixtures for Selenium tests

• Reload all fixtures before every test

   • consistent stat...
Test-driving from Selenium

• Some people like it, some don't

• Pro:

   • Top-down

   • When test passes, feature is do...
Pivotal rake integration

• launches server locally

• runs tests locally and/or remotely
Pivotal Ruby Classes

• selenium_suite.rb

• selenium_helper.rb
Selenium On Rails

• Generates HTML (FIT-style)

• Can't go into your DB or domain

• Any custom test code has to be JavaS...
Q&A
Experimentation
References

• www.openqa.org—selenium-rc <http://www.openqa.org/selenium-rc/>
Timeline

• 90 minutes with Q&A interspersed
Full-stack webapp testing with Selenium and Rails
Upcoming SlideShare
Loading in...5
×

Full-stack webapp testing with Selenium and Rails

36,220

Published on

Want to test your entire Web 2.0 app, from AJAX and DHTML through browser-rendered HTML into a live instance of your application and database? Most web testing frameworks have gaps in their coverage: JUnit and Test::Unit miss the client frontend; JSUnit misses the server backend; web testing frameworks miss some or all of the JavaScript. With Selenium we have a framework that can test the whole application, from browser-executed JavaScript, through a live application backend, then back to assertions on browser-rendered DOM code. Selenium RC takes this further: since you write your tests in your application language, your tests can do data setup and assertions based directly on server-side domain objects that may be inaccessible or only partially accessible from the client side. On our teams we have used and developed a series of helper methods and assertions that allow testing of AJAX and DHTML functions as well.

In this tutorial we outline the architecture of Selenium RC and walk through code and examples illustrating how to do full-stack testing against a Ruby on Rails application.

Published in: Technology
2 Comments
25 Likes
Statistics
Notes
No Downloads
Views
Total Views
36,220
On Slideshare
0
From Embeds
0
Number of Embeds
9
Actions
Shares
0
Downloads
1,101
Comments
2
Likes
25
Embeds 0
No embeds

No notes for slide

Transcript of "Full-stack webapp testing with Selenium and Rails"

  1. 1. Full-stack webapp testing with Selenium and Rails • Alex Chaffee • Brian Takita
  2. 2. Abstract • Want to test your entire Web 2.0 app, from AJAX and DHTML through browser-rendered HTML into a live instance of your application and database? Most web testing frameworks have gaps in their coverage: JUnit and Test::Unit miss the client frontend; JSUnit misses the server backend; web testing frameworks miss some or all of the JavaScript. With Selenium we have a framework that can test the whole application, from browser-executed JavaScript, through a live application backend, then back to assertions on browser-rendered DOM code. Selenium RC takes this further: since you write your tests in your application language, your tests can do data setup and assertions based directly on server-side domain objects that may be inaccessible or only partially accessible from the client side. On our teams we have used and developed a series of helper methods and assertions that allow testing of AJAX and DHTML functions as well. In this tutorial we outline the architecture of Selenium RC and walk through code and examples illustrating how to do full-stack testing against a Ruby on Rails application.
  3. 3. Selenium Features • Selenium runs in the browser • Executes your app in a frame • Simulates user actions via JavaScript • Goes all the way to the server and back • Complementary to JSUnit • JSUnit tests JS pages only, not interaction with server • Fun to watch
  4. 4. First Example • Basic example (login_test.rb) • def test_login open "/" assert_text_present("Please sign in") type "username", "jdoe" type "password", "password" click "submit" assert_text_present("Welcome, John Doe") open "/" assert_text_present("Welcome, John Doe") end • Sort of a DSL ("Selenese") • Written in Ruby • Can also use Java, .NET, Python, Perl, etc.
  5. 5. Accessing Your World • Can access database and/or domain objects • def test_signup open "/signup" assert_text_present("Please sign up") type "username", "noob" type "fullname", "New Bie" type "password", "secret" type "password_verify", "secret" click "submit" assert_text_present("Welcome, New Bie") user = User.find_by_username("noob") assert user.authenticate("secret") end
  6. 6. Or even email! • def test_signup_via_email open "/signup" assert_text_present("Please sign up") type "email", "noob@example.com" click "submit" assert_text_present("An email has been sent to noob@example.com") wait_for { !ActionMailer::Base.deliveries.empty? } user = User.find_by_email("noob@example.com”) assert !user.verified? link = extract_link(ActionMailer::Base.deliveries[0].body) open link wait_for do user.reload user.verified? end end def extract_link(s) /http://S*/.match(s)[0] end
  7. 7. Or even email! • def test_signup_via_email open "/signup" assert_text_present("Please sign up") type "email", "noob@example.com" click "submit" assert_text_present("An email has been sent to noob@example.com") wait_for { !ActionMailer::Base.deliveries.empty? } user = User.find_by_email("noob@example.com”) assert !user.verified? link = extract_link(ActionMailer::Base.deliveries[0].body) open link wait_for do user.reload user.verified? end end def extract_link(s) /http://S*/.match(s)[0] end
  8. 8. Or even email! • def test_signup_via_email open "/signup" assert_text_present("Please sign up") type "email", "noob@example.com" click "submit" assert_text_present("An email has been sent to noob@example.com") wait_for { !ActionMailer::Base.deliveries.empty? } user = User.find_by_email("noob@example.com”) assert !user.verified? link = extract_link(ActionMailer::Base.deliveries[0].body) open link wait_for do user.reload user.verified? end end def extract_link(s) /http://S*/.match(s)[0] end
  9. 9. Or even email! • def test_signup_via_email open "/signup" assert_text_present("Please sign up") type "email", "noob@example.com" click "submit" assert_text_present("An email has been sent to noob@example.com") wait_for { !ActionMailer::Base.deliveries.empty? } user = User.find_by_email("noob@example.com”) assert !user.verified? link = extract_link(ActionMailer::Base.deliveries[0].body) open link wait_for do user.reload user.verified? end end def extract_link(s) /http://S*/.match(s)[0] end
  10. 10. Or even email! • def test_signup_via_email open "/signup" assert_text_present("Please sign up") type "email", "noob@example.com" click "submit" assert_text_present("An email has been sent to noob@example.com") wait_for { !ActionMailer::Base.deliveries.empty? } user = User.find_by_email("noob@example.com”) assert !user.verified? link = extract_link(ActionMailer::Base.deliveries[0].body) open link wait_for do user.reload user.verified? end end def extract_link(s) /http://S*/.match(s)[0] end
  11. 11. And don’t forget to extract a DSL def login(user_name,password) open('/login') type "id=login", user_name type "id=password", password click_and_wait "commit" end def logout open('/logout') end
  12. 12. Selenium RC Architecture: launch rake selenium:test rake selenium:server Launch Launch Selenium Test Suite Selenium Server Browser Launch Browser Launch Browser DOM/JS Webrick running app (in "test" mode) Database Remote (possibly)
  13. 13. Selenium RC Architecture: run test rake selenium:test rake selenium:server Launch Launch Selenium Test Suite Selenium Server Browser def test_foo click click "submit" 'submit' end Selenese Interpreter click 'submit' DOM/JS HTTP Webrick running app HTTP Proxy (in "test" mode) Database Remote (possibly)
  14. 14. Multiple Platform Support • OS (Unix, Windows, Mac) • Browser (IE, Firefox, Safari) • The Selenium Farm (Continuous Integration)
  15. 15. Good Things Come To Those Who Wait • Web 2.0 is Asynchronous and Dynamic • The “A” in "AJAX" • The "D" in "DHTML" • The problem: • Since they happen unpredicatably, and inside the context of a single page, they’re harder to test
  16. 16. Good Things Come To Those Who Wait • wait_for function • executes in Ruby • wait_for_condition action • takes a JavaScript expression, evals it in browser • wait_for_condition is complementary to wait_for • JavaScript can't poll DB or domain objects • Write your conditions in the language you prefer
  17. 17. More examples • Tracker • project_page_editing_test • Peer2Patent • create_prior_art_test • registration_mode_test
  18. 18. What kind of tests? • Integration Tests • Test multiple layers of your app working together • Acceptance Tests • Test specific customer features and/or scenarios • UI tests • Test complicated or detailed UI widgets • Smoke Tests • Special case of Integration Tests • Make sure your app doesn’t blow up
  19. 19. Wait, do you mean Integration Tests or Integration Tests? • Selenium tests are integration tests in the abstract • Not derived from Rails Integration Tests • Rails ITs are faster • Selenium tests are better for testing AJAX, DHTML, widgets, full-stack integration • Where you can do a test in Ruby instead of Selenium... you should.
  20. 20. When to use Selenium? • Test full stack • Test JS->server interaction • Test emails • Smoke testing • Testing in multiple browsers
  21. 21. Particular Strengths of Selenium Tests • AJAX/RJS tests • Test what state the browser is in after the RJS/AJAX call • JS tests • Test JavaScript interacting with your DOM • Not just functional/behavioral JS (which JSUnit is great for)
  22. 22. When not to use Selenium? • For every edge case • Want to test happy path, and one or two error paths • When speed is of the essence • e.g. in a suite you run a lot, or when your tests start taking more than a few minutes • see next slide... • When you’re testing fine-grained features • Selenium tests tend to sprawl out • setup/teardown are more expensive, so you want to test more things per test • Unit tests should be small and isolated
  23. 23. Slow-lenium • Selenium is much faster than a human, but still much slower than unit tests. How do we deal with that? • Fast Suite / Slow Suite • Run Fast suite before checkin • Run Slow suite in Continuous Integration • When appropriate, convert Selenium tests into unit tests • Consider bootstrapping with Selenium tests, then replacing them with isolated tests once your app is up and running
  24. 24. Locators • Specify how to locate an HTML element • id • "id" attribute • e.g. id=userName • name • "name" attribute • followed by optional "value" or "index" filter • e.g. name=flavor value=chocolate
  25. 25. Locators (cont.) • identifier • first id, then name • dom • JavaScript expression • e.g. dom=document.forms['icecreamForm'].flavorDropdown • xpath • e.g. xpath=//a[contains(@href,'#id1')]/@class
  26. 26. Locators (cont.) • link • selects the <A>` element whose Text content matches the given text • e.g. link=Click Here To Continue • css • css selector • e.g. css=a[href="#id3"] • Custom Locators • By default, it's "identifier", • or "xpath" if it starts with "//" • or "dom" if it starts with "document."
  27. 27. Tips • use ids when possible • XPath makes your tests brittle and dependent on page structure • use slow mode for debugging • user_extensions.js
  28. 28. Gotchas • Disable Transactional Fixtures for Selenium tests • Reload all fixtures before every test • consistent state • order-independent • set allow_concurrency=true • cookies stay between tests • can lead to tests passing when run alone, failing when run together • you can eval JS to clear cookies between tests • Firefox profiles
  29. 29. Test-driving from Selenium • Some people like it, some don't • Pro: • Top-down • When test passes, feature is done • Stay close to well-defined customer requirements • Con: • Still vulnerable to edge cases • Brittle in the face of UI changes • You usually iterate through all layers of tests, several times over
  30. 30. Pivotal rake integration • launches server locally • runs tests locally and/or remotely
  31. 31. Pivotal Ruby Classes • selenium_suite.rb • selenium_helper.rb
  32. 32. Selenium On Rails • Generates HTML (FIT-style) • Can't go into your DB or domain • Any custom test code has to be JavaScript • Does it work remotely?
  33. 33. Q&A
  34. 34. Experimentation
  35. 35. References • www.openqa.org—selenium-rc <http://www.openqa.org/selenium-rc/>
  36. 36. Timeline • 90 minutes with Q&A interspersed
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×