Reliable tests with selenium web driver


Published on

Published in: Technology, Education
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • Selenium – Thoughtworks , WebDriver Google – Selenium/WebDriver a hopefully good combination of both
  • 1. Well writtend, high level e2e test can play the role of live documentation.
  • 1. Every time your CI build finishes succesfully a new piece of code can be deployed to production. So let’s say you don’t how e2e tests in place. Get the latests, the app is broken, figure out if it was you, not you, then who, what needs to be done to fix it. Now let’s imagine that all 20 guys on your team got the latest at the same time. 2. Accross the board
  • It takes more time to investiage a broken e2e test than a unit test so every false alarm is costly.
  • Happy path is easy, use unit tests for testing edge cases There is this attitue that tests are not deployed to Production so they don’t need the same attention as prouduction code. Wrong. Does not matter C#, Test or JS Test Recorders – not need to use them and then clean up the mess they create. I would rather start with clean code. It’s faster.
  • it’s a faster, if it takes 5 seconds to load the homepage and every tests needs to load it so each tests is 5 second longer than it needs to be External system, you don’t controll them so they are down your tests fail, you can have integration tests but keep them separate We all know that but with e2e it is even more important
  • Show the app first and explain DB Start with simple test (simple) Add ajax, show how to use (simpleajax) – works and doesn’t, every now and then Add 2s delay ,Debugging, TakeScreenshot (debugging) Fix the code with Thread.Sleep (fixedajax) – what’s wrong with this code, increase to 30s, waste of time Fix properly (reliableajax) (and decrease controller Thread.Sleep) show how it affects test execution time Change controller T.Sleep to 7 s and show TimeoutException, False/TE, bad API Fix with (betterretry) and show with 7s Optional: visibility (anotehr test) Long page load (hardcoded to 60 sec) but sometimes need (eg. Ncover), extensions method We’ve see some problems with finding elelements using CSS selectors If API not good enough, use access to IJavaScriptExecutor
  • Reliable tests with selenium web driver

    1. 1. Reliable end to end tests with Selenium WebDriver Pawel Pabich, Senior Consultant at Readify blog: twitter: @pawelpabich email: [email_address]
    2. 2. My definition of end to end (e2e) test s <ul><ul><li>Automated </li></ul></ul><ul><ul><li>Often start at the UI  </li></ul></ul><ul><ul><li>Same as user acceptance test </li></ul></ul><ul><ul><li>Often used to test user stories </li></ul></ul><ul><ul><li>Often expressed using Given/When/Then </li></ul></ul><ul><ul><li>Can describe what the system does </li></ul></ul><ul><ul><li>Often slower than unit/integration test s </li></ul></ul>
    3. 3. Reliable e2e tests provide a lot of value <ul><ul><li>Assure the code is shippable </li></ul></ul><ul><ul><li>Provide solid safety net and in this way encourage constant refactoring </li></ul></ul><ul><ul><li>Let QA focus on creative, exploratory testing  </li></ul></ul>
    4. 4. Unreliable e2e tests are harmful <ul><ul><li>Take a lot of time to investigate  </li></ul></ul><ul><ul><li>Make the development process slower than if there were no e2e tests at all </li></ul></ul><ul><ul><li>Are extremely frustrating  for all team members </li></ul></ul>
    5. 5. How to write reliable e2e tests 1/2 <ul><ul><li>Do NOT use Thread.Sleep. It is NEVER a good idea </li></ul></ul><ul><ul><li>ALWAYS start with unit tests and fall back to e2e tests only when unit testing is not the right tool </li></ul></ul><ul><ul><li>Focus on the happy path </li></ul></ul><ul><ul><li>Treat test code as if it was Production code </li></ul></ul><ul><ul><li>Do NOT share state between tests </li></ul></ul><ul><ul><li>Do NOT use test recorders </li></ul></ul>
    6. 6. How to write reliable e2e tests 2/2 <ul><ul><li>Navigate directly to the screen under test </li></ul></ul><ul><ul><li>Test only code you own , eg. all external systems are mocked </li></ul></ul><ul><ul><li>Create a separate set of tests for integration with live external systems </li></ul></ul><ul><ul><li>Always make sure the system is in a known initial state </li></ul></ul><ul><ul><li>Test only one scenario at a time </li></ul></ul>
    7. 7. Demo <ul><li>  </li></ul>
    8. 8. Summary <ul><li>If you can't make it reliable do NOT write it at all. You will regret it. </li></ul>Links: Code: Selenium: Our own James Crisp :