Azure Monitor & Application Insight to monitor Infrastructure & Application
Selenium In The Real World
1. Selenium In The Real World
Where are the tests, and how do I run them?
Iain Rose May 2012
2. INTRO
A bit about me
Iain Rose
Software Engineer in Test, xMatters
slides: http://bit.ly/vanq
code: https://github.com/iainrose/vanq-java
twitter: @iainrose
email: irose@xmatters.com
3. AGENDA
Coming up next
• Selenium & friends
• Page objects
• Locators
• Implicit & explicit waits
• Building your tests
• Running your tests
THERE WILL BE CODE!!!
4. W H Y A U T O M AT E ?
Regression testing sucks
5.
6. SELENIUM AND FRIENDS
The world it lives in
• WebDriver API (Java / Ruby / Python / C#)
• Test Frameworks
Java (jUnit, TestNG)
Ruby (Test::Unit, MiniTest, Rspec, Cucumber)
Python (py.test)
C# (nUnit)
• Build Tools (Ant, Maven, Gradle, Rake .... )
• Continuous Integration Server (Jenkins, Bamboo, Cruise Control)
• Browsers (IE, FF, Chrome, Mobile .... )
• Application Under Test
8. R E C O R D & P L AY B A C K ( I S ) F O R D U M M I E S
A shortcut to brittle, flaky tests
9. PA G E O B J E C T S
A test automation design pattern
PAGE
TESTS SELENIUM WEB APP
OBJECTS
10. PA G E O B J E C T S
Usage guidelines
• The public methods represent the services that the page offers
• Try not to expose the internals of the page
• Generally don't make assertions
• Methods return other Page Objects
• Need not represent an entire page
• Different results for the same action are modelled as different methods
http://code.google.com/p/selenium/wiki/PageObjects
11. PA G E O B J E C T E X A M P L E : VA N Q . O R G
Identifying Public Methods
class Home
clickMeetingsTab()
clickPastMeetingsLink()
getDateOfNextMeeting()
getPresenterOfNextMeeting()
isVanqLogoDisplayed()
12. PA G E O B J E C T E X A M P L E : VA N Q . O R G
Sample test method
13. PA G E O B J E C T E X A M P L E : VA N Q . O R G
Sample page object
19. WAIT CONDITIONS
Timing is everything
• Implicit Waits
+ Quick to implement
+ Covers most scenarios
- Slows down tests
- Cannot use to wait for absence of Web Elements
21. WAIT CONDITIONS
Explicit Waits
+ Most efficient use of waits
+ Built in ExpectedConditions class covers majority of scenarios
+ Custom wait conditions can be added to cover any other scenario
- Need to be manually added to each Page Object method
http://selenium.googlecode.com/svn/trunk/docs/api/java/org/openqa/selenium/support/ui/ExpectedConditions.html
22. WAIT CONDITIONS
Usage Guidelines
• Avoid static pauses
• Use long timeouts
• Add explicit waits to end of page object ‘action’ methods
• Always leave your Web Application in known ready state
23. BUILDING THE TESTS
Lessons learnt so far
• Keep tests focused
• Don’t share test data
• Do test data tear down at the start of each test
• Plan to run in parallel from day 1
24. RUNNING THE TESTS
Time for action
• IDE (Run & Debug)
• Command Line (Remote and Local)
• CI Tool
• Selenium Grid
25. GRADLE
Build Tool
• No XML
• Can setup your development IDE (IDEA, Eclipse)
• Dependency management
• 1 line of code runs TestNG & jUnit tests in parallel
• Manage TestNG group inclusions and exclusions
• No XML
• No XML
Automation can help with exploratory testing. Simultaneous testing & learningAutomation can increase learning about the application as you peel the onionRegression testing typically catches less bugs than exploratory testing a product for the first timeAutomated testing by definition is regression testing.Less time spent regression testing means more time exploratory testingAutomation scripts become your exploratory testing notesCan help improve working relationships and communication between testers and developersIt’s fun
Everything I’m going to present is a work in progressWeb Driver is a new tool first stable release was Selenium 2.0 (July 2011)No best practices, find the solution that best fits your application and team dynamicsTesting and checking are different things. Strictly speaking automation is checking
Selenium only automates browsers. Needs some help before it can be used to run tests.Selenium 1 is unsupportedPick the API that makes your test team more productive. Not necessarily the same language that your application is written in.
Beware of automation tools with ‘no coding required’Automation projects are development projects----- Meeting Notes (12-05-28 10:56) -----Lets show an example of how we would record a test against the VanQ.org web site
Issues apply to all record and playback toolsIssues:Browser selection embedded in the testBaseUrl embedded in the testBrittle xpath locators, some are 10 levels deepDuplicated use of locatorsText search scope is too broad, searching entire documentNo assertions = no resultsRecord & Playback can be useful for …Supplementing bug reports, imagine seeing the last ‘works on my machine’ comment ever!Quick demonstrations
Page Object allow us to add a layer of separation between our tests, selenium and the application under testReduces code duplication and allows you to define each action and locator once only, significantly reduces maintenance efforts WHEN things start to changeIncreases test code reaability
Public methods can be getters / setters / mouse driven actions
A web page is a class of objectThe VanQ home page is an instance of a Web Page classThe VanQ home page has propertiesThe VanQ home page offers public facing methods to interact with itThe VanQ home page inherits methods from a parent page object that represents the nav bar and page header / footerThe VanQ home page allows you to navigate to other web pages (represented by other page objects)
Tests are written in the language of your applicationNo Web Driver code in the testsNo details of web application internals in the test
Locators vs Web ElementsWeb Elements are objects on the page you can interact with (click, getText(), etc)Locators are a map to find your Web ElementsPage Objects generally don't make assertionsException to the rule is on transition from one Page Object to anotherRecommend find a strategy for your web application that can be used to determine if the web application is on the correct page. (URL, Title, Page Content, Key Element Locator)
Keep locators as simple as possible and use a few criteria as possible to make a unique match.Good locators will pass the mother in law testRecommend reading up on CSS locators.SauceLabs blog has good articlesCSS can be just as brittle as xPathxPath slow in IE
Casting Web Elements using inline locators cannot be shared across methods. Locators will get duplicatedPage Factory is a support library. Allows Web Elements to be shared across all methods in a page object while eliminating some boiler plate codeFirst 2 methods use your locators to save a Web Element Using By objects means you save the locator itself, not the target Web Element. You use these in your page object methods to find the WebElements as they are neededRecommend using By objects as they are the most flexible and can be reused in lookups and waits
Building your own is trivial but the findElement method returns an exception is Element is not found.By objects can be used by findElements methods which returns a list of all elements found by the provided locatorSaving your locators as By objects in the first place means you don’t need to duplicate your locators if you ever need to check for the presence or absence of a web element on the pageBy objects locators allow you to control exactly when the WebElement is looked up and assigned to a variable. This can avoid stale element exceptions when testing ajax heavy web pages (drag and drop, etc)By objects can be used in explicit wait conditions
Another reason to use By locators is they can be reused in waits which leads to next section
FindElement methods returns 1st Web Element found by locators not necessarily the only Web ElementAlways test your locators, there might be hidden elements on the page you cannot see
Effective use of wait conditions are the single most important feature to learn to.There is no waitForPageToLoad method in WebDriver, even if there was how to define a rule of when page is loaded?Implicit Waits always make the maximum in findElements
Explicit waits have a single purpose and are always used on demand
Efficient becuase they never wait short or longEfficient because are only used when you explicitly instruct the test to do so Another reason to use By objects as your locator strategy
Static pauses will always wait short or wait long. Both badDon’t mix performance tests and functional testsAllow for slow performing test environmentsMake page object methods leave the web application ready to use
Compare to exploratory tests where you do as much as you can at onceBetter to have 15 tests that do a single assertion than 1 test that does 15Use clear, specific names for test methodsDon’t worry about test name lengthUse a BaseTest class to handle browser setup and tear downUse @BeforeClass annotations to handle test data setupDon’t share test data across tests that make data updates
No
Do not show this slide in your presentation, but instead use it to copy and past elements to your slides. This page can also be found in the master slides incase it is deleted.
