Automated Web Testing With Selenium

Automated Web Testing with Selenium
By Jodie Miners
Presented to SBTUG
30 September 2009

What is Selenium?
It’s Free and Open Source
It originally came from Thoughtworks
It comes in 3 flavours
IDE (Integrated Development Environment) – for recording test cases in Firefox
RC (Remote Control) – for running automated tests on any browser
Grid – for large scale test runs of multiple tests in parallel
You can get it here http://www.seleniumhq.org/
“Selenium tests run directly in a browser, just as real users do”

Why use Selenium
You are always doing the same steps over and over again
You need to get to the last step in a wizard to test that step
You want to pre-fill some dummy data
Formal Functional and Regression Testing
Load Testing *

Selenium IDE
A Firefox add-on that allows you to record tests for playing back in any browser.
Very simple to use
Allows non developers to get up and running with automated testing very quickly
Then they can pass on to developers for refinement or inclusion into a continuous build system
Can extend it easily with a little bit of HTML knowledge

Recording and Playback
First Decide what you are testing and work out your test case(s)
IDE Demo
Record a test
Play back a test
Debug a test
Step through a test
Capture screen shot
Asserts

Selenese
The language of selenium. It is quite easy to learn
3 parts to it
Command – eg Type
Locator – eg Input Box
Value – eg “ABC”
Can easily be written in HTML
Things to help build the IDE tests
Assert / Wait for Page to Load / Verify Text Present / Verify element present on each page
Slow things down with pauses and timeouts
Step through firs then run, add pauses and wait’s if it fails because it’s going too fast
Add think times

Selenese Locators
Can Locate page elements by the following 4 methods
ID, DOM, Xpath, CSS
IDE recorderwill choose the best one to use, or you can choose which ever method if you are coding the tests.
Can also use Wildcards and Regular Expressions
See http://seleniumhq.org/docs/04_selenese_commands.html for more info - it’s very comprehensive
Install Firebug to help with working out the locators

Selenium RC
Takes a little time and effort to get it up and running
Can then run your tests scripts from IDE in different browsers
With good results reporting
Or write the tests in code and automate the testing even further
Issues
Need to Install and use Java
Very command line / batch file based
Selenium RC Demo

Demo
Run the test in Firefox
Run the test in IE
Run the test in Chrome
Note that they are all in sequence

Coding your tests
Coding of Selenium testing can be done in
Java, Ruby, Groovy, C#, Perl, PHP, Python
Most examples are Java or Ruby, not that many people using C#
Add If statements
Eg if stock exists then add to cart (rather than having an error message)
Add looping statements
For each product found, add to cart
Easier to add parameterisation or data driven tests
eg have a list of products in a file or table – add each of the products to the cart
(you can do data driven tests in IDE – see here http://wiki.openqa.org/display/SEL/datadriven )
Multiple test runs
Run the test 4 times, each time logged in as a different user
Selenium Code Demo with NUnit

Demo
Show the code
Paste the selenese in
Show NunitvsMSTest?
Run the Nunit test
Have one prepared ready to go

Reporting
There is a great reporting framework if you code in Java or use TestNG
Logging Selenium http://loggingselenium.sourceforge.net/
For C# these are the options:
Just capture the Selenium output to a file
Code a reporting framework yourself
Use the Visual Studio Team System Test Edition framework
Use the NUnit reporting
I use a combination of VSTT reporting and some additional logging to a database that I created myself.

Selenium Grid
Scale out the tests to run on multiple browsers at the same time
Saves time in doing the testing
But you need to rely on the reporting as you can’t watch 5+ browsers do the testing
Great tutorial and demo here http://selenium-grid.seleniumhq.org/
Requires Java SDK and Apache Ant to run

My Grid Setup

Taking the Grid Further
Scale out your grid to the Cloud using Amazon EC2 service
Or use a third party service
Sauce Labs http://www.saucelabs.com
In early beta
Uses your own code
So can set up the code to either run locally or via web
Current special US$100 for 5000 minutes
Lots of Browser / OS combinations
Test Environment Picture
BrowserMobhttp://www.browsermob.com
Time consuming to set up
Some limitations on which selenium commands can be used
Lots of options (eg can test with specific bandwidth limitations)
Can be used as for http requests only, as well as real browser tests
Around US$2.00 per browser per hour

More Advanced Topics
UI Element mapping – How developers can make it easier for end users to build Selenium tests using element locator names that they can understand
http://ttwhy.org/home/blog/2007/05/12/selenium-ui-element-locator/
Using Xpath references with the Ext-JS javascript framework
http://www.xeolabs.com/portal/node/34

Load Testing
Purists say Selenium is not for load testing
That is mainly because of the hardware required to run multiple browsers on the same machine.
I hit limits of 5-8 browsers on a standard desktop machine and 12-15 browsers on a server
Use Browser Mob or SauceLabs to overcome these issues.
Also as Selenium uses Real Browsers it requires real logins to simulate real loads – so you have to log in as real users on your system
It does not do virtual load and virtual users

Recording in IE (just for @aussienick)
Watin for .net
http://watin.sourceforge.net/
Has a built in recorder for IE
There are also Watir (ruby), and Watij (java) variants
http://wtr.rubyforge.org/
Visual Studio Team System 2008 Test Edition (VSTT)
Visual Studio Team System 2010 Test Edition + Team Foundation Server
Looks Fantastic
Will also record and play back windows apps
Must use TFS

References
Selenium Tutorial http://dynamitemap.com/selenium/including a good video
Full list of software testing tools http://www.softwareqatest.com/qatweb1.html
VSTT Quick Reference http://vstt2008qrg.codeplex.com/

X-Path References
There is a great deal of help online for Xpath, here are some of the best sites I’ve found
Xpath reference http://www.w3schools.com/Xpath/default.asp
Some nice simple examples (on the Xpath tab) http://confluence.sakaiproject.org/display/QA/Selenium
Quick Reference Card http://confluence.sakaiproject.org/display/QA/Selenium
MSDN http://msdn.microsoft.com/en-us/library/ms256115.aspx
Great tutorial with lots of examples http://www.zvon.org/xxl/XPathTutorial/General/examples.html
Selenium Wiki Xpath reference http://wiki.openqa.org/display/SEL/Help+With+XPath