Mastering selenium for automated acceptance tests


Published on

Published in: Technology
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • System: tests request, router, controller, view, response Black box: you do not have to be aware of the code behind the system. And thus you do not write your tests accordingly Functional: tests complete gui and behavior
  • System: tests request, router, controller, view, response Black box: you do not have to be aware of the code behind the system. And thus you do not write your tests accordingly Functional: tests complete gui and behavior
  • System: tests request, router, controller, view, response Black box: you do not have to be aware of the code behind the system. And thus you do not write your tests accordingly Functional: tests complete gui and behavior
  • Mastering selenium for automated acceptance tests

    1. Nick Belhomme January 26th, PHPBenelux Conference 2013, Belgium
    2. Nick BelhommeInternational Conference SpeakerContributor to various Open SourceProjectsBuilding ConstructorSoftware Architect / Project LeadLoving HusbandFreelance PHP ConsultantPHP QA trainerYour host for the upcoming hour
    3. Selenium 34 Claims, Benefits: Prevents cancer Aging
    4. Selenium – the web browserautomation tool set
    5. Acceptance Tests● System tests● Black box● Functional testing
    6. Browser Compatibility Tests● Different Browsers● Different Operating Systems
    7. Regression Tests● Internal Changes● Partner Changes
    8. Tools● Humans● Capture and replay● Scripts and bots● Selenium
    9. Manual Testing● Time-intensive● Nerve-stretching● Human error / understanding of the flow● Costly
    10. Capture and Replay● Captures mouse movements and inputs● Replays later● Screenresolution, operating systems, window sizes, makes this very unmaintainable
    11. Scripts and Bots● Bot makes screen-shots, analyzes them for visual blocks● Calculates visual blocks x-y positions and generates mouse clicks on that location and user input.● Generating a test is time-intensive● Error prone on design changes.
    12. Selenium● Records and playback browser tests● Asserts on DOM● Javascript is being executed and testable● Css selectors or xpath position sensitive● Hard to detect layout errors
    13. Learning Curve● Jump in quickly● Steep – If you want to do it right● Mastering it – creating readable, understandable, maintainable and robust tests - takes time and effort.
    14. Worth it Ask a Salary Increase and own a second modest vacation property
    15. And as an added bonus● Better code architecture● More functional code● High browser compatibility● No more regressions● Neat cool stuff to brag about at your local pub!!!
    16. Setting it up client side
    17. Selenium IDE Firefox Plugin:● Easy record and playback● Intelligent field selection will use IDs, names, or XPath as needed● Autocomplete for all common Selenium commands● Walk through tests● Debug and set breakpoints● Save tests as HTML, PHPUnit, or any other format● Support for Selenium user-extensions.js file● Option to automatically assert the title of every page● Easy customization through plugins
    18. Selenium IDE
    19. Test Suite Control Flow
    20. X-forwarding run software on remote machine and display the GUI locally● Linux and Mac OS X support X Forwarding with no extra software ssh -X● Windows needs extra software Xming
    21. Xming (X-forwarding for windows) Xming Windows Start => Xming => XLaunch Display settings dialog box: Multiple windows Display number as 0 Keep clicking next then Finish.
    22. Putty (windows)● Connection => SSH => X11● Enable X11 forwarding● Set X display location as :0.0
    23. Setting it up server side (headless)
    24. Firefox● [64 bit] wget● [32 bit] wget● tar -jxvf firefox-18.0.tar.bz2 -C /opt● ln -s /opt/firefox/firefox /usr/bin/firefox● Firefox (No X-Display needed thanks to forwarding)
    25. Completely headless (no X-forwarding)● Install Xvfb aptitude install Xvfb● Update to latest java version aptitude install sun-java6-jre● Download the latest Selenium Server Jar package wget server-standalone-2.28.0.jar
    26. Test driving Selenium RC (X-forwarding)To test-drive it is always easier to first use X-forwarding:java -jar selenium-server-standalone-2.28.0.jar -htmlSuite*firefox ""“/path/to/selenium/testSuite.html”“/path/where/selenium/should/store/results.html”testSuite.html and related files are the files saved by Selenium IDE
    27. Test driving Selenium RC (X Virtual Frame Buffer)● Open a connection and run Xvfb Xvfb :99● Open another connection and run selenium export DISPLAY=:99 & java -jar selenium-server- standalone-2.28.0.jar -htmlSuite *firefox "" “/path/to/selenium/testSuite.html” “/path/where/selenium/should/store/results.html”
    28. Integrate into Make certain your Jenkins server is started with Xvfb run on :99 ● Check with: ps -ef | grep Xvfb ● If not running then issue a Xvfb :99 CTRL + Z● Log into Jenkins: manage Jenkins -> Manage Plugins install “Hudson Seleniumhq plugin”
    29. Setup a Job● Create a new freestyle job
    30. Problem● remote control is slow at driving the browser.● limited number of concurrent tests on the same remote control before seriously impacting its stability.● Multiple Selenium Remote Controls In Jenkins to work around the limitation on the number of parallel tests.
    31. The Grid● Run tests against multiple browsers● multiple versions of browser● browsers running on different operating systems● To reduce the time it takes for the test suite to complete a test pass.
    32. The Hub● 1 instance● Allocating a Selenium Remote Control to a specific test (transparently)● Limiting the number of concurrent test runs on each Remote Control● Shielding the tests from the actual grid infrastructure
    33. A Node● Remote Control● Multiple instances (across machines preferably)● Multiple Browsers● Can be configured using a json file
    34. Setting up the grid
    35. ● Start the hub java -jar selenium-server-standalone-2.28.0.jar -role hub● Start nodes on separate machines java -jar selenium-server-standalone-2.28.0.jar -role node -hub http://hub-address:4444/grid/register
    36. http://hub-address:4444/grid/console
    37. Optimizing a Node with js ● java -jar selenium-server-standalone-2.28.0.jar -role node -nodeConfig nodeConfig.js //nodeConfig.js{ "class":"org.openqa.grid.common.RegistrationRequest", "capabilities":[ { "platform":"VISTA", "seleniumProtocol":"Selenium", "browserName":"*firefox", "maxInstances":5 }, { "platform":"VISTA", "seleniumProtocol":"Selenium", "browserName":"*iexplore", "maxInstances":1 } ],
    38. "configuration":{ "port":5555, "register":true, "host":"node-address", "proxy":"org.openqa.grid.selenium.proxy.DefaultRemoteProxy", "maxSession":5, "role":"node", "hubHost":"hub-address", "registerCycle":5000, "hub":"http://hub-address:4444/grid/register", "hubPort":4444, "url":"http://node-address:5555", "remoteHost":"http://node-address:5555" }}
    39. http://hub-address:4444/grid/console
    40. Run the suite● No Native HTML Selenese tests support● Web driver needed● PHPUnit
    41. + Selenium Web Driverwget +x phpunit.phar
    42. Selenium-ide-php-formattersConvert Html Selenese tests to PHPUnit testcasesFirefox Addon:
    43. Export the Test Cases
    44. The test case<?phpclass Example extends PHPUnit_Extensions_SeleniumTestCase{ protected function setUp() { $this->setBrowser("*iexplore"); $this->setBrowserUrl(""); } public function testMyTestCase() { $this->open("/"); $this->assertTrue($this->isElementPresent("link=About")); $this->assertTrue($this->isElementPresent("link=Consulting & Training")); $this->assertTrue($this->isElementPresent("link=Blog")); $this->assertTrue($this->isElementPresent("link=Contact")); $name = $this->getEval(""nick"==="nick""); }}?>
    45. Run the test casephp phpunit.phar testcases/testMenu.php Run the test suite php phpunit.phar testcases
    46. Use multiple browserspublic static $browsers = array( array( name => Firefox, browser => *firefox, host => hub-address, port => 4444, timeout => 30000, ), array( name => Internet Explorer, browser => *iexplore, host => hub-address, port => 4444, timeout => 30000, ));protected function setUp(){ $this->setBrowserUrl(;}
    47. Use raw Html Selenese test public function testMenu() { $this->runSelenese(testMenu.html) }Use raw Html Selenese testsuite public static $seleneseDirectory = /path/to/files;
    48. Taking it Parallel
    49. With Paratest Parallel PHPUnit – running tests concurrently● Support for annotated tests● Inheritance● Tests suites● Test methods● Reflection driven
    50. composer install // composer.json{ "require": { "brianium/paratest": "dev-master", "phpunit/dbunit": ">=1.2", "phpunit/php-invoker": "*", "phpunit/phpunit-selenium": ">=1.2", "phpunit/phpunit-story": "*" }}
    51. PHPUnitParatest + PHPUnit
    52. References● X-Forwarding:● Xvfb● Driving Selenium with PHPUnit● Selenium Hq● Composer● Paratest
    53. Troubleshooting● Firefox install dependencies and-running-firefox-on-debian-linux-getting-missing- dependencies-shared-libs● Specify different ports in Jenkins for each RC when not using the grid● Proxy settings in Browser – use X-Forwarding to run Browser and change settings, use profile as a RC param when needed.
    54. Link to slides and other updates is available at
    55. The End! THANK YOU Please rate my talk, Twitter, IRC: NickBelhomme
    56. Flickr Photo CreditsBackgrounds by Flikr