Mastering selenium for automated acceptance tests

20,530 views

Published on

Published in: Technology
0 Comments
16 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
20,530
On SlideShare
0
From Embeds
0
Number of Embeds
1,317
Actions
Shares
0
Downloads
0
Comments
0
Likes
16
Embeds 0
No embeds

No notes for slide
  • System: tests request, router, controller, view, response Black box: you do not have to be aware of the code behind the system. And thus you do not write your tests accordingly Functional: tests complete gui and behavior
  • System: tests request, router, controller, view, response Black box: you do not have to be aware of the code behind the system. And thus you do not write your tests accordingly Functional: tests complete gui and behavior
  • System: tests request, router, controller, view, response Black box: you do not have to be aware of the code behind the system. And thus you do not write your tests accordingly Functional: tests complete gui and behavior
  • Mastering selenium for automated acceptance tests

    1. Nick Belhomme January 26th, PHPBenelux Conference 2013, Belgium
    2. Nick BelhommeInternational Conference SpeakerContributor to various Open SourceProjectsBuilding ConstructorSoftware Architect / Project LeadLoving HusbandFreelance PHP ConsultantPHP QA trainerYour host for the upcoming hour
    3. Selenium 34 Claims, Benefits: Prevents cancer Aging
    4. Selenium – the web browserautomation tool set
    5. Acceptance Tests● System tests● Black box● Functional testing
    6. Browser Compatibility Tests● Different Browsers● Different Operating Systems
    7. Regression Tests● Internal Changes● Partner Changes
    8. Tools● Humans● Capture and replay● Scripts and bots● Selenium
    9. Manual Testing● Time-intensive● Nerve-stretching● Human error / understanding of the flow● Costly
    10. Capture and Replay● Captures mouse movements and inputs● Replays later● Screenresolution, operating systems, window sizes, makes this very unmaintainable
    11. Scripts and Bots● Bot makes screen-shots, analyzes them for visual blocks● Calculates visual blocks x-y positions and generates mouse clicks on that location and user input.● Generating a test is time-intensive● Error prone on design changes.
    12. Selenium● Records and playback browser tests● Asserts on DOM● Javascript is being executed and testable● Css selectors or xpath position sensitive● Hard to detect layout errors
    13. Learning Curve● Jump in quickly● Steep – If you want to do it right● Mastering it – creating readable, understandable, maintainable and robust tests - takes time and effort.
    14. Worth it Ask a Salary Increase and own a second modest vacation property
    15. And as an added bonus● Better code architecture● More functional code● High browser compatibility● No more regressions● Neat cool stuff to brag about at your local pub!!!
    16. Setting it up client side
    17. Selenium IDE Firefox Plugin: http://release.seleniumhq.org/selenium-ide/1.10.0/selenium-ide-1.10.0.xpi● Easy record and playback● Intelligent field selection will use IDs, names, or XPath as needed● Autocomplete for all common Selenium commands● Walk through tests● Debug and set breakpoints● Save tests as HTML, PHPUnit, or any other format● Support for Selenium user-extensions.js file● Option to automatically assert the title of every page● Easy customization through plugins
    18. Selenium IDE
    19. Test Suite Control Flow
    20. X-forwarding run software on remote machine and display the GUI locally● Linux and Mac OS X support X Forwarding with no extra software ssh -X● Windows needs extra software Xming
    21. Xming (X-forwarding for windows)http://sourceforge.net/projects/xming/files/latest/downloadConfigure Xming Windows Start => Xming => XLaunch Display settings dialog box: Multiple windows Display number as 0 Keep clicking next then Finish.
    22. Putty (windows)● Connection => SSH => X11● Enable X11 forwarding● Set X display location as :0.0
    23. Setting it up server side (headless)
    24. Firefox● [64 bit] wget http://download-origin.cdn.mozilla.net/pub/mozilla.org/firefox/relea● [32 bit] wget http://download-origin.cdn.mozilla.net/pub/mozilla.org/firefox/relea● tar -jxvf firefox-18.0.tar.bz2 -C /opt● ln -s /opt/firefox/firefox /usr/bin/firefox● Firefox (No X-Display needed thanks to forwarding)
    25. Completely headless (no X-forwarding)● Install Xvfb aptitude install Xvfb● Update to latest java version aptitude install sun-java6-jre● Download the latest Selenium Server Jar package wget http://selenium.googlecode.com/files/selenium- server-standalone-2.28.0.jar
    26. Test driving Selenium RC (X-forwarding)To test-drive it is always easier to first use X-forwarding:java -jar selenium-server-standalone-2.28.0.jar -htmlSuite*firefox "http://www.domaintocheck.com"“/path/to/selenium/testSuite.html”“/path/where/selenium/should/store/results.html”testSuite.html and related files are the files saved by Selenium IDE
    27. Test driving Selenium RC (X Virtual Frame Buffer)● Open a connection and run Xvfb Xvfb :99● Open another connection and run selenium export DISPLAY=:99 & java -jar selenium-server- standalone-2.28.0.jar -htmlSuite *firefox "http://www.domaintocheck.com" “/path/to/selenium/testSuite.html” “/path/where/selenium/should/store/results.html”
    28. Integrate into Make certain your Jenkins server is started with Xvfb run on :99 ● Check with: ps -ef | grep Xvfb ● If not running then issue a Xvfb :99 CTRL + Z● Log into Jenkins: manage Jenkins -> Manage Plugins install “Hudson Seleniumhq plugin”
    29. Setup a Job● Create a new freestyle job
    30. Problem● remote control is slow at driving the browser.● limited number of concurrent tests on the same remote control before seriously impacting its stability.● Multiple Selenium Remote Controls In Jenkins to work around the limitation on the number of parallel tests.
    31. The Grid● Run tests against multiple browsers● multiple versions of browser● browsers running on different operating systems● To reduce the time it takes for the test suite to complete a test pass.
    32. The Hub● 1 instance● Allocating a Selenium Remote Control to a specific test (transparently)● Limiting the number of concurrent test runs on each Remote Control● Shielding the tests from the actual grid infrastructure
    33. A Node● Remote Control● Multiple instances (across machines preferably)● Multiple Browsers● Can be configured using a json file
    34. Setting up the grid
    35. ● Start the hub java -jar selenium-server-standalone-2.28.0.jar -role hub● Start nodes on separate machines java -jar selenium-server-standalone-2.28.0.jar -role node -hub http://hub-address:4444/grid/register
    36. http://hub-address:4444/grid/console
    37. Optimizing a Node with js ● java -jar selenium-server-standalone-2.28.0.jar -role node -nodeConfig nodeConfig.js //nodeConfig.js{ "class":"org.openqa.grid.common.RegistrationRequest", "capabilities":[ { "platform":"VISTA", "seleniumProtocol":"Selenium", "browserName":"*firefox", "maxInstances":5 }, { "platform":"VISTA", "seleniumProtocol":"Selenium", "browserName":"*iexplore", "maxInstances":1 } ],
    38. "configuration":{ "port":5555, "register":true, "host":"node-address", "proxy":"org.openqa.grid.selenium.proxy.DefaultRemoteProxy", "maxSession":5, "role":"node", "hubHost":"hub-address", "registerCycle":5000, "hub":"http://hub-address:4444/grid/register", "hubPort":4444, "url":"http://node-address:5555", "remoteHost":"http://node-address:5555" }}
    39. http://hub-address:4444/grid/console
    40. Run the suite● No Native HTML Selenese tests support● Web driver needed● PHPUnit
    41. + Selenium Web Driverwget http://pear.phpunit.de/get/phpunit.pharchmod +x phpunit.phar
    42. Selenium-ide-php-formattersConvert Html Selenese tests to PHPUnit testcasesFirefox Addon:https://addons.mozilla.org/en-US/firefox/addon/selenium-ide-php-formatters/
    43. Export the Test Cases
    44. The test case<?phpclass Example extends PHPUnit_Extensions_SeleniumTestCase{ protected function setUp() { $this->setBrowser("*iexplore"); $this->setBrowserUrl("http://www.nickbelhomme.com/"); } public function testMyTestCase() { $this->open("/"); $this->assertTrue($this->isElementPresent("link=About")); $this->assertTrue($this->isElementPresent("link=Consulting & Training")); $this->assertTrue($this->isElementPresent("link=Blog")); $this->assertTrue($this->isElementPresent("link=Contact")); $name = $this->getEval(""nick"==="nick""); }}?>
    45. Run the test casephp phpunit.phar testcases/testMenu.php Run the test suite php phpunit.phar testcases
    46. Use multiple browserspublic static $browsers = array( array( name => Firefox, browser => *firefox, host => hub-address, port => 4444, timeout => 30000, ), array( name => Internet Explorer, browser => *iexplore, host => hub-address, port => 4444, timeout => 30000, ));protected function setUp(){ $this->setBrowserUrl(http://www.example.com/);}
    47. Use raw Html Selenese test public function testMenu() { $this->runSelenese(testMenu.html) }Use raw Html Selenese testsuite public static $seleneseDirectory = /path/to/files;
    48. Taking it Parallel
    49. With Paratest Parallel PHPUnit – running tests concurrently● Support for annotated tests● Inheritance● Tests suites● Test methods● Reflection driven
    50. composer install // composer.json{ "require": { "brianium/paratest": "dev-master", "phpunit/dbunit": ">=1.2", "phpunit/php-invoker": "*", "phpunit/phpunit-selenium": ">=1.2", "phpunit/phpunit-story": "*" }}
    51. PHPUnitParatest + PHPUnit
    52. References● X-Forwarding: http://en.gentoo-wiki.com/wiki/X-Forwarding http://www.straightrunning.com/XmingNotes/● Xvfb http://www.x.org/releases/X11R7.6/doc/man/man1/Xvfb.1.xhtml● Driving Selenium with PHPUnit http://www.phpunit.de/manual/3.8/en/selenium.html● Selenium Hq http://seleniumhq.org/docs/index.jsp● Composer http://getcomposer.org● Paratest http://brianscaturro.com/2012/11/02/parallel-phpunit.html
    53. Troubleshooting● Firefox install dependencies http://www.thinkplexx.com/learn/howto/linux/system/installing- and-running-firefox-on-debian-linux-getting-missing- dependencies-shared-libs● Specify different ports in Jenkins for each RC when not using the grid● Proxy settings in Browser – use X-Forwarding to run Browser and change settings, use profile as a RC param when needed.
    54. Link to slides and other updates is available athttp://www.facebook.com/NickBelhommeFanPage
    55. The End! THANK YOU Please rate my talk https://joind.in/7796contact@nickbelhomme.comSlideshare, Twitter, IRC: NickBelhommehttp://blog.nickbelhomme.com
    56. Flickr Photo CreditsBackgrounds by Flikr http://www.flickr.com/photos/flikr/146538982/ http://www.flickr.com/photos/flikr/147274362/

    ×