Distributed Testing Environment


Published on

Distributed Testing Environment with Jenkins and virtual machines. Presented on 3-rd PTaQ meetup.

Published in: Technology
  • you can check the https://loadfocus.com JMeter Load and Performance service in the cloud on AWS, it's reliable and supports thousands of concurrent users
    Are you sure you want to  Yes  No
    Your message goes here
  • For distributed testing you can also use this plugin https://wiki.jenkins-ci.org/display/JENKINS/BlazeMeter+Plugin. You do not need configure slaves to run distributed load testing as all tests run on configured cloud servers.
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Distributed Testing Environment

  1. 1. Distributed TestingEnvironmentŁukasz Morawski
  2. 2. The Stories• John The Testeris a manual tester,• He is ordered todo an automation,• So he does,• He develop sometests, he runs iton his box
  3. 3. The Stories• The more tests he develops, the more time tests lasts,• At first, it takes couple minutes,• Later, it takes dozen,• It is a lot of time!He cannot afford tostare at running tests
  4. 4. ...and the crucial is...
  5. 5. ...and the crucial is
  6. 6. The problem• QA team needs to provide test feedback as soon as it’s possible,• Automated tests need to be “Fast and Furious”.
  7. 7. Distributed testing environmenton virtual machines.
  8. 8. ...but at the beginning there was         Eternity...• server hosting Hudson,• test execution testbed forSelenium IDE tests,• but one executor meansone test at a time.
  9. 9. VM and Hyper-V• QA team uses Hyper-Vvirtualization software,• It let us to run ~14 VM,• CPU is not the problem,• Memory is the problem,• 80% free memory rule – swapping leads to serious performance lost,• We have GEB and BES servers available for us:• BES – manual tests environments for testers and developers (different OS, different browsers, mobiles),
  10. 10. VM and Hyper-V• GEB – automated tests environments.• Slaves for us and our clients,• Jenkins,• Applications – QA Lab uses this machine to provide tools and services for our test framework. So far we have W3C Markup Validation service – our Validation Tool uses it.
  11. 11. VM and Hyper-V All machines are under Nagios supervision, It let us to control if the environment is up and ready to go, Notifies by e-mail if something is wrong.
  12. 12. VM Future• More tools available for our testing framework (W3C tools),• Different browsers/browsers version,• Maybe different virtualization system – bare metal hyper- visors seems to look promising,• Cloud – as a main resource, as a backup resource.
  13. 13. Centralized Automated testdriving with Jenkins CI• QA Team use Jenkins CI tool to drive tests,• We have Jenkins deployed in a Tomcat container,• We use SLAVE AGENTS as a communication channel,• It’s not greatest solution because CI tools are rather build tools not testing tools.
  14. 14. How we use Jenkins...• We use Views to manage projects
  15. 15. How we use Jenkins... • We use “distributed builds” feature to manage parallel testing (speed up) with several slaves,
  16. 16. How we use Jenkins... • We use labels to nickname a slaves (ff36, ff4, winxp).• We use “Restrict where this project can be run” to configure a test job run.
  17. 17. How we use Jenkins...• For performance testing we use JMeter and JMeter plugin for Jenkins.
  18. 18. How we use Jenkins...• We use “Configuration Matrix” for some jobs.
  19. 19. How we use Jenkins...• We use “Discard old builds” to save disk space.
  20. 20. How we use Jenkins... • We use Jenkins Remote Access API, • We use “Trigger build remotely” to run build from outside:• ...and tools like curl/wget to trigger it:curl --user user:password JENKINS/view/project/job/build?token=token
  21. 21. How we use Jenkins...• We try follow some time line guidelines: 00:00 to 08:00 - Automated/scheduled tests, nightly test, 08:00 to 21:00 - work day, manual/scheduled test, 21:00 to 00:00 - daily/weekly maintenance time,
  22. 22. Maintenance problem• We suffer with maintenance problem while managing so many VM – we use STAF for performing actions on all machines.
  23. 23. Maintenance problem• These machines lives and to prevent deterioration we enforce some maintenance tasks.
  24. 24. Maintenance problem• Daily tasks (21:00): Weekly tasks (21:00 Sunday):• Restart, Cleaner,• Set screen resolution,• Resources. Defragmentation, Anti-Virus Scan, Windows updates.
  25. 25. Jenkins future• Browsers auto updating,• Dynamic slave management,• Custom framework – leave Jenkins bottlenecks behind and build your own,
  26. 26. The answer• Distributed testing environment on virtual machines,• Centralized automated test driving with Jenkins CI.
  27. 27. Benefits of presented solution• Easy way to extend test environment (just clone VM and voila),• ...which let us to speed up tests by splitting and paralleling,• Easy way to restore corrupted system (revert snapshot, copy VM disk file),• HQ for managing tests with different tools (Selenium, Webdriver, Jmeter, Wapiti, Test Complete, AutoIt...),• Built-in features and plugins (e.g.: SVN client),