...but at the beginning there was Eternity...• server hosting Hudson,• test execution testbed forSelenium IDE tests,• but one executor meansone test at a time.
VM and Hyper-V• QA team uses Hyper-Vvirtualization software,• It let us to run ~14 VM,• CPU is not the problem,• Memory is the problem,• 80% free memory rule – swapping leads to serious performance lost,• We have GEB and BES servers available for us:• BES – manual tests environments for testers and developers (different OS, different browsers, mobiles),
VM and Hyper-V• GEB – automated tests environments.• Slaves for us and our clients,• Jenkins,• Applications – QA Lab uses this machine to provide tools and services for our test framework. So far we have W3C Markup Validation service – our Validation Tool uses it.
VM and Hyper-V All machines are under Nagios supervision, It let us to control if the environment is up and ready to go, Notifies by e-mail if something is wrong.
VM Future• More tools available for our testing framework (W3C tools),• Different browsers/browsers version,• Maybe different virtualization system – bare metal hyper- visors seems to look promising,• Cloud – as a main resource, as a backup resource.
Centralized Automated testdriving with Jenkins CI• QA Team use Jenkins CI tool to drive tests,• We have Jenkins deployed in a Tomcat container,• We use SLAVE AGENTS as a communication channel,• It’s not greatest solution because CI tools are rather build tools not testing tools.
How we use Jenkins...• We use Views to manage projects
How we use Jenkins... • We use “distributed builds” feature to manage parallel testing (speed up) with several slaves,
How we use Jenkins... • We use labels to nickname a slaves (ff36, ff4, winxp).• We use “Restrict where this project can be run” to configure a test job run.
How we use Jenkins...• For performance testing we use JMeter and JMeter plugin for Jenkins.
How we use Jenkins...• We use “Configuration Matrix” for some jobs.
How we use Jenkins...• We use “Discard old builds” to save disk space.
How we use Jenkins... • We use Jenkins Remote Access API, • We use “Trigger build remotely” to run build from outside:• ...and tools like curl/wget to trigger it:curl --user user:password JENKINS/view/project/job/build?token=token
How we use Jenkins...• We try follow some time line guidelines: 00:00 to 08:00 - Automated/scheduled tests, nightly test, 08:00 to 21:00 - work day, manual/scheduled test, 21:00 to 00:00 - daily/weekly maintenance time,
Maintenance problem• We suffer with maintenance problem while managing so many VM – we use STAF for performing actions on all machines.
Maintenance problem• These machines lives and to prevent deterioration we enforce some maintenance tasks.
Jenkins future• Browsers auto updating,• Dynamic slave management,• Custom framework – leave Jenkins bottlenecks behind and build your own,
The answer• Distributed testing environment on virtual machines,• Centralized automated test driving with Jenkins CI.
Benefits of presented solution• Easy way to extend test environment (just clone VM and voila),• ...which let us to speed up tests by splitting and paralleling,• Easy way to restore corrupted system (revert snapshot, copy VM disk file),• HQ for managing tests with different tools (Selenium, Webdriver, Jmeter, Wapiti, Test Complete, AutoIt...),• Built-in features and plugins (e.g.: SVN client),