Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Continuous Performance Testing


Published on

The presentation reviews various tools and libraries which help us add continuous performance testing to the development process. Using Hudson CI to manage our build system, JUnit based performance tests, and JMeter and Grinder load tests. See demo project @

Published in: Technology
  • Be the first to like this

Continuous Performance Testing

  1. 1. Continuous Performance Testing
  2. 2. What you are about to witness ● Continious integration bla bla ● What is hudson? ● Performance Unit tests ● Load testing with Jmeter ● Load testing with Grinder
  3. 3. Continuous Integration? ● Scheduled, automated build ● Know of problems early on ● Have a reasonably stable product at any time ● Track various metric (tests, coverage, static code analysis, coding conventions)
  4. 4. Hudson CI ● A breeze to install, simple to configure ● Architecture: – Triggers (scheduling) – Hook to source control – Build tools (executable, ant, maven) – Report generators (test results) – Notifiers (email, RSS, etc.) ● Other goodies: fingerprinting, distribution, ...
  5. 5. Continuous performance testing? ● Track regressions in product performance ● Meet specified performance requirements ● Put guards for bugs resulting in errors threads ● Establish a benchmark for the application ● See (and show) progress
  6. 6. Performance Unit tests ● Junit – not very much (timeout?) ● Roll your own ● Test decorators to help Threading, logging, more detailed requirements: – JUnitPerf (Junit 3.5 – decorate suits() – JUnitBench – ContiPerf
  7. 7. When to test units for performance? ● Sanity tests ● Performance critical units ● Before making changes that might cause regressions ● Note: Junit can be harnessed for sub- system tests, not just units
  8. 8. System tests ● Try to emulate ”Real life” conditions: Test plan, deployment ● Focus on finding out if something is wrong, not what ● Check how to application as a whole is working ● Build to scale when needed ● Update the test as your application changes (example – cached responses)
  9. 9. JMeter ● Tests are written in UI ● Various samplers: Http, DB, java ● Complicated tests are achived with different test plan elements, scripts, plugins ● Distributed agents ● Hudson plugin
  10. 10. Grinder ● Generic and minimal ( == simple) approach ● Jython (python) scripts based ● Instrument tested code ● Instrument your call ● Http - out of the box support ● Distributed test support via console
  11. 11. What we did not cover ● Complex testing architecutre (distributed testing, multiple deployment targets) ● Measuring server metrics (i.e. CPU, IO, Memory) ● Micro performance testing – instrumented code, profiling
  12. 12. Amitay Dobo Http://