Continuous Load Testing with CloudTest and Jenkins

10,180 views

Published on

Two key challenges to continuous load testing are provisioning a test system to handle the load and accessing load generators to drive the traffic.

In this webinar from SOASTA & CloudBees, you will learn how to:

Build realistic automated web performance tests and run them in Jenkins
Architect and launch a test environment that auto-provisions in the cloud
Manage a load generation grid to drive load tests in a lights-out mode
Establish a performance baseline in your daily Jenkins reports

Published in: Technology
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
10,180
On SlideShare
0
From Embeds
0
Number of Embeds
2,457
Actions
Shares
0
Downloads
86
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

Continuous Load Testing with CloudTest and Jenkins

  1. 1. and Present Automating the hard parts, made easier Webinar Copyright © 2014 SOASTA, Inc. - All rights reserved.
  2. 2. Continuous Load Testing TODAY’S PRESENTERS Mark Prichard: CloudBees, Dir. Product Management Mike Ostenberg: SOASTA, Dir. Systems Engineering Brad Johnson: SOASTA, VP Product TWO COMPANIES, A SINGLE SOLUTION - the leader in Mobile and Cloud Testing - the home of Jenkins in the Cloud Together, we offer a complete cloud-based service: Continuous Delivery for Web & Mobile 2 Copyright © 2014 SOASTA, Inc. - All rights reserved.
  3. 3. 3 Copyright © 2014 SOASTA, Inc. - All rights reserved.
  4. 4. • • • • Why should we load test earlier? Building tests and preparing the test environment Connecting your load tests to your CI (Jenkins) Establishing and managing to a performance baseline • Q&A Questions: Submit in the question box during event 4 Copyright © 2014 SOASTA, Inc. - All rights reserved.
  5. 5. 5 Copyright © 2014 SOASTA, Inc. - All rights reserved.
  6. 6. 6 Copyright © 2014 SOASTA, Inc. - All rights reserved.
  7. 7. 7 Copyright © 2014 SOASTA, Inc. - All rights reserved.
  8. 8. “Because it’s hard.” oTools require coding skills we don’t have (C, VB?) oWe don’t have a scalable test environment oWe don’t have hardware for load generation oIt’s a different tool oThere are too many other tests to do 8 Copyright © 2014 SOASTA, Inc. - All rights reserved.
  9. 9. “Because it’s hard.” oTools require coding skills we don’t have (C, VB?) oWe don’t have a scalable test environment oWe don’t have hardware for load generation oIt’s a different tool oThere are too many other tests to do “Because it takes too long.” oScripting takes forever oResults are slow oFast setup of environments is impossible oWe’re dependent on IT or other teams oWe’re too agile 9 Copyright © 2014 SOASTA, Inc. - All rights reserved.
  10. 10. “Because it’s hard.” oTools require coding skills we don’t have (C, VB?) oWe don’t have a scalable test environment oWe don’t have hardware for load generation oIt’s a different tool oThere are too many other tests to do “Because it takes too long.” oScripting takes forever oResults are slow oFast setup of environments is impossible oWe’re dependent on IT or other teams oWe’re too agile “Because no one ever told us to.” oThere are no “performance coverage” requirements oThere isn’t a performance baseline for reference oThere isn’t a defined performance target oNo one is managing to a performance metric 10 Copyright © 2014 SOASTA, Inc. - All rights reserved.
  11. 11. “Because it’s hard.” oTools require coding skills we don’t have (C, VB?) oWe don’t have a scalable test environment oWe don’t have hardware for load generation oIt’s a different tool oThere are too many other tests to do “Because it takes too long.” oScripting takes forever oResults are slow oFast setup of environments is impossible oWe’re dependent on IT or other teams oWe’re too agile “Because no one ever told us to.” oThere are no “performance coverage” requirements oThere isn’t a performance baseline for reference oThere isn’t a defined performance target oNo one is managing to a performance metric Until Now. 11 Copyright © 2014 SOASTA, Inc. - All rights reserved.
  12. 12. 12 Copyright © 2014 SOASTA, Inc. - All rights reserved.
  13. 13. Iteration 1 Iteration 2 Automated unit tests Automated unit & functional tests Iteration 3 Iteration n Regression test suite Regression test suite Load & Performan ce Testing Deploy Issues ? The Issue: • High expectations from tests run with diminishing time windows • Code-level performance bugs found late set entire project back • Separate team and schedule conflict with delivery goals • Major issues escape to production 13 Copyright © 2014 SOASTA, Inc. - All rights reserved.
  14. 14. Iteration 1 Automated unit tests Small perf test Iteration 2 Automated unit & functional tests Small perf tests Iteration 3 Regression test suite Incl: Iteration n Regression test suite Incl: Perf smoke tests Perf smoke tests Deploy End to End Production Scale Test Production perf tests DevOps Development/Test team Performance/Ops team The Solution: • Run incremental performance tests automatically with each iteration • Establish a trend line for performance early • Developers find code and architecture issues early • Performance teams focuse on end-to-end infrastructure and user related testing • Minimize production escapes and implement production testing 14 Copyright © 2014 SOASTA, Inc. - All rights reserved.
  15. 15. Manage with a new metric! I JUS COD T MADE E CH A AN G E Jenkins launches suitable cloudbased test environment Identify bottlenecks & inform teams, just like unit and functional tests in Jenkins Daily (or more frequent) updates keep teams “performance focused” CloudTest runs load tests via Jenkins 15 Copyright © 2014 SOASTA, Inc. - All rights reserved.
  16. 16. 16 Copyright © 2014 SOASTA, Inc. - All rights reserved.
  17. 17. Mark Prichard @mqprichard 17 Copyright © 2014 SOASTA, Inc. - All rights reserved.
  18. 18. Have you met Jenkins? o #1 OSS CI server o Written in Java o Highly Extensible o About 8 years old o Easy to install/use o Extensible via 1000+ plugins o Widely adopted o 70K+ installations o CloudBees knows Jenkins o On-premise, cloud and hybrid o Expert support o Enterprise plugins
  19. 19. Continuous Integration and Automated Testing o Let developers focus on developing o Leave building, packaging, testing, distributing to a computer o Make the right thing the easiest thing – don’t think about it o Many, widely-adopted plugins o Safety in numbers – hugely active community o Dashboard view of the state of your builds o You don’t have to be an expert at everything o Test Anywhere o Deploy on-premise or in the cloud for testing o Spin up VMs or EC2 Instances as build executors o We can do it for you, if you like…
  20. 20. Setting up the Test Envrionment • CloudBees ClickStarts – A complete project in just two clicks! • Bees Shop Demo • Git repo, database, build and deploy in seconds • Online examples: https://partnerdemo.ci.cloudbees.com
  21. 21. o Demo: • Bees Shop Overview • SOASTA Cloud Test: Setting up a load test up for unattended runs • Jenkins Performance Data: • Reviewing Performance Trending Data: Track performance of each build • Reviewing Performance Pass/Fail Data: Failing • Setting up Continuous Load Testing • Install Plot Plugin • Download GitHub: https://github.com/mostenberg/SOASTAPERFPERL • Add steps to plot build data for any metrics of interest. • Publish Junit test results to PASS/FAIL based on performance thresholds: SOASTA/5_PERF_THRESHOLD_RESULTS.xml 21 Copyright © 2014 SOASTA, Inc. - All rights reserved.
  22. 22. Mike Ostenberg 22 Copyright © 2014 SOASTA, Inc. - All rights reserved.
  23. 23. TouchTest CloudTest Continuous Testing Load & Performance mPulse Real User Monitoring 23 Copyright © 2014 SOASTA, Inc. - All rights reserved.
  24. 24. SOASTA Jenkins Plug-in • Open Source: online at jenkins-ci.org • Automatically available on CloudBees DEV@cloud • Complete control over CloudTest/TouchTest Automation from within your Jenkins builds: • Totally integrated into Jenkins environment
  25. 25. Try it out yourself! • CloudBees ClickStarts – A complete project in just two clicks! • Example configurations all online: • https://partnerdemo.ci.cloudbees.com/ • Bees-Shop-PerformanceTest
  26. 26. Thanks & Q&A RESOURCES www.SOASTA.com Knowledge Center •White Papers •Webinar Recordings •Case Studies CloudLink Community •Support •Tutorials •Video Contact SOASTA: www.soasta.com/cloudtest/ info@soasta.com 866.344.8766 Follow us: www.CloudBees.com blog.cloudbees.com •Look for Mark’s post on this topic developer.cloudbees.com •Platform Services •Frameworks •ClickStarts Contact CloudBees www.cloudbees.com info@cloudbees.com 1.781.404.5100 Follow us: twitter.com/cloudtest twitter.com/cloudbees facebook.com/cloudtest facebook.com/CloudBees 26 Copyright © 2014 SOASTA, Inc. - All rights reserved.

×