Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
and
Present

Automating
the hard
parts, made
easier

Webinar

Copyright © 2014 SOASTA, Inc. - All rights reserved.
Continuous Load Testing
TODAY’S PRESENTERS
Mark Prichard: CloudBees, Dir. Product Management
Mike Ostenberg: SOASTA, Dir. ...
3
Copyright © 2014 SOASTA, Inc. - All rights reserved.
•
•
•
•

Why should we load test earlier?
Building tests and preparing the test environment
Connecting your load tests to ...
5
Copyright © 2014 SOASTA, Inc. - All rights reserved.
6
Copyright © 2014 SOASTA, Inc. - All rights reserved.
7
Copyright © 2014 SOASTA, Inc. - All rights reserved.
“Because it’s hard.”

oTools require coding skills we don’t have (C, VB?)
oWe don’t have a scalable test environment
oWe d...
“Because it’s hard.”

oTools require coding skills we don’t have (C, VB?)
oWe don’t have a scalable test environment
oWe d...
“Because it’s hard.”

oTools require coding skills we don’t have (C, VB?)
oWe don’t have a scalable test environment
oWe d...
“Because it’s hard.”

oTools require coding skills we don’t have (C, VB?)
oWe don’t have a scalable test environment
oWe d...
12
Copyright © 2014 SOASTA, Inc. - All rights reserved.
Iteration 1

Iteration 2

Automated unit tests

Automated unit &
functional tests

Iteration 3

Iteration n

Regression te...
Iteration 1
Automated unit tests

Small perf test

Iteration 2
Automated unit &
functional tests
Small perf tests

Iterati...
Manage with a new metric!

I JUS
COD T MADE
E CH
A
AN G
E

Jenkins launches
suitable cloudbased test
environment

Identify...
16
Copyright © 2014 SOASTA, Inc. - All rights reserved.
Mark
Prichard

@mqprichard

17
Copyright © 2014 SOASTA, Inc. - All rights reserved.
Have you met Jenkins?
o #1 OSS CI server
o Written in Java
o Highly Extensible

o About 8 years old
o Easy to install/use
...
Continuous Integration and Automated
Testing
o Let developers focus on developing

o Leave building, packaging, testing, d...
Setting up the Test Envrionment
• CloudBees ClickStarts – A complete project in just two clicks!
• Bees Shop Demo
• Git re...
o Demo:

• Bees Shop Overview
• SOASTA Cloud Test: Setting up a load test up for unattended runs
• Jenkins Performance Dat...
Mike Ostenberg

22
Copyright © 2014 SOASTA, Inc. - All rights reserved.
TouchTest

CloudTest

Continuous
Testing

Load &
Performance

mPulse
Real User
Monitoring

23
Copyright © 2014 SOASTA, Inc...
SOASTA Jenkins Plug-in
• Open Source: online at jenkins-ci.org
• Automatically available on CloudBees DEV@cloud
• Complete...
Try it out yourself!
• CloudBees ClickStarts – A complete project in just two clicks!
• Example configurations all online:...
Thanks & Q&A
RESOURCES
www.SOASTA.com

Knowledge Center
•White Papers
•Webinar
Recordings
•Case Studies
CloudLink
Communit...
Upcoming SlideShare
Loading in …5
×

Continuous Load Testing with CloudTest and Jenkins

10,423 views

Published on

Two key challenges to continuous load testing are provisioning a test system to handle the load and accessing load generators to drive the traffic.

In this webinar from SOASTA & CloudBees, you will learn how to:

Build realistic automated web performance tests and run them in Jenkins
Architect and launch a test environment that auto-provisions in the cloud
Manage a load generation grid to drive load tests in a lights-out mode
Establish a performance baseline in your daily Jenkins reports

Published in: Technology
  • Be the first to comment

Continuous Load Testing with CloudTest and Jenkins

  1. 1. and Present Automating the hard parts, made easier Webinar Copyright © 2014 SOASTA, Inc. - All rights reserved.
  2. 2. Continuous Load Testing TODAY’S PRESENTERS Mark Prichard: CloudBees, Dir. Product Management Mike Ostenberg: SOASTA, Dir. Systems Engineering Brad Johnson: SOASTA, VP Product TWO COMPANIES, A SINGLE SOLUTION - the leader in Mobile and Cloud Testing - the home of Jenkins in the Cloud Together, we offer a complete cloud-based service: Continuous Delivery for Web & Mobile 2 Copyright © 2014 SOASTA, Inc. - All rights reserved.
  3. 3. 3 Copyright © 2014 SOASTA, Inc. - All rights reserved.
  4. 4. • • • • Why should we load test earlier? Building tests and preparing the test environment Connecting your load tests to your CI (Jenkins) Establishing and managing to a performance baseline • Q&A Questions: Submit in the question box during event 4 Copyright © 2014 SOASTA, Inc. - All rights reserved.
  5. 5. 5 Copyright © 2014 SOASTA, Inc. - All rights reserved.
  6. 6. 6 Copyright © 2014 SOASTA, Inc. - All rights reserved.
  7. 7. 7 Copyright © 2014 SOASTA, Inc. - All rights reserved.
  8. 8. “Because it’s hard.” oTools require coding skills we don’t have (C, VB?) oWe don’t have a scalable test environment oWe don’t have hardware for load generation oIt’s a different tool oThere are too many other tests to do 8 Copyright © 2014 SOASTA, Inc. - All rights reserved.
  9. 9. “Because it’s hard.” oTools require coding skills we don’t have (C, VB?) oWe don’t have a scalable test environment oWe don’t have hardware for load generation oIt’s a different tool oThere are too many other tests to do “Because it takes too long.” oScripting takes forever oResults are slow oFast setup of environments is impossible oWe’re dependent on IT or other teams oWe’re too agile 9 Copyright © 2014 SOASTA, Inc. - All rights reserved.
  10. 10. “Because it’s hard.” oTools require coding skills we don’t have (C, VB?) oWe don’t have a scalable test environment oWe don’t have hardware for load generation oIt’s a different tool oThere are too many other tests to do “Because it takes too long.” oScripting takes forever oResults are slow oFast setup of environments is impossible oWe’re dependent on IT or other teams oWe’re too agile “Because no one ever told us to.” oThere are no “performance coverage” requirements oThere isn’t a performance baseline for reference oThere isn’t a defined performance target oNo one is managing to a performance metric 10 Copyright © 2014 SOASTA, Inc. - All rights reserved.
  11. 11. “Because it’s hard.” oTools require coding skills we don’t have (C, VB?) oWe don’t have a scalable test environment oWe don’t have hardware for load generation oIt’s a different tool oThere are too many other tests to do “Because it takes too long.” oScripting takes forever oResults are slow oFast setup of environments is impossible oWe’re dependent on IT or other teams oWe’re too agile “Because no one ever told us to.” oThere are no “performance coverage” requirements oThere isn’t a performance baseline for reference oThere isn’t a defined performance target oNo one is managing to a performance metric Until Now. 11 Copyright © 2014 SOASTA, Inc. - All rights reserved.
  12. 12. 12 Copyright © 2014 SOASTA, Inc. - All rights reserved.
  13. 13. Iteration 1 Iteration 2 Automated unit tests Automated unit & functional tests Iteration 3 Iteration n Regression test suite Regression test suite Load & Performan ce Testing Deploy Issues ? The Issue: • High expectations from tests run with diminishing time windows • Code-level performance bugs found late set entire project back • Separate team and schedule conflict with delivery goals • Major issues escape to production 13 Copyright © 2014 SOASTA, Inc. - All rights reserved.
  14. 14. Iteration 1 Automated unit tests Small perf test Iteration 2 Automated unit & functional tests Small perf tests Iteration 3 Regression test suite Incl: Iteration n Regression test suite Incl: Perf smoke tests Perf smoke tests Deploy End to End Production Scale Test Production perf tests DevOps Development/Test team Performance/Ops team The Solution: • Run incremental performance tests automatically with each iteration • Establish a trend line for performance early • Developers find code and architecture issues early • Performance teams focuse on end-to-end infrastructure and user related testing • Minimize production escapes and implement production testing 14 Copyright © 2014 SOASTA, Inc. - All rights reserved.
  15. 15. Manage with a new metric! I JUS COD T MADE E CH A AN G E Jenkins launches suitable cloudbased test environment Identify bottlenecks & inform teams, just like unit and functional tests in Jenkins Daily (or more frequent) updates keep teams “performance focused” CloudTest runs load tests via Jenkins 15 Copyright © 2014 SOASTA, Inc. - All rights reserved.
  16. 16. 16 Copyright © 2014 SOASTA, Inc. - All rights reserved.
  17. 17. Mark Prichard @mqprichard 17 Copyright © 2014 SOASTA, Inc. - All rights reserved.
  18. 18. Have you met Jenkins? o #1 OSS CI server o Written in Java o Highly Extensible o About 8 years old o Easy to install/use o Extensible via 1000+ plugins o Widely adopted o 70K+ installations o CloudBees knows Jenkins o On-premise, cloud and hybrid o Expert support o Enterprise plugins
  19. 19. Continuous Integration and Automated Testing o Let developers focus on developing o Leave building, packaging, testing, distributing to a computer o Make the right thing the easiest thing – don’t think about it o Many, widely-adopted plugins o Safety in numbers – hugely active community o Dashboard view of the state of your builds o You don’t have to be an expert at everything o Test Anywhere o Deploy on-premise or in the cloud for testing o Spin up VMs or EC2 Instances as build executors o We can do it for you, if you like…
  20. 20. Setting up the Test Envrionment • CloudBees ClickStarts – A complete project in just two clicks! • Bees Shop Demo • Git repo, database, build and deploy in seconds • Online examples: https://partnerdemo.ci.cloudbees.com
  21. 21. o Demo: • Bees Shop Overview • SOASTA Cloud Test: Setting up a load test up for unattended runs • Jenkins Performance Data: • Reviewing Performance Trending Data: Track performance of each build • Reviewing Performance Pass/Fail Data: Failing • Setting up Continuous Load Testing • Install Plot Plugin • Download GitHub: https://github.com/mostenberg/SOASTAPERFPERL • Add steps to plot build data for any metrics of interest. • Publish Junit test results to PASS/FAIL based on performance thresholds: SOASTA/5_PERF_THRESHOLD_RESULTS.xml 21 Copyright © 2014 SOASTA, Inc. - All rights reserved.
  22. 22. Mike Ostenberg 22 Copyright © 2014 SOASTA, Inc. - All rights reserved.
  23. 23. TouchTest CloudTest Continuous Testing Load & Performance mPulse Real User Monitoring 23 Copyright © 2014 SOASTA, Inc. - All rights reserved.
  24. 24. SOASTA Jenkins Plug-in • Open Source: online at jenkins-ci.org • Automatically available on CloudBees DEV@cloud • Complete control over CloudTest/TouchTest Automation from within your Jenkins builds: • Totally integrated into Jenkins environment
  25. 25. Try it out yourself! • CloudBees ClickStarts – A complete project in just two clicks! • Example configurations all online: • https://partnerdemo.ci.cloudbees.com/ • Bees-Shop-PerformanceTest
  26. 26. Thanks & Q&A RESOURCES www.SOASTA.com Knowledge Center •White Papers •Webinar Recordings •Case Studies CloudLink Community •Support •Tutorials •Video Contact SOASTA: www.soasta.com/cloudtest/ info@soasta.com 866.344.8766 Follow us: www.CloudBees.com blog.cloudbees.com •Look for Mark’s post on this topic developer.cloudbees.com •Platform Services •Frameworks •ClickStarts Contact CloudBees www.cloudbees.com info@cloudbees.com 1.781.404.5100 Follow us: twitter.com/cloudtest twitter.com/cloudbees facebook.com/cloudtest facebook.com/CloudBees 26 Copyright © 2014 SOASTA, Inc. - All rights reserved.

×