How To Introduce Cloud Based Load Testing to Your Jenkins Continuous Delivery Pipeline

1 © 2015 SOASTA Inc.- All rights reserved.
In This Webinar
TODAY’S PRESENTERS

Tom Chavez: SOASTA, Sr. Product MarkeEng Manager
Andy Pemberton: CloudBees, Senior Director, SoluEons Architecture & ConsulEng

TWO COMPANIES, A SINGLE SOLUTION

SOASTA: The leader in Cloud and Mobile TesEng
CloudBees: The Enterprise Jenkins Company

Together, we oﬀer a complete cloud-based service:
ConEnuous Load TesEng & ConEnuous Delivery with Jenkins

QuesEons?
Ask in QuesEon window

Poll QuesEon
What best describes your status with ConEnuous IntegraEon / ConEnuous
Delivery (CI/CD)? (single answer)

a)  Determining if CI/CD is a good opEon
a)  ImplemenEng CI/CD
a)  PracEcing CI
b)  PracEcing CD
c)  What is CI/CD?

Agenda

•  Load tesEng: why?
•  Why don’t we load test earlier?
•  ConEnuous Load and Performance TesEng
•  CloudBees and the Enterprise Jenkins PlaYorm
•  ConnecEng load tests to your CD Pipeline
•  Q&A

QuesEons:
Submit in the quesEon box during event

Why Should We Load Test?
o  Ensure that site will perform well at load
•  Test hardware configuraEons
•  Test applicaEon, database, components
•  Test third party partners
o  To be ready for peak season
•  Black Friday, Cyber Monday, Christmas
•  Tax filing, Insurance filing, SuperBowl commercial

Why Should We Load Test?

7 © 2015 SOASTA, Inc.-All rights reserved.
Why Don’t We Load Test Earlier?
Copyright © 2014 SOASTA, Inc. - All rights reserved.

Why DON’T We Load Test Earlier?
“Because it’s hard.”
o  Tools require coding skills we don’t have (C, VB?)
o  We don’t have a scalable test environment
o  We don’t have hardware for load generation
o  It’s a different tool
o  There are too many other tests to do

“Because it takes too long.”
o  Scripting takes forever
o  Results are slow
o  Fast setup of environments is impossible
o  We’re dependent on IT or other teams
o  We’re too agile

“Because no one ever told us to.”
o  There are no “performance coverage” requirements
o  There isn’t a performance baseline for reference
o  There isn’t a defined performance target
o  No one is managing to a performance metric

“Because no one ever told us to.”
o  There are no “performance coverage” requirements
o  There isn’t a performance baseline for reference
o  There isn’t a defined performance target
o  No one is managing to a performance metric
Until Now.

Poll QuesEon
What do you see holding back a more iteraEve load tesEng approach? (check all
that apply)

a)  Skills gaps
b)  Lack of test environment(s)
c)  Slow test development
d)  Test tools limitaEons
e)  3rd Party Dependencies
f)  Undeﬁned Performance Requirements
g)  Lack of Management Push
h)  Other

Load TesEng Has Been an Elite Event
SThe Issue:
• High expectaEons from tests run with diminishing Eme windows
• Code-level performance bugs found late set enEre project back
• Separate team and schedule conﬂict with delivery goals
• Major issues escape to producEon
Iteration 1
Automated unit
tests
Iteration 2
Automated unit &
functional tests
Iteration 3
Regression test
suite
Iteration n
Regression test
suite
Load &
Performance
Testing
Deploy
Issues?

ConEnuous Load TesEng
Validate performance piece by piece
Iteration 1
Automated unit
tests
Deploy
Production perf
tests
Iteration 2
Automated unit &
functional tests
Iteration 3
Regression test
suite
Incl:
Iteration n
Regression test
suite
Incl:
End to End
Production Scale
Test
Small perf test Small perf tests
Perf smoke tests Perf smoke tests
The SoluEon:
•  Run incremental performance tests automaEcally with each iteraEon
•  Establish a trend line for performance early
•  Developers ﬁnd code and architecture issues early
•  Performance teams focus on end-to-end infrastructure and user related tesEng
•  Minimize producEon escapes and implement producEon tesEng
Development/Test team Performance/Ops teamDevOps

ConEnuous Performance (for Web & Mobile)
The Path to a Performance Culture
I JUST MADE A
CODE CHANGE
CloudTest runs load
tests via JenkinsDaily (or more frequent) updates keep
teams “performance focused”
Identify
bottlenecks &
inform teams,
just like unit
and functional
tests in
Jenkins
Jenkins launches
suitable cloud-
based test
environment
Manage with a new metric!

Poll QuesEon
What types of Load TesEng are you doing today? (Single Answer)

a)  Not doing any load tesEng
b)  Non-Web Protocols (not mobile)
c)  Primarily web protocols
d)  Primarily mobile web traﬃc
e)  Both web and mobile
f)  All web, mobile & non-web protocols

17 © 2015 SOASTA, Inc. All rights reserved.
Enabling ConEnuous Performance
TesEng with Jenkins Pipeline
Andy Pemberton
Twitter: @apemberton
Email: apemberton@cloudbees.com

Have you met Jenkins?
#1 CI/CD Automation Engine
•  Open Source
•  Highly Extensible
About 8 years old
Easy to install/use
1000+ plugins
Widely adopted
•  100K+ installations

Jenkins is the #1 Automation Platform Used for CI
Continuous Integration
Developer
Source
Code
Control
System
Compile
/BuildCode
Scan TestsCode
Commit
Results
Report
BuildCommit Test

Jenkins is Also Your Key to Continuous Delivery
Developer
Source
Code
Control
System
Compile
/BuildCode
Scan TestsCode
Commit
Results
Report
BuildCommit Test Stage Deploy
Deploy
Prod
Deploy
Stage
Complex Delivery Pipelines
Delivery of App and Config
Continuous Delivery
ProdDev

Over 1000 Jenkins Plugins for Tool Integration
Integrated Development Environments
(IDEs), editors/compilers/debuggers,
code review and collaboration tools.
Source code control
systems and related asset
management tools.
Automated API-based
testing frameworks.
Release
management and
release automation
software
Automated API-based
testing frameworks, static
code analysis tools.
Load Testing Tools
and Security Scanning
technology
Automated and
manual UAT tools
Release management,
release automation, and
change control software.
Environment and Configuration
Management SoftwareDevelop
Commit
Build &
Integrate
Unit Tests
Release
(to Test)
Automated
Regression
Tests
Release
(to Prod)
User
Acceptance
Tests
Load &
Security
Tests

Jenkins is the Hub of the CD Ecosystem
On-premise or
in the cloud
Plug-ins for all
your tools
RunDeployStageTestBuildCommit

Are You Using Today?
23
Chances are someone in your
organization is.

24

25
How Do You Manage Jenkins and Testing at
Enterprise Scale?

About CloudBees
Jenkins founder, Kohsuke Kawaguchi, CloudBees CTO
We optimize Jenkins for the enterprise
•  Jenkins Operations Center
•  Jenkins OSS Support
•  Enterprise-class plugins
•  Commit code and support the Jenkins Community
•  Jenkins training and consulting
•  Jenkins User Conferences (JUC) and newsletters
…we are the Jenkins Enterprise experts!

CloudBees Jenkins Enterprise
27
Professional support
High availability
Large installations
Security
Optimized utilization
Continuous Delivery

Additional Jenkins Services from CloudBees
28
Jenkins Certified Training
•  Delivered via CloudBees partners
Jenkins Consulting
•  Delivered via CloudBees Professional Services Team
Jenkins Newsletter
•  Community news and expert tips
Jenkins User Conferences
•  Learn from other community members

How Can CloudBees Jenkins Enterprise Help with Your CD
Strategy?
29
PipelineStage View
Provides deep insight into the execution of your workflow stages so
you can detect problems and performance issues
Manage complex “level-of-detail” testing pipelines
Jenkins Pipeline
Created by CloudBees to provide Jenkins with a way to define
flexible delivery pipelines that allow for a rich set of new
delivery scenarios
Manage and share “testing-as-code”
Dynamic Environment Creation
Spin up new test/deployment environments automatically as part of
your delivery pipeline
Deployment Tracking and Verfiication
Track the history of your deployments – including who kicked
off a deployment and what code changes were included in it
Create pipelines that leverage verification tests to determine
the quality of a new deployment
Jenkins Analytics
Monitor and analyze performance across all your Jenkins instances
Monitor, measure and manage adoption of testing, CI and CD best
practicesRestart builds, tests, and deployments where they
break
Leverage pipeline checkpoints to restart delivery pipelines
without a full (expensive) rebuild
...central source of orchestration to bridge dev, test, and ops alike
And more…

#Jenkins2
Greatly Enhanced UI/UX for easier startup and beOer security

#Jenkins2
Jenkins Pipeline-as-Code front and center

Jenkins Pipeline and Pipeline Stage View Enable Full
Continuous Delivery Pipelines
Continuous Delivery
ProdDev
Developer
Source
Code
Control
System
Code
Commit

33
CloudBees Jenkins Pipeline Stage View
Restartable Checkpoints
Artifact Downloads

Monitoring, Visualization and Analytics
Jenkins Pipeline Stage View
Visibility into Pipeline status
Jenkins Analytics
Cross-Team Build Analytics
… among many other features, tailored for enterprise
organizations using Jenkins @ scale

ConEnuous Load TesEng
o  SOASTA Cloud Test Components / Architecture
o  Demo:
•  CloudTest UI Overview
•  Jenkins Setup
•  Downloading and installing CloudTest plug-in
•  CreaEng a Performance Job in Jenkins
•  CreaEng a Pipeline in Jenkins
•  Reviewing Results
What are we going to show

Architecture:
Jenkins
Jill
SriRaj
DevOps Team 1
CloudTest
Manager
Always up at:
Cloudtestmanager.soasta.com
CloudTestserver
any IP Address/ Domainname
Up ‘as needed’ at
“US Grid1”
2 Azure California
2 AWS Virginia
2x Google Iowa
“WW Grid1”
2 Azure California
2 AWS Virginia
2x Google Iowa
CloudTest“Main” server
dev.acme.com
CPU,DISK,
Memory,I/O
Etc.

Jenkins Enterprise on
CloudBees
Jill
SriRaj
DevOps Team 1
CloudTest
Manager
Always up at:
Cloudtestmanager.soasta.com
dev.acme.com
“US Grid1”
2 Azure California
2 AWS Virginia
2x Google Iowa
CloudTest“Main” server
Demo Preview:

Demo

Resources
Contact SOASTA:
www.soasta.com/cloudtest/
info@soasta.com
866.344.8766
Follow us:
twiOer.com/SoastaInc
facebook.com/cloudtest
Knowledge Center
•  White Papers
•  Webinar Recordings
•  Case Studies
CloudLink Community
•  Support
•  Tutorials
•  Video

Contact CloudBees
www.cloudbees.com
info@cloudbees.com
1.781.404.5100
Follow us:
twiOer.com/cloudbees
facebook.com/CloudBees
www.SOASTA.com

www.CloudBees.com
Try it @ www.cloudbees.com/get-started

Documenta]on
•  CloudBees Jenkins Platform
•  Workflow in Jenkins
White Papers:
•  Making the Leap from Continuous Integration to
Continuous Delivery
•  The Business Value of Continuous Delivery

How to Introduce Cloud Based Load TesEng
to your Jenkins ConEnuous Delivery Pipeline
April 20, 2016
Tom Chavez, SOASTA @TomChavez
Andy Pemberton, CloudBees @apemberton

How To Introduce Cloud Based Load Testing to Your Jenkins Continuous Delivery Pipeline

More Related Content

What's hot

Viewers also liked

Similar to How To Introduce Cloud Based Load Testing to Your Jenkins Continuous Delivery Pipeline

More from Jennifer Finney

Recently uploaded

How To Introduce Cloud Based Load Testing to Your Jenkins Continuous Delivery Pipeline