Using Jenkins Enterprise to effectively manage the Jenkins ecosystem across the Enterprise. The document discusses using Jenkins Enterprise and Job DSL plugins to automate the creation of Jenkins infrastructure including folders, jobs and views/pipelines. It proposes treating all Jenkins build and deployment infrastructure as code by defining application details in a JSON file and using a Groovy job DSL framework to automatically generate the infrastructure from the JSON file. This would standardize the Jenkins environment and reduce manual setup efforts.

1. Using Jenkins Enterprise to effectively manage the Jenkins
ecosystem across the Enterprise
Sarah Banks, Kostas Gaitanos, Raj Menattamai

2. Footer
#jenkinsconfAbout FINRA www.finra.org
2
Financial Industry
Regulatory Authority
Ø Largest
independent
regulator
for
all
securi2es
firms
doing
business
in
the
U.S.
Ø >4,000
brokerage
firms
Ø ~161,000
branch
offices
Ø ~637,700
registered
securi2es
representa2ves
Our Mission:
Ø Investor
Protec2on
Ø Market
Integrity
Our
Technology
is
powerful
enough
to
look
across
markets
and
detect
poten2al
fraud.
We
oversee
50
billion
market
transac2ons
every
day
In
2014
,
we
expelled
18
firms
from
the
industry,
suspended
705
brokers
and
barred
481
brokers
from
doing
business.
We
also
fined
firms
more
than
$134
million,
and
ordered
res2tu2on
of
$32.3
million
to
investors
who
had
been
harmed.
Computerized
cer:fica:on
&
con:nued
educa:on.
Series
7,
63…etc.
Educating
& informing investors
Actively supporting
firms’ compliance efforts
Providing independent,
vigorous regulation
Inviting active
industry involvement
& input

3. Footer
#jenkinsconfMetrics for Jenkins Enterprise based ecosystem activities
# of Apps # of Folders # of Jobs # of Builds/yr
(estimate)
# of
Deployments/yr
(estimate)
~110 ~610 ~1,400* ~100K
promotable
builds
~75K (across all
envs)
3
* # of jobs active over the past quarter; valid jobs requiring maintenance ~ 2.5K
>100 plugins currently utilized

4. Footer
#jenkinsconfDiscussion areas
4
• Enable deployments to higher environments via the build
pipeline
– Development have the ability to deploy to all lower environments
including QC
– Enable Prod Deployment group to execute production deployments
using the exact same approach as for the lower environments.
• Treat all of Jenkins build-deploy infra as software/source
code
– Automatic creation of all infra (i.e folders, jobs, views/pipelines)
– Use the same framework to create all that make up an application

5. Footer
#jenkinsconfEnable deployments to higher environments via the build pipeline
5

6. Footer
#jenkinsconfEnable deployments to higher environments via the build pipeline
6
Folder Template when instantiated will create:
v Deployment jobs for lower environments
• Job template to create standard XL Deploy deployment job *
v Deployment job for QA with a child job “promote_to_prod”
• Keep the build forever
• Job template to create standard XL Deploy deployment job AND a child job *
v Deployment job for Prod
• Job template to create a Prod specific XL Deploy deployment job *
v PROD folder with a Prod Deployment job
§ RBAC is set
§ promote_to_prod & prod_deploy jobs are created *
* jobs are created based on corresponding job templates

7. Footer
#jenkinsconfFolder template groovy code handling RBAC
7
• <com.cloudbees.hudson.plugins.folder.Folder plugin="cloudbees-folder@4.5">
• <actions/>
• <description></description>
• <properties>
• <com.cloudbees.jenkins.plugins.foldersplus.SecurityGrantsFolderProperty plugin="cloudbees-folders-plus@2.9">
• <securityGrants/>
• </com.cloudbees.jenkins.plugins.foldersplus.SecurityGrantsFolderProperty>
• <com.cloudbees.hudson.plugins.folder.properties.FolderCredentialsProvider_-FolderCredentialsProperty>
• <domainCredentialsMap class="hudson.util.CopyOnWriteMap$Hash">
• <entry>
• <com.cloudbees.plugins.credentials.domains.Domain plugin="credentials@1.16.1">
• <specifications/>
• </com.cloudbees.plugins.credentials.domains.Domain>
• <java.util.concurrent.CopyOnWriteArrayList/>
• </entry>
• </domainCredentialsMap>
• </com.cloudbees.hudson.plugins.folder.properties.FolderCredentialsProvider_-FolderCredentialsProperty>
• <com.cloudbees.hudson.plugins.folder.properties.EnvVarsFolderProperty plugin="cloudbees-folders-plus@2.9">
• <properties></properties>
• </com.cloudbees.hudson.plugins.folder.properties.EnvVarsFolderProperty>
• <com.cloudbees.hudson.plugins.folder.properties.SubItemFilterProperty plugin="cloudbees-folders-plus@2.9"/>
• <com.cloudbees.hudson.plugins.folder.properties.FolderProxyGroupContainer plugin="nectar-rbac@4.12">
• <groups>
• <nectar.plugins.rbac.groups.Group>
• <name>PPL_ROLE_TEC_ART_LOCAL</name>
• <member>PPL_ROLE_TEC_ART</member>
• <role>develop_prod</role>
• </nectar.plugins.rbac.groups.Group>
• </groups>
• </com.cloudbees.hudson.plugins.folder.properties.FolderProxyGroupContainer>
• </properties>

8. Footer
#jenkinsconfTreat all of Jenkins build-deploy infra as software/source code
8
The goal is to create all of our build-deploy infrastructure automatically:
Ø Folders
Ø Jobs
Ø Views/Pipelines
v Create a single framework ( groovy job DSL) for all application-components that
make an application
v All of the application related info (i.e folders, environments, pipeline views,
components, parameters for plugins used etc…) is captured in a JSON file
v JSON file is the input to the framework, as the framework effectively becomes the
“seed” job; when “seed” job runs it will create all of folders, jobs views/pipelines for
the application

9. Footer
#jenkinsconf
9
Properties JSON
"AGS": {
"scm": "svn",
"name": "ENTERPRISEDESKTOP",
"rootFolder": "FLDR_ENTERPRISEDESKTOP_DSL_DEMO",
"DevQCDeployJobFolder":FLDR_ENTERPRISEDESKTOP_DSL_DEMO/deployment_jobs",
"DevQCDeployJobFolderTemplate": "FLDR_DEPLOY_JOBS_use_with_Job_DSL",
"ProdDeployJobFolder": "FLDR_ENTERPRISEDESKTOP_DSL_DEMO/deployment_jobs/
PROD_JOBS",
"ProdDeployJobFolderTemplate": "FLDR_DEPLOY_JOBS_use_with_Job_DSL/PROD_JOBS",
"DevQC": [
{ "name": "DEV_deploy" },
:
:
JobDSLmethods
(class Folders {
static void createParentFolders(def asgProperties, def AGS, def AGSFolder,…)
{ Folders.createAGSFolder(AGSFolder, AGS)
:
:
}
static void createAGSFolder(def folder, def AGS) {
folder.with {
:
: :
}
JobDSL Framework
import jobdslmethods.enterprise.folder.*
import groovy.json.JsonSlurper
:
:
Folders.createParentFolders(asgProperties, AGS, AGSFolder, DeploymentJobFolder, DeploymentJobProdFolder)
:

10. Footer
#jenkinsconfMetrics discussion
10
Template Plugins Manual
setup of pipeline * 5 mins 30 mins
~4 hrs
Job DSL & Template
Plugins
Manual
Jenkins infra as
code^
20 mins 4-5 hours
* Times are per job; on an average each app contains 12-13 jobs
~ 1week upfront investment to design/implement
^ metrics captured based on applications with ~20 jobs/pipelines
~3 weeks upfront investment to design/implement