This is a series that we will dedicate to the CI/CD pipeline and how you can use tools from ClusterHQ to help gain fungibility, efficiency and mobility to your CI/CD infrastructure and build pipelines. The series will start with a focus on running Jenkins and how to provide flexibility and mobility running your master and its associated configuration. Then, the series will slowly move into other topics such as limiting non-deterministic failures, fungibility for slaves, greater efficiency for popular plug-ins and overall a more deterministic and repeatable Jenkins infrastructure.

1. Running your Jenkins
infrastructure with ClusterHQ
@RyanWallner
@ClusterHQ

2. We will cover...
● What is Flocker?
● Why use Flocker with your Jenkins Master?
● 3 main ways to run Jenkins according to DockerHub
● Fungible Jenkins Master
● Separate volumes
● Backups
@RyanWallner
@ClusterHQ

3. Flocker is a container data volume manager that allows
stateful services to run inside containers in production.
@RyanWallner
@ClusterHQ

4. Data associated with your Jenkins build pipeline
● Logs
● Job output
● Job configuration
● Update data
● Artifacts
● Libraries
● More!
@RyanWallner
@ClusterHQ

5. Running your Jenkins master in a container?
Consider how this data is resilient and mobile when the
container restarts, gets rescheduled or when the host running
Jenkins fails.
@RyanWallner
@ClusterHQ

6. Three main ways to run Jenkins according to DockerHub...
1. All of your data resides inside the container’s own
file-system storage. Most vulnerable to loss.
@RyanWallner
@ClusterHQ

7. 2. Your data is placed into a Docker volume. This means that
the data is within /var/lib/docker/volumes/ but won’t be safe if
you lose your Docker host.
Three main ways to run Jenkins according to DockerHub...
@RyanWallner
@ClusterHQ

8. Three main ways to run Jenkins according to DockerHub...
3. Mounting in a directory that exists on the Docker host
outside of the Docker file-system. Leaves data vulnerable to
node failure and container movement issues.
@RyanWallner
@ClusterHQ

9. Fungible Jenkins Master
Start a Jenkins master
with a Flocker volume
using Docker Compose
@RyanWallner
@ClusterHQ

10. Fungible Jenkins Master
Find your Jenkins admin password by running a command
on your Jenkins container
@RyanWallner
@ClusterHQ

11. Fungible Jenkins Master
You will be prompted to intall plugins. All plugins will
be stored in /var/jenkins_home/plugins within your
Flocker volume that’s mounted at /var/jenkins_home/
@RyanWallner
@ClusterHQ

12. Fungible Jenkins Master
You have now added some data to your Jenkins home
directory.
@RyanWallner
@ClusterHQ

13. We can also verify that our Flocker volume is mounted so all
this data is safely persisted to our EBS Flocker volume.
Fungible Jenkins Master
@RyanWallner
@ClusterHQ

14. Fungible Jenkins Master
Now ready to handle failover. Instruct docker-compose
to stop and rm -f the Jenkins build and let it start on a
new host.
@RyanWallner
@ClusterHQ

15. Fungible Jenkins Master
Add a scheduling filter telling it not to deploy to the
server.
@RyanWallner
@ClusterHQ

16. Fungible Jenkins Master
Restart your Jenkins master backup.
See it on a new node.
@RyanWallner
@ClusterHQ
Note: I am using a fake IP in the ouput,
yours would be different

17. More options...
@RyanWallner
@ClusterHQ

18. Separate Volumes
Create separate
volumes for
different parts of
Jenkins config
@RyanWallner
@ClusterHQ

19. Using a Flocker volume that had plugins pre-loaded,
plugin installation times decreased 4X.
Separate Volumes
@RyanWallner
@ClusterHQ

20. Jenkins Backup Plugin
Docker Compose file
with a separate
backup volume that
can be used to store
our Jenkins home.
@RyanWallner
@ClusterHQ

21. Backups
After you run this Jenkins master, you can configure
the backup manager with your Flocker volume.
@RyanWallner
@ClusterHQ

22. Backups
Recovering your
Jenkins master by
expanding a backup
in a Flocker volume
and using it at your
Jenkins home.
@RyanWallner
@ClusterHQ

23. You can use the
jenkins_backups
folder as your
$JENKINS_HOME
Backups
@RyanWallner
@ClusterHQ

24. Bring your Jenkins server
up and you can login like
normal from a point in
time backup from a
Flocker backup volume.
Backups
@RyanWallner
@ClusterHQ

25. Stay tuned for more on running your CI/CD infrastructure
with container-based data management tools from
ClusterHQ.
@RyanWallner
@ClusterHQ