This document discusses using Docker for continuous integration and continuous delivery (CI/CD) processes. It covers using Docker for coding, building, and running Jenkins with Docker. Docker provides benefits like easy to prepare development environments, faster builds with no library dependencies, increased robustness, and compatibility with existing CI tools. The document provides examples of building Docker from within a Docker container, running Jenkins to build Docker images, and running Jenkins in Docker containers with plugins for building and executing tasks inside Docker.

1. CI/CD with Docker
Henry Huang

2. Agengda
• Docker brings benefits on CI/CD process
• Coding: IDE with Docker
• Build with Docker
• Jenkins with Docker
– 1 Master x Slaves
– Masters x Slaves

3. Docker Provides
• Easy to prepare dev build environments
• No languages/libraries dependencies
• Build starts in seconds - more faster
• Increased robustness
• Simplified deployment on CI nodes
• Faster Rollback & Canary Release
• Easy to scale on CI nodes
• Well compatible with existing CI tools

4. CI/CD Process

5. Coding
• IDE with Docker
– Can run your code any where
If it runs on your laptop it will run on the integration
server too
– Don't have to bother about libraries, conflicts or
installing GCC, etc.
– Easy to build with different language packs or
libraries, etc.

6. Coding
• Requirements for IDE with Docker
– Dockerfile
• Syntax highlighting
• Autocomplete
• Syntax validation
– Compose yml file
• Define inter container relation as links and volumes
• Run multiple containers with one click
– IDE build system
• Run containers from the IDE with different language packs
or library requirement, etc.
– IDE runtime system
• Launch runtime environment inside the container for
further troubleshooting and testing

10. Docker Plugin for IDE
• Doclipser
• Sublime Docker
• IntelliJ IDEA 14.1
• Eclipse JBossTools
• Visual Studio 2015 RCTools for Docker -
Preview extension
Summary Page: http://domeide.github.io/

11. Build
• Currently, we cannot meet the CI principles:
– Every commit (to baseline) should be built
– Keep the build fast
– Make it easy to get the latest deliverables
• Problems:
– [Dragged] Update dev build environment
– [Dragged/Fragile] Build the source files
– [Complicated] Get the latest/specified
deliverables

12. Build with Docker
• Easy to prepare dev build environments
• No languages/libraries dependencies
• Build starts in seconds - more faster
• Increased robustness
• Simplified deployment on CI nodes
• Faster Rollback & Canary Release
• Easy to scale on CI nodes
• Well compatible with existing CI tools

13. Example: Way of Building Docker
• Chicken and Egg
– build the latest docker binary in old version of docker container
• Dockerfile: provides the necessary dependencies and
environment to build
– COPY source files into Docker build image
• hack/make.sh: provides the build script for
– binary, cross, unit test, integration-cli-test, etc.
• Verify the new version via dind (docker-in-docker)
• Deliverables
– Docker Images
– Binary
• mounting host folder to the dev container
• Via `docker cp`

14. dind - build docker
# git clone https://github.com/docker/docker.git
# cd docker; docker build -t docker-dev .
# docker run --privileged --rm -ti docker-dev /bin/bash
root@383dd9ee7613:/go/src/github.com/docker/docker# hack/make.sh binary
---> Making bundle: binary (in bundles/1.8.0-dev/binary)
Building: bundles/1.8.0-dev/binary/docker-1.8.0-dev
Created binary: bundles/1.8.0-dev/binary/docker-1.8.0-dev
root@383dd9ee7613:/go/src/github.com/docker/docker# docker
bash: docker: command not found
… …
root@383dd9ee7613:/go/src/github.com/docker/docker# docker -dD
root@383dd9ee7613:/go/src/github.com/docker/docker# docker run hello-world

15. dind - hello world

16. Jenkins to build docker
• https://jenkins.dockerproject.org/job/Docker%20Master/

17. Docker Registry
• Docker Registry
– V1: named as registry (python, boto)
• https://github.com/docker/docker-registry
– V2: named as distribution (re-write from scratch, golang)
• https://github.com/docker/distribution
• Storage model
– inmemory
– Filesystem
– S3
– azure: Microsoft Azure Blob Storage
– rados: Ceph Object Storage
• Frontend UI
– https://github.com/kwk/docker-registry-frontend
– No supportV2

18. Docker Registry
# http://docs.docker.com/registry/deploying/
• Take Dockerfile
– HTTP
• Add “--insecure-registry” into docker daemon options
# docker pull private-registry.iwsaas:5000/centos:latest
– HTTPS: SSL private key and certificate
• Take Compose
– Extra data volumes, etc.

19. Multiple Registries
• Mirroring (still in proposal)
• Cross repository (included in the v2.1 milestone)

20. Jenkins with Docker
• Build/Push docker images
– Execute docker CLI in jobs: case #1
– Docker build step plugin: case #2
• https://wiki.jenkins-ci.org/display/JENKINS/Docker+build+step+plugin
• Execute tasks inside docker container
– Docker plugin (dind): case #3
• https://wiki.jenkins-ci.org/display/JENKINS/Docker+Plugin
– Patched docker plugin (master and slave in the same
container level): case #4
• https://github.com/henrysher/docker-plugin

21. Demo for case 4
• Add new options into docker daemon marked in red:
OPTIONS='-H tcp://0.0.0.0:4243 -H unix:///var/run/docker.sock --selinux-
enabled'
• Run the docker image “iwsaas/jenkins” as a Jenkins container
# docker run -d -p 8080:8080 --name jenkins iwsaas/jenkins
b675af71332782a9bbfe6d92469d9a280ae751593b460520ac6bd6ab047b8b70
• Configure Docker Plugins
– Docker URL(docker0 virtual bridge address)
• http://172.17.42.1:4243
– DockerTemplate (iwsaas/jenkins-slave)
• Default username/password: jenkins/jenkins
• Create a “test” job and execute some basic commands

24. Case 4 - Console Output
• Execute the command inside the container
– Container ID: e380cfb050a1

25. Jenkins Problems
• Multiple Masters
– Sync & Standby
• Resource isolation & allocation (for slaves)
– Multiple tasks running on one slave
– CPU, Memory, Disk, etc. per task?

26. Mesos + Marathon +Docker+ Jenkins
• Mesos: kernel of distribute systems
• Marathon: init/upstart for mesos

27. Mesos + Marathon + Docker + Jenkins
• Mesos: kernel of distribute systems
• Marathon: init/upstart for mesos