Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Scaling Jenkins with Docker and Kubernetes


Published on

Docker is revolutionizing the way people think about applications and deployments. It provides a simple way to run and distribute Linux containers for a variety of use cases, from lightweight virtual machines to complex distributed micro-services architectures. Kubernetes is an open source project to manage a cluster of Linux containers as a single system, managing and running Docker containers across multiple Docker hosts, offering co-location of containers, service discovery and replication control. It was started by Google and now it is supported by Microsoft, RedHat, IBM and Docker Inc amongst others. Jenkins Continuous Integration environment can be dynamically scaled by using the Kubernetes and Docker plugins, using containers to run slaves and jobs, and also isolate job execution.

Published in: Software

Scaling Jenkins with Docker and Kubernetes

  1. 1. Scaling Jenkins with Docker and Kubernetes Carlos Sanchez
  2. 2. About me Senior Software Engineer @ CloudBees Author of Jenkins Kubernetes plugin Long time OSS contributor at Apache Maven, Eclipse, Puppet,…
  3. 3. Containers & micro services
  4. 4. @YourTwitterHandle Docker Linux containers Union File System File System Users Processes Network
  5. 5. But it is not trivial
  6. 6. Kubernetes
  7. 7. Kubernetes Container cluster orchestration Docker containers across multiple hosts (nodes or minions) Higher level API Enforced state Monitoring of endpoints
  8. 8. Master Kubernetes API Server scheduling and synchronization etcd Kubernetes Controller Manager Server implements replication algorithm watching etcd
  9. 9. Node Docker Kubelet ensures state of Pods Kubernetes Proxy simple network proxy etcd SkyDNS ElasticSearch + Kibana
  10. 10. Providers GKE Azure Vmware Rackspace oVirt Vagrant CloudStack Ubuntu
  11. 11. Pod Group of colocated containers Same network namespace/IP Environment variables Shared volumes host mounted empty volumes GCE data disks AWS EBS volumes nfs glusterfs secrets
  12. 12. @YourTwitterHandle Pods Node client kubelet pods Master Node kubelet pods Node kubelet pods … … …
  13. 13. Pod kind: "Pod" apiVersion: "v1" metadata: name: "jenkins" labels: name: "jenkins" spec: containers: - name: "jenkins" image: "csanchez/jenkins-swarm:1.625.1- for-volumes" ports: - containerPort: 8080 - containerPort: 50000 volumeMounts: - name: "jenkins-data" mountPath: "/var/jenkins_home" volumes: - name: "jenkins-data" hostPath: path: "/home/docker/jenkins"
  14. 14. Replication controller Ensure a number of pods are running Pod templates Rolling update podpod pod container container container
  15. 15. @YourTwitterHandle Replication controllers Node client podA Master Node podA Node podB podA podB
  16. 16. Replication controller apiVersion: "v1" kind: "ReplicationController" metadata: name: "jenkins" labels: name: "jenkins" spec: replicas: 1 template: metadata: name: "jenkins" labels: name: "jenkins" spec: containers: - name: "jenkins" image: "csanchez/jenkins-swarm:1.625.1-for-volumes" ports: - containerPort: 8080 - containerPort: 50000 volumeMounts: - name: "jenkins-data" mountPath: "/var/jenkins_home" volumes: - name: "jenkins-data" hostPath: path: "/var/jenkins"
  17. 17. Services Pod discovery IP per service Route to pods selected with labels Can create a load balancer in GCE and AWS
  18. 18. @YourTwitterHandle Services client pod Service pod pod
  19. 19. Services apiVersion: "v1" kind: "Service" metadata: name: "jenkins" spec: type: "LoadBalancer" selector: name: "jenkins" ports: - name: "http" port: 80 targetPort: 8080 protocol: "TCP" - name: "slave" port: 50000 protocol: "TCP"
  20. 20. Kubernetes cluster with docker-compose # Docker Compose definition for a one node Kubernetes cluster # Based on Docker Cookbook example # etcd: image: kubernetes/etcd: net: "host" command: /usr/local/bin/etcd --addr= --bind-addr= --data- dir=/var/etcd/data master: image: net: "host" volumes: - /var/run/docker.sock:/var/run/docker.sock command: /hyperkube kubelet --api_servers=http://localhost:8080 --v=2 -- address= --enable_server --hostname_override= --config=/etc/kubernetes/ manifests proxy: image: net: "host" privileged: true command: /hyperkube proxy --master= --v=2
  21. 21. Kubernetes and Jenkins
  22. 22. @YourTwitterHandle Initial solution Pod Jenkins master Replication Controller Slaves slave 3 slave 2Google Persistent Disk slave 1
  23. 23. @YourTwitterHandle slave 2
 (on demand pod) Replication controller Improved solution Replication controller Jenkins master Google Persistent Disk slave 1
 (on demand pod) CloudBees Jenkins Operations Center Google Persistent Disk Replication controller Jenkins master Replication controller Jenkins master Replication controller Jenkins master Replication controllers Jenkins master slave 3
 (on demand pod)
  24. 24. Kubernetes Jenkins plugin
  25. 25. Kubernetes Jenkins plugin As a plugin on demand slaves
  26. 26. Roadmap Stable API When Kubernetes Java lib is stable Using new Jenkins Cloud/Containers APIs Use jobs in Kubernetes 1.1
  27. 27. Example code and slides Available at
  28. 28. Gracias!