This document discusses continuous deployment of Clojure services to Kubernetes using Docker and GitLab CI. It provides an overview of Docker, Kubernetes, deploying a sample Clojure service, and configuring GitLab CI for continuous integration and deployment. The sample Clojure service is built as a Docker image, tested using GitLab CI, and deployed to Kubernetes clusters for testing and production using configuration files and GitLab CI pipelines.

1. Continuous Deployment
with Kubernetes, Docker
and GitLab CI
@alexander_kiel
Clojure Berlin 2016

3. Outline
• Continuous Deployment why?
• Docker
• Kubernetes
• Sample Clojure Service
• Deploy with GitLabCI

4. Continuous Deployment
• What do we want?
• Increase responsiveness
• Decrease time to market
• Gain confidence by deploying often in small amounts
• How to achieve that?
• Automate everything
• Always deploy the master into production
• Use feature toggles when needed

5. Simple Git Workflow
• Works for in-house apps
• not for libs or shipping apps
• No versions, no tags, just SHA’s
• Latest commit on master is always
deployed to production
• Feature/fix branches are merged
when ready
master
feature/fix
branches
1ebb95d
be61dda
6e4010d

6. Docker
• Like VM’s but much more light-weight and shippable
• Runs on Linux, executes processes in an isolated environment
(resource limitation, filesystem, network)
• Container principle: Can contain everything, but looks the
same from the outside
• A container platform can run every container
• Developers have max. freedom what to do
• In contrast: PaaS like Heroku - has to support the language

7. Kubernetes
• Container runtime platform
• Originally designed by Google - now Open Source
• One of the most active projects on GitHub - 20,000
stars, 40,000 commits, 15,000 issues, 200 releases
• Alternatives: Apache Mesos, Docker Swarm (lacks
features)

8. Kubernetes Architecture
k8s-master-1
k8s-master-2
k8s-master-3
load-balancer-1
load-balancer-2
DNS RR
k8s-worker-1
proxy
app-1
k8s-worker-2
proxy
app-2
k8s-worker-n
proxy
app-k
etcd cluster
quorum
HAProxy
• Runs on VMware ESX
• CoreOS Linux
• Single YAML file as configuration
• Everything in containers

9. Kubernetes - Pods
• A Pod is a deployable unit in
Kubernetes
• Pods can contain multiple
containers
• Containers inside a Pod share
on port space, can use
localhost and can
communicate via IPC and
shared memory
• Idea: one process per
container - many cooperating
processes in one Pod
apiVersion: v1
kind: Pod
metadata:
name: <pod-name>
labels:
<key>: <value>
spec:
containers:
- name: <container-name>
image: <container-image>
ports:
- containerPort: 80
env:
- name: <key>
value: <value>

10. Kubernetes - Deployments
• A Deployment ensures that
certain number of Pods are
always running
• It consists of a Pod template
and the number of replicas
• It supports hot-redeployments
by changing parts of the Pod
template
• Horizontal scaling is possible
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: <deployment-name>
spec:
replicas: 2
template:
metadata:
labels:
<key>: <value>
spec:
containers:
- name: <container-name>
image: <container-image>
ports:
- containerPort: 80
env:
- name: <key>
value: <value>

11. Kubernetes - Services
• Kubernetes uses an overlay
network to provide different address
spaces (we use flannel)
• Every Pod has an IP address - but it
changes every time one is created
• Services provide a stable IP
address for groups of Pods
• Service names are resolvable by an
internal DNS
• Service selectors are used to match
Pods according to there labels
apiVersion: v1
kind: Service
metadata:
name: clojure-berlin-2016
labels:
app: lens
spec:
type: NodePort
ports:
- port: 80
targetPort: 80
protocol: TCP
selector:
service: clojure-
berlin-2016

12. Kubernetes - External Access
• Kubernetes networks are internal
only
• External access through load
balancers necessary
• Certain Platforms like Google
Compute Engine provide load
balancer integration with Kubernetes
• We have our own solution as a
combination of HAProxy and
Kubernetes NodePort
• Kubernetes Services with type
NodePort are exposed on every
worker under a certain port
frontend http
bind 0.0.0.0:80
mode http
option httplog
acl host_clj hdr(host)
clj.<domain>
use_backend clj if host_clj
backend clj
mode http
balance roundrobin
option httplog
server worker-1 <ip>:32599 check
server worker-2 <ip>:32599 check

13. Deployment Lifecycle
GitLab CI
Source Code
build
test
Kubernetes
Test
Cluster
Kubernetes
Prod
Cluster
automatic deployment
manual
deployment
git
push

14. Sample Clojure Service
• .gitlab-ci.yml
• Like .travis.yml contains instructions for GitLabCI
how to test, build and deploy
• Dockerfile
• Instructions for Docker how to build the image of
the app
• Artifact of the build is a docker image - not
uberjar
• kube-deployment.yml
• Kubernetes deployment instructions
• kube-svc.yml
• Kubernetes service description
https://github.com/alexanderkiel/clojure-berlin-2016

15. The Core Namespace
(ns clojure-berlin-2016.core
(:require [aleph.http :as http]
[clojure.core.async :refer [<!! chan]]))
(defn -main [& args]
(-> (fn [_]
{:status 200
:body "Clojure Berlin 2016"})
(http/start-server {:port 8080}))
(<!! (chan)))
• A simple web server returning "Clojure Berlin 2016"

16. The Leiningen Project File
(defproject clojure-berlin-2016 "<VERSION>"
:dependencies [[aleph "0.4.1"]
[org.clojure/clojure "1.8.0"]
[org.clojure/core.async "0.2.395"]]
:main clojure-berlin-2016.core)
• <VERSION> is replaced at build time by the Git SHA
• :main is for lein run to work

17. .gitlab-ci.yml - test/build
image: clojure:lein-2.7.1
stages:
- test
- build
- deploy
test:
stage: test
tags:
- docker
script:
- lein test
build:
stage: build
tags:
- docker
script:
- sed -i "s/<VERSION>/$CI_BUILD_REF/" project.clj
- docker build -t clojure-berlin-2016:$CI_BUILD_REF .
- docker push clojure-berlin-2016:$CI_BUILD_REF

18. .gitlab-ci.yml - deploy branch
deploy-branch:
stage: deploy
environment: test
image: dreg.life.uni-leipzig.local/kubectl:0.4
tags:
- docker
script:
- sed -i "s/<VERSION>/$CI_BUILD_REF/" kube-deployment.yml
- kubectl config use-context gitlab-ci-test
- kubectl apply -f kube-deployment.yml
except:
- master
when: manual
• Used to test a feature/fix branch in a full environment

19. .gitlab-ci.yml - deploy test
deploy-master:
stage: deploy
environment: test
image: dreg.life.uni-leipzig.local/kubectl:0.4
tags:
- docker
script:
- sed -i "s/<VERSION>/$CI_BUILD_REF/" kube-deployment.yml
- kubectl config use-context gitlab-ci-test
- kubectl apply -f kube-deployment.yml
only:
- master

20. .gitlab-ci.yml - deploy prod
deploy-prod:
stage: deploy
environment: prod
image: dreg.life.uni-leipzig.local/kubectl:0.4
tags:
- docker
script:
- sed -i "s/<VERSION>/$CI_BUILD_REF/" kube-deployment.yml
- kubectl config use-context gitlab-ci-prod-a
- kubectl apply -f kube-deployment.yml
only:
- master
when: manual

21. Docker file
FROM clojure:lein-2.7.1
COPY src /app/src
COPY project.clj /app/
WORKDIR /app
RUN lein with-profile production deps
EXPOSE 80
CMD ["lein", "with-profile", "production", "run"]
• Just copy the sources into the container
• Use Leiningen itself to run in production

22. kube-deployment.yml
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: clojure-berlin-2016
spec:
replicas: 2
template:
metadata:
labels:
app: lens
service: clojure-berlin-2016
spec:
containers:
- name: clojure-berlin-2016
image: dreg.life.uni-leipzig.local/clojure-berlin-2016:<VERSION>
ports:
- containerPort: 8080
resources:
requests:
cpu: "125m"
memory: "1Gi"
limits:
cpu: 1
memory: "2Gi"

23. kube-svc.yml
apiVersion: v1
kind: Service
metadata:
name: clojure-berlin-2016
labels:
app: lens
spec:
type: NodePort
ports:
- port: 80
targetPort: 8080
protocol: TCP
selector:
service: clojure-berlin-2016

24. Steps to Follow
• Create the Kubernetes Service
• kubectl create -f kube-svc.yml
• Edit HAProxy Config
• add rules and backend for the service
• Push to GitLab
• git push

25. Pipeline in GitLab CI

26. Deployment in GitLabCI

27. Environments in GitLabCI
• Very good visibility of wich commit is deployed in
which environment right now
• Manual deployment to prod possible

28. Environment History
• Easy to see when what commit was deployed
• Rollback possible

29. Numbers
• Our team has 4 developers
• We run 2 Kubernetes clusters (test and prod) with
about 96 GB RAM and and 24 vCPU’s each
• We run about 60 pods in production
• We have other services like central log aggregation
running using Fluentd and Elasticsearch/Kibana

30. Thank You
• Sample Project on Github
https://github.com/alexanderkiel/clojure-berlin-2016
• Twitter
@alexander_kiel
• Mail
alexanderkiel@gmx.net