1. Create a Varnish cluster in Kubernetes
for Drupal caching
Vadym Myrgorod
Slides: http://bit.ly/k8s-varnish-drupal

2. Vadym Myrgorod
Using Drupal since 2008
Web App Developer
Howard Hughes Medical Institute
@dealancer1
https://www.myrgorod.net

3. I will make a quick intro into Varnish and Kubernetes.
We will learn how to configure Varnish cluster for Drupal 8
and run it on Kubernetes platform.
In this session

4. what is
Varnish?

5. More users means slower website

6. Slower website means bigger bounce rate

7. 1. Cutting visitors off
2. Arranging visitors in an online queue
3. Buying more hardware
Approaches we won’t cover in this session

8. In computing, a cache is a hardware or software
component that stores data so that future requests for
that data can be served faster; the data stored in a cache
might be the result of an earlier computation or a copy of
data stored elsewhere.
Caching

9. Levels of caching
1
Opcode Caching
Compile PHP script into opcode and cache it.
Examples: OPcache extension.
3
Drupal Caching
Store blocks, views and other Drupal
components in memory.
Examples: Memcache, Redis.
5
Content Delivery Network
Store static assets in geographically
distributed caching system.
Examples: Cloudflare, Fastly, Akamai.
2
Database Caching
Store results of SQL queries in memory.
Examples: memcached.
4
Static Content Caching
Store static assets in memory.
Examples: Varnish, Nginx.

10. ● Aggregated CSS files
● Aggregated JS files
● Images of various image styles
● HTML pages (for anonymous users)
Static content generated by Drupal

11. ● Caching HTTP reverse proxy
● Speeds up delivery with a factor of 300 - 1000x
● Varnish Configuration Language (VCL)
● Can be scaled into Varnish Cluster

12. Backend - server which is providing
the content Varnish will accelerate.
Frontend - client facing Varnish
server.
Terms
Backend
Frontend

13. vcl 4.0;
backend default {
.host = "127.0.0.1";
.port = "8080";
}
sub vcl_recv {
if (req.http.host == "www.example.com") {
set req.backend_hint = default;
}
}
See following VCL example for Drupal 8 and 7:
https://www.varnish-software.com/wiki/content/tutorials/drupal/drupal_vcl.html
Minimal Varnish VCL
/etc/varnish/default.vcl

14. now let’s talk about
Kubernetes

15. 1. You want to manage services you are running e.g. MySQL,
Varnish, Redis, Apache Solr, Elasticsearch...
2. You want to have an ability to migrate from one cloud
provider to another quickly.
3. You have on-premises infrastructure you need to utilize.
4. You want to use multicloud or hybrid-cloud approaches.
5. You want to have some of the cool features that
Kubernetes provides.
When using Kubernetes is a good idea

16. 1. Scaling: runs more containers if needed.
2. Healing: restarts containers when they are down.
3. Deployment strategies: rolling updates, blue/green, canary...
4. Service discovery and load balancing: distributes access to
containers by service name.
5. Security: secret management, security policies.
6. Storage orchestration: cloud, NFS, or storage providers.
7. Flexibility and extensibility: Go is widely used in k8s world.
Things Kubernetes is good at

17. ● Resource - endpoint in k8s API that stores a
collection of certain kind. Declared using YAML
syntax. Example: config map, secret, volume, volume
claim, pod, deployment, stateful set, service, ingress,
etc...
● Config map - stores software configuration that can
be mounted as files or passed as env variables.
● Volume - used to preserve file system after container
is restarted.
● Pod - k8s resource that represents running
container (or set set of containers).
Kubernetes resources
Config map
Pod
Deployment
Service
Ingress
Stateful Set
Volume

18. ● Deployment - ensures certain amount of pods are
up and running.
● Stateful Set - acts as deployment for stateful
services that have specific requirements to storage
and pod identity.
● Service - provides a way to access deployments or
stateful sets using different behaviours. Creates a
common DNS record that can be used to access
pods and does load balancing.
● Ingress - manages external access to services using
various routing rules.
Kubernetes resources
Config map
Pod
Deployment
Service
Ingress
Stateful Set
Volume

19. nginx-deployment.yaml:
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-deployment
labels:
app: nginx
spec:
replicas: 2
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx:1.14.2
ports:
- containerPort: 80
Minimal Kubernetes deployment and service
nginx-service.yaml:
apiVersion: v1
kind: Service
metadata:
name: nginx-service
spec:
selector:
app: nginx
type: NodePort
ports:
- protocol: TCP
port: 8080
targetPort: 80
nodePort: 30080

20. 1. Apply deployment
$ kubectl apply -f nginx-deployment.yaml
2. Check deployments
$ kubectl get deployments
NAME READY UP-TO-DATE AVAILABLE AGE
nginx-deployment 0/2 0 0 1s
3. Check pods
$ kubectl get pods --show-labels
NAME READY STATUS RESTARTS AGE LABELS
nginx-deployment-75675f5897-7ci7o 1/1 Running 0 18s app=nginx,pod-template-hash=3123191453
nginx-deployment-75675f5897-kzszj 1/1 Running 0 18s app=nginx,pod-template-hash=3123191453
Minimal Kubernetes deployment and service

21. 4. Apply service
$ kubectl apply -f nginx-service.yaml
5. Check services
$ kubectl get services
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 41d
nginx-service NodePort 10.98.115.133 <none> 8080:30080/TCP 5m11s
6. Access nginx using node port
$ curl localhost:30080
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
...
Minimal Kubernetes deployment and service

22. running
Varnish in Kubernetes

23. To run Varnish in k8s we need to declare app deployment, app
service, Varnish deployment, Varnish service, config maps and
ingress.
Example: https://github.com/dealancer/k8s-varnish-test.
The simplest approach

24. varnish-deployment.yaml:
apiVersion: apps/v1
kind: Deployment
metadata:
name: varnish-deployment
labels:
app: varnish
spec:
replicas: 1
selector:
matchLabels:
app: varnish
template:
metadata:
labels:
app: varnish
The simplest approach
spec:
containers:
- name: varnish
image: varnish:6.6.0
env:
- name: CACHE_SIZE
value: 128m
- name: VCL_CONFIG
value: /etc/varnish/configmap/default.vcl
volumeMounts:
- name: varnish-config
mountPath: /etc/varnish/configmap
ports:
- containerPort: 80
volumes:
- name: varnish-config
configMap:
name: varnish-configmap

25. varnish-configmap.yaml:
apiVersion: v1
kind: ConfigMap
metadata:
name: varnish-configmap
labels:
app: varnish
data:
default.vcl: |
vcl 4.0;
backend default {
.host = "nginx-service";
.port = "8080";
}
sub vcl_recv {
set req.backend_hint = default;
}
The simplest approach
varnish-service.yaml:
apiVersion: v1
kind: Service
metadata:
name: varnish-service
spec:
selector:
app: nginx
type: NodePort
ports:
- protocol: TCP
port: 80
targetPort: 80
nodePort: 30081

26. Questions to this approach:
1. Can we eliminate App Service and let Varnish talk to
application pods (backends) directly?
2. How do we scale Varnish pods (frontends)?
3. How do we shard cache across multiple Varnish pods?
4. How do we invalidate cache in multiple Varnish pods?
The simplest approach

27. kube-httpcache is an open Source Kubernetes controller written in Go to run Varnish
cluster. It is a free solution comparing to Varnish Cache Plus, but it requires to configure
Varnish in certain way using VCL.
GitHub: https://github.com/mittwald/kube-httpcache.
Features:
● Monitors backend (app) and frontend (Varnish) pods.
● Dynamically update Varnish VCL and reload it on the the fly in all Varnish pods.
● Supports Go-template syntax in VCL file.
● Sends cache invalidation requests to all Varnish pods using Signaller component.
kube-httpcache

28. spec:
containers:
- name: cache
image: quay.io/mittwald/kube-httpcache:stable
imagePullPolicy: Always
args:
- -admin-addr=0.0.0.0
- -admin-port=6083
- -signaller-enable
- -signaller-port=8090
- -frontend-watch
- -frontend-namespace=$(NAMESPACE)
- -frontend-service=frontend-service
- -backend-watch
- -backend-namespace=$(NAMESPACE)
- -backend-service=backend-service
- -varnish-secret-file=/etc/varnish/k8s-secret/secret
- -varnish-vcl-template=/etc/varnish/tmpl/default.vcl.tmpl
- -varnish-storage=malloc,128M
env:
- name: NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
volumeMounts:
- name: template
mountPath: /etc/varnish/tmpl
- name: secret
mountPath: /etc/varnish/k8s-secret
varnish-service.yaml (part)

29. {{ range .Backends }}
backend be-{{ .Name }} {
.host = "{{ .Host }}";
.port = "{{ .Port }}";
}
{{- end }}
{{ range .Frontends }}
backend {{ .Name }} {
.host = "{{ .Host }}";
.port = "{{ .Port }}";
}
{{- end }}
sub vcl_init {
new lb = directors.round_robin();
{{ range .Backends -}}
lb.add_backend(be-{{ .Name }});
{{ end }}
}
sub vcl_recv {
set req.backend_hint = lb.backend();
}
Eliminate the need in backend service

30. In this approach cache is sharded across multiple Varnish frontends.
1. Varnish Service routes requests to a random Varnish frontend.
2. A corresponding Varnish frontend (x-shard) determined based on the
URL using hash director:
a. http://example.com/foo -> frontend 1
b. http://example.com/bar -> frontend 2
c. http://example.com/bar2 -> frontend 1
3. If x-shard is not the current Varnish frontend, request is forwarded to
x-shard.
4. Otherwise, request is handled by the current Varnish node:
a. If request is cacheable and cache exist, cached content is returned.
b. If cache does not exists, Varnish requests a backend to save cache
and serve cached content.
c. If request is cacheable request is routed to a backend.
See https://info.varnish-software.com/blog/creating-self-routing-varnish-cluster
Running Varnish cluster (sharded cache)

31. sub vcl_init {
# ...
new cluster = directors.hash();
{{ range .Frontends -}}
cluster.add_backend({{ .Name }}, 1);
{{ end }}
}
sub vcl_recv {
set req.backend_hint = lb.backend();
# ...
unset req.http.x-cache;
set req.backend_hint = cluster.backend(req.url);
set req.http.x-shard = req.backend_hint;
if (req.http.x-shard != server.identity) {
return(pass);
}
set req.backend_hint = lb.backend();
# ...
return(hash);
}
Running Varnish cluster (sharded cache)

32. Problem
In case if you horizontally scale your Varnish cluster, a
hash director will recalculate hashes for existing URLs
and will route requests to different Varnish frontends
which do not store cached content for these URLs.
Thus users will experience slowdown.
Solution
Consistent caching mechanism is provided by Varnish
through shard director. Consistent caching represents
URLs and frontends as points on the ring. Adding new
frontend, simply adds a new point on the ring.
Scaling Varnish cluster

33. sub vcl_init {
# ...
new cluster = directors.shard();
{{ range .Frontends -}}
cluster.add_backend({{ .Name }});
{{ end }}
cluster.set_warmup(180);
}
sub vcl_recv {
set req.backend_hint = lb.backend();
# ...
unset req.http.x-cache;
set req.backend_hint = cluster.backend(by=URL);
set req.http.x-shard = req.backend_hint;
if (req.http.x-shard != server.identity) {
return(pass);
}
set req.backend_hint = lb.backend();
# ...
return(hash);
}
Scaling Varnish cluster

34. Varnish Signaller:
● Is aware of all Varnish frontends running
● acts similarly to Varnish Broadcaster (component of Varnish Plus)
● broadcasts flush cache request to all Varnish frontends
● runs on port 8090
BAN requests
$ curl -H "X-Url: /path" -X BAN http://cache-service:8090
$ curl -H "Cache-Tags: node-1" -X BAN http://cache-service:8090
PURGE requests
$ curl -H "X-Host: www.example.com" -X PURGE http://cache-service:8090/path
Invalidating cache

35. sub vcl_recv {
# ...
if (req.method == "PURGE") {
if (client.ip !~ privileged) {
return (synth(403, "Not allowed."));
}
if (req.http.X-Host) {
set req.http.host = req.http.X-Host;
}
return (purge);
}
if (req.method == "BAN") {
if (client.ip !~ privileged) {
return (synth(403, "Not allowed."));
}
if (req.http.Cache-Tags) {
ban("obj.http.Cache-Tags ~ " + req.http.Cache-Tags);
return (synth(200, "Ban added " + req.http.host));
}
if (req.http.X-Url) {
ban("obj.http.X-Url == " + req.http.X-Url);
return (synth(200, "Ban added " + req.http.host));
}
return (synth(403, "Cache-Tags or X-Url header missing."));
}
# ...
}
Invalidating cache

36. 1. Kubernetes Playground
https://www.katacoda.com/courses/kubernetes/playground
2. Kubernetes Patterns Book
https://www.redhat.com/cms/managed-files/cm-oreilly-kubernetes-patterns-ebook-f19824-2
01910-en.pdf
3. Article this presentation is based on
https://dealancer.medium.com/creating-a-scalable-and-resilient-varnish-cluster-using-kuber
netes-853f03ec9731
4. kube-httpcache project
https://github.com/mittwald/kube-httpcache
5. Complete VCL file for Drupal 8 and 9 (IMPORTANT)
https://gist.github.com/dealancer/968297d6ddd93df80d012af7f4093294
Resources

37. configuring
Drupal

38. 1. Purge module - https://www.drupal.org/project/purge
Clreans external caching systems, reverse proxies and CDNs
2. Varnish purger - https://www.drupal.org/project/varnish_purge
Extension of Purge module to clear Varnish cache
3. Configuration URL: /admin/config/development/performance/purge
Configuring Drupal

39. Configuring Drupal

40. Configuring Drupal

41. Configuring Drupal

42. Q&A?
Slides: http://bit.ly/k8s-varnish-drupal
Please, take a survey!

43. Thanks!
Slides: http://bit.ly/k8s-varnish-drupal
Please, take a survey!