Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Create a Varnish cluster in Kubernetes for Drupal caching - DrupalCon North America 2021

Varnish is a caching proxy usually used for high profile Drupal sites. However, configuring Varnish is not an easy task that requires a lot of work. It is even more difficult when it comes to creating a scalable cluster of Varnish nodes.
Fortunately, there is a solution. I’ve been working on kube-httpcache project ( that takes care of many things such as routing, scaling, broadcasting, config-reloading, etc...
If you need to run more than one instance of Varnish, this session is for you. You will learn how to:
* Launch a single instance of Varnish in Kubernetes.
* Configure Varnish for Drupal.
* Scale Varnish from 1 to N nodes as part of the cluster.
* Make your Varnish cluster resilient.
* Reload Varnish configs on the fly.
* Properly invalidate cache for multiple Varnish nodes.
This session requires some basic understanding of Docker and Kubernetes; however, I will provide some intro if you are new to it.
Join this session and enjoy!

  • Be the first to comment

  • Be the first to like this

Create a Varnish cluster in Kubernetes for Drupal caching - DrupalCon North America 2021

  1. 1. Create a Varnish cluster in Kubernetes for Drupal caching Vadym Myrgorod Slides:
  2. 2. Vadym Myrgorod Using Drupal since 2008 Web App Developer Howard Hughes Medical Institute @dealancer1
  3. 3. I will make a quick intro into Varnish and Kubernetes. We will learn how to configure Varnish cluster for Drupal 8 and run it on Kubernetes platform. In this session
  4. 4. what is Varnish?
  5. 5. More users means slower website
  6. 6. Slower website means bigger bounce rate
  7. 7. 1. Cutting visitors off 2. Arranging visitors in an online queue 3. Buying more hardware Approaches we won’t cover in this session
  8. 8. In computing, a cache is a hardware or software component that stores data so that future requests for that data can be served faster; the data stored in a cache might be the result of an earlier computation or a copy of data stored elsewhere. Caching
  9. 9. Levels of caching 1 Opcode Caching Compile PHP script into opcode and cache it. Examples: OPcache extension. 3 Drupal Caching Store blocks, views and other Drupal components in memory. Examples: Memcache, Redis. 5 Content Delivery Network Store static assets in geographically distributed caching system. Examples: Cloudflare, Fastly, Akamai. 2 Database Caching Store results of SQL queries in memory. Examples: memcached. 4 Static Content Caching Store static assets in memory. Examples: Varnish, Nginx.
  10. 10. ● Aggregated CSS files ● Aggregated JS files ● Images of various image styles ● HTML pages (for anonymous users) Static content generated by Drupal
  11. 11. ● Caching HTTP reverse proxy ● Speeds up delivery with a factor of 300 - 1000x ● Varnish Configuration Language (VCL) ● Can be scaled into Varnish Cluster
  12. 12. Backend - server which is providing the content Varnish will accelerate. Frontend - client facing Varnish server. Terms Backend Frontend
  13. 13. vcl 4.0; backend default { .host = ""; .port = "8080"; } sub vcl_recv { if ( == "") { set req.backend_hint = default; } } See following VCL example for Drupal 8 and 7: Minimal Varnish VCL /etc/varnish/default.vcl
  14. 14. now let’s talk about Kubernetes
  15. 15. 1. You want to manage services you are running e.g. MySQL, Varnish, Redis, Apache Solr, Elasticsearch... 2. You want to have an ability to migrate from one cloud provider to another quickly. 3. You have on-premises infrastructure you need to utilize. 4. You want to use multicloud or hybrid-cloud approaches. 5. You want to have some of the cool features that Kubernetes provides. When using Kubernetes is a good idea
  16. 16. 1. Scaling: runs more containers if needed. 2. Healing: restarts containers when they are down. 3. Deployment strategies: rolling updates, blue/green, canary... 4. Service discovery and load balancing: distributes access to containers by service name. 5. Security: secret management, security policies. 6. Storage orchestration: cloud, NFS, or storage providers. 7. Flexibility and extensibility: Go is widely used in k8s world. Things Kubernetes is good at
  17. 17. ● Resource - endpoint in k8s API that stores a collection of certain kind. Declared using YAML syntax. Example: config map, secret, volume, volume claim, pod, deployment, stateful set, service, ingress, etc... ● Config map - stores software configuration that can be mounted as files or passed as env variables. ● Volume - used to preserve file system after container is restarted. ● Pod - k8s resource that represents running container (or set set of containers). Kubernetes resources Config map Pod Deployment Service Ingress Stateful Set Volume
  18. 18. ● Deployment - ensures certain amount of pods are up and running. ● Stateful Set - acts as deployment for stateful services that have specific requirements to storage and pod identity. ● Service - provides a way to access deployments or stateful sets using different behaviours. Creates a common DNS record that can be used to access pods and does load balancing. ● Ingress - manages external access to services using various routing rules. Kubernetes resources Config map Pod Deployment Service Ingress Stateful Set Volume
  19. 19. nginx-deployment.yaml: apiVersion: apps/v1 kind: Deployment metadata: name: nginx-deployment labels: app: nginx spec: replicas: 2 selector: matchLabels: app: nginx template: metadata: labels: app: nginx spec: containers: - name: nginx image: nginx:1.14.2 ports: - containerPort: 80 Minimal Kubernetes deployment and service nginx-service.yaml: apiVersion: v1 kind: Service metadata: name: nginx-service spec: selector: app: nginx type: NodePort ports: - protocol: TCP port: 8080 targetPort: 80 nodePort: 30080
  20. 20. 1. Apply deployment $ kubectl apply -f nginx-deployment.yaml 2. Check deployments $ kubectl get deployments NAME READY UP-TO-DATE AVAILABLE AGE nginx-deployment 0/2 0 0 1s 3. Check pods $ kubectl get pods --show-labels NAME READY STATUS RESTARTS AGE LABELS nginx-deployment-75675f5897-7ci7o 1/1 Running 0 18s app=nginx,pod-template-hash=3123191453 nginx-deployment-75675f5897-kzszj 1/1 Running 0 18s app=nginx,pod-template-hash=3123191453 Minimal Kubernetes deployment and service
  21. 21. 4. Apply service $ kubectl apply -f nginx-service.yaml 5. Check services $ kubectl get services NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE kubernetes ClusterIP <none> 443/TCP 41d nginx-service NodePort <none> 8080:30080/TCP 5m11s 6. Access nginx using node port $ curl localhost:30080 <!DOCTYPE html> <html> <head> <title>Welcome to nginx!</title> ... Minimal Kubernetes deployment and service
  22. 22. running Varnish in Kubernetes
  23. 23. To run Varnish in k8s we need to declare app deployment, app service, Varnish deployment, Varnish service, config maps and ingress. Example: The simplest approach
  24. 24. varnish-deployment.yaml: apiVersion: apps/v1 kind: Deployment metadata: name: varnish-deployment labels: app: varnish spec: replicas: 1 selector: matchLabels: app: varnish template: metadata: labels: app: varnish The simplest approach spec: containers: - name: varnish image: varnish:6.6.0 env: - name: CACHE_SIZE value: 128m - name: VCL_CONFIG value: /etc/varnish/configmap/default.vcl volumeMounts: - name: varnish-config mountPath: /etc/varnish/configmap ports: - containerPort: 80 volumes: - name: varnish-config configMap: name: varnish-configmap
  25. 25. varnish-configmap.yaml: apiVersion: v1 kind: ConfigMap metadata: name: varnish-configmap labels: app: varnish data: default.vcl: | vcl 4.0; backend default { .host = "nginx-service"; .port = "8080"; } sub vcl_recv { set req.backend_hint = default; } The simplest approach varnish-service.yaml: apiVersion: v1 kind: Service metadata: name: varnish-service spec: selector: app: nginx type: NodePort ports: - protocol: TCP port: 80 targetPort: 80 nodePort: 30081
  26. 26. Questions to this approach: 1. Can we eliminate App Service and let Varnish talk to application pods (backends) directly? 2. How do we scale Varnish pods (frontends)? 3. How do we shard cache across multiple Varnish pods? 4. How do we invalidate cache in multiple Varnish pods? The simplest approach
  27. 27. kube-httpcache is an open Source Kubernetes controller written in Go to run Varnish cluster. It is a free solution comparing to Varnish Cache Plus, but it requires to configure Varnish in certain way using VCL. GitHub: Features: ● Monitors backend (app) and frontend (Varnish) pods. ● Dynamically update Varnish VCL and reload it on the the fly in all Varnish pods. ● Supports Go-template syntax in VCL file. ● Sends cache invalidation requests to all Varnish pods using Signaller component. kube-httpcache
  28. 28. spec: containers: - name: cache image: imagePullPolicy: Always args: - -admin-addr= - -admin-port=6083 - -signaller-enable - -signaller-port=8090 - -frontend-watch - -frontend-namespace=$(NAMESPACE) - -frontend-service=frontend-service - -backend-watch - -backend-namespace=$(NAMESPACE) - -backend-service=backend-service - -varnish-secret-file=/etc/varnish/k8s-secret/secret - -varnish-vcl-template=/etc/varnish/tmpl/default.vcl.tmpl - -varnish-storage=malloc,128M env: - name: NAMESPACE valueFrom: fieldRef: fieldPath: metadata.namespace volumeMounts: - name: template mountPath: /etc/varnish/tmpl - name: secret mountPath: /etc/varnish/k8s-secret varnish-service.yaml (part)
  29. 29. {{ range .Backends }} backend be-{{ .Name }} { .host = "{{ .Host }}"; .port = "{{ .Port }}"; } {{- end }} {{ range .Frontends }} backend {{ .Name }} { .host = "{{ .Host }}"; .port = "{{ .Port }}"; } {{- end }} sub vcl_init { new lb = directors.round_robin(); {{ range .Backends -}} lb.add_backend(be-{{ .Name }}); {{ end }} } sub vcl_recv { set req.backend_hint = lb.backend(); } Eliminate the need in backend service
  30. 30. In this approach cache is sharded across multiple Varnish frontends. 1. Varnish Service routes requests to a random Varnish frontend. 2. A corresponding Varnish frontend (x-shard) determined based on the URL using hash director: a. -> frontend 1 b. -> frontend 2 c. -> frontend 1 3. If x-shard is not the current Varnish frontend, request is forwarded to x-shard. 4. Otherwise, request is handled by the current Varnish node: a. If request is cacheable and cache exist, cached content is returned. b. If cache does not exists, Varnish requests a backend to save cache and serve cached content. c. If request is cacheable request is routed to a backend. See Running Varnish cluster (sharded cache)
  31. 31. sub vcl_init { # ... new cluster = directors.hash(); {{ range .Frontends -}} cluster.add_backend({{ .Name }}, 1); {{ end }} } sub vcl_recv { set req.backend_hint = lb.backend(); # ... unset req.http.x-cache; set req.backend_hint = cluster.backend(req.url); set req.http.x-shard = req.backend_hint; if (req.http.x-shard != server.identity) { return(pass); } set req.backend_hint = lb.backend(); # ... return(hash); } Running Varnish cluster (sharded cache)
  32. 32. Problem In case if you horizontally scale your Varnish cluster, a hash director will recalculate hashes for existing URLs and will route requests to different Varnish frontends which do not store cached content for these URLs. Thus users will experience slowdown. Solution Consistent caching mechanism is provided by Varnish through shard director. Consistent caching represents URLs and frontends as points on the ring. Adding new frontend, simply adds a new point on the ring. Scaling Varnish cluster
  33. 33. sub vcl_init { # ... new cluster = directors.shard(); {{ range .Frontends -}} cluster.add_backend({{ .Name }}); {{ end }} cluster.set_warmup(180); } sub vcl_recv { set req.backend_hint = lb.backend(); # ... unset req.http.x-cache; set req.backend_hint = cluster.backend(by=URL); set req.http.x-shard = req.backend_hint; if (req.http.x-shard != server.identity) { return(pass); } set req.backend_hint = lb.backend(); # ... return(hash); } Scaling Varnish cluster
  34. 34. Varnish Signaller: ● Is aware of all Varnish frontends running ● acts similarly to Varnish Broadcaster (component of Varnish Plus) ● broadcasts flush cache request to all Varnish frontends ● runs on port 8090 BAN requests $ curl -H "X-Url: /path" -X BAN http://cache-service:8090 $ curl -H "Cache-Tags: node-1" -X BAN http://cache-service:8090 PURGE requests $ curl -H "X-Host:" -X PURGE http://cache-service:8090/path Invalidating cache
  35. 35. sub vcl_recv { # ... if (req.method == "PURGE") { if (client.ip !~ privileged) { return (synth(403, "Not allowed.")); } if (req.http.X-Host) { set = req.http.X-Host; } return (purge); } if (req.method == "BAN") { if (client.ip !~ privileged) { return (synth(403, "Not allowed.")); } if (req.http.Cache-Tags) { ban("obj.http.Cache-Tags ~ " + req.http.Cache-Tags); return (synth(200, "Ban added " +; } if (req.http.X-Url) { ban("obj.http.X-Url == " + req.http.X-Url); return (synth(200, "Ban added " +; } return (synth(403, "Cache-Tags or X-Url header missing.")); } # ... } Invalidating cache
  36. 36. 1. Kubernetes Playground 2. Kubernetes Patterns Book 01910-en.pdf 3. Article this presentation is based on netes-853f03ec9731 4. kube-httpcache project 5. Complete VCL file for Drupal 8 and 9 (IMPORTANT) Resources
  37. 37. configuring Drupal
  38. 38. 1. Purge module - Clreans external caching systems, reverse proxies and CDNs 2. Varnish purger - Extension of Purge module to clear Varnish cache 3. Configuration URL: /admin/config/development/performance/purge Configuring Drupal
  39. 39. Configuring Drupal
  40. 40. Configuring Drupal
  41. 41. Configuring Drupal
  42. 42. Q&A? Slides: Please, take a survey!
  43. 43. Thanks! Slides: Please, take a survey!