This document provides an overview of a technical meetup about Varnish Cache. It includes an agenda covering an introduction to Varnish Cache, use cases and architectures, and limitations and pitfalls. Example use cases discussed are caching static resources on a separate domain, caching REST API responses, and caching full websites. The presentation covers Varnish configuration, caching strategies, and cache invalidation techniques.

1. APACHE SLING & FRIENDS TECH MEETUP
BERLIN, 23-25 SEPTEMBER 2013
Become happier by using Varnish Cache
Stefan Maurer
Senior Software Engineer, Namics AG

2. Agenda
adaptTo() 2013 2
 Varnish Cache at a glance
 Use Cases and Architectures
 Limitations and Pitfalls

3. Varnish Cache at a glance
adaptTo() 2013 3
 Reverse Proxy Cache
 1.0.0 in 2006
 Current version is 3.0.4
 Free under BSD License
 Available for various Linux Distributions,
Max OS X, Open Solaris, Solaris 10,
FreeBSD and Windows (with Cygwin)

4. Varnish Cache at a glance
adaptTo() 2013 4
 Designed as HTTP accelerator – not
more, not less
 “Varnish is a modern program, designed
and written for modern operating systems”
https://www.varnish-cache.org/trac/wiki/VarnishFeatures
 Storage
 malloc or file
 Load balancing
 ESI (subset)

5. Varnish Cache at a glance
adaptTo() 2013 5
 Process architecture
 Management process and child process
 Configuration change without restart
Source: www.varnish-software.com/static/book/Tuning.html

6. Varnish Cache at a glance
adaptTo() 2013 6
 Configuration
 Startup parameters
 Varnish Configuration Language (VCL)
sub vcl_fetch {
if (req.url ~ ".jpg$") {
set beresp.ttl = 60s;
}
}

7. Varnish Cache at a glance
adaptTo() 2013 Source: www.varnish-software.com/static/book/VCL_Basics.html

8. Varnish Cache at a glance
adaptTo() 2013 8
 Logging
 Logs to shared memory
 Separate application for display and write
back

9. Agenda
adaptTo() 2013 9
 Varnish Cache at a glance
 Use Cases and Architectures
 Limitations and Pitfalls

10. Use Cases and Architectures
adaptTo() 2013 10
 Use case 1: Static resources
 Use case 2: REST API calls
 Use case 3: Full website

11. Use Case 1: Static Resources
adaptTo() 2013 11
 Use Case
 Cache resources like CSS / JS / static
images “forever”
 Benefit with Varnish
 Internal network traffic decreases
significant
 Do not bore backend systems ;-)
 Solution
 Cookie-less domain sc.customer.com
served by Varnish

12. CQ5 Publisher
Dispatcher
www.customer.com
Varnish Cache
sc.customer.com
Use Case 1: Static Resources
adaptTo() 2013 12
 Architecture Sample 1
Browser
Varnish Cache
sc.customer.com
Dispatcher
www.customer.com
Load Balancer
CQ5 Publisher

13. Use Case 1: Static Resources
adaptTo() 2013 13
 Architecture Sample 2
Browser
Varnish Cache
sc.customer.com
Dispatcher
www.customer.com
Load Balancer
www.customer.com
CQ5 Publisher
Dispatcher
www.customer.com
CQ5 Publisher

14. Use Case 1: Static Resources
adaptTo() 2013 14
 Why not directly connect Varnish with
CQ5 Publisher?
 Possible. But requires configuration for
restricted paths like /system/console
 Changes in CQ
 Load resources from static domain with a
timestamp:
<script
type="text/javascript"
src="http://sc.customer.com/etc/designs/customer/clientlib.201309231000.js"
></script>
<img src="http://sc.customer.com/content/dam/logo.201309231000.png" />

15. Use Case 1: Static Resources
adaptTo() 2013 15
backend default {
.host = "dispatcher-hostname";
.port = "80";
}
sub vcl_recv {
if (req.url !~ "^/etc/designs/" && req.url !~ "^/content/dam/"){
error 405 "Not allowed.";
}
unset req.http.cookie;
}
sub vcl_fetch {
unset beresp.http.set-cookie;
unset beresp.http.expires;
unset beresp.http.Etag;
set beresp.http.Cache-Control = "max-age=86400";
set beresp.ttl = 4w;
set beresp.http.magicmarker = "1";
}
sub vcl_deliver {
if (resp.http.magicmarker) {
unset resp.http.magicmarker;
set resp.http.age = "0";
}
}

16. Use Cases and Architectures
adaptTo() 2013 16
 Use case 1: Static resources
 Use case 2: REST API calls
 Use case 3: Full website

17. Use Case 2: REST API Calls
adaptTo() 2013 17
 Use Case
 Cache requests to backend APIs
 Benefit with Varnish
 Reduce load on backend systems
 Reduce response time
 Handle backend downtime
 Custom time to life for each backend
 Solution
 Call backend through Varnish Cache

18. Use Case 2: REST API Calls
adaptTo() 2013 18
 Sample: Yahoo Finance API
 http://finance.yahoo.com/d/quotes.csv?s=ADBE&f=snd1l1c1p2
 Last trade date, price, change and percents
 But: has limit of requests/day
 CQ Publisher calls Varnish with prefix
“finance”
 http://varnish/finance/d/quotes.csv?s=ADBE&f=snd1l1c1p2
 Varnish executes API Call and caches
the response for one hour

19. Use Case 2: REST API Calls
adaptTo() 2013 19
 Architecture
Varnish Cache
Yahoo Finance API
CQ5 Publisher
Backend 2..n

20. Use Case 2: REST API Calls
adaptTo() 2013 20
backend yahoofinance {
.host = "finance.yahoo.com";
.port = "80";
}
sub vcl_recv {
if (req.url ~ "^/finance") {
set req.backend = yahoofinance;
set req.url = regsub(req.url, "^/finance", "");
unset req.http.cookie;
}
}
sub vcl_fetch {
if(req.backend == yahoofinance) {
set beresp.ttl = 1h;
unset beresp.http.set-cookie;
return(deliver);
}
}

21. Use Cases and Architectures
adaptTo() 2013 21
 Use case 1: Static resources
 Use case 2: REST API calls
 Use case 3: Full website

22. Use Case 3: Full Website
adaptTo() 2013 22
 Use Case
 Cache generated HTML and related
resources
 Benefit with Varnish
 Reduce load on backend systems
 Reduce response time
 Granular cache invalidation

23. CQ5 Publisher
Dispatcher
Use Case 3: Full Website
adaptTo() 2013 23
 Architecture Sample 1
Browser
Varnish Cache
Dispatcher
CQ5 Publisher

24. CQ5 Publisher
Varnish Cache
Use Case 3: Full Website
Dispatcher
adaptTo() 2013 24
 Architecture Sample 2
Browser
Varnish Cache
Load Balancer
CQ5 Publisher
Dispatcher

25. Use Case 3: Full Website
adaptTo() 2013 25
 Cache Invalidation
Short TTL (1 minute) Long TTL (1 month)
Frequently called pages + +
Seldom called pages - +
Instant invalidation - +
Low complexity + -

26. Use Case 3: Full Website
adaptTo() 2013 26
 Excludes from cache
sub vcl_recv {
if (req.url ~ "?"
|| req.url ~ "/cug_") {
return(pass);
}
}
sub vcl_fetch {
if (beresp.http.Dispatcher ~ "no-cache"
|| beresp.http.cache-control ~ "(no-cache|private)"
|| beresp.http.pragma ~ "no-cache") {
set beresp.ttl = 0s;
}
else {
set beresp.ttl = 4w;
}
}

27. Use Case 3: Full Website
adaptTo() 2013 28
 Invalidate a page with “smart bans”
acl purgers {
"127.0.0.1";
"192.168.0.0"/24;
}
sub vcl_recv {
if (req.http.X-Purge-URL) {
if (!client.ip ~ purgers) {
error 405 "Method not allowed";
}
ban("obj.http.x-url == " + req.http.X-Purge-URL);
error 200 "Banned URL";
}
}
sub vcl_fetch {
set beresp.http.x-url = req.url;
}

28. Use Case 3: Full Website
adaptTo() 2013 29
 Invalidate from CQ Publisher with
Modification Listener
 Modify: /content/customer/en/news/adaptto
 Risky: Purge only page and sub pages
/content/customer/en/news/adaptto.*
 Safe: Purge whole language tree
/content/customer/en.*

29. Agenda
adaptTo() 2013 30
 Varnish Cache at a glance
 Use Cases and Architectures
 Limitations and Pitfalls

30. Limitations and Pitfalls
adaptTo() 2013 31
 SSL termination
 Problem
Varnish has no SSL termination
 Justification
Varnish source code: 80’000 lines of code
OpenSSL: 340’000 lines of code
 Solution
Use another tier in front of Varnish (nginx,
pond, …)

31. Limitations and Pitfalls
adaptTo() 2013 32
 Avoid “double purge”
 Assumed situation
2 Publisher and 1 Varnish
 Problem
Modification listener is executed on both
Publishers  2 purge requests
 Solution
Define “master” publisher and handle
failover

32. Limitations and Pitfalls
adaptTo() 2013 33
 Proxy
 Assumed situation
- Varnish is used to cache REST API Calls
- Your customer force to use proxy for all
external requests
 Problem
Varnish has no configuration for backend
through proxy
 Solution
Set proxy as backend and tweak URL

33. Limitations and Pitfalls
adaptTo() 2013 34
 Proxy solution
backend proxy {
.host = "corporate-proxy-hostname";
.port = "8080";
}
sub vcl_recv {
set req.backend = proxy;
set req.http.X-Forwarded-For = client.ip;
set req.http.host = "dispatcher-hostname";
set req.http.port = 80;
set req.url = "http://" + req.http.host + ":" +
req.http.port + req.url;
}

34. adaptTo() 2013 35
Thank you

35. Namics
adaptTo() 2012 36
 5 Locations in Germany and Switzerland
Frankfurt, Hamburg, München, Zürich, St. Gallen
 Many interesting projects with CQ5 and
other technologies
 400 + 1 (?) Employees
www.namics.com/jobs

36. Resources and Further Links
adaptTo() 2013 37
 https://www.varnish-
cache.org/trac/wiki/VCLExampleLongerCaching
 https://www.varnish-software.com/static/book/Tuning.html
 https://www.varnish-
software.com/static/book/_images/vcl.png
 http://blog.namics.com/2013/01/www-zh-ch-ist-50x-schneller-
und-stabiler.html
 http://blog.namics.com/2010/12/varnish-cache.html

37. Variable availability in VCL
adaptTo() 2012 38