Varnish Summit 2016

1. 1

2. Do you
speak VCL?
2

3. The Varnish Book
Dedicated writer and
maintainer
Francisco Velázquez
Curriculum Manager
francisco@varnish-software.com
CDNs with Varnish
Build your own CDN with
Varnish
Who am I?

4. Agenda
● What is VCL?
● Motivate from learning VCL
● Understanding VCL in Varnish
● VCL as finite state machine
● How to design and test your VCL code?
● VCL tips and tricks
● Typical mistakes in VCL
● Conclusions
4

5. What is
VCL?
5

6. Varnish Configuration Language (VCL)
● VCL is a domain specific
language based on C
● Programming language
specialized to express solutions
for caching the Web
● Rule-based language
6

7. Why should you program in VCL?
● Varnish covers most common caching needs out-of-the-box, but
does not intend to be a one-fit-all solution!
● VCL allows you to:
○ define your own rules
○ specify when to use and alternate them on run time
7

8. Cacher
Web Application Firewall
Hotlinking Protector
DDoS Attack Defender
Load Balancer
Authentication and
Authorization Mechanism
HTTP Router
and more...
VCL makes Varnish to act as:
VCL
8

9. VCL Characteristics (1/2)
● Simple syntax, but pay attention to the semantics of HTTP
● No loops
● Functionality grouped in subroutines that
○ do not take arguments nor return values
○ exchange data only through HTTP headers
● VCL can manipulate HTTP headers
○ Override TTLs
○ Strip cookies
○ Rewrite URLs
● Extensible via Varnish modules (VMODs)
9

10. VCL Characteristics (2/2)
● Load multiple VCLs and switch between them instantly and seamlessly
● No server restart required
● Includes built-in functions that allow you to:
○ modify strings
○ invalidate caches
● Supports regular expressions (regex)
● Languages evolve, so VCL does!
○ varnish3to4 script
10

11. Varnish Finite State
Machine (FSM)
● States in Varnish FSM are per HTTP
request
○ HTTP is a stateless protocol
● State preservation across
transactions use mechanisms such
as cookies
11

12. Built-in VCL
subroutines per state
● File location: varnish-cache/bin/varnishd/builtin.vcl
12

13. Your VCL code *always*
precedes the built-in VCL
code!
13

14. Your VCL code
Built-in VCL code
14

15. Understanding how VCL works in Varnish
varnishd
VCC Process
Manager
Process
CLI
...
Cacher
Process
Storage
...
Shared Memory
Log
C-compiler Shared Object
One
binary
program
VAC
varnishadm
varnishlog
...
15

16. How to design your VCL code?
● Design your rules first
○ Create Varnish Test Cases (VTC) in varnishtest
○ Attend to Arianna’s presentation after break
● Be sure you understand the HTTP caching headers and objects’ lifetime
○ Refer to The Varnish Book
○ Refer to RFC7232 and RFC7234
● Get ideas from VCL snippets you find in Internet, but be critical!
● Test your VCL!
○ Use varnishtest
16

17. ● Design a plan for
cache invalidation
Source: https://www.posterlounge.co.uk/spiderman-
pr28529.html
17

18. Cache Invalidation Alternatives
Bans Purge Soft Purge Hashtwo –
Surrogate keys
Force Cache
Misses
Targets Patterns matching Specific object Specific object Objects with
common key
Specific object
Frees
memory
After a request hits an
object or the ban luker
invalidates cached object
Immediately After grace
time
Immediately No
Scalability High High High High Low
VCL Yes Yes Yes Yes Yes
CLI Yes No No No No
Table 18 in The Varnish Book 18

19. VCL tips and tricks
● Command to print VCL code compiled to C language and exit:
○ $ varnishd -C -f filename
○ Useful to check whether your VCL code compiles correctly
● Data types:
○ TIME + DURATION is allowed
○ TIME - TIME = DURATION is allowed
○ TIME + TIME is not allowed!
○ Use the Varnish Standard Module (std VMOD)
○ If not in std VMOD, do not assume automatic data type conversions, test
them!
● Check operator precedence and more at varnish-cache.org
19

20. Typical
mistakes20

21. Typical Mistakes in VCL (1/3)
● Regular expressions
● Cache with inefficient cache invalidation plan
● Skipping the built-in VCL by calling return(action)
○ Caching cookies inadvertently
● Order query parameters of a URL
○ example.com/test?var0=0&var1=1
○ example.com/test?var1=1&var0=0
○ Call querysort(STRING) from std VMOD
● Copy/paste VCL snippets without understanding them
21

22. Typical Mistakes in VCL (2/3)
● Create uncontrolled variations of hash key values for the same object
○ Vary: User-Agent
○ Vary: Cookie
22

23. Typical Mistakes in VCL (3/3)
● Understand request serialization and the hit-for-pass object
● Let vcl_backend_response execute!
○ Otherwise, never set beresp.ttl = 0
○ Slows down site response if skipped
● Varnish 3 has a hit_for_pass return action
● Varnish 4 beresp.uncacheable = true;
23

24. Ask for Help and Share
● Contact Varnish Support
● IRC channel #varnish on irc.linpro.net
● Mailing lists: https://www.varnish-cache.org/trac/wiki/MailingLists
● Consult The Varnish Book
● https://www.varnish-cache.org/trac/wiki/VCLExamples
● https://www.varnish-cache.org/docs/4.0/reference/vcl.html
24

25. Take away points
● It is worth it to learn VCL
● VCL has a simple syntax
● Design Varnish Test Cases and test your VCL in varnishtest
● Let the built-in VCL code execute!
25