#drupaldevdays birds of a feather talk on Varnish and ESI - Edge Side Includes - at Brussels.

1. Varnish BOF!

2. What's this talk about? A basic intro to Varnish and reverse proxies Some cool ESI stuff How to setup, implement + configure Varnish/ESI in Drupal Teh Awesomeness of speed

3. Why Varnish?

4. Drupal performs…just…

5. How much faster? Drupal 20 reqs/sec? Varnish 200 reqs/sec?

6. When personalisation goes wrong

7. Dries logs in

8. Drupal creates a personalised page Logged in as dries

9. The page gets cached Varnish

10. When webchick hits the site

11. The page is fetched from the cache Varnish

12. The personalisation is incorrect Logged in as dries

13. WTF? WTF?

14. No-cache headers can prevent this Varnish

15. But now it's slow :-(

16. EdgeSideIncludes

17. ESI: Tales from the past The idea originated in SSI SSI, but moved to the edge ESI is a W3C standard ESI is a bunch of xml tags/attributes. Supported by: Varnish (partial support – no ESI variables) Akamai Some Nginx support ???

18. Dries is visiting the site

19. The page is requested Varnish http://example.com/node/1

20. The page includes an ESI tag <esisrc= "http://example.com/esi/user/0" />

21. The proxy sends a second request http://example.com/esi/user/0 Varnish http://example.com/node/1

22. The ESI URL sends a HTML snip <div id="userinfo"> <a title="View &amp; edit your user profile">Logged in as dries</a> <a href="/logout">Logout</a> </div>

23. The ESI code is merged with the page

24. Proxy forwards the page on http://example.com/esi/user/0 Varnish http://example.com/node/1

25. This doesn’t sound quicker

26. The Importance of Tests It's important to measure the results(I'll come to measuring tools later!) The ESI snippets can be cached too

27. Cached ESIs

28. 1st Page = 2 Requests http://example.com/esi/user/0 Varnish http://example.com/node/1

29. 2nd Page = 1 request Varnish http://example.com/node/2

30. 1 request for each new page 10 pages = 11 requests Each page is cached The ESI tag is cached

31. The next visitor gets the page for free Varnish Page fetched from cachehttp://example.com/node/1

32. 1st page = 1 request (for the ESI) http://example.com/esi/user/0 Varnish Page fetched from cachehttp://example.com/node/1

33. Following pages all served from cache Varnish

34. 1 request for each new page 10 pages = 11 requests The ESI tag is cached Each page is cached Subsequent requests have 1 request for the ESI tag 20 pages = 12 requests (10 pages + 2 ESI tags)

35. Fast enough yet? Warm the cache

36. A modern infrastructure Varnish HA Proxy TehIntertubes Service webheadadmin/cron/emails/??

37. Spidering the site Varnish HA Proxy TehIntertubes Link-checker /spider app. Service webheadadmin/cron/emails/??

38. Dries is back for more

39. Each user generates 1 request http://example.com/esi/user/0 Varnish Page fetched from cachehttp://example.com/node/1

40. 5 users, 40 pages 5 backend requests!

41. Theory Over

42. Basics Varnish+Drupal

43. Installing Varnish sudo apt-get install varnishsudo yum install varnish Edit /etc/default/varnish Edit /etc/varnish/default.vcl Usual config: Varnish listens on port 80 Apache listens on port 8080 (or any available port) SSL is more complex – sometimes Pound is used for SSL decryption

44. VCLs are awesome! Varnish configuration is controlled by VCL: Varnish Configuration Language. Documented athttp://www.varnish-cache.org/trac/wiki/VCL Drupal VCL examples can be found in: Mercury (Amazon AMI) ESI module You can write C code in VCLs. Oh my.

45. Keeping the cache current Varnish admin port Varnish

46. Varnish Management Port The Varnish admin port uses a plain-text protocol to command Varnish Test by:telnet localhost 6093 List commands by typing help Purge cached pages: url.purge <regexp>E.g. url.purge .* The Varnish module provides an API.

47. Use with caution! The default behaviour of the Varnish module is toclear the entire cachewhen you edit a node.

48. Enter…the Expires module Came from the Boost module Clears specific cache pages – not the entire cache Supports path-alias URLs as well as node/1 paths. Provides hook_expire_cache_alterto add URLs (e.g. nodes displayed in a page-view) Supports other cache modules, not just Varnish.

49. I can haz ESI? The ESI module provides block integration

50. Configuring ESI Download the module Configure the blocks to serve via ESI, and their TTL Modify the Varnish VCL (the ESI module comes with example VCLs to use Coming soon: D7 Panels integration

51. Benchmarking Some tools: J-meter ab (apache-benchmark) Seige Approaches: Hammer your URL(s)ab –n 100 –c5 http://example.com/ Replay your log files

52. Pitfalls await VCLs are complex Per-user cache rules are even scarier! Varnish may be slower Risk exposing private data

53. Alternatives Boost module No need to install a new service on the server Static file cache Advanced cache module Authcache module Other proxies: Squid Nginx Akamai?

54. Summary

55. How hard?

56. Varnish Varnish module Expire module ESI module Teh Awesomeness

57. Thanks! Marcus Deglos e: marcus@deglos.com t: @manarth irc: manarth

58. Picture credits Thanks to these CC-licencedflickr users Mac Book Pro - http://www.flickr.com/photos/wicho/ F1 car - http://www.flickr.com/photos/w3i_yu/ Banger car - http://www.flickr.com/photos/photoplod Dries - http://www.flickr.com/photos/boris Dries+laptop - http://www.flickr.com/photos/azugaldia Webchick - http://www.flickr.com/photos/mortendk Webchick - http://www.flickr.com/photos/puregin Fire - http://www.flickr.com/photos/benwatts Hourglass - http://www.flickr.com/photos/opendemocracy Stopwatch - http://www.flickr.com/photos/purplemattfish