Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Varnish<br />BOF!<br />
What's this talk about?<br />A basic intro to Varnish and reverse proxies<br />Some cool ESI stuff<br />How to setup, impl...
Why Varnish?<br />
Drupal performs…just…<br />
How much faster?<br />Drupal<br />20 reqs/sec?<br />Varnish<br />200 reqs/sec?<br />
When personalisation goes wrong<br />
Dries logs in<br />
Drupal creates a personalised page<br />Logged in as dries<br />
The page gets cached<br />Varnish<br />
When webchick hits the site<br />
The page is fetched from the cache<br />Varnish<br />
The personalisation is incorrect<br />Logged in as dries<br />
WTF?<br />WTF?<br />
No-cache headers can prevent this<br />Varnish<br />
But now it's slow :-(<br />
EdgeSideIncludes<br />
ESI: Tales from the past<br />The idea originated in SSI<br />SSI, but moved to the edge<br />ESI is a W3C standard<br />E...
Dries is visiting the site<br />
The page is requested<br />Varnish<br />http://example.com/node/1<br />
The page includes an ESI tag<br /><esisrc=  "http://example.com/esi/user/0"  /><br />
The proxy sends a second request<br />http://example.com/esi/user/0<br />Varnish<br />http://example.com/node/1<br />
The ESI URL sends a HTML snip<br /><div id="userinfo"><br />  <a title="View &amp; edit your user profile">Logged in as dr...
The ESI code is merged with the page<br />
Proxy forwards the page on<br />http://example.com/esi/user/0<br />Varnish<br />http://example.com/node/1<br />
This doesn’t sound quicker<br />
The Importance of Tests<br />It's important to measure the results(I'll come to measuring tools later!)<br />The ESI snipp...
Cached ESIs<br />
1st Page = 2 Requests<br />http://example.com/esi/user/0<br />Varnish<br />http://example.com/node/1<br />
2nd Page = 1 request<br />Varnish<br />http://example.com/node/2<br />
1 request for each new page<br />10 pages = 11 requests<br />Each page is cached<br />The ESI tag is cached<br />
The next visitor gets the page for free<br />Varnish<br />Page fetched from cachehttp://example.com/node/1<br />
1st page = 1 request (for the ESI)<br />http://example.com/esi/user/0<br />Varnish<br />Page fetched from cachehttp://exam...
Following pages all served from cache<br />Varnish<br />
1 request for each new page<br />10 pages = 11 requests<br />The ESI tag is cached<br />Each page is cached<br />Subsequen...
Fast enough yet?<br />Warm the cache<br />
A modern infrastructure<br />Varnish<br />HA Proxy<br />TehIntertubes<br />Service webheadadmin/cron/emails/??<br />
Spidering the site<br />Varnish<br />HA Proxy<br />TehIntertubes<br />Link-checker /spider app.<br />Service webheadadmin/...
Dries is back for more<br />
Each user generates 1 request<br />http://example.com/esi/user/0<br />Varnish<br />Page fetched from cachehttp://example.c...
5 users, 40 pages<br />5 backend requests!<br />
Theory Over<br />
Basics Varnish+Drupal<br />
Installing Varnish<br />sudo apt-get install varnishsudo yum install varnish<br />Edit /etc/default/varnish<br />Edit /etc...
VCLs are awesome!<br />Varnish configuration is controlled by VCL: Varnish Configuration Language.<br />Documented athttp:...
Keeping the cache current<br />Varnish admin port<br />Varnish<br />
Varnish Management Port<br />The Varnish admin port uses a plain-text protocol to command Varnish<br />Test by:telnet loca...
Use with caution!<br />The default behaviour of the Varnish module is toclear the entire cachewhen you edit a node.<br />
Enter…the Expires module<br />Came from the Boost module<br />Clears specific cache pages – not the entire cache<br />Supp...
I can haz ESI?<br />The ESI module provides block integration<br />
Configuring ESI<br />Download the module<br />Configure the blocks to serve via ESI, and their TTL<br />Modify the Varnish...
Benchmarking<br />Some tools:<br />J-meter<br />ab (apache-benchmark)<br />Seige<br />Approaches:<br />Hammer your URL(s)a...
Pitfalls await<br />VCLs are complex<br />Per-user cache rules are even scarier!<br />Varnish may be slower<br />Risk expo...
Alternatives<br />Boost module<br />No need to install a new service on the server<br />Static file cache<br />Advanced ca...
Summary<br />
How hard?<br />
Varnish<br />Varnish module<br />Expire module<br />ESI module<br />Teh Awesomeness<br />
Thanks!<br />Marcus Deglos<br />e: marcus@deglos.com<br />t: @manarth<br />irc: manarth<br />
Picture credits<br />Thanks to these CC-licencedflickr users<br />Mac Book Pro - http://www.flickr.com/photos/wicho/<br />...
Upcoming SlideShare
Loading in …5
×

Varnish bof

#drupaldevdays birds of a feather talk on Varnish and ESI - Edge Side Includes - at Brussels.

  • Login to see the comments

Varnish bof

  1. 1. Varnish<br />BOF!<br />
  2. 2. What's this talk about?<br />A basic intro to Varnish and reverse proxies<br />Some cool ESI stuff<br />How to setup, implement + configure Varnish/ESI in Drupal<br />Teh Awesomeness of speed<br />
  3. 3. Why Varnish?<br />
  4. 4. Drupal performs…just…<br />
  5. 5. How much faster?<br />Drupal<br />20 reqs/sec?<br />Varnish<br />200 reqs/sec?<br />
  6. 6. When personalisation goes wrong<br />
  7. 7. Dries logs in<br />
  8. 8. Drupal creates a personalised page<br />Logged in as dries<br />
  9. 9. The page gets cached<br />Varnish<br />
  10. 10. When webchick hits the site<br />
  11. 11. The page is fetched from the cache<br />Varnish<br />
  12. 12. The personalisation is incorrect<br />Logged in as dries<br />
  13. 13. WTF?<br />WTF?<br />
  14. 14. No-cache headers can prevent this<br />Varnish<br />
  15. 15. But now it's slow :-(<br />
  16. 16. EdgeSideIncludes<br />
  17. 17. ESI: Tales from the past<br />The idea originated in SSI<br />SSI, but moved to the edge<br />ESI is a W3C standard<br />ESI is a bunch of xml tags/attributes.<br />Supported by:<br />Varnish (partial support – no ESI variables)<br />Akamai<br />Some Nginx support<br />???<br />
  18. 18. Dries is visiting the site<br />
  19. 19. The page is requested<br />Varnish<br />http://example.com/node/1<br />
  20. 20. The page includes an ESI tag<br /><esisrc= "http://example.com/esi/user/0" /><br />
  21. 21. The proxy sends a second request<br />http://example.com/esi/user/0<br />Varnish<br />http://example.com/node/1<br />
  22. 22. The ESI URL sends a HTML snip<br /><div id="userinfo"><br /> <a title="View &amp; edit your user profile">Logged in as dries</a><br /> <a href="/logout">Logout</a><br /></div><br />
  23. 23. The ESI code is merged with the page<br />
  24. 24. Proxy forwards the page on<br />http://example.com/esi/user/0<br />Varnish<br />http://example.com/node/1<br />
  25. 25. This doesn’t sound quicker<br />
  26. 26. The Importance of Tests<br />It's important to measure the results(I'll come to measuring tools later!)<br />The ESI snippets can be cached too<br />
  27. 27. Cached ESIs<br />
  28. 28. 1st Page = 2 Requests<br />http://example.com/esi/user/0<br />Varnish<br />http://example.com/node/1<br />
  29. 29. 2nd Page = 1 request<br />Varnish<br />http://example.com/node/2<br />
  30. 30. 1 request for each new page<br />10 pages = 11 requests<br />Each page is cached<br />The ESI tag is cached<br />
  31. 31. The next visitor gets the page for free<br />Varnish<br />Page fetched from cachehttp://example.com/node/1<br />
  32. 32. 1st page = 1 request (for the ESI)<br />http://example.com/esi/user/0<br />Varnish<br />Page fetched from cachehttp://example.com/node/1<br />
  33. 33. Following pages all served from cache<br />Varnish<br />
  34. 34. 1 request for each new page<br />10 pages = 11 requests<br />The ESI tag is cached<br />Each page is cached<br />Subsequent requests have 1 request for the ESI tag<br />20 pages = 12 requests (10 pages + 2 ESI tags)<br />
  35. 35. Fast enough yet?<br />Warm the cache<br />
  36. 36. A modern infrastructure<br />Varnish<br />HA Proxy<br />TehIntertubes<br />Service webheadadmin/cron/emails/??<br />
  37. 37. Spidering the site<br />Varnish<br />HA Proxy<br />TehIntertubes<br />Link-checker /spider app.<br />Service webheadadmin/cron/emails/??<br />
  38. 38. Dries is back for more<br />
  39. 39. Each user generates 1 request<br />http://example.com/esi/user/0<br />Varnish<br />Page fetched from cachehttp://example.com/node/1<br />
  40. 40. 5 users, 40 pages<br />5 backend requests!<br />
  41. 41. Theory Over<br />
  42. 42. Basics Varnish+Drupal<br />
  43. 43. Installing Varnish<br />sudo apt-get install varnishsudo yum install varnish<br />Edit /etc/default/varnish<br />Edit /etc/varnish/default.vcl<br />Usual config:<br />Varnish listens on port 80<br />Apache listens on port 8080 (or any available port)<br />SSL is more complex – sometimes Pound is used for SSL decryption<br />
  44. 44. VCLs are awesome!<br />Varnish configuration is controlled by VCL: Varnish Configuration Language.<br />Documented athttp://www.varnish-cache.org/trac/wiki/VCL<br />Drupal VCL examples can be found in:<br />Mercury (Amazon AMI)<br />ESI module<br />You can write C code in VCLs. Oh my.<br />
  45. 45. Keeping the cache current<br />Varnish admin port<br />Varnish<br />
  46. 46. Varnish Management Port<br />The Varnish admin port uses a plain-text protocol to command Varnish<br />Test by:telnet localhost 6093<br />List commands by typing help<br />Purge cached pages: url.purge <regexp>E.g. url.purge .*<br />The Varnish module provides an API.<br />
  47. 47. Use with caution!<br />The default behaviour of the Varnish module is toclear the entire cachewhen you edit a node.<br />
  48. 48. Enter…the Expires module<br />Came from the Boost module<br />Clears specific cache pages – not the entire cache<br />Supports path-alias URLs as well as node/1 paths.<br />Provides hook_expire_cache_alterto add URLs (e.g. nodes displayed in a page-view)<br />Supports other cache modules, not just Varnish.<br />
  49. 49. I can haz ESI?<br />The ESI module provides block integration<br />
  50. 50. Configuring ESI<br />Download the module<br />Configure the blocks to serve via ESI, and their TTL<br />Modify the Varnish VCL (the ESI module comes with example VCLs to use<br />Coming soon:<br />D7<br />Panels integration<br />
  51. 51. Benchmarking<br />Some tools:<br />J-meter<br />ab (apache-benchmark)<br />Seige<br />Approaches:<br />Hammer your URL(s)ab –n 100 –c5 http://example.com/<br />Replay your log files<br />
  52. 52. Pitfalls await<br />VCLs are complex<br />Per-user cache rules are even scarier!<br />Varnish may be slower<br />Risk exposing private data<br />
  53. 53. Alternatives<br />Boost module<br />No need to install a new service on the server<br />Static file cache<br />Advanced cache module<br />Authcache module<br />Other proxies:<br />Squid<br />Nginx<br />Akamai?<br />
  54. 54. Summary<br />
  55. 55. How hard?<br />
  56. 56. Varnish<br />Varnish module<br />Expire module<br />ESI module<br />Teh Awesomeness<br />
  57. 57. Thanks!<br />Marcus Deglos<br />e: marcus@deglos.com<br />t: @manarth<br />irc: manarth<br />
  58. 58. Picture credits<br />Thanks to these CC-licencedflickr users<br />Mac Book Pro - http://www.flickr.com/photos/wicho/<br />F1 car - http://www.flickr.com/photos/w3i_yu/<br />Banger car - http://www.flickr.com/photos/photoplod<br />Dries - http://www.flickr.com/photos/boris<br />Dries+laptop - http://www.flickr.com/photos/azugaldia<br />Webchick - http://www.flickr.com/photos/mortendk<br />Webchick - http://www.flickr.com/photos/puregin<br />Fire - http://www.flickr.com/photos/benwatts<br />Hourglass - http://www.flickr.com/photos/opendemocracy<br />Stopwatch - http://www.flickr.com/photos/purplemattfish<br />

×