Varnish bof

5,295 views
5,112 views

Published on

#drupaldevdays birds of a feather talk on Varnish and ESI - Edge Side Includes - at Brussels.

Published in: Technology
2 Comments
8 Likes
Statistics
Notes
  • ESI isn't currently used on D.O. - the ESI module's being revamped for Drupal 7 (and to provide a new extensible ESI architecture in the module)...once the new module's ready I'll be writing a blogpost for http://deglos.com/ to demo it.
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Hot stuff, thanks :)

    EDIT: Could the author further explain if ESI is actually used - or not used - for authenticated users on drupal.org? Or was this just how it 'could' be done?

    I will be implementing Varnish+ESI for authenticated users on a really big site, and have found that detailed info of real ESI implementations with Drupal still seem very rare.
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Views
Total views
5,295
On SlideShare
0
From Embeds
0
Number of Embeds
290
Actions
Shares
0
Downloads
49
Comments
2
Likes
8
Embeds 0
No embeds

No notes for slide
  • - Old hands: feel free to switch off for a while- New to Varnish? Welcome along
  • You've heard that varnish is *quicker* - so how is it quicker?
  • - So what happens when you make a request?
  • > Pic of absurd personalisation
  • What if it's *really* private info?This *does* happen!
  • - Add EGGTIMER pic
  • Page is requested,Gets a response
  • - Add a pic of a page with an ESI TAG
  • - Requests the ESI URL
  • - More requests makes it slower!
  • This shows how important *measurements* areI'll talk about measurements laterESIs can be cached
  • - Warm the cache
  • Varnish bof

    1. 1. Varnish<br />BOF!<br />
    2. 2. What's this talk about?<br />A basic intro to Varnish and reverse proxies<br />Some cool ESI stuff<br />How to setup, implement + configure Varnish/ESI in Drupal<br />Teh Awesomeness of speed<br />
    3. 3. Why Varnish?<br />
    4. 4. Drupal performs…just…<br />
    5. 5. How much faster?<br />Drupal<br />20 reqs/sec?<br />Varnish<br />200 reqs/sec?<br />
    6. 6. When personalisation goes wrong<br />
    7. 7. Dries logs in<br />
    8. 8. Drupal creates a personalised page<br />Logged in as dries<br />
    9. 9. The page gets cached<br />Varnish<br />
    10. 10. When webchick hits the site<br />
    11. 11. The page is fetched from the cache<br />Varnish<br />
    12. 12. The personalisation is incorrect<br />Logged in as dries<br />
    13. 13. WTF?<br />WTF?<br />
    14. 14. No-cache headers can prevent this<br />Varnish<br />
    15. 15. But now it's slow :-(<br />
    16. 16. EdgeSideIncludes<br />
    17. 17. ESI: Tales from the past<br />The idea originated in SSI<br />SSI, but moved to the edge<br />ESI is a W3C standard<br />ESI is a bunch of xml tags/attributes.<br />Supported by:<br />Varnish (partial support – no ESI variables)<br />Akamai<br />Some Nginx support<br />???<br />
    18. 18. Dries is visiting the site<br />
    19. 19. The page is requested<br />Varnish<br />http://example.com/node/1<br />
    20. 20. The page includes an ESI tag<br /><esisrc= "http://example.com/esi/user/0" /><br />
    21. 21. The proxy sends a second request<br />http://example.com/esi/user/0<br />Varnish<br />http://example.com/node/1<br />
    22. 22. The ESI URL sends a HTML snip<br /><div id="userinfo"><br /> <a title="View &amp; edit your user profile">Logged in as dries</a><br /> <a href="/logout">Logout</a><br /></div><br />
    23. 23. The ESI code is merged with the page<br />
    24. 24. Proxy forwards the page on<br />http://example.com/esi/user/0<br />Varnish<br />http://example.com/node/1<br />
    25. 25. This doesn’t sound quicker<br />
    26. 26. The Importance of Tests<br />It's important to measure the results(I'll come to measuring tools later!)<br />The ESI snippets can be cached too<br />
    27. 27. Cached ESIs<br />
    28. 28. 1st Page = 2 Requests<br />http://example.com/esi/user/0<br />Varnish<br />http://example.com/node/1<br />
    29. 29. 2nd Page = 1 request<br />Varnish<br />http://example.com/node/2<br />
    30. 30. 1 request for each new page<br />10 pages = 11 requests<br />Each page is cached<br />The ESI tag is cached<br />
    31. 31. The next visitor gets the page for free<br />Varnish<br />Page fetched from cachehttp://example.com/node/1<br />
    32. 32. 1st page = 1 request (for the ESI)<br />http://example.com/esi/user/0<br />Varnish<br />Page fetched from cachehttp://example.com/node/1<br />
    33. 33. Following pages all served from cache<br />Varnish<br />
    34. 34. 1 request for each new page<br />10 pages = 11 requests<br />The ESI tag is cached<br />Each page is cached<br />Subsequent requests have 1 request for the ESI tag<br />20 pages = 12 requests (10 pages + 2 ESI tags)<br />
    35. 35. Fast enough yet?<br />Warm the cache<br />
    36. 36. A modern infrastructure<br />Varnish<br />HA Proxy<br />TehIntertubes<br />Service webheadadmin/cron/emails/??<br />
    37. 37. Spidering the site<br />Varnish<br />HA Proxy<br />TehIntertubes<br />Link-checker /spider app.<br />Service webheadadmin/cron/emails/??<br />
    38. 38. Dries is back for more<br />
    39. 39. Each user generates 1 request<br />http://example.com/esi/user/0<br />Varnish<br />Page fetched from cachehttp://example.com/node/1<br />
    40. 40. 5 users, 40 pages<br />5 backend requests!<br />
    41. 41. Theory Over<br />
    42. 42. Basics Varnish+Drupal<br />
    43. 43. Installing Varnish<br />sudo apt-get install varnishsudo yum install varnish<br />Edit /etc/default/varnish<br />Edit /etc/varnish/default.vcl<br />Usual config:<br />Varnish listens on port 80<br />Apache listens on port 8080 (or any available port)<br />SSL is more complex – sometimes Pound is used for SSL decryption<br />
    44. 44. VCLs are awesome!<br />Varnish configuration is controlled by VCL: Varnish Configuration Language.<br />Documented athttp://www.varnish-cache.org/trac/wiki/VCL<br />Drupal VCL examples can be found in:<br />Mercury (Amazon AMI)<br />ESI module<br />You can write C code in VCLs. Oh my.<br />
    45. 45. Keeping the cache current<br />Varnish admin port<br />Varnish<br />
    46. 46. Varnish Management Port<br />The Varnish admin port uses a plain-text protocol to command Varnish<br />Test by:telnet localhost 6093<br />List commands by typing help<br />Purge cached pages: url.purge <regexp>E.g. url.purge .*<br />The Varnish module provides an API.<br />
    47. 47. Use with caution!<br />The default behaviour of the Varnish module is toclear the entire cachewhen you edit a node.<br />
    48. 48. Enter…the Expires module<br />Came from the Boost module<br />Clears specific cache pages – not the entire cache<br />Supports path-alias URLs as well as node/1 paths.<br />Provides hook_expire_cache_alterto add URLs (e.g. nodes displayed in a page-view)<br />Supports other cache modules, not just Varnish.<br />
    49. 49. I can haz ESI?<br />The ESI module provides block integration<br />
    50. 50. Configuring ESI<br />Download the module<br />Configure the blocks to serve via ESI, and their TTL<br />Modify the Varnish VCL (the ESI module comes with example VCLs to use<br />Coming soon:<br />D7<br />Panels integration<br />
    51. 51. Benchmarking<br />Some tools:<br />J-meter<br />ab (apache-benchmark)<br />Seige<br />Approaches:<br />Hammer your URL(s)ab –n 100 –c5 http://example.com/<br />Replay your log files<br />
    52. 52. Pitfalls await<br />VCLs are complex<br />Per-user cache rules are even scarier!<br />Varnish may be slower<br />Risk exposing private data<br />
    53. 53. Alternatives<br />Boost module<br />No need to install a new service on the server<br />Static file cache<br />Advanced cache module<br />Authcache module<br />Other proxies:<br />Squid<br />Nginx<br />Akamai?<br />
    54. 54. Summary<br />
    55. 55. How hard?<br />
    56. 56. Varnish<br />Varnish module<br />Expire module<br />ESI module<br />Teh Awesomeness<br />
    57. 57. Thanks!<br />Marcus Deglos<br />e: marcus@deglos.com<br />t: @manarth<br />irc: manarth<br />
    58. 58. Picture credits<br />Thanks to these CC-licencedflickr users<br />Mac Book Pro - http://www.flickr.com/photos/wicho/<br />F1 car - http://www.flickr.com/photos/w3i_yu/<br />Banger car - http://www.flickr.com/photos/photoplod<br />Dries - http://www.flickr.com/photos/boris<br />Dries+laptop - http://www.flickr.com/photos/azugaldia<br />Webchick - http://www.flickr.com/photos/mortendk<br />Webchick - http://www.flickr.com/photos/puregin<br />Fire - http://www.flickr.com/photos/benwatts<br />Hourglass - http://www.flickr.com/photos/opendemocracy<br />Stopwatch - http://www.flickr.com/photos/purplemattfish<br />

    ×