Varnish bof
Upcoming SlideShare
Loading in...5
×
 

Varnish bof

on

  • 4,744 views

#drupaldevdays birds of a feather talk on Varnish and ESI - Edge Side Includes - at Brussels.

#drupaldevdays birds of a feather talk on Varnish and ESI - Edge Side Includes - at Brussels.

Statistics

Views

Total Views
4,744
Views on SlideShare
4,479
Embed Views
265

Actions

Likes
7
Downloads
43
Comments
2

2 Embeds 265

http://www.scoop.it 264
http://ericope.com 1

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
  • ESI isn't currently used on D.O. - the ESI module's being revamped for Drupal 7 (and to provide a new extensible ESI architecture in the module)...once the new module's ready I'll be writing a blogpost for http://deglos.com/ to demo it.
    Are you sure you want to
    Your message goes here
    Processing…
  • Hot stuff, thanks :)

    EDIT: Could the author further explain if ESI is actually used - or not used - for authenticated users on drupal.org? Or was this just how it 'could' be done?

    I will be implementing Varnish+ESI for authenticated users on a really big site, and have found that detailed info of real ESI implementations with Drupal still seem very rare.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • - Old hands: feel free to switch off for a while- New to Varnish? Welcome along
  • You've heard that varnish is *quicker* - so how is it quicker?
  • - So what happens when you make a request?
  • > Pic of absurd personalisation
  • What if it's *really* private info?This *does* happen!
  • - Add EGGTIMER pic
  • Page is requested,Gets a response
  • - Add a pic of a page with an ESI TAG
  • - Requests the ESI URL
  • - More requests makes it slower!
  • This shows how important *measurements* areI'll talk about measurements laterESIs can be cached
  • - Warm the cache

Varnish bof Varnish bof Presentation Transcript

  • Varnish
    BOF!
  • What's this talk about?
    A basic intro to Varnish and reverse proxies
    Some cool ESI stuff
    How to setup, implement + configure Varnish/ESI in Drupal
    Teh Awesomeness of speed
  • Why Varnish?
  • Drupal performs…just…
  • How much faster?
    Drupal
    20 reqs/sec?
    Varnish
    200 reqs/sec?
  • When personalisation goes wrong
  • Dries logs in
  • Drupal creates a personalised page
    Logged in as dries
  • The page gets cached
    Varnish
  • When webchick hits the site
  • The page is fetched from the cache
    Varnish
  • The personalisation is incorrect
    Logged in as dries
  • WTF?
    WTF?
  • No-cache headers can prevent this
    Varnish
  • But now it's slow :-(
  • EdgeSideIncludes
  • ESI: Tales from the past
    The idea originated in SSI
    SSI, but moved to the edge
    ESI is a W3C standard
    ESI is a bunch of xml tags/attributes.
    Supported by:
    Varnish (partial support – no ESI variables)
    Akamai
    Some Nginx support
    ???
  • Dries is visiting the site
  • The page is requested
    Varnish
    http://example.com/node/1
  • The page includes an ESI tag
    <esisrc= "http://example.com/esi/user/0" />
  • The proxy sends a second request
    http://example.com/esi/user/0
    Varnish
    http://example.com/node/1
  • The ESI URL sends a HTML snip
    <div id="userinfo">
    <a title="View &amp; edit your user profile">Logged in as dries</a>
    <a href="/logout">Logout</a>
    </div>
  • The ESI code is merged with the page
  • Proxy forwards the page on
    http://example.com/esi/user/0
    Varnish
    http://example.com/node/1
  • This doesn’t sound quicker
  • The Importance of Tests
    It's important to measure the results(I'll come to measuring tools later!)
    The ESI snippets can be cached too
  • Cached ESIs
  • 1st Page = 2 Requests
    http://example.com/esi/user/0
    Varnish
    http://example.com/node/1
  • 2nd Page = 1 request
    Varnish
    http://example.com/node/2
  • 1 request for each new page
    10 pages = 11 requests
    Each page is cached
    The ESI tag is cached
  • The next visitor gets the page for free
    Varnish
    Page fetched from cachehttp://example.com/node/1
  • 1st page = 1 request (for the ESI)
    http://example.com/esi/user/0
    Varnish
    Page fetched from cachehttp://example.com/node/1
  • Following pages all served from cache
    Varnish
  • 1 request for each new page
    10 pages = 11 requests
    The ESI tag is cached
    Each page is cached
    Subsequent requests have 1 request for the ESI tag
    20 pages = 12 requests (10 pages + 2 ESI tags)
  • Fast enough yet?
    Warm the cache
  • A modern infrastructure
    Varnish
    HA Proxy
    TehIntertubes
    Service webheadadmin/cron/emails/??
  • Spidering the site
    Varnish
    HA Proxy
    TehIntertubes
    Link-checker /spider app.
    Service webheadadmin/cron/emails/??
  • Dries is back for more
  • Each user generates 1 request
    http://example.com/esi/user/0
    Varnish
    Page fetched from cachehttp://example.com/node/1
  • 5 users, 40 pages
    5 backend requests!
  • Theory Over
  • Basics Varnish+Drupal
  • Installing Varnish
    sudo apt-get install varnishsudo yum install varnish
    Edit /etc/default/varnish
    Edit /etc/varnish/default.vcl
    Usual config:
    Varnish listens on port 80
    Apache listens on port 8080 (or any available port)
    SSL is more complex – sometimes Pound is used for SSL decryption
  • VCLs are awesome!
    Varnish configuration is controlled by VCL: Varnish Configuration Language.
    Documented athttp://www.varnish-cache.org/trac/wiki/VCL
    Drupal VCL examples can be found in:
    Mercury (Amazon AMI)
    ESI module
    You can write C code in VCLs. Oh my.
  • Keeping the cache current
    Varnish admin port
    Varnish
  • Varnish Management Port
    The Varnish admin port uses a plain-text protocol to command Varnish
    Test by:telnet localhost 6093
    List commands by typing help
    Purge cached pages: url.purge <regexp>E.g. url.purge .*
    The Varnish module provides an API.
  • Use with caution!
    The default behaviour of the Varnish module is toclear the entire cachewhen you edit a node.
  • Enter…the Expires module
    Came from the Boost module
    Clears specific cache pages – not the entire cache
    Supports path-alias URLs as well as node/1 paths.
    Provides hook_expire_cache_alterto add URLs (e.g. nodes displayed in a page-view)
    Supports other cache modules, not just Varnish.
  • I can haz ESI?
    The ESI module provides block integration
  • Configuring ESI
    Download the module
    Configure the blocks to serve via ESI, and their TTL
    Modify the Varnish VCL (the ESI module comes with example VCLs to use
    Coming soon:
    D7
    Panels integration
  • Benchmarking
    Some tools:
    J-meter
    ab (apache-benchmark)
    Seige
    Approaches:
    Hammer your URL(s)ab –n 100 –c5 http://example.com/
    Replay your log files
  • Pitfalls await
    VCLs are complex
    Per-user cache rules are even scarier!
    Varnish may be slower
    Risk exposing private data
  • Alternatives
    Boost module
    No need to install a new service on the server
    Static file cache
    Advanced cache module
    Authcache module
    Other proxies:
    Squid
    Nginx
    Akamai?
  • Summary
  • How hard?
  • Varnish
    Varnish module
    Expire module
    ESI module
    Teh Awesomeness
  • Thanks!
    Marcus Deglos
    e: marcus@deglos.com
    t: @manarth
    irc: manarth
  • Picture credits
    Thanks to these CC-licencedflickr users
    Mac Book Pro - http://www.flickr.com/photos/wicho/
    F1 car - http://www.flickr.com/photos/w3i_yu/
    Banger car - http://www.flickr.com/photos/photoplod
    Dries - http://www.flickr.com/photos/boris
    Dries+laptop - http://www.flickr.com/photos/azugaldia
    Webchick - http://www.flickr.com/photos/mortendk
    Webchick - http://www.flickr.com/photos/puregin
    Fire - http://www.flickr.com/photos/benwatts
    Hourglass - http://www.flickr.com/photos/opendemocracy
    Stopwatch - http://www.flickr.com/photos/purplemattfish