Marcus Deglos
@manarth
#cdl2011
> Old
           faithful, serving
           sites since 1996.
         > Quick
         > Simple
Apache
Apache's model


             Web server
Apache's model with PHP


                      Web server




               PHP engine
Do it with Drupal


                   Web server
                                  DB


                                index.php


            PHP engine
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML+RDFa 1.0//EN"

"http://www.w3.org/MarkUp/DTD/xhtml-rdfa-1.dtd">
<html

                        Example HTML
xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" version="XHTML+RDFa 1.0" dir="ltr"

xmlns:content="http://purl.org/rss/1.0/modules/content/"

xmlns:dc="http://purl.org/dc/terms/"
 xmlns:foaf="http://xmlns.com/foaf/0.1/"

xmlns:og="http://ogp.me/ns#"
 xmlns:rdfs="http://www.w3.org/2000/01/rdf-schema#"

xmlns:sioc="http://rdfs.org/sioc/ns#"
 xmlns:sioct="http://rdfs.org/sioc/types#"

xmlns:skos="http://www.w3.org/2004/02/skos/core#"

xmlns:xsd="http://www.w3.org/2001/XMLSchema#">

<head
profile="http://www.w3.org/1999/xhtml/vocab">
 <meta http-equiv="Content-Type"
content="text/html; charset=utf-8" />
<link rel="shortcut icon"
href="http://testbed.local/misc/favicon.ico" type="image/vnd.microsoft.icon" />
<meta
name="Generator" content="Drupal 7 (http://drupal.org)" />
 <title>Welcome to Site-Install |
Site-Install</title>




                HTML from homepage after
                          running drushsi
Drag race results
> Prepared in
          advance
        > Typically uses a
          build-script
        > Produces static
          pages
        > Quick
        > No complex
          behaviour
        > Boost module is an
Baked     example
> Pages created on
  the fly
> CMS or other
  complex dynamic
  system
> Allows complex
  behaviour
> Drupal does this
> Slow
                     Fried
> Software
                     package
                     (daemon) for
                     linux
                   > Reverse proxy,
                     such as Squid,
                     Nginx and Zeus
What is Varnish?   > Cache
How Varnish works
   GET /blog
   Host   deglos.com
   User-AgentMozilla/5.0 (Macintosh; Intel
   Mac OS X 10.6; rv:8.0.1) Gecko/20100101
   Firefox/8.0.1
   Accept
      text/html,application/xhtml+xml,applic
   ation/xml;q=0.9,*/*;q=0.8
   Varnish                Drupal
   Accept-Language en-us,en;q=0.5
   Accept-Encoding gzip, deflate
   Accept-Charset    ISO-8859-1,utf-
   8;q=0.7,*;q=0.7
   Connectionkeep-alive
Varnish is slower
  If Varnish passes every request to Drupal
  – and acts just as a MITM – your site will
  be slower.



  Varnish               Drupal
Varnish needs to cache



     Varnish           Drupal




               Cache
Varnish + cache is quicker
   10x faster
   (or more)



          Varnish           Drupal




                    Cache
Personalisation is a problem



        Varnish           Drupal




                  Cache
The Uncacheable



  Varnish           Drupal




            Cache
Example VCL:
#This is a basic VCL configuration file for
  varnish.
backend default {
  .host = "127.0.0.1";
  .port = "80";
}

sub vcl_fetch {
  unset obj.http.Cache-Control;
  unset obj.http.Expires;
  set obj.ttl = 86400s;
}
vcl.list
telnet localhost6082
Trying ::1...
Connected to localhost.
Escape character is '^]'.
vcl.list
200 23
active          1 boot
help
telnet localhost6082
Trying ::1...
Connected to localhost.
Escape character is '^]'.
help
200 376
help [command]
ping [timestamp]
status
start
stop
stats
...
url.purge
telnet localhost6082
Trying ::1...
Connected to localhost.
Escape character is '^]'.
url.purge .*
200 0
Setting up Varnish
> Download varnish daemon
  debian: apt-get install varnish
  redhat:yum install varnish
> Re-configure Apache to listen on port 8080
  (or other non-standard port)
> Configure Varnish to listen on port 80
> Configure Varnish VCL to connect to
  Apache
> VCL tweaks (default does not cache
  Drupal)
Serving ESIs



Varnish           Drupal




          Cache
Serving ESIs



Varnish           Drupal




          Cache
Serving ESIs
<div id="nav-masthead" class="grid-12">
<ul>
<li class="homepage active first active">
<a href="/home" class="active active">Drupal Homepage</a>
</li>
<li class="dashboard last">
<a href="/user/dashboard">Your Dashboard</a>
</li>
</ul>                   Varnish                      Drupal
<div id="userinfo">
<a href="/user" title="View &amp; edit your user profile">Logged in as
manarth</a>
<a href="/logout">Log out</a>
</div>
</div><!-- /#nav-masthead -->            Cache
Serving ESIs
<div id="nav-masthead" class="grid-12">
<ul>
<li class="homepage active first active">
<a href="/home" class="active active">Drupal Homepage</a>
</li>
<li class="dashboard last">
<a href="/user/dashboard">Your Dashboard</a>
</li>
</ul>                 Varnish                 Drupal
<esisrc="http://example.com/esi/userinfo" />




</div><!-- /#nav-masthead -->     Cache
Serving ESIs

<div id="userinfo">
<a href="/user" title="View &amp; edit your user profile">Logged in
as manarth</a>
<a href="/logout">Log out</a>
</div>
                 Varnish                     Drupal




                                  Cache
What do you need in order to do
             ESIs?
> Replace personalisationwith ESI tags
  Theme-alters?
> Provide backend hook_menuhandlers for
  each ESI tag
> Ensure that Varnish is caching
> Rule 35 applies
Configuring ESI Blocks
Flickr Credits
> Babbage engine
  http://www.flickr.com/photos/melgart/4444251
  225
> Cookies
  http://www.flickr.com/photos/sabine01/24855
  02578
> Frying wok
  http://www.flickr.com/photos/18403292@N00/
  3048993
> Varnish
  http://www.flickr.com/photos/dawniecakes/46
  86467791
Drupal, varnish, esi - Toulouse November 2

Drupal, varnish, esi - Toulouse November 2

  • 2.
  • 4.
    > Old faithful, serving sites since 1996. > Quick > Simple Apache
  • 5.
    Apache's model Web server
  • 6.
    Apache's model withPHP Web server PHP engine
  • 7.
    Do it withDrupal Web server DB index.php PHP engine
  • 8.
    <!DOCTYPE html PUBLIC"-//W3C//DTD XHTML+RDFa 1.0//EN"
 "http://www.w3.org/MarkUp/DTD/xhtml-rdfa-1.dtd">
<html Example HTML xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" version="XHTML+RDFa 1.0" dir="ltr"
 xmlns:content="http://purl.org/rss/1.0/modules/content/"
 xmlns:dc="http://purl.org/dc/terms/"
 xmlns:foaf="http://xmlns.com/foaf/0.1/"
 xmlns:og="http://ogp.me/ns#"
 xmlns:rdfs="http://www.w3.org/2000/01/rdf-schema#"
 xmlns:sioc="http://rdfs.org/sioc/ns#"
 xmlns:sioct="http://rdfs.org/sioc/types#"
 xmlns:skos="http://www.w3.org/2004/02/skos/core#"
 xmlns:xsd="http://www.w3.org/2001/XMLSchema#">

<head profile="http://www.w3.org/1999/xhtml/vocab">
 <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<link rel="shortcut icon" href="http://testbed.local/misc/favicon.ico" type="image/vnd.microsoft.icon" />
<meta name="Generator" content="Drupal 7 (http://drupal.org)" />
 <title>Welcome to Site-Install | Site-Install</title> HTML from homepage after running drushsi
  • 9.
  • 13.
    > Prepared in advance > Typically uses a build-script > Produces static pages > Quick > No complex behaviour > Boost module is an Baked example
  • 14.
    > Pages createdon the fly > CMS or other complex dynamic system > Allows complex behaviour > Drupal does this > Slow Fried
  • 16.
    > Software package (daemon) for linux > Reverse proxy, such as Squid, Nginx and Zeus What is Varnish? > Cache
  • 17.
    How Varnish works GET /blog Host deglos.com User-AgentMozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:8.0.1) Gecko/20100101 Firefox/8.0.1 Accept text/html,application/xhtml+xml,applic ation/xml;q=0.9,*/*;q=0.8 Varnish Drupal Accept-Language en-us,en;q=0.5 Accept-Encoding gzip, deflate Accept-Charset ISO-8859-1,utf- 8;q=0.7,*;q=0.7 Connectionkeep-alive
  • 18.
    Varnish is slower If Varnish passes every request to Drupal – and acts just as a MITM – your site will be slower. Varnish Drupal
  • 19.
    Varnish needs tocache Varnish Drupal Cache
  • 20.
    Varnish + cacheis quicker 10x faster (or more) Varnish Drupal Cache
  • 21.
    Personalisation is aproblem Varnish Drupal Cache
  • 22.
    The Uncacheable Varnish Drupal Cache
  • 27.
    Example VCL: #This isa basic VCL configuration file for varnish. backend default { .host = "127.0.0.1"; .port = "80"; } sub vcl_fetch { unset obj.http.Cache-Control; unset obj.http.Expires; set obj.ttl = 86400s; }
  • 30.
    vcl.list telnet localhost6082 Trying ::1... Connectedto localhost. Escape character is '^]'. vcl.list 200 23 active 1 boot
  • 31.
    help telnet localhost6082 Trying ::1... Connectedto localhost. Escape character is '^]'. help 200 376 help [command] ping [timestamp] status start stop stats ...
  • 32.
    url.purge telnet localhost6082 Trying ::1... Connectedto localhost. Escape character is '^]'. url.purge .* 200 0
  • 34.
    Setting up Varnish >Download varnish daemon debian: apt-get install varnish redhat:yum install varnish > Re-configure Apache to listen on port 8080 (or other non-standard port) > Configure Varnish to listen on port 80 > Configure Varnish VCL to connect to Apache > VCL tweaks (default does not cache Drupal)
  • 37.
  • 38.
  • 39.
    Serving ESIs <div id="nav-masthead"class="grid-12"> <ul> <li class="homepage active first active"> <a href="/home" class="active active">Drupal Homepage</a> </li> <li class="dashboard last"> <a href="/user/dashboard">Your Dashboard</a> </li> </ul> Varnish Drupal <div id="userinfo"> <a href="/user" title="View &amp; edit your user profile">Logged in as manarth</a> <a href="/logout">Log out</a> </div> </div><!-- /#nav-masthead --> Cache
  • 40.
    Serving ESIs <div id="nav-masthead"class="grid-12"> <ul> <li class="homepage active first active"> <a href="/home" class="active active">Drupal Homepage</a> </li> <li class="dashboard last"> <a href="/user/dashboard">Your Dashboard</a> </li> </ul> Varnish Drupal <esisrc="http://example.com/esi/userinfo" /> </div><!-- /#nav-masthead --> Cache
  • 41.
    Serving ESIs <div id="userinfo"> <ahref="/user" title="View &amp; edit your user profile">Logged in as manarth</a> <a href="/logout">Log out</a> </div> Varnish Drupal Cache
  • 42.
    What do youneed in order to do ESIs? > Replace personalisationwith ESI tags Theme-alters? > Provide backend hook_menuhandlers for each ESI tag > Ensure that Varnish is caching > Rule 35 applies
  • 49.
  • 52.
    Flickr Credits > Babbageengine http://www.flickr.com/photos/melgart/4444251 225 > Cookies http://www.flickr.com/photos/sabine01/24855 02578 > Frying wok http://www.flickr.com/photos/18403292@N00/ 3048993 > Varnish http://www.flickr.com/photos/dawniecakes/46 86467791

Editor's Notes

  • #4 Back, to the dawn of the internet, when email was in fidonet and 56k modems were all the rage.
  • #6 Add animation through the tubes.Dries makes requestGoes through internetHits serverHanded to apacheApache looks up from diskMight add some headersReturns the fileDodges the lolcat on the way back
  • #7 Add animation through the tubes.Dries makes requestGoes through internetHits serverHanded to apacheApache recognises it&apos;s .php and passes to ModPHP straight awayModPHP looks up from diskHands to lexical parserProcessesReturns the file to apacheDodges the lolcat on the way back
  • #8 Add animation through the tubes.Dries makes requestGoes through internetHits serverHanded to apacheApache processes .htaccess rulesEvery URL is index.phpPassed to ModPHP straight awayModPHP looks up from diskHands to lexical parserProcesses, invokes Drupal bootstrapReturns the file to apacheDodges the lolcat on the way back
  • #9 Benchmark:ApachePlain old PHP handler (no processing)Drupal
  • #10 Benchmark:ApachePlain old PHP handler (no processing)DrupalPHP parser adds 20%
  • #12 Refer to concepts such as baked vs fried
  • #13 Baked or fried?
  • #14  Prepared in advance No personalisation (more choc chips) Ready to eat
  • #15  One serving cooked at a time Wait a bit longer to eat Cooked freshPersonalised to taste
  • #50 - Looks at the hook_block information and uses appropriate cache rules – per user, etc.