Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Varnish	
  in	
  ac*on

                         Thijs	
  Feryn
                         Evangelist
                      ...
About	
  me




 I’m	
  an	
  evangelist	
  at	
  Combell
About	
  me




 I’m	
  a	
  board	
  member	
  at	
  PHPBenelux
<?php
$country = new Country();
$country->setName('Belgium');
$country->setHasGovernment(false);
I	
  live	
  in	
  the	
  wonderful	
  city	
  of	
  Bruges
      MPBecker	
  -­‐	
  Bruges	
  by	
  Night	
  hVp://www.fli...
Follow	
  me	
  on	
  TwiVer:	
  @ThijsFeryn

Give	
  me	
  feedback:	
  hVp://joind.in/2882

  Read	
  my	
  blog:	
  hVp...
Varnish
Varnish




          Reverse	
  proxy?

          Caching	
  proxy?

          Loadbalancer?
Varnish




          HTTP	
  accelerator	
  !
Caching
Caching




     Storing	
  computed	
  data	
  for	
  faster	
  
        serving	
  of	
  future	
  requests
Basically




       Pucng	
  stuff	
  in	
  boxes	
  for	
  later	
  use
Why?
Protect	
  your	
  server
In	
  the	
  beginning	
  there	
  was	
  ...
Browser	
  cache
Browser	
  cache:	
  expira*on




     Expires:	
  Sun,	
  25	
  Jun	
  2006	
  14:57:12	
  

        Cache-­‐Control:	
 ...
Browser	
  cache:	
  valida*on



 Last-­‐Modified:	
  Sun,	
  25	
  Jun	
  2006	
  14:57:12	
  GMT

            ETag:	
  “...
Browser	
  cache:	
  valida*on



 Last-­‐Modified:	
  Sun,	
  25	
  Jun	
  2006	
  14:57:12	
  GMT

             ETag:	
  ...
Browser	
  cache:	
  in	
  PHP
Browser	
  cache:	
  in	
  PHP


<?php
$seconds = 10;
$expires = gmdate('D, j M Y H:i:s T', time() + $seconds);
header("Ex...
Browser	
  cache:	
  in	
  Apache
Browser	
  cache:	
  in	
  Apache



ExpiresAc*ve	
  on
ExpiresDefault	
  "access	
  plus	
  1	
  month"
ExpiresDefault	
 ...
Browser	
  cache	
  is	
  your	
  friend	
  !
Problems	
  with	
  browser	
  cache
Problems	
  with	
  browser	
  cache


                 Mul*ple	
  standards

           Can	
  be	
  ignored	
  by	
  bro...
Solu*ons:	
  use	
  a	
  caching/reverse	
  proxy!



                    Use	
  a	
  caching/reverse	
  proxy

          ...
Info



•Website:	
  h$p://www.varnish-­‐cache.org
•Wiki:	
  h$p://www.varnish-­‐cache.org/trac/wiki
•Examples:	
  h$p://w...
Setup	
  Varnish
Setup	
  Varnish

    curl http://repo.varnish-cache.org/debian/
    GPG-key.txt | apt-key add -

    echo "deb http://rep...
Setup	
  Varnish

    START=yes

    DAEMON_OPTS="-a :80 
    -T localhost:6082 
    -f /etc/varnish/default.vcl 
    -s f...
Setup	
  the	
  backend
Setup	
  the	
  backend




        Listen 8080




   In	
  “/etc/apache2/ports.conf”
Setup	
  the	
  backend




        backend default {
              .host = "127.0.0.1";
              .port = "8080";
   ...
Out	
  of	
  the	
  box	
  Varnish	
  will	
  only	
  ...
...	
  obey	
  your	
  caching	
  headers
...	
  and	
  cache	
  sta*c	
  files
Time	
  for	
  ac*on	
  !
Request                          vcl_recv




                                             Yes         Cacheable?
        ...
Hits	
  &	
  misses
Logging	
  &	
  sta*s*cs	
  
Varnishstat




  Real	
  *me	
  stats	
  of	
  running	
  varnishd	
  instance

  hVp://www.varnish-­‐cache.org/docs/2.1/...
Hitrate	
  ratio:	
  	
  	
  	
  	
  	
  	
  10	
  	
  	
  	
  	
  	
  100	
  	
  	
  	
  	
  	
  121
Hitrate	
  avg:	
  	...
Varnishlog




  Display	
  requests	
  of	
  running	
  varnishd	
  instance

  hVp://www.varnish-­‐cache.org/docs/2.1/re...
Varnishlog

                 Notable	
  tags

• ReqEnd       • TxHeader          • RxHeader
• ReqStart     • TxLostHeader ...
Varnishlog




   varnishlog	
  -­‐c	
  -­‐o	
  RxRequest	
  POST


                  All	
  POST	
  requests
Varnishlog




  varnishlog	
  -­‐c	
  -­‐i	
  RxHeader	
  -­‐I	
  ^User-­‐Agent


                All	
  user	
  agents
Varnishlog




 varnishlog	
  -­‐i	
  TxURL


          All	
  URL’s	
  that	
  “missed”
Varnishlog




  varnishlog	
  	
  -­‐o	
  VCL_call	
  hit	
  |	
  grep	
  RxURL


                      All	
  URL’s	
  t...
Varnishlog


 varnishlog	
  	
  -­‐w	
  /tmp/yourfile.log


                    Write	
  log	
  to	
  file

  varnishlog	
  ...
 	
  	
  12	
  SessionOpen	
  	
  c	
  172.16.26.1	
  50396	
  :8080
	
  	
  	
  12	
  ReqStart	
  	
  	
  	
  	
  c	
  17...
 	
  	
  14	
  TxRequest	
  	
  	
  	
  b	
  GET
	
  	
  	
  14	
  TxURL	
  	
  	
  	
  	
  	
  	
  	
  b	
  /test.php
	
 ...
 	
  	
  12	
  TTL	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  c	
  668213522	
  RFC	
  120	
  1287492889	
  0	
  0	
  0	
  0
	...
Varnishtop




  Read	
  memory	
  logs	
  and	
  presents	
  them	
  as	
  a	
  
  con*nuously	
  updated	
  list	
  of	
...
Varnishtop




  varnishtop	
  -­‐i	
  RxHeader	
  -­‐I	
  ^User-­‐Agent


             Get	
  the	
  top	
  user	
  agents
Varnishtop




  varnishtop	
  -­‐i	
  RxRequest


        Get	
  the	
  top	
  request	
  methods
Varnishtop




  varnishtop	
  -­‐i	
  TxURL


                Get	
  the	
  top	
  misses
list	
  length	
  90	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
...
Varnishncsa




  Read	
  memory	
  logs	
  and	
  presents	
  them	
  in	
  the	
  
  Apache	
  /	
  NCSA	
  “combined”	
...
Varnishncsa


  varnishncsa	
  -­‐D	
  -­‐a	
  -­‐w	
  /var/log/some.log

  172.16.26.1	
  -­‐	
  -­‐	
  [19/Oct/2010:16:0...
Varnishreplay




 The	
  varnishreplay	
  u*lity	
  parses	
  varnish	
  logs	
  and	
  
 aVempts	
  to	
  reproduce	
  t...
Varnishreplay




  varnishlog	
  	
  -­‐w	
  /tmp/yourfile.log

  varnishreplay	
  	
  -­‐r	
  /tmp/yourfile.log
Management	
  via	
  telnet
Management	
  via	
  telnet

  root@dev:/#	
  telnet	
  localhost	
  6082
  Trying	
  ::1...
  Connected	
  to	
  localhos...
Management	
  via	
  telnet

                       Notable	
  ac*ons
• vcl.load	
  <configname>	
  <filename>
• vcl.use	
  ...
More	
  ac*on?
Request                          vcl_recv




                                             Yes         Cacheable?
        ...
Cache	
  sta*c	
  files


sub vcl_recv {
  if ( (req.request == "GET" || req.request == "HEAD")
&& req.url ~ ".(png|gif|jpe...
Provide	
  HIT/MISS	
  informa*on


sub vcl_deliver {
    if (obj.hits > 0) {
        set resp.http.X-Cache = "HIT";
    }...
Manipulate	
  TTL



sub vcl_fetch {
        set beresp.ttl = 10s;
}
Cache	
  ...	
  even	
  with	
  cookies

sub vcl_recv {
   if (req.request == "GET" || req.request ==
"HEAD") {
     retur...
Remove	
  cookies


 sub vcl_recv {
     unset req.http.cookie;
 }
 sub vcl_fetch {
     unset beresp.http.set-cookie;
 }
Remove	
  Google	
  Analy*cs	
  cookies

sub vcl_recv {
if (req.http.Cookie) {
    set req.http.Cookie = regsuball
(req.ht...
Mul*ple	
  backends:	
  hostname	
  filtering

backend default {
    .host = "default.server.com";
    .port = "80";
}
back...
www.example.com   example.com




Default backend   Other backend
Grace	
  mode



sub vcl_recv {
    if (req.backend.healthy) {
        set req.grace = 1s;
    } else {
        set req.gr...
Mul*ple	
  backends:	
  health	
  checks

backend default {
    .host = "default.server.com";
    .port = "80";
    .probe...
Mul*ple	
  backends:	
  health	
  checks

backend other {
    .host = "other.server.com";
    .port = "80";
    .probe = {...
Mul*ple	
  backends:	
  health	
  checks
sub vcl_recv {
  if (req.restarts == 0) {
    set req.backend = default;
  } else...
Default backend   Other backend
Default backend   Other backend
Loadbalancing	
  via	
  directors



director back round-robin {
        {.backend = default;}
        {.backend = other;}...
50%      50%




Default backend   Other backend
Loadbalancing	
  via	
  directors



director back random {
  {.backend = default; .weight=1;}
  {.backend = other; .weigh...
33%      66%




Default backend   Other backend
Purging

acl purge_acl {
  "localhost";
  "1.2.3.4";
  "some.host.ext";
}
sub vcl_recv {
  if (req.request == "PURGE") {
 ...
Edge	
  Side	
  Includes:	
  the	
  final	
  piece	
  of	
  the	
  puzzle
header.php



menu.php          main.php



           footer.php
<html>
<body>
    <table>
      <tr>
        <td colspan="2" >
           <?php include('header.php') ?>
        </td>
   ...
No	
  caching
   TTL	
  10s
                header.php

                                             TTL	
  2s
menu.php   ...
<html>
<body>
    <table>
      <tr>
        <td colspan="2" >
          <esi:include src="/header.php"   />
        </td>...
<?php
function esi($file){
    if(array_key_exists('HTTP_X_VARNISH',$_SERVER)){
        header("esi-enabled: 1");
        ...
<?php
class My_View_Helper_Esi extends Zend_View_Helper_Abstract
{
  protected $_requestHeader;
  protected $_responseHead...
...
protected function _getRequestHeader()
{
  if($this->_requestHeader === null || $this-
>_requestHeader === false){
   ...
...
   protected function _dispatch($controller,$action)
   {
     $request = new Zend_Controller_Request_Simple
($action,...
<html>
<body>
 <table>
  <tr>
   <td colspan="2" >
     <?php echo $this->esi('index',‘header’) ?>
   </td>
  </tr>
  <tr>...
sub vcl_recv {
  if(req.url == "/header.php") {
    return (pass);
  }
  if(req.url ~ "^/(footer|main|menu).php$") {
    r...
Q&A
Varnish in action confoo11
Varnish in action confoo11
Varnish in action confoo11
Varnish in action confoo11
Upcoming SlideShare
Loading in …5
×

of

Varnish in action confoo11 Slide 1 Varnish in action confoo11 Slide 2 Varnish in action confoo11 Slide 3 Varnish in action confoo11 Slide 4 Varnish in action confoo11 Slide 5 Varnish in action confoo11 Slide 6 Varnish in action confoo11 Slide 7 Varnish in action confoo11 Slide 8 Varnish in action confoo11 Slide 9 Varnish in action confoo11 Slide 10 Varnish in action confoo11 Slide 11 Varnish in action confoo11 Slide 12 Varnish in action confoo11 Slide 13 Varnish in action confoo11 Slide 14 Varnish in action confoo11 Slide 15 Varnish in action confoo11 Slide 16 Varnish in action confoo11 Slide 17 Varnish in action confoo11 Slide 18 Varnish in action confoo11 Slide 19 Varnish in action confoo11 Slide 20 Varnish in action confoo11 Slide 21 Varnish in action confoo11 Slide 22 Varnish in action confoo11 Slide 23 Varnish in action confoo11 Slide 24 Varnish in action confoo11 Slide 25 Varnish in action confoo11 Slide 26 Varnish in action confoo11 Slide 27 Varnish in action confoo11 Slide 28 Varnish in action confoo11 Slide 29 Varnish in action confoo11 Slide 30 Varnish in action confoo11 Slide 31 Varnish in action confoo11 Slide 32 Varnish in action confoo11 Slide 33 Varnish in action confoo11 Slide 34 Varnish in action confoo11 Slide 35 Varnish in action confoo11 Slide 36 Varnish in action confoo11 Slide 37 Varnish in action confoo11 Slide 38 Varnish in action confoo11 Slide 39 Varnish in action confoo11 Slide 40 Varnish in action confoo11 Slide 41 Varnish in action confoo11 Slide 42 Varnish in action confoo11 Slide 43 Varnish in action confoo11 Slide 44 Varnish in action confoo11 Slide 45 Varnish in action confoo11 Slide 46 Varnish in action confoo11 Slide 47 Varnish in action confoo11 Slide 48 Varnish in action confoo11 Slide 49 Varnish in action confoo11 Slide 50 Varnish in action confoo11 Slide 51 Varnish in action confoo11 Slide 52 Varnish in action confoo11 Slide 53 Varnish in action confoo11 Slide 54 Varnish in action confoo11 Slide 55 Varnish in action confoo11 Slide 56 Varnish in action confoo11 Slide 57 Varnish in action confoo11 Slide 58 Varnish in action confoo11 Slide 59 Varnish in action confoo11 Slide 60 Varnish in action confoo11 Slide 61 Varnish in action confoo11 Slide 62 Varnish in action confoo11 Slide 63 Varnish in action confoo11 Slide 64 Varnish in action confoo11 Slide 65 Varnish in action confoo11 Slide 66 Varnish in action confoo11 Slide 67 Varnish in action confoo11 Slide 68 Varnish in action confoo11 Slide 69 Varnish in action confoo11 Slide 70 Varnish in action confoo11 Slide 71 Varnish in action confoo11 Slide 72 Varnish in action confoo11 Slide 73 Varnish in action confoo11 Slide 74 Varnish in action confoo11 Slide 75 Varnish in action confoo11 Slide 76 Varnish in action confoo11 Slide 77 Varnish in action confoo11 Slide 78 Varnish in action confoo11 Slide 79 Varnish in action confoo11 Slide 80 Varnish in action confoo11 Slide 81 Varnish in action confoo11 Slide 82 Varnish in action confoo11 Slide 83 Varnish in action confoo11 Slide 84 Varnish in action confoo11 Slide 85 Varnish in action confoo11 Slide 86 Varnish in action confoo11 Slide 87 Varnish in action confoo11 Slide 88 Varnish in action confoo11 Slide 89 Varnish in action confoo11 Slide 90 Varnish in action confoo11 Slide 91 Varnish in action confoo11 Slide 92 Varnish in action confoo11 Slide 93 Varnish in action confoo11 Slide 94 Varnish in action confoo11 Slide 95 Varnish in action confoo11 Slide 96 Varnish in action confoo11 Slide 97 Varnish in action confoo11 Slide 98 Varnish in action confoo11 Slide 99 Varnish in action confoo11 Slide 100 Varnish in action confoo11 Slide 101 Varnish in action confoo11 Slide 102
Upcoming SlideShare
Vfinal V1
Next
Download to read offline and view in fullscreen.

4 Likes

Share

Download to read offline

Varnish in action confoo11

Download to read offline

The slides for my Varnish talk at Confoo 2011 in Montreal Canada

Related Books

Free with a 30 day trial from Scribd

See all

Varnish in action confoo11

  1. 1. Varnish  in  ac*on Thijs  Feryn Evangelist +32  (0)9  218  79  06 thijs@combellgroup.com Confoo Friday  March  11th  2011 Montrea,  Canada
  2. 2. About  me I’m  an  evangelist  at  Combell
  3. 3. About  me I’m  a  board  member  at  PHPBenelux
  4. 4. <?php $country = new Country(); $country->setName('Belgium'); $country->setHasGovernment(false);
  5. 5. I  live  in  the  wonderful  city  of  Bruges MPBecker  -­‐  Bruges  by  Night  hVp://www.flickr.com/photos/galverson2/3715965933
  6. 6. Follow  me  on  TwiVer:  @ThijsFeryn Give  me  feedback:  hVp://joind.in/2882 Read  my  blog:  hVp://blog.feryn.eu
  7. 7. Varnish
  8. 8. Varnish Reverse  proxy? Caching  proxy? Loadbalancer?
  9. 9. Varnish HTTP  accelerator  !
  10. 10. Caching
  11. 11. Caching Storing  computed  data  for  faster   serving  of  future  requests
  12. 12. Basically Pucng  stuff  in  boxes  for  later  use
  13. 13. Why?
  14. 14. Protect  your  server
  15. 15. In  the  beginning  there  was  ...
  16. 16. Browser  cache
  17. 17. Browser  cache:  expira*on Expires:  Sun,  25  Jun  2006  14:57:12   Cache-­‐Control:  max-­‐age=86400,   must-­‐revalidate
  18. 18. Browser  cache:  valida*on Last-­‐Modified:  Sun,  25  Jun  2006  14:57:12  GMT ETag:  “686897696a7c876b7e”
  19. 19. Browser  cache:  valida*on Last-­‐Modified:  Sun,  25  Jun  2006  14:57:12  GMT ETag:  “686897696a7c876b7e” Compared  with  the  “If-­‐None-­‐Match”  header.   Throws  “HTTP/1.0  304  Not  Modified”  accordingly
  20. 20. Browser  cache:  in  PHP
  21. 21. Browser  cache:  in  PHP <?php $seconds = 10; $expires = gmdate('D, j M Y H:i:s T', time() + $seconds); header("Expires: $expires"); header("Cache-Control: max-age=$seconds, must-revalidate"); ?> <ul>     <li>Current date/time: <?= gmdate('D, j M Y H:i:s T')?></li>     <li>Expires: <?= $expires ?></li>     <li>Cache-control: <?= $seconds?> seconds</li> </ul>
  22. 22. Browser  cache:  in  Apache
  23. 23. Browser  cache:  in  Apache ExpiresAc*ve  on ExpiresDefault  "access  plus  1  month" ExpiresDefault  "now  plus  4  weeks" ExpiresDefault  "modifica*on  plus  30  days" ExpiresByType  text/html  "access  plus  1  month  15  days  2   hours"
  24. 24. Browser  cache  is  your  friend  !
  25. 25. Problems  with  browser  cache
  26. 26. Problems  with  browser  cache Mul*ple  standards Can  be  ignored  by  browser Force  refresh Cache  per  client
  27. 27. Solu*ons:  use  a  caching/reverse  proxy! Use  a  caching/reverse  proxy Varnish  does  that It’ll  make  your  site  fly!
  28. 28. Info •Website:  h$p://www.varnish-­‐cache.org •Wiki:  h$p://www.varnish-­‐cache.org/trac/wiki •Examples:  h$p://www.varnish-­‐cache.org/trac/wiki/VCLExamples •DocumentaHon  site:  h$p://www.varnish-­‐cache.org/docs/2.1  
  29. 29. Setup  Varnish
  30. 30. Setup  Varnish curl http://repo.varnish-cache.org/debian/ GPG-key.txt | apt-key add - echo "deb http://repo.varnish-cache.org/ debian/ $(lsb_release -s -c) varnish-2.1" >> /etc/apt/sources.list apt-get update apt-get install varnish
  31. 31. Setup  Varnish START=yes DAEMON_OPTS="-a :80 -T localhost:6082 -f /etc/varnish/default.vcl -s file,/var/lib/varnish/$INSTANCE/ varnish_storage.bin,1G" In  “/etc/default/varnish”
  32. 32. Setup  the  backend
  33. 33. Setup  the  backend Listen 8080 In  “/etc/apache2/ports.conf”
  34. 34. Setup  the  backend backend default { .host = "127.0.0.1"; .port = "8080"; } In  “/etc/varnish/default.vcl”
  35. 35. Out  of  the  box  Varnish  will  only  ...
  36. 36. ...  obey  your  caching  headers
  37. 37. ...  and  cache  sta*c  files
  38. 38. Time  for  ac*on  !
  39. 39. Request vcl_recv Yes Cacheable? vcl_hash Yes No In cache? No vcl_hit() vcl_miss() vcl_fetch() vcl_deliver() Response
  40. 40. Hits  &  misses
  41. 41. Logging  &  sta*s*cs  
  42. 42. Varnishstat Real  *me  stats  of  running  varnishd  instance hVp://www.varnish-­‐cache.org/docs/2.1/reference/varnishstat.html
  43. 43. Hitrate  ratio:              10            100            121 Hitrate  avg:          0.5365      0.5287      0.5277                9957                  0.00                  4.10  Client  connections  accepted              58932                67.92                24.28  Client  requests  received              30483                  1.00                12.56  Cache  hits                8592                  0.00                  3.54  Cache  hits  for  pass              18991                66.92                  7.82  Cache  misses              28449                66.92                11.72  Backend  connections  success              22562                66.92                  9.30  Backend  connections  reuses              28357                66.92                11.68  Backend  connections  recycles                  488                    .                        .      N  struct  srcaddr                      7                    .                        .      N  active  struct  srcaddr                    81                    .                        .      N  struct  sess_mem                    21                    .                        .      N  struct  sess                4398                    .                        .      N  struct  object                4401                    .                        .      N  struct  objecthead                9220                    .                        .      N  struct  smf                  355                    .                        .      N  small  free  smf                      6                    .                        .      N  large  free  smf                      2                    .                        .      N  struct  vbe_conn                    10                    .                        .      N  struct  bereq                    17                    .                        .      N  worker  threads                    28                  0.00                  0.01  N  worker  threads  created                  164                  0.00                  0.07  N  overflowed  work  requests                      1                    .                        .      N  backends              14608                    .                        .      N  expired  objects              24113                    .                        .      N  LRU  moved  objects              53778                66.92                22.16  Objects  sent  with  write                9957                  0.00                  4.10  Total  Sessions              58935                67.92                24.28  Total  Requests                9458                  0.00                  3.90  Total  pass              28445                66.92                11.72  Total  fetch        22496844          23506.62            9269.40  Total  header  bytes      606897837        263427.01        250060.91  Total  body  bytes                1384                  0.00                  0.57  Session  Closed
  44. 44. Varnishlog Display  requests  of  running  varnishd  instance hVp://www.varnish-­‐cache.org/docs/2.1/reference/varnishlog.html
  45. 45. Varnishlog Notable  tags • ReqEnd • TxHeader • RxHeader • ReqStart • TxLostHeader • RxLostHeader • Hit • TxProtocol • RxProtocol • Error • TxRequest • RxRequest • VCL_call • TxResponse • RxResponse • VCL_return • TxStatus • RxStatus • VCL_acl • TxURL • RxURL
  46. 46. Varnishlog varnishlog  -­‐c  -­‐o  RxRequest  POST All  POST  requests
  47. 47. Varnishlog varnishlog  -­‐c  -­‐i  RxHeader  -­‐I  ^User-­‐Agent All  user  agents
  48. 48. Varnishlog varnishlog  -­‐i  TxURL All  URL’s  that  “missed”
  49. 49. Varnishlog varnishlog    -­‐o  VCL_call  hit  |  grep  RxURL All  URL’s  that  “hit”
  50. 50. Varnishlog varnishlog    -­‐w  /tmp/yourfile.log Write  log  to  file varnishlog    -­‐r  /tmp/yourfile.log Read  log  from  file
  51. 51.      12  SessionOpen    c  172.16.26.1  50396  :8080      12  ReqStart          c  172.16.26.1  50396  668213522      12  RxRequest        c  GET      12  RxURL                c  /test.php      12  RxProtocol      c  HTTP/1.1      12  RxHeader          c  Host:  varnish.dev:8080      12  RxHeader          c  User-­‐Agent:  Mozilla/5.0  (Macintosh;  U;  Intel  Mac  OS  X  10.6;  nl;   rv:1.9.2.10)  Gecko/20100914  Firefox/3.6.10      12  RxHeader          c  Accept:  text/html,application/xhtml+xml,application/xml;q=0.9,*/ *;q=0.8      12  RxHeader          c  Accept-­‐Language:  nl,en-­‐us;q=0.7,en;q=0.3      12  RxHeader          c  Accept-­‐Encoding:  gzip,deflate      12  RxHeader          c  Accept-­‐Charset:  ISO-­‐8859-­‐1,utf-­‐8;q=0.7,*;q=0.7      12  RxHeader          c  Keep-­‐Alive:  115      12  RxHeader          c  Connection:  keep-­‐alive      12  RxHeader          c  Cookie:  PHPSESSID=2n2pkit81qdgk6k4trf1crft16      12  VCL_call          c  recv      12  VCL_return      c  pass      12  VCL_call          c  hash      12  VCL_return      c  hash      12  VCL_call          c  pass      12  VCL_return      c  pass      14  BackendClose  -­‐  default      14  BackendOpen    b  default  127.0.0.1  34267  127.0.0.1  80      12  Backend            c  14  default  default
  52. 52.      14  TxRequest        b  GET      14  TxURL                b  /test.php      14  TxProtocol      b  HTTP/1.1      14  TxHeader          b  Host:  varnish.dev:8080      14  TxHeader          b  User-­‐Agent:  Mozilla/5.0  (Macintosh;  U;  Intel  Mac  OS  X  10.6;  nl;   rv:1.9.2.10)  Gecko/20100914  Firefox/3.6.10      14  TxHeader          b  Accept:  text/html,application/xhtml+xml,application/xml;q=0.9,*/ *;q=0.8      14  TxHeader          b  Accept-­‐Language:  nl,en-­‐us;q=0.7,en;q=0.3      14  TxHeader          b  Accept-­‐Encoding:  gzip,deflate      14  TxHeader          b  Accept-­‐Charset:  ISO-­‐8859-­‐1,utf-­‐8;q=0.7,*;q=0.7      14  TxHeader          b  Cookie:  PHPSESSID=2n2pkit81qdgk6k4trf1crft16      14  TxHeader          b  X-­‐Forwarded-­‐For:  172.16.26.1      14  TxHeader          b  X-­‐Varnish:  668213522      14  RxProtocol      b  HTTP/1.1      14  RxStatus          b  200      14  RxResponse      b  OK      14  RxHeader          b  Date:  Tue,  19  Oct  2010  12:54:49  GMT      14  RxHeader          b  Server:  Apache/2.2.16  (Ubuntu)      14  RxHeader          b  X-­‐Powered-­‐By:  PHP/5.3.3-­‐1ubuntu9      14  RxHeader          b  Vary:  Accept-­‐Encoding      14  RxHeader          b  Content-­‐Encoding:  gzip      14  RxHeader          b  Content-­‐Length:  36      14  RxHeader          b  Content-­‐Type:  text/html
  53. 53.      12  TTL                    c  668213522  RFC  120  1287492889  0  0  0  0      12  VCL_call          c  fetch      12  TTL                    c  668213522  VCL  10  1287492889      12  VCL_return      c  pass      12  ObjProtocol    c  HTTP/1.1      12  ObjStatus        c  200      12  ObjResponse    c  OK      12  ObjHeader        c  Date:  Tue,  19  Oct  2010  12:54:49  GMT      12  ObjHeader        c  Server:  Apache/2.2.16  (Ubuntu)      12  ObjHeader        c  X-­‐Powered-­‐By:  PHP/5.3.3-­‐1ubuntu9      12  ObjHeader        c  Vary:  Accept-­‐Encoding      12  ObjHeader        c  Content-­‐Encoding:  gzip      12  ObjHeader        c  Content-­‐Type:  text/html      14  Length              b  36      14  BackendReuse  b  default      12  VCL_call          c  deliver      12  VCL_return      c  deliver      12  TxProtocol      c  HTTP/1.1      12  TxStatus          c  200      12  TxResponse      c  OK      12  TxHeader          c  Server:  Apache/2.2.16  (Ubuntu)      12  TxHeader          c  X-­‐Powered-­‐By:  PHP/5.3.3-­‐1ubuntu9      12  TxHeader          c  Vary:  Accept-­‐Encoding      12  TxHeader          c  Content-­‐Encoding:  gzip      12  TxHeader          c  Content-­‐Type:  text/html      12  TxHeader          c  Content-­‐Length:  36      12  TxHeader          c  Date:  Tue,  19  Oct  2010  12:54:49  GMT      12  TxHeader          c  X-­‐Varnish:  668213522      12  TxHeader          c  Age:  0      12  TxHeader          c  Via:  1.1  varnish
  54. 54. Varnishtop Read  memory  logs  and  presents  them  as  a   con*nuously  updated  list  of  commonly   occuring  log  entries hVp://www.varnish-­‐cache.org/docs/2.1/reference/varnishtop.html
  55. 55. Varnishtop varnishtop  -­‐i  RxHeader  -­‐I  ^User-­‐Agent Get  the  top  user  agents
  56. 56. Varnishtop varnishtop  -­‐i  RxRequest Get  the  top  request  methods
  57. 57. Varnishtop varnishtop  -­‐i  TxURL Get  the  top  misses
  58. 58. list  length  90                                                                                                                                                     dev        76.42  VCL_return          pass              50.95  RxProtocol          HTTP/1.1        50.95  TxProtocol          HTTP/1.1        27.54  Length                  110        25.48  Debug                    herding        25.47  RxRequest            GET        25.47  RxHeader              Host:  varnish.dev:8080        25.47  RxHeader              User-­‐Agent:  Mozilla/5.0  (Macintosh;  U;  Intel  Mac  OS  X  10.6;  nl; rv:1.9.2.10)  Gecko/20100914  Firefox/3.6.10        25.47  RxHeader              Accept:  text/html,application/xhtml+xml,application/ xml;q=0.9,*/*;q=0.8        25.47  RxHeader              Accept-­‐Language:  nl,en-­‐us;q=0.7,en;q=0.3        25.47  RxHeader              Accept-­‐Encoding:  gzip,deflate        25.47  RxHeader              Accept-­‐Charset:  ISO-­‐8859-­‐1,utf-­‐8;q=0.7,*;q=0.7        25.47  RxHeader              Keep-­‐Alive:  115        25.47  RxHeader              Connection:  keep-­‐alive        25.47  RxHeader              Cookie:  PHPSESSID=2n2pkit81qdgk6k4trf1crft16        25.47  VCL_call              recv        25.47  VCL_call              hash        25.47  VCL_return          hash        25.47  VCL_call              pass        25.47  Backend                14  default  default        25.47  TxRequest            GET        25.47  TxHeader              Host:  varnish.dev:8080        25.47  TxHeader              User-­‐Agent:  Mozilla/5.0  (Macintosh;  U;  Intel  Mac  OS  X  10.6;  nl; rv:1.9.2.10)  Gecko/20100914  Firefox/3.6.10        25.47  TxHeader              Accept:  text/html,application/xhtml+xml,application/
  59. 59. Varnishncsa Read  memory  logs  and  presents  them  in  the   Apache  /  NCSA  “combined”  log  format hVp://www.varnish-­‐cache.org/docs/2.1/reference/varnishncsa.html
  60. 60. Varnishncsa varnishncsa  -­‐D  -­‐a  -­‐w  /var/log/some.log 172.16.26.1  -­‐  -­‐  [19/Oct/2010:16:08:02  +0200]  "GET  hVp:// varnish.dev:8080/  HTTP/1.1"  200  97  "-­‐"  "Mozilla/5.0  (Macintosh;  U;   Intel  Mac  OS  X  10.6;  nl;  rv:1.9.2.10)  Gecko/20100914  Firefox/3.6.10" 172.16.26.1  -­‐  -­‐  [19/Oct/2010:16:08:10  +0200]  "GET  hVp:// varnish.dev:8080/test.php  HTTP/1.1"  200  36  "-­‐"  "Mozilla/5.0   (Macintosh;  U;  Intel  Mac  OS  X  10.6;  nl;  rv:1.9.2.10)  Gecko/20100914   Firefox/3.6.10"
  61. 61. Varnishreplay The  varnishreplay  u*lity  parses  varnish  logs  and   aVempts  to  reproduce  the  traffic hVp://www.varnish-­‐cache.org/docs/2.1/reference/varnishreplay.html
  62. 62. Varnishreplay varnishlog    -­‐w  /tmp/yourfile.log varnishreplay    -­‐r  /tmp/yourfile.log
  63. 63. Management  via  telnet
  64. 64. Management  via  telnet root@dev:/#  telnet  localhost  6082 Trying  ::1... Connected  to  localhost. Escape  character  is  '^]'. 200  154           -­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐ Varnish  HTTP  accelerator  CLI. -­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐ Type  'help'  for  command  list. Type  'quit'  to  close  CLI  session.
  65. 65. Management  via  telnet Notable  ac*ons • vcl.load  <configname>  <filename> • vcl.use  <configname> • vcl.discard  <configname> Avoid  interrup*on • vcl.list • vcl.show  <configname>
  66. 66. More  ac*on?
  67. 67. Request vcl_recv Yes Cacheable? vcl_hash Yes No In cache? No vcl_hit() vcl_miss() vcl_fetch() vcl_deliver() Response
  68. 68. Cache  sta*c  files sub vcl_recv { if ( (req.request == "GET" || req.request == "HEAD") && req.url ~ ".(png|gif|jpe?g|swf|css|js|html?|sstm) $" ) { return(lookup); } else { return(pass); } }
  69. 69. Provide  HIT/MISS  informa*on sub vcl_deliver { if (obj.hits > 0) { set resp.http.X-Cache = "HIT"; } else { set resp.http.X-Cache = "MISS"; } }
  70. 70. Manipulate  TTL sub vcl_fetch { set beresp.ttl = 10s; }
  71. 71. Cache  ...  even  with  cookies sub vcl_recv { if (req.request == "GET" || req.request == "HEAD") { return (lookup); } } sub vcl_hash { if (req.http.Cookie) { set req.hash += req.http.Cookie; } }
  72. 72. Remove  cookies sub vcl_recv { unset req.http.cookie; } sub vcl_fetch { unset beresp.http.set-cookie; }
  73. 73. Remove  Google  Analy*cs  cookies sub vcl_recv { if (req.http.Cookie) { set req.http.Cookie = regsuball (req.http.Cookie,"(^|; ) *__utm.=[^;]+;? *","1"); if (req.http.Cookie == "") { remove req.http.Cookie; } }
  74. 74. Mul*ple  backends:  hostname  filtering backend default { .host = "default.server.com"; .port = "80"; } backend other { .host = "other.server.com"; .port = "80"; } sub vcl_recv { if(req.http.host ~ "^www."){ set req.backend= default; } else { set req.backend = other; } }
  75. 75. www.example.com example.com Default backend Other backend
  76. 76. Grace  mode sub vcl_recv { if (req.backend.healthy) { set req.grace = 1s; } else { set req.grace = 10s; } } Keep  reading  from  cache,   even  when  expired
  77. 77. Mul*ple  backends:  health  checks backend default { .host = "default.server.com"; .port = "80"; .probe = { .url = "/"; .timeout = 50 ms; .interval = 1s; .window = 2; .threshold = 2; } }
  78. 78. Mul*ple  backends:  health  checks backend other { .host = "other.server.com"; .port = "80"; .probe = { .url = "/"; .timeout = 50 ms; .interval = 1s; .window = 2; .threshold = 2; } }
  79. 79. Mul*ple  backends:  health  checks sub vcl_recv { if (req.restarts == 0) { set req.backend = default; } else { set req.backend = other; } return(pass); } sub vcl_error { if (obj.status == 503 && req.restarts < 4) { restart; } }
  80. 80. Default backend Other backend
  81. 81. Default backend Other backend
  82. 82. Loadbalancing  via  directors director back round-robin { {.backend = default;} {.backend = other;} } sub vcl_recv { set req.backend = back; return(pass); }
  83. 83. 50% 50% Default backend Other backend
  84. 84. Loadbalancing  via  directors director back random { {.backend = default; .weight=1;} {.backend = other; .weight=2;} } sub vcl_recv { set req.backend = back; return(pass); }
  85. 85. 33% 66% Default backend Other backend
  86. 86. Purging acl purge_acl { "localhost"; "1.2.3.4"; "some.host.ext"; } sub vcl_recv { if (req.request == "PURGE") { if (!client.ip ~ purge_acl) { error 405 "Not allowed."; } purge_url(req.url); error 200 "Purged."; } return(lookup); } curl  -­‐X  PURGE  hVp://yoursite.com/yourpage
  87. 87. Edge  Side  Includes:  the  final  piece  of  the  puzzle
  88. 88. header.php menu.php main.php footer.php
  89. 89. <html> <body>     <table>       <tr>         <td colspan="2" > <?php include('header.php') ?> </td>       </tr>       <tr>         <td><?php include('menu.php') ?></td>         <td><?php include('main.php') ?></td>       </tr>             <tr>         <td colspan="2" > <?php include('footer.php') ?> </td>       </tr>           </table>   </body> </html>
  90. 90. No  caching TTL  10s header.php TTL  2s menu.php main.php footer.php TTL  5s
  91. 91. <html> <body> <table> <tr> <td colspan="2" > <esi:include src="/header.php" /> </td> </tr> <tr> <td><esi:include src="/menu.php" /></td> <td><esi:include src="/main.php" /></td> </tr> <tr> <td colspan="2" > <esi:include src="/footer.php" /> </td> </tr> </table> </body>
  92. 92. <?php function esi($file){     if(array_key_exists('HTTP_X_VARNISH',$_SERVER)){         header("esi-enabled: 1");         $url = 'http://'.$_SERVER['HTTP_HOST'].substr($_SERVER ['SCRIPT_NAME'],0,1+strrpos($_SERVER['SCRIPT_NAME'],'/')). $file;         echo "<esi:include src="$url" />".PHP_EOL;     } else {         include($file);     } }
  93. 93. <?php class My_View_Helper_Esi extends Zend_View_Helper_Abstract { protected $_requestHeader;   protected $_responseHeader;   protected $_frontController;   public function esi ($controller = 'index', $action='index', $requestHeader = 'X_VARNISH',  $responseHeader = 'esi-enabled: 1')   {       $this->_requestHeader = $requestHeader;       $this->_responseHeader = $responseHeader;       $this->_frontController = Zend_Controller_Front::getInstance();       if($this->_getRequestHeader()){           $this->_getResponseHeader();           return $this->_buildEsi($controller,$action);       } else {           return $this->_dispatch($controller, $action);       }   } ...
  94. 94. ... protected function _getRequestHeader() { if($this->_requestHeader === null || $this- >_requestHeader === false){    return false;   }           $request = $this->_frontController->getRequest();   if($request->getHeader($this->_requestHeader) === false){       return false;   } else {       return true;   } } protected function _getResponseHeader() {   if($this->_responseHeader !== null && $this- >_responseHeader !== false){   header($this->_responseHeader);   } } ...
  95. 95. ... protected function _dispatch($controller,$action) { $request = new Zend_Controller_Request_Simple ($action, $controller);     $response = new Zend_Controller_Response_Http();     $dispatcher = $this->_frontController->getDispatcher();     $dispatcher->dispatch($request, $response);     return $response->getBody(); } protected function _buildEsi($controller,$action) { $router = $this->_frontController->getRouter();   $url = $router->assemble(array('controller'=> $controller,'action'=>$action));    return "<esi:include src="{$url}"/>"; } }
  96. 96. <html> <body> <table>   <tr>    <td colspan="2" > <?php echo $this->esi('index',‘header’) ?> </td>   </tr>   <tr>   <td><?php echo $this->esi('index',‘menu’) ?></td>    <td><?php echo $this->esi('index',‘main’) ?></td>   </tr>         <tr>    <td colspan="2" > <?php echo $this->esi('index',‘footer’) ?> </td>   </tr>        </table>
  97. 97. sub vcl_recv { if(req.url == "/header.php") { return (pass); } if(req.url ~ "^/(footer|main|menu).php$") { return (lookup); } } sub vcl_fetch { if (beresp.http.esi-enabled == "1") { esi; unset beresp.http.esi-enabled; } if(req.url == "/footer.php") { set beresp.ttl = 5s; } if(req.url == "/menu.php") { set beresp.ttl = 10s; } if(req.url == "/main.php") { set beresp.ttl = 2s; } }
  98. 98. Q&A
  • pims

    Nov. 5, 2012
  • thanatas11

    Mar. 16, 2012
  • KristofC

    Mar. 24, 2011
  • chrifs

    Mar. 16, 2011

The slides for my Varnish talk at Confoo 2011 in Montreal Canada

Views

Total views

3,334

On Slideshare

0

From embeds

0

Number of embeds

69

Actions

Downloads

38

Shares

0

Comments

0

Likes

4

×