Varnish, the high performance valhalla?
Upcoming SlideShare
Loading in...5
×
 

Like this? Share it with your network

Share

Varnish, the high performance valhalla?

on

  • 7,105 views

Varnish, the high-performance HTTP accelerator. Is it really a high performance valhalla?

Varnish, the high-performance HTTP accelerator. Is it really a high performance valhalla?

Statistics

Views

Total Views
7,105
Views on SlideShare
5,548
Embed Views
1,557

Actions

Likes
8
Downloads
65
Comments
0

13 Embeds 1,557

http://weblogs.vpro.nl 1457
http://redactie-live.test.vpro.nl 31
http://lanyrd.com 23
http://lanyrd.dev:8000 22
http://paper.li 16
https://www.google.nl 1
http://redactie-test.vpro.omroep.nl 1
http://translate.googleusercontent.com 1
http://a0.twimg.com 1
http://127.0.0.1:8795 1
http://tweetedtimes.com 1
http://twitter.com 1
http://digg.com 1
More...

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Varnish, the high performance valhalla? Presentation Transcript

  • 1. Varnish, the high performance valhalla? Kings of Code 2011
  • 2. JEROEN VAN DIJK∂ CTO @ Enrise∂ Email : jeroen@enrise.com∂ Twitter: @neorey∂ Open source addict 2
  • 3. JEROEN VAN DIJK∂ CTO @ Enrise∂ Email : jeroen@enrise.com∂ Twitter: @neorey∂ Open source addict 2
  • 4. THE ENRISE RESTAURANT 3
  • 5. THE ENRISE RESTAURANT∂ We want to prepare the best dishes 3
  • 6. THE ENRISE RESTAURANT∂ We want to prepare the best dishes∂ With the best ingredients 3
  • 7. THE ENRISE RESTAURANT∂ We want to prepare the best dishes∂ With the best ingredients∂ To create a magical client experience!∂ Engineers are our top chefs! 3
  • 8. BEFORE WE START WHO KNOWS RFC 2616? 4
  • 9. BEFORE WE START WHO KNOWS HTTP? 5
  • 10. BEFORE WE START WHO THINKS HE/SHE KNOWS HTTP? 6
  • 11. STANDARD PROXY∂ Wikipedia : Proxy 7
  • 12. REVERSE PROXY∂ Wikipedia : Reverse proxy 8
  • 13. REVERSE PROXY∂ Wikipedia : Reverse proxy 8
  • 14. REVERSE PROXIES∂ Wikipedia : Reverse proxy 9
  • 15. REVERSE PROXIES∂ Wikipedia : Reverse proxy 9
  • 16. REVERSE PROXIES∂ Wikipedia : Reverse proxy 9
  • 17. WHAT IS VARNISH? (1)∂ NOT a webserver! 10
  • 18. WHAT IS VARNISH? (1)∂ Caching reverse proxy∂ NOT a webserver! 10
  • 19. WHAT IS VARNISH? (1)∂ Caching reverse proxy∂ Load balancer∂ NOT a webserver! 10
  • 20. WHAT IS VARNISH? (1)∂ Caching reverse proxy∂ Load balancer∂ Fail over system∂ NOT a webserver! 10
  • 21. WHAT IS VARNISH? (1)∂ Caching reverse proxy∂ Load balancer∂ Fail over system∂ HTTP Accelerator∂ NOT a webserver! 10
  • 22. WHAT IS VARNISH? (2) 11
  • 23. WHAT IS VARNISH? (2)∂ Originally built for a Norwegian newspaper 11
  • 24. WHAT IS VARNISH? (2)∂ Originally built for a Norwegian newspaper∂ > 545k new hostnames in December 2010 § NetCraft monthly survey 11
  • 25. WHAT IS VARNISH? (2)∂ Originally built for a Norwegian newspaper∂ > 545k new hostnames in December 2010 § NetCraft monthly survey∂ Known users § Wikileaks § Facebook § Twitter § SlashDot∂ Massive growth in usage 11
  • 26. WHAT IS VARNISH? (3)∂ Scaling limitations currently unknown 12
  • 27. WHAT IS VARNISH? (3)∂ Designed for 64bit architecture∂ Scaling limitations currently unknown 12
  • 28. WHAT IS VARNISH? (3)∂ Designed for 64bit architecture∂ Highly scalable∂ Scaling limitations currently unknown 12
  • 29. WHAT IS VARNISH? (3)∂ Designed for 64bit architecture∂ Highly scalable §Varnish in front of Varnish(es)∂ Scaling limitations currently unknown 12
  • 30. WHAT IS VARNISH? (3)∂ Designed for 64bit architecture∂ Highly scalable §Varnish in front of Varnish(es)∂ Perfect browser cache solution∂ Scaling limitations currently unknown 12
  • 31. WHAT IS VARNISH? (3)∂ Designed for 64bit architecture∂ Highly scalable §Varnish in front of Varnish(es)∂ Perfect browser cache solution §Prevent those hard refresh calls!∂ Scaling limitations currently unknown 12
  • 32. INTERNALS (1)∂ Possible request paths 13
  • 33. INTERNALS (2)∂ Optimal request path 14
  • 34. INTERNALS (3)∂ Caching request path 15
  • 35. INTERNALS (4)∂ Prevent caching 16
  • 36. INTERNALS (5)∂ Bypassing Varnish 17
  • 37. INTERNALS (6)∂ Errors or issueing restarts 18
  • 38. VARNISH CONFIGURATION LANGUAGE (1)∂ Compiled to C when run 19
  • 39. VARNISH CONFIGURATION LANGUAGE (1)∂ 11 subroutines (vcl_recv, vcl_hash, etc ...)∂ Compiled to C when run 19
  • 40. VARNISH CONFIGURATION LANGUAGE (1)∂ 11 subroutines (vcl_recv, vcl_hash, etc ...)∂ Backend(s)∂ Compiled to C when run 19
  • 41. VARNISH CONFIGURATION LANGUAGE (1)∂ 11 subroutines (vcl_recv, vcl_hash, etc ...)∂ Backend(s)∂ Directors (random, round robin, hash)∂ Compiled to C when run 19
  • 42. VARNISH CONFIGURATION LANGUAGE (1)∂ 11 subroutines (vcl_recv, vcl_hash, etc ...)∂ Backend(s)∂ Directors (random, round robin, hash)∂ Access control lists∂ Compiled to C when run 19
  • 43. VARNISH CONFIGURATION LANGUAGE (1)∂ 11 subroutines (vcl_recv, vcl_hash, etc ...)∂ Backend(s)∂ Directors (random, round robin, hash)∂ Access control lists∂ Saint and grace mode∂ Compiled to C when run 19
  • 44. VARNISH CONFIGURATION LANGUAGE (1)∂ 11 subroutines (vcl_recv, vcl_hash, etc ...)∂ Backend(s)∂ Directors (random, round robin, hash)∂ Access control lists∂ Saint and grace mode∂ Custom subroutines∂ Compiled to C when run 19
  • 45. VARNISH CONFIGURATION LANGUAGE (1)∂ 11 subroutines (vcl_recv, vcl_hash, etc ...)∂ Backend(s)∂ Directors (random, round robin, hash)∂ Access control lists∂ Saint and grace mode∂ Custom subroutines∂ Inline C∂ Compiled to C when run 19
  • 46. VARNISH CONFIGURATION LANGUAGE (1)∂ 11 subroutines (vcl_recv, vcl_hash, etc ...)∂ Backend(s)∂ Directors (random, round robin, hash)∂ Access control lists∂ Saint and grace mode∂ Custom subroutines∂ Inline C∂ Since v3.0 : VMODS∂ Compiled to C when run 19
  • 47. VARNISH CONFIGURATION LANGUAGE (2) if (req.request == “GET” && req.url ~ “.(gif|jpg|jpeg|bmp|png|ico)$”) { unset req.http.cookie; set req.grace = 1m; return(lookup); }∂ Cache images in vcl_recv 20
  • 48. VARNISH CONFIGURATION LANGUAGE (3)if (req.http.cookie) { set req.http.Cookie = regsuball(req.http.Cookie, "__utm.=[^;]+(; )?", ""); if (req.http.cookie ~ "^ *$") { remove req.http.cookie; }}∂ Remove Google Analytics in vcl_recv 21
  • 49. VARNISH CONFIGURATION LANGUAGE (4)backend default { .host="192.168.1.10"; .port="8080"; .probe={ .url=”/test.php”; }}backend second { .host="192.168.1.20"; .port="8080";}∂ Multiple backend configuration 22
  • 50. VARNISH CONFIGURATION LANGUAGE (5)director balance random { { .backend = default; .weight = 1; } { .backend = second; .weight = 2; }}sub vcl_recv { set.req.backend = balance; return(pass);}∂ Load balanced 33 / 66 % config 23
  • 51. VARNISH CONFIGURATION LANGUAGE (6)if (obj.hits > 0) { #if hit add hit count set resp.http.X-Cache = "HIT-" obj.hits;} else { set resp.http.X-Cache = "MISS”;}∂ Adding headers on response 24
  • 52. VARNISHING WORDPRESS (1) 25
  • 53. VARNISHING WORDPRESS (1)∂ Siege on standard WordPress homepage 25
  • 54. VARNISHING WORDPRESS (1)∂ Siege on standard WordPress homepage §No Varnish 25
  • 55. VARNISHING WORDPRESS (1)∂ Siege on standard WordPress homepage §No Varnish §Default Varnish installation 25
  • 56. VARNISHING WORDPRESS (1)∂ Siege on standard WordPress homepage §No Varnish §Default Varnish installation §Custom VCL configuration 25
  • 57. VARNISHING WORDPRESS (2)∂ No Varnish 26
  • 58. VARNISHING WORDPRESS (2)∂ No Varnish 26
  • 59. VARNISHING WORDPRESS (2)∂ No Varnish 26
  • 60. VARNISHING WORDPRESS (3)∂ Default Varnish 27
  • 61. VARNISHING WORDPRESS (3)∂ Default Varnish 27
  • 62. VARNISHING WORDPRESS (3)∂ Default Varnish 27
  • 63. VARNISHING WORDPRESS (4)# Drop any cookies sent to WordPress.sub vcl_recv { unset req.http.cookie;}# Drop any cookies WordPress tries to send backsub vcl_fetch { unset beresp.http.set-cookie;}∂ Kill those cookies!! 28
  • 64. VARNISHING WORDPRESS (5)∂ Custom Varnish 29
  • 65. VARNISHING WORDPRESS (5)∂ Custom Varnish 29
  • 66. VARNISHING WORDPRESS (5)∂ Custom Varnish 29
  • 67. OTHER PITFALLS 30
  • 68. OTHER PITFALLS∂ Accept headers §application/vnd.kingsofcode.speakers+xml; §return “Vary: Accept” header 30
  • 69. OTHER PITFALLS∂ Accept headers §application/vnd.kingsofcode.speakers+xml; §return “Vary: Accept” header∂ Accept-encoding header 30
  • 70. OTHER PITFALLS∂ Accept headers §application/vnd.kingsofcode.speakers+xml; §return “Vary: Accept” header∂ Accept-encoding header∂ Vary: User-Agent header∂ Not specifically Varnish issues 30
  • 71. EDGE SIDE INCLUDES (1)∂ Markup language∂ Co-authored Akamai, IBM, Oracle & more∂ Extensive features∂ Very powerful for content assembly 31
  • 72. EDGE SIDE INCLUDES (2)∂ if ... else § <esi:choose><esi:when> ... <esi:otherwise>∂ Variables § <esi:assign> and <esi:vars>∂ Varnish implements only one feature! § <esi:include src=”uri” />∂ Examples? http://esi-examples.akamai.com 32
  • 73. EDGE SIDE INCLUDES (3)if (req.url ~ "^/url/to/esi/snippets") { unset beresp.http.set-cookie;}## enable esi on header from backendif (beresp.http.enable-esi == "1") { set beresp.do_esi = true; //varnish 3.0 unset beresp.http.enable-esi;}∂ VCL config for ESI 33
  • 74. EDGE SIDE INCLUDES (4)∂ Where are the ESI’s? 34
  • 75. EDGE SIDE INCLUDES (5)∂ Different TTL’s for content snippets 35
  • 76. TOOLS∂Varnishstat∂Varnishsizes∂Varnishtop∂Varnishadm∂Varnishlog∂ You can’t go without 36
  • 77. VARNISHLOG∂ Log hits § varnishlog -o VCL_Call hit | grep RxURL∂ Log misses § varnishlog -i TxURL 37
  • 78. VARNISHLOG∂ Request of a gif image by client 38
  • 79. VARNISHLOG∂ Request of a gif image by client 38
  • 80. VARNISHLOG∂ Request of a gif image by client 38
  • 81. VARNISHLOG∂ Request of a gif image by client 38
  • 82. CAVEATS∂ Unsuitable for 32 bit∂ Beware of cookie monster applications∂ Check your API design∂ Avoid large files∂ Minimize cookie usage in ESI∂ No SSL support 39
  • 83. ∂ Questions? Thank you!