Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Nginx and friends - putting a turbo button on your site

3,849 views

Published on

Whenever you build a dynamic site with user-specific content, each hit to the site causes a hit to the .Net/PHP/Ruby/Python backend, potentially causing scalability issues. In this talk, we’ll look at a new way of handling user-specific content which takes the load away from the backend. Be prepared to learn about some exciting tools that add a ‘turbo’ button to your infrastructure

Published in: Technology

Nginx and friends - putting a turbo button on your site

  1. 1. Nginx and friendsaka Adding a turbo button to your site Wim Godden Cu.be Solutions
  2. 2. Who am I ?Wim Godden (@wimgtr)Owner of Cu.be Solutions (http://cu.be)Open source developer since 1997Developer of OpenXZend Certified EngineerZend Framework Certified EngineerMySQL Certified Developer
  3. 3. Who are you ?Developers ?System/network engineers ?Managers ?
  4. 4. The Microsoft box
  5. 5. Whats outside the box ?Operating systems ?Database servers ?Webservers ?Reverse proxies ?
  6. 6. Website X Header Latest news Article content page Navigation Article content
  7. 7. What is caching ? x = 5, y = 2 Same result n = 50 CACHE select * from article join user Doesnt change on article.user_id = user.id all the time order by created desc limit 10
  8. 8. Caching storage - Memcache(d)Facebook, Twitter, YouTube, … → need we say more ?Distributed memory caching systemMultiple machines ↔ 1 big memory-based hash-tableKey-value storage system Keys - max. 250bytes Values - max. 1Mbyte
  9. 9. Caching storage - Memcache(d)Facebook, Twitter, YouTube, … → need we say more ?Distributed memory caching systemKey-value storage system Keys - max. 250bytes Values - max. 1MbyteExtremely fast... non-blocking, UDP (!)
  10. 10. Memcache - installation & running itInstallation Distribution package PECL Windows : binariesRunning No config-files memcached -d -m <mem> -l <ip> -p <port> ex. : memcached -d -m 2048 -l 172.16.1.91 -p 11211
  11. 11. Caching storage - Memcache - some notesNot fault-tolerant Its a cache ! Lose session data Lose shopping cart data ...
  12. 12. Caching storage - Memcache - some notesNot fault-tolerant Its a cache ! Lose session data Lose shopping cart data …Firewall your Memcache port !
  13. 13. Memcache in codeInitialization :MemcachedClient.Setup("MyCache", new string[]{ "server1.example.com", "server2.example.com"});Usage :MemcachedClient cache = MemcachedClient.GetInstance("MyCache");string myData = cache.Get("myKey") as string;if (myData == null) { Miss myData = GetMyDataFromDB(); cache.Set("myKey", myData);}// Use myData
  14. 14. Memcache slabs (or why Memcache says its full when its not)Multiple slabs of different sizes : Slab 1 : 400 bytes Slab 2 : 480 bytes (400 * 1.2) Slab 3 : 576 bytes (480 * 1.2) (and so on...)Multiplier (1.2 here) can be configuredStore a lot of very large objects→ Large slabs : full→ Rest : free→ Eviction of data !
  15. 15. Memcache - Is it working ?Connect to it using telnet STAT pid 2941 STAT uptime 10878 STAT time 1296074240 "stats" command → STAT version 1.4.5 STAT pointer_size 64 STAT rusage_user 20.089945 Use Cacti or other monitoring tools STAT rusage_system 58.499106 STAT curr_connections 16 STAT total_connections 276950 STAT connection_structures 96 STAT cmd_get 276931 STAT cmd_set 584148 STAT cmd_flush 0 STAT get_hits 211106 STAT get_misses 65825 STAT delete_misses 101 STAT delete_hits 276829 STAT incr_misses 0 STAT incr_hits 0 STAT decr_misses 0 STAT decr_hits 0 STAT cas_misses 0 STAT cas_hits 0 STAT cas_badval 0 STAT auth_cmds 0 STAT auth_errors 0 STAT bytes_read 613193860 STAT bytes_written 553991373 STAT limit_maxbytes 268435456 STAT accepting_conns 1 STAT listen_disabled_num 0 STAT threads 4 STAT conn_yields 0 STAT bytes 20418140 STAT curr_items 65826 STAT total_items 553856 STAT evictions 0 STAT reclaimed 0
  16. 16. Website X – news is changing ! Header Latest news Article content page Navigation Article content
  17. 17. cache.Delete("Latest-news")
  18. 18. Updating data
  19. 19. Updating data LCD_Popular_Product_List
  20. 20. Adding/updating datacache.Delete("LCD_Popular_Product_List")
  21. 21. Adding/updating data
  22. 22. Adding/updating data - Why it crashed
  23. 23. Adding/updating data - Why it crashed
  24. 24. Adding/updating data - Why it crashed
  25. 25. Cache stampeding
  26. 26. Cache stampeding
  27. 27. Memcache code ? Memcache code Visitor interface Admin interface DB
  28. 28. Cache warmup scriptsUsed to fill your cache when its emptyRun it before starting Webserver !2 ways : Visit all URLs Error-prone Hard to maintain Call all cache-updating methods Make sure you have a warmup script !
  29. 29. Cache stampeding - what about locking ?Seems like a nice idea, but...While lock in placeWhat if the process that created the lock fails ?
  30. 30. Website X Header Latest news Article content page Navigation Article content
  31. 31. Website X Header (TTL = 2h) Latest news Article content page Navigation (TTL = 2h) Article content
  32. 32. Website X Header (TTL = 2h) Latest news (TTL = 2m) Article content page Navigation (TTL = 2h) Article content (TTL = 15m)
  33. 33. Move the cache
  34. 34. VarnishNot just a load balancerReverse proxy cache / http accelerator / …Caches (parts of) pages in memory
  35. 35. Varnish - ESI In your article page output : Header (TTL : 60 min) <esi:include src="/top"/> /top <esi:include src="/nav"/> Latest news (TTL : 2 min) /news <esi:include src="/news"/> <esi:include src="/article/732"/>Navigation Article content page (TTL : In your Varnish config : 60 min) Article content (TTL : 15 min) sub vcl_fetch { /nav /article/732 if (req.url == "/news") { esi; /* Do ESI processing */ set obj.ttl = 2m; } elseif (req.url == "/nav") { esi; set obj.ttl = 1h; } elseif …. …. }
  36. 36. A simple benchmark – 2KByte JPEG Apache 2.2 4210 IIS 7.5 3960 Varnish 3.0 11400
  37. 37. A dynamically generated, but static page Apache 2.2 + PHP (3 DB queries) 18 IIS 7.5 + .Net (3 DB queries) 16 Varnish 3.0 11400
  38. 38. Varnish - what can/cant be cached ?Can : Static pages Images, js, css Static parts of pages that dont change often (ESI)Cant : POST requests Very large files (its not a file server !) Requests with Set-Cookie User-specific content
  39. 39. ESI → no caching on user-specific content ? Logged in as : Wim Godden TTL = 0s ? 5 messages TTL=1h TTL = 5min
  40. 40. Dynamic contentRequest Varnish Ideallyuser X : /top /top request to webserver sameuser Y : /top /top request to webserver sameuser X : /top /top request to webserver if unchanged : served from cacheuser Y : /top /top request to webserver if unchanged : served from cache
  41. 41. NginxWeb serverReverse proxyLightweight, fast12.2% of all Websites
  42. 42. NginxNo threads, event-drivenUses epoll / kqueueLow memory footprint10000 active connections = normal
  43. 43. Coming to Nginx soon... Logged in as : Wim Godden 5 messages Menu NEWS
  44. 44. Coming to Nginx soon... <esim:include src="/top" session="1" ttl="1h" /> <esim:include src="/menu" <esim:include src="/news" ttl="5m" /> ttl="1h" />
  45. 45. Requesting /page (1st time) Nginx 1 3 Shared memory /page 2 4 /page
  46. 46. Requesting /page ESI subrequests (1st time) Nginx 1 2 3 /menu /news /top ($sessionid$)
  47. 47. Requesting /page (next time) Nginx 1 Shared memory /page 2 /page /menu /news /top ($sessionid$)
  48. 48. New message arrives... POST /send o ... se i nt t(. e rt ..) ins DB top ($sessionid$)
  49. 49. AdvantagesNo repeated hits to webserver anymore ! Only the initial hit (unless you pre-heat/warm up the cache !) No hits for user-specific content Not even for non-specific content
  50. 50. News added addnews() method o ... se i nt t(. e rt ..) ins DB /news
  51. 51. AdvantagesNo repeated hits to webserver anymore ! Only the initial hit (unless you pre-heat/warm up the cache !) Not for user-specific content Not even for non-specific content No TTLs for non-specific content Imagine doing it for the bid status on Ebay items ;-)
  52. 52. How many Memcache requests ? Logged in as : Wim Godden <esim:include src="/top" session="1" ttl="1h" /> 5 messages <esim:include src="/menu" <esim:include src="/news" ttl="5m" /> ttl="1h" />
  53. 53. Why Nginx ?Native Memcache supportExcellent and superfast subrequest system Including parallel subrequestsHandles thousands of connections per worker With minimal memory footprint
  54. 54. Whats the result ?
  55. 55. FiguresFirst customer : No. of web servers : 18 → 4 No. of db servers : 6 → 2 Total : 24 → 6 (75% reduction !)Second customer (already using Nginx + Memcache) : No. of web servers : 72 → 8 No. of db servers : 15 → 4 Total : 87 → 12 (86% reduction !)
  56. 56. AvailabilityStable at 2 customersStill under heavy developmentBeta : July 2012Final : Sep 2012
  57. 57. So...
  58. 58. So...Theres life outside the MS-box And its pretty awesome ;-)Need caching in .Net (for DB queries, config-files, etc.) ? Memcache Velocity ?Caching static pages → Varnish with ESICaching for authenticated users → Nginx with Dynamic ESITheres a lot more outside the box → Maybe some other time ;-)
  59. 59. Questions ?
  60. 60. Questions ?
  61. 61. ContactTwitter @wimgtrWeb http://techblog.wimgodden.beSlides http://www.slideshare.net/wimgE-mail wim.godden@cu.be
  62. 62. Thanks ! Please rate my talk :http://speakerrate.com/talks/11081

×