Nginx and friends - putting a turbo button on your site

3,694 views

Published on

Whenever you build a dynamic site with user-specific content, each hit to the site causes a hit to the .Net/PHP/Ruby/Python backend, potentially causing scalability issues. In this talk, we’ll look at a new way of handling user-specific content which takes the load away from the backend. Be prepared to learn about some exciting tools that add a ‘turbo’ button to your infrastructure

Published in: Technology

Nginx and friends - putting a turbo button on your site

  1. 1. Nginx and friendsaka Adding a turbo button to your site Wim Godden Cu.be Solutions
  2. 2. Who am I ?Wim Godden (@wimgtr)Owner of Cu.be Solutions (http://cu.be)Open source developer since 1997Developer of OpenXZend Certified EngineerZend Framework Certified EngineerMySQL Certified Developer
  3. 3. Who are you ?Developers ?System/network engineers ?Managers ?
  4. 4. The Microsoft box
  5. 5. Whats outside the box ?Operating systems ?Database servers ?Webservers ?Reverse proxies ?
  6. 6. Website X Header Latest news Article content page Navigation Article content
  7. 7. What is caching ? x = 5, y = 2 Same result n = 50 CACHE select * from article join user Doesnt change on article.user_id = user.id all the time order by created desc limit 10
  8. 8. Caching storage - Memcache(d)Facebook, Twitter, YouTube, … → need we say more ?Distributed memory caching systemMultiple machines ↔ 1 big memory-based hash-tableKey-value storage system Keys - max. 250bytes Values - max. 1Mbyte
  9. 9. Caching storage - Memcache(d)Facebook, Twitter, YouTube, … → need we say more ?Distributed memory caching systemKey-value storage system Keys - max. 250bytes Values - max. 1MbyteExtremely fast... non-blocking, UDP (!)
  10. 10. Memcache - installation & running itInstallation Distribution package PECL Windows : binariesRunning No config-files memcached -d -m <mem> -l <ip> -p <port> ex. : memcached -d -m 2048 -l 172.16.1.91 -p 11211
  11. 11. Caching storage - Memcache - some notesNot fault-tolerant Its a cache ! Lose session data Lose shopping cart data ...
  12. 12. Caching storage - Memcache - some notesNot fault-tolerant Its a cache ! Lose session data Lose shopping cart data …Firewall your Memcache port !
  13. 13. Memcache in codeInitialization :MemcachedClient.Setup("MyCache", new string[]{ "server1.example.com", "server2.example.com"});Usage :MemcachedClient cache = MemcachedClient.GetInstance("MyCache");string myData = cache.Get("myKey") as string;if (myData == null) { Miss myData = GetMyDataFromDB(); cache.Set("myKey", myData);}// Use myData
  14. 14. Memcache slabs (or why Memcache says its full when its not)Multiple slabs of different sizes : Slab 1 : 400 bytes Slab 2 : 480 bytes (400 * 1.2) Slab 3 : 576 bytes (480 * 1.2) (and so on...)Multiplier (1.2 here) can be configuredStore a lot of very large objects→ Large slabs : full→ Rest : free→ Eviction of data !
  15. 15. Memcache - Is it working ?Connect to it using telnet STAT pid 2941 STAT uptime 10878 STAT time 1296074240 "stats" command → STAT version 1.4.5 STAT pointer_size 64 STAT rusage_user 20.089945 Use Cacti or other monitoring tools STAT rusage_system 58.499106 STAT curr_connections 16 STAT total_connections 276950 STAT connection_structures 96 STAT cmd_get 276931 STAT cmd_set 584148 STAT cmd_flush 0 STAT get_hits 211106 STAT get_misses 65825 STAT delete_misses 101 STAT delete_hits 276829 STAT incr_misses 0 STAT incr_hits 0 STAT decr_misses 0 STAT decr_hits 0 STAT cas_misses 0 STAT cas_hits 0 STAT cas_badval 0 STAT auth_cmds 0 STAT auth_errors 0 STAT bytes_read 613193860 STAT bytes_written 553991373 STAT limit_maxbytes 268435456 STAT accepting_conns 1 STAT listen_disabled_num 0 STAT threads 4 STAT conn_yields 0 STAT bytes 20418140 STAT curr_items 65826 STAT total_items 553856 STAT evictions 0 STAT reclaimed 0
  16. 16. Website X – news is changing ! Header Latest news Article content page Navigation Article content
  17. 17. cache.Delete("Latest-news")
  18. 18. Updating data
  19. 19. Updating data LCD_Popular_Product_List
  20. 20. Adding/updating datacache.Delete("LCD_Popular_Product_List")
  21. 21. Adding/updating data
  22. 22. Adding/updating data - Why it crashed
  23. 23. Adding/updating data - Why it crashed
  24. 24. Adding/updating data - Why it crashed
  25. 25. Cache stampeding
  26. 26. Cache stampeding
  27. 27. Memcache code ? Memcache code Visitor interface Admin interface DB
  28. 28. Cache warmup scriptsUsed to fill your cache when its emptyRun it before starting Webserver !2 ways : Visit all URLs Error-prone Hard to maintain Call all cache-updating methods Make sure you have a warmup script !
  29. 29. Cache stampeding - what about locking ?Seems like a nice idea, but...While lock in placeWhat if the process that created the lock fails ?
  30. 30. Website X Header Latest news Article content page Navigation Article content
  31. 31. Website X Header (TTL = 2h) Latest news Article content page Navigation (TTL = 2h) Article content
  32. 32. Website X Header (TTL = 2h) Latest news (TTL = 2m) Article content page Navigation (TTL = 2h) Article content (TTL = 15m)
  33. 33. Move the cache
  34. 34. VarnishNot just a load balancerReverse proxy cache / http accelerator / …Caches (parts of) pages in memory
  35. 35. Varnish - ESI In your article page output : Header (TTL : 60 min) <esi:include src="/top"/> /top <esi:include src="/nav"/> Latest news (TTL : 2 min) /news <esi:include src="/news"/> <esi:include src="/article/732"/>Navigation Article content page (TTL : In your Varnish config : 60 min) Article content (TTL : 15 min) sub vcl_fetch { /nav /article/732 if (req.url == "/news") { esi; /* Do ESI processing */ set obj.ttl = 2m; } elseif (req.url == "/nav") { esi; set obj.ttl = 1h; } elseif …. …. }
  36. 36. A simple benchmark – 2KByte JPEG Apache 2.2 4210 IIS 7.5 3960 Varnish 3.0 11400
  37. 37. A dynamically generated, but static page Apache 2.2 + PHP (3 DB queries) 18 IIS 7.5 + .Net (3 DB queries) 16 Varnish 3.0 11400
  38. 38. Varnish - what can/cant be cached ?Can : Static pages Images, js, css Static parts of pages that dont change often (ESI)Cant : POST requests Very large files (its not a file server !) Requests with Set-Cookie User-specific content
  39. 39. ESI → no caching on user-specific content ? Logged in as : Wim Godden TTL = 0s ? 5 messages TTL=1h TTL = 5min
  40. 40. Dynamic contentRequest Varnish Ideallyuser X : /top /top request to webserver sameuser Y : /top /top request to webserver sameuser X : /top /top request to webserver if unchanged : served from cacheuser Y : /top /top request to webserver if unchanged : served from cache
  41. 41. NginxWeb serverReverse proxyLightweight, fast12.2% of all Websites
  42. 42. NginxNo threads, event-drivenUses epoll / kqueueLow memory footprint10000 active connections = normal
  43. 43. Coming to Nginx soon... Logged in as : Wim Godden 5 messages Menu NEWS
  44. 44. Coming to Nginx soon... <esim:include src="/top" session="1" ttl="1h" /> <esim:include src="/menu" <esim:include src="/news" ttl="5m" /> ttl="1h" />
  45. 45. Requesting /page (1st time) Nginx 1 3 Shared memory /page 2 4 /page
  46. 46. Requesting /page ESI subrequests (1st time) Nginx 1 2 3 /menu /news /top ($sessionid$)
  47. 47. Requesting /page (next time) Nginx 1 Shared memory /page 2 /page /menu /news /top ($sessionid$)
  48. 48. New message arrives... POST /send o ... se i nt t(. e rt ..) ins DB top ($sessionid$)
  49. 49. AdvantagesNo repeated hits to webserver anymore ! Only the initial hit (unless you pre-heat/warm up the cache !) No hits for user-specific content Not even for non-specific content
  50. 50. News added addnews() method o ... se i nt t(. e rt ..) ins DB /news
  51. 51. AdvantagesNo repeated hits to webserver anymore ! Only the initial hit (unless you pre-heat/warm up the cache !) Not for user-specific content Not even for non-specific content No TTLs for non-specific content Imagine doing it for the bid status on Ebay items ;-)
  52. 52. How many Memcache requests ? Logged in as : Wim Godden <esim:include src="/top" session="1" ttl="1h" /> 5 messages <esim:include src="/menu" <esim:include src="/news" ttl="5m" /> ttl="1h" />
  53. 53. Why Nginx ?Native Memcache supportExcellent and superfast subrequest system Including parallel subrequestsHandles thousands of connections per worker With minimal memory footprint
  54. 54. Whats the result ?
  55. 55. FiguresFirst customer : No. of web servers : 18 → 4 No. of db servers : 6 → 2 Total : 24 → 6 (75% reduction !)Second customer (already using Nginx + Memcache) : No. of web servers : 72 → 8 No. of db servers : 15 → 4 Total : 87 → 12 (86% reduction !)
  56. 56. AvailabilityStable at 2 customersStill under heavy developmentBeta : July 2012Final : Sep 2012
  57. 57. So...
  58. 58. So...Theres life outside the MS-box And its pretty awesome ;-)Need caching in .Net (for DB queries, config-files, etc.) ? Memcache Velocity ?Caching static pages → Varnish with ESICaching for authenticated users → Nginx with Dynamic ESITheres a lot more outside the box → Maybe some other time ;-)
  59. 59. Questions ?
  60. 60. Questions ?
  61. 61. ContactTwitter @wimgtrWeb http://techblog.wimgodden.beSlides http://www.slideshare.net/wimgE-mail wim.godden@cu.be
  62. 62. Thanks ! Please rate my talk :http://speakerrate.com/talks/11081

×