Nginx and friends - putting a turbo button on your site

  • 2,857 views
Uploaded on

Whenever you build a dynamic site with user-specific content, each hit to the site causes a hit to the .Net/PHP/Ruby/Python backend, potentially causing scalability issues. In this talk, we’ll look at …

Whenever you build a dynamic site with user-specific content, each hit to the site causes a hit to the .Net/PHP/Ruby/Python backend, potentially causing scalability issues. In this talk, we’ll look at a new way of handling user-specific content which takes the load away from the backend. Be prepared to learn about some exciting tools that add a ‘turbo’ button to your infrastructure

More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
2,857
On Slideshare
0
From Embeds
0
Number of Embeds
3

Actions

Shares
Downloads
23
Comments
0
Likes
3

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Nginx and friendsaka Adding a turbo button to your site Wim Godden Cu.be Solutions
  • 2. Who am I ?Wim Godden (@wimgtr)Owner of Cu.be Solutions (http://cu.be)Open source developer since 1997Developer of OpenXZend Certified EngineerZend Framework Certified EngineerMySQL Certified Developer
  • 3. Who are you ?Developers ?System/network engineers ?Managers ?
  • 4. The Microsoft box
  • 5. Whats outside the box ?Operating systems ?Database servers ?Webservers ?Reverse proxies ?
  • 6. Website X Header Latest news Article content page Navigation Article content
  • 7. What is caching ? x = 5, y = 2 Same result n = 50 CACHE select * from article join user Doesnt change on article.user_id = user.id all the time order by created desc limit 10
  • 8. Caching storage - Memcache(d)Facebook, Twitter, YouTube, … → need we say more ?Distributed memory caching systemMultiple machines ↔ 1 big memory-based hash-tableKey-value storage system Keys - max. 250bytes Values - max. 1Mbyte
  • 9. Caching storage - Memcache(d)Facebook, Twitter, YouTube, … → need we say more ?Distributed memory caching systemKey-value storage system Keys - max. 250bytes Values - max. 1MbyteExtremely fast... non-blocking, UDP (!)
  • 10. Memcache - installation & running itInstallation Distribution package PECL Windows : binariesRunning No config-files memcached -d -m <mem> -l <ip> -p <port> ex. : memcached -d -m 2048 -l 172.16.1.91 -p 11211
  • 11. Caching storage - Memcache - some notesNot fault-tolerant Its a cache ! Lose session data Lose shopping cart data ...
  • 12. Caching storage - Memcache - some notesNot fault-tolerant Its a cache ! Lose session data Lose shopping cart data …Firewall your Memcache port !
  • 13. Memcache in codeInitialization :MemcachedClient.Setup("MyCache", new string[]{ "server1.example.com", "server2.example.com"});Usage :MemcachedClient cache = MemcachedClient.GetInstance("MyCache");string myData = cache.Get("myKey") as string;if (myData == null) { Miss myData = GetMyDataFromDB(); cache.Set("myKey", myData);}// Use myData
  • 14. Memcache slabs (or why Memcache says its full when its not)Multiple slabs of different sizes : Slab 1 : 400 bytes Slab 2 : 480 bytes (400 * 1.2) Slab 3 : 576 bytes (480 * 1.2) (and so on...)Multiplier (1.2 here) can be configuredStore a lot of very large objects→ Large slabs : full→ Rest : free→ Eviction of data !
  • 15. Memcache - Is it working ?Connect to it using telnet STAT pid 2941 STAT uptime 10878 STAT time 1296074240 "stats" command → STAT version 1.4.5 STAT pointer_size 64 STAT rusage_user 20.089945 Use Cacti or other monitoring tools STAT rusage_system 58.499106 STAT curr_connections 16 STAT total_connections 276950 STAT connection_structures 96 STAT cmd_get 276931 STAT cmd_set 584148 STAT cmd_flush 0 STAT get_hits 211106 STAT get_misses 65825 STAT delete_misses 101 STAT delete_hits 276829 STAT incr_misses 0 STAT incr_hits 0 STAT decr_misses 0 STAT decr_hits 0 STAT cas_misses 0 STAT cas_hits 0 STAT cas_badval 0 STAT auth_cmds 0 STAT auth_errors 0 STAT bytes_read 613193860 STAT bytes_written 553991373 STAT limit_maxbytes 268435456 STAT accepting_conns 1 STAT listen_disabled_num 0 STAT threads 4 STAT conn_yields 0 STAT bytes 20418140 STAT curr_items 65826 STAT total_items 553856 STAT evictions 0 STAT reclaimed 0
  • 16. Website X – news is changing ! Header Latest news Article content page Navigation Article content
  • 17. cache.Delete("Latest-news")
  • 18. Updating data
  • 19. Updating data LCD_Popular_Product_List
  • 20. Adding/updating datacache.Delete("LCD_Popular_Product_List")
  • 21. Adding/updating data
  • 22. Adding/updating data - Why it crashed
  • 23. Adding/updating data - Why it crashed
  • 24. Adding/updating data - Why it crashed
  • 25. Cache stampeding
  • 26. Cache stampeding
  • 27. Memcache code ? Memcache code Visitor interface Admin interface DB
  • 28. Cache warmup scriptsUsed to fill your cache when its emptyRun it before starting Webserver !2 ways : Visit all URLs Error-prone Hard to maintain Call all cache-updating methods Make sure you have a warmup script !
  • 29. Cache stampeding - what about locking ?Seems like a nice idea, but...While lock in placeWhat if the process that created the lock fails ?
  • 30. Website X Header Latest news Article content page Navigation Article content
  • 31. Website X Header (TTL = 2h) Latest news Article content page Navigation (TTL = 2h) Article content
  • 32. Website X Header (TTL = 2h) Latest news (TTL = 2m) Article content page Navigation (TTL = 2h) Article content (TTL = 15m)
  • 33. Move the cache
  • 34. VarnishNot just a load balancerReverse proxy cache / http accelerator / …Caches (parts of) pages in memory
  • 35. Varnish - ESI In your article page output : Header (TTL : 60 min) <esi:include src="/top"/> /top <esi:include src="/nav"/> Latest news (TTL : 2 min) /news <esi:include src="/news"/> <esi:include src="/article/732"/>Navigation Article content page (TTL : In your Varnish config : 60 min) Article content (TTL : 15 min) sub vcl_fetch { /nav /article/732 if (req.url == "/news") { esi; /* Do ESI processing */ set obj.ttl = 2m; } elseif (req.url == "/nav") { esi; set obj.ttl = 1h; } elseif …. …. }
  • 36. A simple benchmark – 2KByte JPEG Apache 2.2 4210 IIS 7.5 3960 Varnish 3.0 11400
  • 37. A dynamically generated, but static page Apache 2.2 + PHP (3 DB queries) 18 IIS 7.5 + .Net (3 DB queries) 16 Varnish 3.0 11400
  • 38. Varnish - what can/cant be cached ?Can : Static pages Images, js, css Static parts of pages that dont change often (ESI)Cant : POST requests Very large files (its not a file server !) Requests with Set-Cookie User-specific content
  • 39. ESI → no caching on user-specific content ? Logged in as : Wim Godden TTL = 0s ? 5 messages TTL=1h TTL = 5min
  • 40. Dynamic contentRequest Varnish Ideallyuser X : /top /top request to webserver sameuser Y : /top /top request to webserver sameuser X : /top /top request to webserver if unchanged : served from cacheuser Y : /top /top request to webserver if unchanged : served from cache
  • 41. NginxWeb serverReverse proxyLightweight, fast12.2% of all Websites
  • 42. NginxNo threads, event-drivenUses epoll / kqueueLow memory footprint10000 active connections = normal
  • 43. Coming to Nginx soon... Logged in as : Wim Godden 5 messages Menu NEWS
  • 44. Coming to Nginx soon... <esim:include src="/top" session="1" ttl="1h" /> <esim:include src="/menu" <esim:include src="/news" ttl="5m" /> ttl="1h" />
  • 45. Requesting /page (1st time) Nginx 1 3 Shared memory /page 2 4 /page
  • 46. Requesting /page ESI subrequests (1st time) Nginx 1 2 3 /menu /news /top ($sessionid$)
  • 47. Requesting /page (next time) Nginx 1 Shared memory /page 2 /page /menu /news /top ($sessionid$)
  • 48. New message arrives... POST /send o ... se i nt t(. e rt ..) ins DB top ($sessionid$)
  • 49. AdvantagesNo repeated hits to webserver anymore ! Only the initial hit (unless you pre-heat/warm up the cache !) No hits for user-specific content Not even for non-specific content
  • 50. News added addnews() method o ... se i nt t(. e rt ..) ins DB /news
  • 51. AdvantagesNo repeated hits to webserver anymore ! Only the initial hit (unless you pre-heat/warm up the cache !) Not for user-specific content Not even for non-specific content No TTLs for non-specific content Imagine doing it for the bid status on Ebay items ;-)
  • 52. How many Memcache requests ? Logged in as : Wim Godden <esim:include src="/top" session="1" ttl="1h" /> 5 messages <esim:include src="/menu" <esim:include src="/news" ttl="5m" /> ttl="1h" />
  • 53. Why Nginx ?Native Memcache supportExcellent and superfast subrequest system Including parallel subrequestsHandles thousands of connections per worker With minimal memory footprint
  • 54. Whats the result ?
  • 55. FiguresFirst customer : No. of web servers : 18 → 4 No. of db servers : 6 → 2 Total : 24 → 6 (75% reduction !)Second customer (already using Nginx + Memcache) : No. of web servers : 72 → 8 No. of db servers : 15 → 4 Total : 87 → 12 (86% reduction !)
  • 56. AvailabilityStable at 2 customersStill under heavy developmentBeta : July 2012Final : Sep 2012
  • 57. So...
  • 58. So...Theres life outside the MS-box And its pretty awesome ;-)Need caching in .Net (for DB queries, config-files, etc.) ? Memcache Velocity ?Caching static pages → Varnish with ESICaching for authenticated users → Nginx with Dynamic ESITheres a lot more outside the box → Maybe some other time ;-)
  • 59. Questions ?
  • 60. Questions ?
  • 61. ContactTwitter @wimgtrWeb http://techblog.wimgodden.beSlides http://www.slideshare.net/wimgE-mail wim.godden@cu.be
  • 62. Thanks ! Please rate my talk :http://speakerrate.com/talks/11081