Nginx and friendsaka Adding a turbo button to your site                                             Wim Godden            ...
Who am I ?Wim Godden (@wimgtr)Owner of Cu.be Solutions (http://cu.be)Open source developer since 1997Developer of OpenXZen...
Who are you ?Developers ?System/network engineers ?Managers ?
The Microsoft box
Whats outside the box ?Operating systems ?Database servers ?Webservers ?Reverse proxies ?
Website X                          Header                               Latest news                    Article content pag...
What is caching ?                                             x = 5, y = 2                                                ...
Caching storage - Memcache(d)Facebook, Twitter, YouTube, … → need we say more ?Distributed memory caching systemMultiple m...
Caching storage - Memcache(d)Facebook, Twitter, YouTube, … → need we say more ?Distributed memory caching systemKey-value ...
Memcache - installation & running itInstallation   Distribution package   PECL   Windows : binariesRunning   No config-fil...
Caching storage - Memcache - some notesNot fault-tolerant   Its a cache !   Lose session data   Lose shopping cart data   ...
Caching storage - Memcache - some notesNot fault-tolerant   Its a cache !   Lose session data   Lose shopping cart data   ...
Memcache in codeInitialization :MemcachedClient.Setup("MyCache", new string[]{ "server1.example.com", "server2.example.com...
Memcache slabs            (or why Memcache says its full when its not)Multiple slabs of different sizes :   Slab 1 : 400 b...
Memcache - Is it working ?Connect to it using telnet               STAT pid 2941                                         S...
Website X – news is changing !                           Header                                Latest news                ...
cache.Delete("Latest-news")
Updating data
Updating data                LCD_Popular_Product_List
Adding/updating datacache.Delete("LCD_Popular_Product_List")
Adding/updating data
Adding/updating data - Why it crashed
Adding/updating data - Why it crashed
Adding/updating data - Why it crashed
Cache stampeding
Cache stampeding
Memcache code ?    Memcache code     Visitor interface        Admin interface                         DB
Cache warmup scriptsUsed to fill your cache when its emptyRun it before starting Webserver !2 ways :   Visit all URLs     ...
Cache stampeding - what about locking ?Seems like a nice idea, but...While lock in placeWhat if the process that created t...
Website X                          Header                               Latest news                    Article content pag...
Website X                          Header                        (TTL = 2h)                               Latest news     ...
Website X                          Header                        (TTL = 2h)                         Latest news (TTL = 2m)...
Move the cache
VarnishNot just a load balancerReverse proxy cache / http accelerator / …Caches (parts of) pages in memory
Varnish - ESI                                                In your article page output :             Header (TTL : 60 mi...
A simple benchmark – 2KByte JPEG         Apache 2.2    4210         IIS 7.5       3960         Varnish 3.0   11400
A dynamically generated, but static page     Apache 2.2 + PHP (3 DB queries)   18     IIS 7.5 + .Net (3 DB queries)     16...
Varnish - what can/cant be cached ?Can :   Static pages   Images, js, css   Static parts of pages that dont change often (...
ESI → no caching on user-specific content ?                                           Logged in as : Wim Godden           ...
Dynamic contentRequest         Varnish                     Ideallyuser X : /top   /top request to webserver   sameuser Y :...
NginxWeb serverReverse proxyLightweight, fast12.2% of all Websites
NginxNo threads, event-drivenUses epoll / kqueueLow memory footprint10000 active connections = normal
Coming to Nginx soon...                                 Logged in as : Wim Godden                                         ...
Coming to Nginx soon...                      <esim:include src="/top" session="1" ttl="1h" />      <esim:include      src=...
Requesting /page (1st time)                   Nginx                                              1                        ...
Requesting /page ESI subrequests (1st time)                  Nginx                                              1         ...
Requesting /page (next time)                  Nginx                                         1                     Shared m...
New message arrives...                          POST /send                               o   ...   se                     ...
AdvantagesNo repeated hits to webserver anymore !   Only the initial hit (unless you pre-heat/warm up the cache !)   No hi...
News added                  addnews() method                              o   ...      se                          i nt   ...
AdvantagesNo repeated hits to webserver anymore !   Only the initial hit (unless you pre-heat/warm up the cache !)   Not f...
How many Memcache requests ?                                                                         Logged in as : Wim Go...
Why Nginx ?Native Memcache supportExcellent and superfast subrequest system   Including parallel subrequestsHandles thousa...
Whats the result ?
FiguresFirst customer :   No. of web servers : 18 → 4   No. of db servers : 6 → 2   Total : 24 → 6 (75% reduction !)Second...
AvailabilityStable at 2 customersStill under heavy developmentBeta : July 2012Final : Sep 2012
So...
So...Theres life outside the MS-box   And its pretty awesome ;-)Need caching in .Net (for DB queries, config-files, etc.) ...
Questions ?
Questions ?
ContactTwitter    @wimgtrWeb       http://techblog.wimgodden.beSlides        http://www.slideshare.net/wimgE-mail        w...
Thanks !        Please rate my talk :http://speakerrate.com/talks/11081
Upcoming SlideShare
Loading in...5
×

Nginx and friends - putting a turbo button on your site

3,103

Published on

Whenever you build a dynamic site with user-specific content, each hit to the site causes a hit to the .Net/PHP/Ruby/Python backend, potentially causing scalability issues. In this talk, we’ll look at a new way of handling user-specific content which takes the load away from the backend. Be prepared to learn about some exciting tools that add a ‘turbo’ button to your infrastructure

Published in: Technology
0 Comments
4 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
3,103
On Slideshare
0
From Embeds
0
Number of Embeds
4
Actions
Shares
0
Downloads
24
Comments
0
Likes
4
Embeds 0
No embeds

No notes for slide

Nginx and friends - putting a turbo button on your site

  1. 1. Nginx and friendsaka Adding a turbo button to your site Wim Godden Cu.be Solutions
  2. 2. Who am I ?Wim Godden (@wimgtr)Owner of Cu.be Solutions (http://cu.be)Open source developer since 1997Developer of OpenXZend Certified EngineerZend Framework Certified EngineerMySQL Certified Developer
  3. 3. Who are you ?Developers ?System/network engineers ?Managers ?
  4. 4. The Microsoft box
  5. 5. Whats outside the box ?Operating systems ?Database servers ?Webservers ?Reverse proxies ?
  6. 6. Website X Header Latest news Article content page Navigation Article content
  7. 7. What is caching ? x = 5, y = 2 Same result n = 50 CACHE select * from article join user Doesnt change on article.user_id = user.id all the time order by created desc limit 10
  8. 8. Caching storage - Memcache(d)Facebook, Twitter, YouTube, … → need we say more ?Distributed memory caching systemMultiple machines ↔ 1 big memory-based hash-tableKey-value storage system Keys - max. 250bytes Values - max. 1Mbyte
  9. 9. Caching storage - Memcache(d)Facebook, Twitter, YouTube, … → need we say more ?Distributed memory caching systemKey-value storage system Keys - max. 250bytes Values - max. 1MbyteExtremely fast... non-blocking, UDP (!)
  10. 10. Memcache - installation & running itInstallation Distribution package PECL Windows : binariesRunning No config-files memcached -d -m <mem> -l <ip> -p <port> ex. : memcached -d -m 2048 -l 172.16.1.91 -p 11211
  11. 11. Caching storage - Memcache - some notesNot fault-tolerant Its a cache ! Lose session data Lose shopping cart data ...
  12. 12. Caching storage - Memcache - some notesNot fault-tolerant Its a cache ! Lose session data Lose shopping cart data …Firewall your Memcache port !
  13. 13. Memcache in codeInitialization :MemcachedClient.Setup("MyCache", new string[]{ "server1.example.com", "server2.example.com"});Usage :MemcachedClient cache = MemcachedClient.GetInstance("MyCache");string myData = cache.Get("myKey") as string;if (myData == null) { Miss myData = GetMyDataFromDB(); cache.Set("myKey", myData);}// Use myData
  14. 14. Memcache slabs (or why Memcache says its full when its not)Multiple slabs of different sizes : Slab 1 : 400 bytes Slab 2 : 480 bytes (400 * 1.2) Slab 3 : 576 bytes (480 * 1.2) (and so on...)Multiplier (1.2 here) can be configuredStore a lot of very large objects→ Large slabs : full→ Rest : free→ Eviction of data !
  15. 15. Memcache - Is it working ?Connect to it using telnet STAT pid 2941 STAT uptime 10878 STAT time 1296074240 "stats" command → STAT version 1.4.5 STAT pointer_size 64 STAT rusage_user 20.089945 Use Cacti or other monitoring tools STAT rusage_system 58.499106 STAT curr_connections 16 STAT total_connections 276950 STAT connection_structures 96 STAT cmd_get 276931 STAT cmd_set 584148 STAT cmd_flush 0 STAT get_hits 211106 STAT get_misses 65825 STAT delete_misses 101 STAT delete_hits 276829 STAT incr_misses 0 STAT incr_hits 0 STAT decr_misses 0 STAT decr_hits 0 STAT cas_misses 0 STAT cas_hits 0 STAT cas_badval 0 STAT auth_cmds 0 STAT auth_errors 0 STAT bytes_read 613193860 STAT bytes_written 553991373 STAT limit_maxbytes 268435456 STAT accepting_conns 1 STAT listen_disabled_num 0 STAT threads 4 STAT conn_yields 0 STAT bytes 20418140 STAT curr_items 65826 STAT total_items 553856 STAT evictions 0 STAT reclaimed 0
  16. 16. Website X – news is changing ! Header Latest news Article content page Navigation Article content
  17. 17. cache.Delete("Latest-news")
  18. 18. Updating data
  19. 19. Updating data LCD_Popular_Product_List
  20. 20. Adding/updating datacache.Delete("LCD_Popular_Product_List")
  21. 21. Adding/updating data
  22. 22. Adding/updating data - Why it crashed
  23. 23. Adding/updating data - Why it crashed
  24. 24. Adding/updating data - Why it crashed
  25. 25. Cache stampeding
  26. 26. Cache stampeding
  27. 27. Memcache code ? Memcache code Visitor interface Admin interface DB
  28. 28. Cache warmup scriptsUsed to fill your cache when its emptyRun it before starting Webserver !2 ways : Visit all URLs Error-prone Hard to maintain Call all cache-updating methods Make sure you have a warmup script !
  29. 29. Cache stampeding - what about locking ?Seems like a nice idea, but...While lock in placeWhat if the process that created the lock fails ?
  30. 30. Website X Header Latest news Article content page Navigation Article content
  31. 31. Website X Header (TTL = 2h) Latest news Article content page Navigation (TTL = 2h) Article content
  32. 32. Website X Header (TTL = 2h) Latest news (TTL = 2m) Article content page Navigation (TTL = 2h) Article content (TTL = 15m)
  33. 33. Move the cache
  34. 34. VarnishNot just a load balancerReverse proxy cache / http accelerator / …Caches (parts of) pages in memory
  35. 35. Varnish - ESI In your article page output : Header (TTL : 60 min) <esi:include src="/top"/> /top <esi:include src="/nav"/> Latest news (TTL : 2 min) /news <esi:include src="/news"/> <esi:include src="/article/732"/>Navigation Article content page (TTL : In your Varnish config : 60 min) Article content (TTL : 15 min) sub vcl_fetch { /nav /article/732 if (req.url == "/news") { esi; /* Do ESI processing */ set obj.ttl = 2m; } elseif (req.url == "/nav") { esi; set obj.ttl = 1h; } elseif …. …. }
  36. 36. A simple benchmark – 2KByte JPEG Apache 2.2 4210 IIS 7.5 3960 Varnish 3.0 11400
  37. 37. A dynamically generated, but static page Apache 2.2 + PHP (3 DB queries) 18 IIS 7.5 + .Net (3 DB queries) 16 Varnish 3.0 11400
  38. 38. Varnish - what can/cant be cached ?Can : Static pages Images, js, css Static parts of pages that dont change often (ESI)Cant : POST requests Very large files (its not a file server !) Requests with Set-Cookie User-specific content
  39. 39. ESI → no caching on user-specific content ? Logged in as : Wim Godden TTL = 0s ? 5 messages TTL=1h TTL = 5min
  40. 40. Dynamic contentRequest Varnish Ideallyuser X : /top /top request to webserver sameuser Y : /top /top request to webserver sameuser X : /top /top request to webserver if unchanged : served from cacheuser Y : /top /top request to webserver if unchanged : served from cache
  41. 41. NginxWeb serverReverse proxyLightweight, fast12.2% of all Websites
  42. 42. NginxNo threads, event-drivenUses epoll / kqueueLow memory footprint10000 active connections = normal
  43. 43. Coming to Nginx soon... Logged in as : Wim Godden 5 messages Menu NEWS
  44. 44. Coming to Nginx soon... <esim:include src="/top" session="1" ttl="1h" /> <esim:include src="/menu" <esim:include src="/news" ttl="5m" /> ttl="1h" />
  45. 45. Requesting /page (1st time) Nginx 1 3 Shared memory /page 2 4 /page
  46. 46. Requesting /page ESI subrequests (1st time) Nginx 1 2 3 /menu /news /top ($sessionid$)
  47. 47. Requesting /page (next time) Nginx 1 Shared memory /page 2 /page /menu /news /top ($sessionid$)
  48. 48. New message arrives... POST /send o ... se i nt t(. e rt ..) ins DB top ($sessionid$)
  49. 49. AdvantagesNo repeated hits to webserver anymore ! Only the initial hit (unless you pre-heat/warm up the cache !) No hits for user-specific content Not even for non-specific content
  50. 50. News added addnews() method o ... se i nt t(. e rt ..) ins DB /news
  51. 51. AdvantagesNo repeated hits to webserver anymore ! Only the initial hit (unless you pre-heat/warm up the cache !) Not for user-specific content Not even for non-specific content No TTLs for non-specific content Imagine doing it for the bid status on Ebay items ;-)
  52. 52. How many Memcache requests ? Logged in as : Wim Godden <esim:include src="/top" session="1" ttl="1h" /> 5 messages <esim:include src="/menu" <esim:include src="/news" ttl="5m" /> ttl="1h" />
  53. 53. Why Nginx ?Native Memcache supportExcellent and superfast subrequest system Including parallel subrequestsHandles thousands of connections per worker With minimal memory footprint
  54. 54. Whats the result ?
  55. 55. FiguresFirst customer : No. of web servers : 18 → 4 No. of db servers : 6 → 2 Total : 24 → 6 (75% reduction !)Second customer (already using Nginx + Memcache) : No. of web servers : 72 → 8 No. of db servers : 15 → 4 Total : 87 → 12 (86% reduction !)
  56. 56. AvailabilityStable at 2 customersStill under heavy developmentBeta : July 2012Final : Sep 2012
  57. 57. So...
  58. 58. So...Theres life outside the MS-box And its pretty awesome ;-)Need caching in .Net (for DB queries, config-files, etc.) ? Memcache Velocity ?Caching static pages → Varnish with ESICaching for authenticated users → Nginx with Dynamic ESITheres a lot more outside the box → Maybe some other time ;-)
  59. 59. Questions ?
  60. 60. Questions ?
  61. 61. ContactTwitter @wimgtrWeb http://techblog.wimgodden.beSlides http://www.slideshare.net/wimgE-mail wim.godden@cu.be
  62. 62. Thanks ! Please rate my talk :http://speakerrate.com/talks/11081
  1. Gostou de algum slide específico?

    Recortar slides é uma maneira fácil de colecionar informações para acessar mais tarde.

×