Caching and tuning funfor high scalabilityWim GoddenCu.be Solutions
Who am I ?Wim Godden (@wimgtr)Founder of Cu.be Solutions (http://cu.be)Open Source developer since 1997Developer of OpenXZ...
Who are you ?Developers ?System/network engineers ?Managers ?Caching experience ?
Goals of this tutorialEverything about caching and tuningA few techniquesHow-toHow-NOT-to→ Increase reliability, performan...
LAMP
Architecture
Test page3 DB-queriesselect firstname, lastname, email from user where user_id = 5;select title, createddate, body from ar...
Our base benchmarkApachebench = useful enoughResult ?Single webserver ProxyStatic PHP Static PHPApache + PHP 3900 17.5 670...
CachingCaching
What is caching ?CACHECACHE
What is caching ?x = 5, y = 2n = 50Same resultCACHECACHEselect*fromarticlejoin useron article.user_id = user.idorder bycre...
Theory of cachingDBCache$data = get(key)falseGET /pagePageselect data fromtable$data = returned resultset(key, $data)if ($...
Theory of cachingDBCacheHIT
Caching techniques#1 : Store entire pages#2 : Store part of a page (block)#3 : Store data retrieval (SQL ?)#4 : Store comp...
How to find cacheable dataNew projects : start from cache everythingExisting projects :Look at MySQL slow query logMake a ...
Caching storage - DiskData with few updates : goodCaching SQL queries : preferably notDONT use NFS or other network file s...
Caching storage - Disk / ramdiskLocal5 Webservers → 5 local cachesHow will you keep them synchronized ?→ Dont say NFS or r...
Caching storage - Memcache(d)Facebook, Twitter, YouTube, … → need we say more ?Distributed memory caching systemMultiple m...
Caching storage - Memcache(d)Facebook, Twitter, YouTube, … → need we say more ?Distributed memory caching systemKey-value ...
Memcache - where to install
Memcache - where to install
Memcache - installation & running itInstallationDistribution packagePECLWindows : binariesRunningNo config-filesmemcached ...
Caching storage - Memcache - some notesNot fault-tolerantIts a cache !Lose session dataLose shopping cart data...
Caching storage - Memcache - some notesNot fault-tolerantIts a cache !Lose session dataLose shopping cart data…Firewall yo...
Memcache in code<?php$memcache = new Memcache();$memcache->addServer(172.16.0.1, 11211);$memcache->addServer(172.16.0.2, 1...
Benchmark with MemcacheSingle webserver ProxyStatic PHP Static PHPApache + PHP 3900 17.5 6700 17.5Apache + PHP + MC 3900 5...
Memcache slabs(or why Memcache says its full when its not)Multiple slabs of different sizes :Slab 1 : 400 bytesSlab 2 : 48...
Memcache - Is it working ?Connect to it using telnet"stats" command →Use Cacti or other monitoring toolsSTAT pid 2941STAT ...
Memcache - backing up
Memcache - tipPage with multiple blocks ?→ use Memcached::getMulti()But : what if you get some hits and some misses ?getMu...
Updating data
Updating dataLCD_Popular_Product_List
Adding/updating data$memcache->delete(LCD_Popular_Product_List);
Adding/updating data
Adding/updating data - Why it crashed
Adding/updating data - Why it crashed
Adding/updating data - Why it crashed
Cache stampeding
Cache stampeding
Memcache code ?DBVisitor interface Admin interfaceMemcache code
Cache warmup scriptsUsed to fill your cache when its emptyRun it before starting Webserver !2 ways :Visit all URLsError-pr...
Cache stampeding - what about locking ?Seems like a nice idea, but...While lock in placeWhat if the process that created t...
LAMP...→ LAMMP→ LNMMP
NginxWeb serverReverse proxyLightweight, fast12.81% of all Websites
NginxNo threads, event-drivenUses epoll / kqueueLow memory footprint10000 active connections = normal
Nginx - Configurationserver {listen 80;server_name www.domain.ext *.domain.ext;index index.html;root /home/domain.ext/www;...
Nginx with PHP-FPMSince PHP 5.3.3Runs on port 9000Nginx connects using fastcgi methodlocation / {fastcgi_pass 127.0.0.1:90...
Nginx + PHP-FPM featuresGraceful upgradeSpawn new processes under high loadChrootSlow request log !
Nginx + PHP-FPM featuresGraceful upgradeSpawn new processes under high loadChrootSlow request log !fastcgi_finish_request(...
Nginx + PHP-FPM - performance ?Single webserver ProxyStatic PHP Static PHPApache + PHP 3900 17.5 6700 17.5Apache + PHP + M...
Nginx + PHP-FPM - performance ?Single webserver ProxyStatic PHP Static PHPApache + PHP 3900 17.5 6700 17.5Apache + PHP + M...
Reverse proxy time...
VarnishNot just a load balancerReverse proxy cache / http accelerator / …Caches (parts of) pages in memoryCareful :uses th...
Varnish - backends + load balancingbackend server1 {.host = "192.168.0.10";}backend server2 {.host = "192.168.0.11";}direc...
Varnish - VCLVarnish Configuration LanguageDSL (Domain Specific Language)→ compiled to CHooks into each requestDefines :Ba...
Varnish - whatever you wantReal-time statistics (varnishtop, varnishhist, ...)ESI
Article content pageArticle content (TTL : 15 min)/article/732Varnish - ESIPerfect for caching pagesHeader (TTL : 60 min)/...
Varnish with ESI - hold on tight !Single webserver ProxyStatic PHP Static PHPApache + PHP 3900 17.5 6700 17.5Apache + PHP ...
Varnish - what can/cant be cached ?Can :Static pagesImages, js, cssPages or parts of pages that dont change often (ESI)Can...
ESI → no caching on user-specific content ?Logged in as : Wim Godden5 messagesTTL = 5minTTL=1hTTL = 0s ?
Coming soon...Based on NginxReduces load by 50 – 95%Requires code changes !Well-built project → few changesEffect on webse...
Whats the result ?
Whats the result ?
FiguresSecond customer (already using Nginx + Memcache) :No. of web servers : 72 → 8No. of db servers : 15 → 4Total : 87 →...
AvailabilityOld system : stable at 4 customersTotal rebuild : still under heavy developmentBeta : Sep 2013Final : End 2013
PHP speed - some tipsUpgrade PHP - every minor release has 5-15% speed gain !Use an opcode cache (APC, eAccelerator, XCache)
DB speed - some tipsUse same types for joinsi.e. dont join decimal with intRAND() is evil !count(*) is evil in InnoDB with...
Caching & Tuning @ frontendhttp://www.websiteoptimization.com/speed/tweak/average-web-page/
Frontend tuning1. You optimize backend2. Frontend engineers messes up → havoc on backend3. Dont forget : frontend sends re...
Tuning frontendMinimize requestsCombine CSS/JavaScript files
Tuning frontendMinimize requestsCombine CSS/JavaScript filesUse CSS Sprites
CSS Sprites
Tuning content - CSS sprites
Tuning content - CSS sprites11 images11 HTTP requests24KByte1 image1 HTTP requests14KByte
Tuning frontendMinimize requestsCombine CSS/JavaScript filesUse CSS Sprites (horizontally if possible)Put CSS at topPut Ja...
What else can kill your site ?Redirect loopsMultiple requestsMore load on WebserverMore PHP to processAdditional latency f...
Above all else... be prepared !Have a monitoring systemUse a cache abstraction layer (disk → Memcache)Dont install for the...
So...CacheBut : never delete, always push !Have a warmup scriptMonitor your cacheHave an abstraction layerApache = fine, N...
Questions ?
Questions ?
ContactTwitter @wimgtrWeb http://techblog.wimgodden.beSlides http://www.slideshare.net/wimgE-mail wim.godden@cu.bePlease.....
Thanks !Please...Rate my talk : http://joind.in/8228
Caching and tuning fun for high scalability
Upcoming SlideShare
Loading in...5
×

Caching and tuning fun for high scalability

1,855

Published on

Caching has been a 'hot' topic for a few years. But caching takes more than merely taking data and putting it in a cache : the right caching techniques can improve performance and reduce load significantly. But we'll also look at some major pitfalls, showing that caching the wrong way can bring down your site. If you're looking for a clear explanation about various caching techniques and tools like Memcached, Nginx and Varnish, as well as ways to deploy them in an efficient way, this talk is for you.

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
1,855
On Slideshare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
0
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Transcript of "Caching and tuning fun for high scalability"

  1. 1. Caching and tuning funfor high scalabilityWim GoddenCu.be Solutions
  2. 2. Who am I ?Wim Godden (@wimgtr)Founder of Cu.be Solutions (http://cu.be)Open Source developer since 1997Developer of OpenXZend Certified EngineerZend Framework Certified EngineerMySQL Certified Developer
  3. 3. Who are you ?Developers ?System/network engineers ?Managers ?Caching experience ?
  4. 4. Goals of this tutorialEverything about caching and tuningA few techniquesHow-toHow-NOT-to→ Increase reliability, performance and scalability5 visitors/day → 5 million visitors/day(Dont expect miracle cure !)
  5. 5. LAMP
  6. 6. Architecture
  7. 7. Test page3 DB-queriesselect firstname, lastname, email from user where user_id = 5;select title, createddate, body from article order by createddate desc limit 5;select title, createddate, body from article order by score desc limit 5;Page just outputs result
  8. 8. Our base benchmarkApachebench = useful enoughResult ?Single webserver ProxyStatic PHP Static PHPApache + PHP 3900 17.5 6700 17.5Limit :CPU, networkor diskLimit :database
  9. 9. CachingCaching
  10. 10. What is caching ?CACHECACHE
  11. 11. What is caching ?x = 5, y = 2n = 50Same resultCACHECACHEselect*fromarticlejoin useron article.user_id = user.idorder bycreated desclimit10Doesnt changeall the time
  12. 12. Theory of cachingDBCache$data = get(key)falseGET /pagePageselect data fromtable$data = returned resultset(key, $data)if ($data == false)
  13. 13. Theory of cachingDBCacheHIT
  14. 14. Caching techniques#1 : Store entire pages#2 : Store part of a page (block)#3 : Store data retrieval (SQL ?)#4 : Store complex processing result#? : Your call !When you have data, think :Creating time ?Modification frequency ?Retrieval frequency ?
  15. 15. How to find cacheable dataNew projects : start from cache everythingExisting projects :Look at MySQL slow query logMake a complete query log (dont forget to turn it off !)→ Use Percona Toolkit (pt-query-digest)Check page loading times
  16. 16. Caching storage - DiskData with few updates : goodCaching SQL queries : preferably notDONT use NFS or other network file systemshigh latencypossible problem for sessions : locking issues !
  17. 17. Caching storage - Disk / ramdiskLocal5 Webservers → 5 local cachesHow will you keep them synchronized ?→ Dont say NFS or rsync !
  18. 18. Caching storage - Memcache(d)Facebook, Twitter, YouTube, … → need we say more ?Distributed memory caching systemMultiple machines ↔ 1 big memory-based hash-tableKey-value storage systemKeys - max. 250bytesValues - max. 1Mbyte
  19. 19. Caching storage - Memcache(d)Facebook, Twitter, YouTube, … → need we say more ?Distributed memory caching systemKey-value storage systemKeys - max. 250bytesValues - max. 1MbyteExtremely fast... non-blocking, UDP (!)
  20. 20. Memcache - where to install
  21. 21. Memcache - where to install
  22. 22. Memcache - installation & running itInstallationDistribution packagePECLWindows : binariesRunningNo config-filesmemcached -d -m <mem> -l <ip> -p <port>ex. : memcached -d -m 2048 -l 172.16.1.91 -p 11211
  23. 23. Caching storage - Memcache - some notesNot fault-tolerantIts a cache !Lose session dataLose shopping cart data...
  24. 24. Caching storage - Memcache - some notesNot fault-tolerantIts a cache !Lose session dataLose shopping cart data…Firewall your Memcache port !
  25. 25. Memcache in code<?php$memcache = new Memcache();$memcache->addServer(172.16.0.1, 11211);$memcache->addServer(172.16.0.2, 11211);$myData = $memcache->get(myKey);if ($myData === false) {$myData = GetMyDataFromDB();// Put it in Memcache as myKey, without compression, with no expiration$memcache->set(myKey, $myData, false, 0);}echo $myData;
  26. 26. Benchmark with MemcacheSingle webserver ProxyStatic PHP Static PHPApache + PHP 3900 17.5 6700 17.5Apache + PHP + MC 3900 55 6700 108
  27. 27. Memcache slabs(or why Memcache says its full when its not)Multiple slabs of different sizes :Slab 1 : 400 bytesSlab 2 : 480 bytes (400 * 1.2)Slab 3 : 576 bytes (480 * 1.2) (and so on...)Multiplier (1.2 here) can be configuredStore a lot of very large objects→ Large slabs : full→ Rest : free→ Eviction of data !
  28. 28. Memcache - Is it working ?Connect to it using telnet"stats" command →Use Cacti or other monitoring toolsSTAT pid 2941STAT uptime 10878STAT time 1296074240STAT version 1.4.5STAT pointer_size 64STAT rusage_user 20.089945STAT rusage_system 58.499106STAT curr_connections 16STAT total_connections 276950STAT connection_structures 96STAT cmd_get 276931STAT cmd_set 584148STAT cmd_flush 0STAT get_hits 211106STAT get_misses 65825STAT delete_misses 101STAT delete_hits 276829STAT incr_misses 0STAT incr_hits 0STAT decr_misses 0STAT decr_hits 0STAT cas_misses 0STAT cas_hits 0STAT cas_badval 0STAT auth_cmds 0STAT auth_errors 0STAT bytes_read 613193860STAT bytes_written 553991373STAT limit_maxbytes 268435456STAT accepting_conns 1STAT listen_disabled_num 0STAT threads 4STAT conn_yields 0STAT bytes 20418140STAT curr_items 65826STAT total_items 553856STAT evictions 0STAT reclaimed 0
  29. 29. Memcache - backing up
  30. 30. Memcache - tipPage with multiple blocks ?→ use Memcached::getMulti()But : what if you get some hits and some misses ?getMulti($array)Hashingalgorithm
  31. 31. Updating data
  32. 32. Updating dataLCD_Popular_Product_List
  33. 33. Adding/updating data$memcache->delete(LCD_Popular_Product_List);
  34. 34. Adding/updating data
  35. 35. Adding/updating data - Why it crashed
  36. 36. Adding/updating data - Why it crashed
  37. 37. Adding/updating data - Why it crashed
  38. 38. Cache stampeding
  39. 39. Cache stampeding
  40. 40. Memcache code ?DBVisitor interface Admin interfaceMemcache code
  41. 41. Cache warmup scriptsUsed to fill your cache when its emptyRun it before starting Webserver !2 ways :Visit all URLsError-proneHard to maintainCall all cache-updating methodsMake sure you have a warmup script !
  42. 42. Cache stampeding - what about locking ?Seems like a nice idea, but...While lock in placeWhat if the process that created the lock fails ?
  43. 43. LAMP...→ LAMMP→ LNMMP
  44. 44. NginxWeb serverReverse proxyLightweight, fast12.81% of all Websites
  45. 45. NginxNo threads, event-drivenUses epoll / kqueueLow memory footprint10000 active connections = normal
  46. 46. Nginx - Configurationserver {listen 80;server_name www.domain.ext *.domain.ext;index index.html;root /home/domain.ext/www;}server {listen 80;server_name photo.domain.ext;index index.html;root /home/domain.ext/photo;}
  47. 47. Nginx with PHP-FPMSince PHP 5.3.3Runs on port 9000Nginx connects using fastcgi methodlocation / {fastcgi_pass 127.0.0.1:9000;fastcgi_index index.php;include fastcgi_params;fastcgi_param SCRIPT_NAME $fastcgi_script_name;fastcgi_param SCRIPT_FILENAME /home/www.4developers.pl/$fastcgi_script_name;fastcgi_param SERVER_NAME $host;fastcgi_intercept_errors on;}
  48. 48. Nginx + PHP-FPM featuresGraceful upgradeSpawn new processes under high loadChrootSlow request log !
  49. 49. Nginx + PHP-FPM featuresGraceful upgradeSpawn new processes under high loadChrootSlow request log !fastcgi_finish_request() → offline processing
  50. 50. Nginx + PHP-FPM - performance ?Single webserver ProxyStatic PHP Static PHPApache + PHP 3900 17.5 6700 17.5Apache + PHP + MC 3900 55 6700 108Nginx + PHP-FPM + MC 11700 57 11200 112Limit :single-threadedApachebench
  51. 51. Nginx + PHP-FPM - performance ?Single webserver ProxyStatic PHP Static PHPApache + PHP 3900 17.5 6700 17.5Apache + PHP + MC 3900 55 6700 108Apache (tuned) + PHP/MC 10600 55 11400 108Nginx + PHP-FPM + MC 11700 57 11200 112Limit :single-threadedApachebench
  52. 52. Reverse proxy time...
  53. 53. VarnishNot just a load balancerReverse proxy cache / http accelerator / …Caches (parts of) pages in memoryCareful :uses threads (like Apache)Nginx usually scales better (but doesnt have VCL)
  54. 54. Varnish - backends + load balancingbackend server1 {.host = "192.168.0.10";}backend server2 {.host = "192.168.0.11";}director example_director round-robin {{.backend = server1;}{.backend = server2;}}
  55. 55. Varnish - VCLVarnish Configuration LanguageDSL (Domain Specific Language)→ compiled to CHooks into each requestDefines :Backends (web servers)ACLsLoad balancing strategyCan be reloaded while running
  56. 56. Varnish - whatever you wantReal-time statistics (varnishtop, varnishhist, ...)ESI
  57. 57. Article content pageArticle content (TTL : 15 min)/article/732Varnish - ESIPerfect for caching pagesHeader (TTL : 60 min)/topLatest news (TTL : 2 min) /newsNavigation(TTL :60 min)/navIn your article page output :<esi:include src="/top"/><esi:include src="/nav"/><esi:include src="/news"/><esi:include src="/article/732"/>In your Varnish config :sub vcl_fetch {if (req.url == "/news") {esi; /* Do ESI processing */set obj.ttl = 2m;} elseif (req.url == "/nav") {esi;set obj.ttl = 1m;} elseif ….….}
  58. 58. Varnish with ESI - hold on tight !Single webserver ProxyStatic PHP Static PHPApache + PHP 3900 17.5 6700 17.5Apache + PHP + MC 3900 55 6700 108Nginx + PHP-FPM + MC 11700 57 11200 112Varnish - - 11200 4200
  59. 59. Varnish - what can/cant be cached ?Can :Static pagesImages, js, cssPages or parts of pages that dont change often (ESI)Cant :POST requestsVery large files (its not a file server !)Requests with Set-CookieUser-specific content
  60. 60. ESI → no caching on user-specific content ?Logged in as : Wim Godden5 messagesTTL = 5minTTL=1hTTL = 0s ?
  61. 61. Coming soon...Based on NginxReduces load by 50 – 95%Requires code changes !Well-built project → few changesEffect on webservers and database servers
  62. 62. Whats the result ?
  63. 63. Whats the result ?
  64. 64. FiguresSecond customer (already using Nginx + Memcache) :No. of web servers : 72 → 8No. of db servers : 15 → 4Total : 87 → 12 (86% reduction !)Latest customer :Total no. of servers : 1350 → 38072% reduction → €1.5 million / year
  65. 65. AvailabilityOld system : stable at 4 customersTotal rebuild : still under heavy developmentBeta : Sep 2013Final : End 2013
  66. 66. PHP speed - some tipsUpgrade PHP - every minor release has 5-15% speed gain !Use an opcode cache (APC, eAccelerator, XCache)
  67. 67. DB speed - some tipsUse same types for joinsi.e. dont join decimal with intRAND() is evil !count(*) is evil in InnoDB without a where clause !Persistent connect is sort-of evil
  68. 68. Caching & Tuning @ frontendhttp://www.websiteoptimization.com/speed/tweak/average-web-page/
  69. 69. Frontend tuning1. You optimize backend2. Frontend engineers messes up → havoc on backend3. Dont forget : frontend sends requests to backend !SO...Care about frontendTest frontendCheck what requests frontend sends to backend
  70. 70. Tuning frontendMinimize requestsCombine CSS/JavaScript files
  71. 71. Tuning frontendMinimize requestsCombine CSS/JavaScript filesUse CSS Sprites
  72. 72. CSS Sprites
  73. 73. Tuning content - CSS sprites
  74. 74. Tuning content - CSS sprites11 images11 HTTP requests24KByte1 image1 HTTP requests14KByte
  75. 75. Tuning frontendMinimize requestsCombine CSS/JavaScript filesUse CSS Sprites (horizontally if possible)Put CSS at topPut JavaScript at bottomMax. no connectionsEspecially if JavaScript does Ajax (advertising-scripts, …) !Avoid iFramesAgain : max no. of connectionsDont scale images in HTMLHave a favicon.ico (dont 404 it !)→ see my blog
  76. 76. What else can kill your site ?Redirect loopsMultiple requestsMore load on WebserverMore PHP to processAdditional latency for visitorTry to avoid redirects anyway→ In ZF : use $this->_forward instead of $this->_redirectWatch your logs, but equally important...Watch the logging process →Logging = disk I/O → can kill your server !
  77. 77. Above all else... be prepared !Have a monitoring systemUse a cache abstraction layer (disk → Memcache)Dont install for the worst → prepare for the worstHave a test-setupHave fallbacks→ Turn off non-critical functionality
  78. 78. So...CacheBut : never delete, always push !Have a warmup scriptMonitor your cacheHave an abstraction layerApache = fine, Nginx = betterStatic pages ? Use VarnishTune your frontend → impact on backend !
  79. 79. Questions ?
  80. 80. Questions ?
  81. 81. ContactTwitter @wimgtrWeb http://techblog.wimgodden.beSlides http://www.slideshare.net/wimgE-mail wim.godden@cu.bePlease...Rate my talk : http://joind.in/8228
  82. 82. Thanks !Please...Rate my talk : http://joind.in/8228

×