Caching and tuning fun  for high scalability                           Wim Godden                         Cu.be Solutions
Who am I ? Wim Godden (@wimgtr) Founder of Cu.be Solutions (http://cu.be) Open Source developer since 1997 Developer of Op...
Who are you ? Developers ? System/network engineers ? Managers ? Caching experience ?
Goals of this tutorial Everything about caching and tuning A few techniques    How-to    How-NOT-to → Increase reliability...
LAMP
Architecture
Test page 3 DB-queries    select firstname, lastname, email from user where user_id = 5;    select title, createddate, bod...
Our base benchmark  Apachebench = useful enough  Result ?                      Single webserver                    Proxy  ...
Caching
What is caching ?                    CACHE                     CACHE
What is caching ?                                             x = 5, y = 2                                                ...
Theory of caching                                                                          DB                             ...
Theory of caching                            DB                    HIT                          Cache
Caching techniques                 #1 : Store entire pages            #2 : Store part of a page (block)            #3 : St...
How to find cacheable data New projects : start from cache everything Existing projects :    Look at MySQL slow query log ...
Caching storage - Disk Data with few updates : good Caching SQL queries : preferably not DONT use NFS or other network fil...
Caching storage - Disk / ramdisk Local    5 Webservers → 5 local caches    How will you keep them synchronized ?         →...
Caching storage - Memcache(d) Facebook, Twitter, YouTube, … → need we say more ? Distributed memory caching system Multipl...
Caching storage - Memcache(d) Facebook, Twitter, YouTube, … → need we say more ? Distributed memory caching system Key-val...
Memcache - where to install
Memcache - where to install
Memcache - installation & running it Installation    Distribution package    PECL    Windows : binaries Running    No conf...
Caching storage - Memcache - some notes Not fault-tolerant    Its a cache !    Lose session data    Lose shopping cart dat...
Caching storage - Memcache - some notes Not fault-tolerant    Its a cache !    Lose session data    Lose shopping cart dat...
Memcache in code<?php$memcache = new Memcache();$memcache->addServer(172.16.0.1, 11211);$memcache->addServer(172.16.0.2, 1...
Benchmark with Memcache                      Single webserver             Proxy                    Static         PHP    S...
Memcache slabs           (or why Memcache says its full when its not) Multiple slabs of different sizes :    Slab 1 : 400 ...
Memcache - Is it working ? Connect to it using telnet               STAT pid 2941                                         ...
Memcache - backing up
Memcache - tip Page with multiple blocks ? → use Memcached::getMulti()                    Hashing getMulti($array)        ...
Updating data
Updating data                LCD_Popular_Product_List
Adding/updating data$memcache->delete(LCD_Popular_Product_List);
Adding/updating data
Adding/updating data - Why it crashed
Adding/updating data - Why it crashed
Adding/updating data - Why it crashed
Cache stampeding
Cache stampeding
Memcache code ?    Memcache code     Visitor interface        Admin interface                         DB
Cache warmup scripts Used to fill your cache when its empty Run it before starting Webserver ! 2 ways :    Visit all URLs ...
Cache stampeding - what about locking ? Seems like a nice idea, but... While lock in place What if the process that create...
LAMP...          → LAMMP          → LNMMP
Nginx Web server Reverse proxy Lightweight, fast 12.81% of all Websites
Nginx No threads, event-driven Uses epoll / kqueue Low memory footprint 10000 active connections = normal
Nginx - Configuration        server {         listen        80;         server_name   www.domain.ext *.domain.ext;        ...
Nginx with PHP-FPM  Since PHP 5.3.3  Runs on port 9000  Nginx connects using fastcgi methodlocation / {      fastcgi_pass ...
Nginx + PHP-FPM features Graceful upgrade Spawn new processes under high load Chroot Slow request log !
Nginx + PHP-FPM features Graceful upgrade Spawn new processes under high load Chroot Slow request log ! fastcgi_finish_req...
Nginx + PHP-FPM - performance ?                         Single webserver                    Proxy                       St...
Nginx + PHP-FPM - performance ?                            Single webserver                    Proxy                      ...
Reverse proxy time...
Varnish Not just a load balancer Reverse proxy cache / http accelerator / … Caches (parts of) pages in memory Careful :   ...
Varnish - backends + load balancing backend server1 {     .host = "192.168.0.10"; } backend server2 {     .host = "192.168...
Varnish - VCL Varnish Configuration Language DSL (Domain Specific Language)    → compiled to C Hooks into each request Def...
Varnish - whatever you want Real-time statistics (varnishtop, varnishhist, ...) ESI
Varnish - ESI  Perfect for caching pages             Header (TTL : 60 min)              In your article page output :     ...
Varnish with ESI - hold on tight !                         Single webserver             Proxy                       Static...
Varnish - what can/cant be cached ? Can :    Static pages    Images, js, css    Pages or parts of pages that dont change o...
ESI → no caching on user-specific content ?                                           Logged in as : Wim Godden           ...
Coming soon... Based on Nginx Reduces load by 50 – 95%   Requires code changes !   Well-built project → few changes   Effe...
Whats the result ?
Whats the result ?
Figures Second customer (already using Nginx + Memcache) :    No. of web servers : 72 → 8    No. of db servers : 15 → 4   ...
Availability Old system : stable at 4 customers Total rebuild : still under heavy development Beta : Sep 2013 Final : End ...
PHP speed - some tips Upgrade PHP - every minor release has 5-15% speed gain ! Use an opcode cache (APC, eAccelerator, XCa...
DB speed - some tips Use same types for joins    i.e. dont join decimal with int RAND() is evil ! count(*) is evil in Inno...
Caching & Tuning @ frontend        http://www.websiteoptimization.com/speed/tweak/average-web-page/
Frontend tuning 1. You optimize backend 2. Frontend engineers messes up → havoc on backend 3. Dont forget : frontend sends...
Tuning frontend Minimize requests   Combine CSS/JavaScript files
Tuning frontend Minimize requests   Combine CSS/JavaScript files   Use CSS Sprites
CSS Sprites
Tuning content - CSS sprites
Tuning content - CSS sprites         11 images             1 image         11 HTTP requests      1 HTTP requests         2...
Tuning frontend Minimize requests    Combine CSS/JavaScript files    Use CSS Sprites (horizontally if possible) Put CSS at...
What else can kill your site ? Redirect loops    Multiple requests        More load on Webserver        More PHP to proces...
Above all else... be prepared ! Have a monitoring system Use a cache abstraction layer (disk → Memcache) Dont install for ...
So... Cache    But : never delete, always push !    Have a warmup script    Monitor your cache    Have an abstraction laye...
Questions ?
Questions ?
Contact Twitter          @wimgtr Web              http://techblog.wimgodden.be Slides           http://www.slideshare.net/...
Thanks !               Please...Rate my talk : http://spkr8.com/t/21151
Caching and tuning fun for high scalability
Upcoming SlideShare
Loading in...5
×

Caching and tuning fun for high scalability

3,271

Published on

Caching has been a ‘hot’ topic for a few years. But caching takes more than merely taking data and putting it in a cache: the right caching techniques can improve performance and reduce load significantly. But we’ll also look at some major pitfalls, showing that caching the wrong way can bring down your site. If you’re looking for a clear explanation about various caching techniques and tools like Memcached, Nginx and Varnish, as well as ways to deploy them in an efficient way, this talk is for you.

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
3,271
On Slideshare
0
From Embeds
0
Number of Embeds
7
Actions
Shares
0
Downloads
18
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Caching and tuning fun for high scalability

  1. 1. Caching and tuning fun for high scalability Wim Godden Cu.be Solutions
  2. 2. Who am I ? Wim Godden (@wimgtr) Founder of Cu.be Solutions (http://cu.be) Open Source developer since 1997 Developer of OpenX, PHPCompatibility, ... Speaker at PHP and Open Source conferences
  3. 3. Who are you ? Developers ? System/network engineers ? Managers ? Caching experience ?
  4. 4. Goals of this tutorial Everything about caching and tuning A few techniques How-to How-NOT-to → Increase reliability, performance and scalability 5 visitors/day → 5 million visitors/day (Dont expect miracle cure !)
  5. 5. LAMP
  6. 6. Architecture
  7. 7. Test page 3 DB-queries select firstname, lastname, email from user where user_id = 5; select title, createddate, body from article order by createddate desc limit 5; select title, createddate, body from article order by score desc limit 5; Page just outputs result
  8. 8. Our base benchmark Apachebench = useful enough Result ? Single webserver Proxy Static PHP Static PHPApache + PHP 3900 17.5 6700 17.5 Limit : Limit : CPU, network database or disk
  9. 9. Caching
  10. 10. What is caching ? CACHE CACHE
  11. 11. What is caching ? x = 5, y = 2 Same result n = 50 CACHE CACHE select * from article join user Doesnt change on article.user_id = user.id all the time order by created desc limit 10
  12. 12. Theory of caching DB le tab fro m ult ta d re s da turne e ct re sel ta = $da if ($data == false) Page GET /page set( key , $da fal $data) ta = se get( key ) Cache
  13. 13. Theory of caching DB HIT Cache
  14. 14. Caching techniques #1 : Store entire pages #2 : Store part of a page (block) #3 : Store data retrieval (SQL ?) #4 : Store complex processing result #? : Your call ! When you have data, think : Creating time ? Modification frequency ? Retrieval frequency ?
  15. 15. How to find cacheable data New projects : start from cache everything Existing projects : Look at MySQL slow query log Make a complete query log (dont forget to turn it off !) → Use Percona Toolkit (pt-query-digest) Check page loading times
  16. 16. Caching storage - Disk Data with few updates : good Caching SQL queries : preferably not DONT use NFS or other network file systems high latency possible problem for sessions : locking issues !
  17. 17. Caching storage - Disk / ramdisk Local 5 Webservers → 5 local caches How will you keep them synchronized ? → Dont say NFS or rsync !
  18. 18. Caching storage - Memcache(d) Facebook, Twitter, YouTube, … → need we say more ? Distributed memory caching system Multiple machines ↔ 1 big memory-based hash-table Key-value storage system Keys - max. 250bytes Values - max. 1Mbyte
  19. 19. Caching storage - Memcache(d) Facebook, Twitter, YouTube, … → need we say more ? Distributed memory caching system Key-value storage system Keys - max. 250bytes Values - max. 1Mbyte Extremely fast... non-blocking, UDP (!)
  20. 20. Memcache - where to install
  21. 21. Memcache - where to install
  22. 22. Memcache - installation & running it Installation Distribution package PECL Windows : binaries Running No config-files memcached -d -m <mem> -l <ip> -p <port> ex. : memcached -d -m 2048 -l 172.16.1.91 -p 11211
  23. 23. Caching storage - Memcache - some notes Not fault-tolerant Its a cache ! Lose session data Lose shopping cart data ...
  24. 24. Caching storage - Memcache - some notes Not fault-tolerant Its a cache ! Lose session data Lose shopping cart data … Firewall your Memcache port !
  25. 25. Memcache in code<?php$memcache = new Memcache();$memcache->addServer(172.16.0.1, 11211);$memcache->addServer(172.16.0.2, 11211);$myData = $memcache->get(myKey);if ($myData === false) { $myData = GetMyDataFromDB(); // Put it in Memcache as myKey, without compression, with no expiration $memcache->set(myKey, $myData, false, 0);}echo $myData;
  26. 26. Benchmark with Memcache Single webserver Proxy Static PHP Static PHPApache + PHP 3900 17.5 6700 17.5Apache + PHP + MC 3900 55 6700 108
  27. 27. Memcache slabs (or why Memcache says its full when its not) Multiple slabs of different sizes : Slab 1 : 400 bytes Slab 2 : 480 bytes (400 * 1.2) Slab 3 : 576 bytes (480 * 1.2) (and so on...) Multiplier (1.2 here) can be configured Store a lot of very large objects → Large slabs : full → Rest : free → Eviction of data !
  28. 28. Memcache - Is it working ? Connect to it using telnet STAT pid 2941 STAT uptime 10878 STAT time 1296074240 "stats" command → STAT version 1.4.5 STAT pointer_size 64 STAT rusage_user 20.089945 Use Cacti or other monitoring tools STAT rusage_system 58.499106 STAT curr_connections 16 STAT total_connections 276950 STAT connection_structures 96 STAT cmd_get 276931 STAT cmd_set 584148 STAT cmd_flush 0 STAT get_hits 211106 STAT get_misses 65825 STAT delete_misses 101 STAT delete_hits 276829 STAT incr_misses 0 STAT incr_hits 0 STAT decr_misses 0 STAT decr_hits 0 STAT cas_misses 0 STAT cas_hits 0 STAT cas_badval 0 STAT auth_cmds 0 STAT auth_errors 0 STAT bytes_read 613193860 STAT bytes_written 553991373 STAT limit_maxbytes 268435456 STAT accepting_conns 1 STAT listen_disabled_num 0 STAT threads 4 STAT conn_yields 0 STAT bytes 20418140 STAT curr_items 65826 STAT total_items 553856 STAT evictions 0 STAT reclaimed 0
  29. 29. Memcache - backing up
  30. 30. Memcache - tip Page with multiple blocks ? → use Memcached::getMulti() Hashing getMulti($array) algorithm But : what if you get some hits and some misses ?
  31. 31. Updating data
  32. 32. Updating data LCD_Popular_Product_List
  33. 33. Adding/updating data$memcache->delete(LCD_Popular_Product_List);
  34. 34. Adding/updating data
  35. 35. Adding/updating data - Why it crashed
  36. 36. Adding/updating data - Why it crashed
  37. 37. Adding/updating data - Why it crashed
  38. 38. Cache stampeding
  39. 39. Cache stampeding
  40. 40. Memcache code ? Memcache code Visitor interface Admin interface DB
  41. 41. Cache warmup scripts Used to fill your cache when its empty Run it before starting Webserver ! 2 ways : Visit all URLs Error-prone Hard to maintain Call all cache-updating methods Make sure you have a warmup script !
  42. 42. Cache stampeding - what about locking ? Seems like a nice idea, but... While lock in place What if the process that created the lock fails ?
  43. 43. LAMP... → LAMMP → LNMMP
  44. 44. Nginx Web server Reverse proxy Lightweight, fast 12.81% of all Websites
  45. 45. Nginx No threads, event-driven Uses epoll / kqueue Low memory footprint 10000 active connections = normal
  46. 46. Nginx - Configuration server { listen 80; server_name www.domain.ext *.domain.ext; index index.html; root /home/domain.ext/www; } server { listen 80; server_name photo.domain.ext; index index.html; root /home/domain.ext/photo; }
  47. 47. Nginx with PHP-FPM Since PHP 5.3.3 Runs on port 9000 Nginx connects using fastcgi methodlocation / { fastcgi_pass 127.0.0.1:9000; fastcgi_index index.php; include fastcgi_params; fastcgi_param SCRIPT_NAME $fastcgi_script_name; fastcgi_param SCRIPT_FILENAME /home/www.4developers.pl/$fastcgi_script_name; fastcgi_param SERVER_NAME $host; fastcgi_intercept_errors on;}
  48. 48. Nginx + PHP-FPM features Graceful upgrade Spawn new processes under high load Chroot Slow request log !
  49. 49. Nginx + PHP-FPM features Graceful upgrade Spawn new processes under high load Chroot Slow request log ! fastcgi_finish_request() → offline processing
  50. 50. Nginx + PHP-FPM - performance ? Single webserver Proxy Static PHP Static PHPApache + PHP 3900 17.5 6700 17.5Apache + PHP + MC 3900 55 6700 108Nginx + PHP-FPM + MC 11700 57 11200 112 Limit : single-threaded Apachebench
  51. 51. Nginx + PHP-FPM - performance ? Single webserver Proxy Static PHP Static PHPApache + PHP 3900 17.5 6700 17.5Apache + PHP + MC 3900 55 6700 108Apache (tuned) + PHP/MC 10600 55 11400 108Nginx + PHP-FPM + MC 11700 57 11200 112 Limit : single-threaded Apachebench
  52. 52. Reverse proxy time...
  53. 53. Varnish Not just a load balancer Reverse proxy cache / http accelerator / … Caches (parts of) pages in memory Careful : uses threads (like Apache) Nginx usually scales better (but doesnt have VCL)
  54. 54. Varnish - backends + load balancing backend server1 { .host = "192.168.0.10"; } backend server2 { .host = "192.168.0.11"; } director example_director round-robin { { .backend = server1; } { .backend = server2; } }
  55. 55. Varnish - VCL Varnish Configuration Language DSL (Domain Specific Language) → compiled to C Hooks into each request Defines : Backends (web servers) ACLs Load balancing strategy Can be reloaded while running
  56. 56. Varnish - whatever you want Real-time statistics (varnishtop, varnishhist, ...) ESI
  57. 57. Varnish - ESI Perfect for caching pages Header (TTL : 60 min) In your article page output : /top <esi:include src="/top"/> <esi:include src="/nav"/> Latest news (TTL : 2 min) /news <esi:include src="/news"/> <esi:include src="/article/732"/>Navigation Article content page In your Varnish config : (TTL : sub vcl_fetch { 60 min) Article content (TTL : 15 min) if (req.url == "/news") { /nav /article/732 esi; /* Do ESI processing */ set obj.ttl = 2m; } elseif (req.url == "/nav") { esi; set obj.ttl = 1m; } elseif …. …. }
  58. 58. Varnish with ESI - hold on tight ! Single webserver Proxy Static PHP Static PHPApache + PHP 3900 17.5 6700 17.5Apache + PHP + MC 3900 55 6700 108Nginx + PHP-FPM + MC 11700 57 11200 112Varnish - - 11200 4200
  59. 59. Varnish - what can/cant be cached ? Can : Static pages Images, js, css Pages or parts of pages that dont change often (ESI) Cant : POST requests Very large files (its not a file server !) Requests with Set-Cookie User-specific content
  60. 60. ESI → no caching on user-specific content ? Logged in as : Wim Godden TTL = 0s ? 5 messages TTL=1h TTL = 5min
  61. 61. Coming soon... Based on Nginx Reduces load by 50 – 95% Requires code changes ! Well-built project → few changes Effect on webservers and database servers
  62. 62. Whats the result ?
  63. 63. Whats the result ?
  64. 64. Figures Second customer (already using Nginx + Memcache) : No. of web servers : 72 → 8 No. of db servers : 15 → 4 Total : 87 → 12 (86% reduction !) Latest customer : Total no. of servers : 1350 → 380 72% reduction → €1.5 million / year vBulletin test project : Load dropped by 98% on webservers and db-servers !
  65. 65. Availability Old system : stable at 4 customers Total rebuild : still under heavy development Beta : Sep 2013 Final : End 2013
  66. 66. PHP speed - some tips Upgrade PHP - every minor release has 5-15% speed gain ! Use an opcode cache (APC, eAccelerator, XCache)
  67. 67. DB speed - some tips Use same types for joins i.e. dont join decimal with int RAND() is evil ! count(*) is evil in InnoDB without a where clause ! Persistent connect is sort-of evil
  68. 68. Caching & Tuning @ frontend http://www.websiteoptimization.com/speed/tweak/average-web-page/
  69. 69. Frontend tuning 1. You optimize backend 2. Frontend engineers messes up → havoc on backend 3. Dont forget : frontend sends requests to backend ! SO... Care about frontend Test frontend Check what requests frontend sends to backend
  70. 70. Tuning frontend Minimize requests Combine CSS/JavaScript files
  71. 71. Tuning frontend Minimize requests Combine CSS/JavaScript files Use CSS Sprites
  72. 72. CSS Sprites
  73. 73. Tuning content - CSS sprites
  74. 74. Tuning content - CSS sprites 11 images 1 image 11 HTTP requests 1 HTTP requests 24KByte 14KByte
  75. 75. Tuning frontend Minimize requests Combine CSS/JavaScript files Use CSS Sprites (horizontally if possible) Put CSS at top Put JavaScript at bottom Max. no connections Especially if JavaScript does Ajax (advertising-scripts, …) ! Avoid iFrames Again : max no. of connections Dont scale images in HTML Have a favicon.ico (dont 404 it !) → see my blog
  76. 76. What else can kill your site ? Redirect loops Multiple requests More load on Webserver More PHP to process Additional latency for visitor Try to avoid redirects anyway → In ZF : use $this->_forward instead of $this->_redirect Watch your logs, but equally important... Watch the logging process → Logging = disk I/O → can kill your server !
  77. 77. Above all else... be prepared ! Have a monitoring system Use a cache abstraction layer (disk → Memcache) Dont install for the worst → prepare for the worst Have a test-setup Have fallbacks → Turn off non-critical functionality
  78. 78. So... Cache But : never delete, always push ! Have a warmup script Monitor your cache Have an abstraction layer Apache = fine, Nginx = better Static pages ? Use Varnish Tune your frontend → impact on backend !
  79. 79. Questions ?
  80. 80. Questions ?
  81. 81. Contact Twitter @wimgtr Web http://techblog.wimgodden.be Slides http://www.slideshare.net/wimg E-mail wim.godden@cu.be Please... Rate my talk : http://spkr8.com/t/21151
  82. 82. Thanks ! Please...Rate my talk : http://spkr8.com/t/21151
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×