1.
Caching and tuning fun for high scalability Wim Godden Cu.be Solutions

2.
Who am I ?Wim Godden (@wimgtr)Owner of Cu.be Solutions (http://cu.be)Open Source developer since 1997Developer of OpenXZend Certified EngineerZend Framework Certified EngineerMySQL Certified Developer

3.
Who are you ?Developers ?System/network engineers ?Managers ?Caching experience ?

4.
Goals of this tutorialEverything about caching and tuningA few techniques How-to How-NOT-to→ Increase reliability, performance and scalability 5 visitors/day → 5 million visitors/day (Dont expect miracle cure !)

5.
LAMP

6.
Architecture

7.
Test page3 DB-queries select firstname, lastname, email from user where user_id = 5; select title, createddate, body from article order by createddate desc limit 5; select title, createddate, body from article order by score desc limit 5;Page just outputs result

8.
Our base benchmarkApachebench = useful enoughResult ? Single webserver Proxy Static PHP Static PHPApache + PHP 3900 17.5 6700 17.5 Limit : Limit : CPU, network database or disk

9.
Caching

10.
What is caching ? CACHE

11.
What is caching ? x = 5, y = 2 Same result n = 50 CACHE select * from article join user Doesnt change on article.user_id = user.id all the time order by created desc limit 10

12.
Theory of caching DB le t ab fro m ul t res ata ned e ct d retur sel ta = $da if ($data == false) Page GET /page set( key , $ $da fals data) ta = e get( key ) Cache

13.
Theory of caching DB HIT Cache

14.
Caching techniques #1 : Store entire pages #2 : Store part of a page (block) #3 : Store data retrieval (SQL ?) #4 : Store complex processing result #? : Your call ! When you have data, think : Creating time ? Modification frequency ? Retrieval frequency ?

15.
How to find cacheable dataNew projects : start from cache everythingExisting projects : Look at MySQL slow query log Make a complete query log (dont forget to turn it off !) Check page loading times

16.
Caching storage - DiskData with few updates : goodCaching SQL queries : preferably notDONT use NFS or other network file systems especially for sessions locking issues ! high latency

17.
Caching storage - Disk / ramdiskLocal 5 Webservers → 5 local caches How will you keep them synchronized ? → Dont say NFS or rsync !

18.
Caching storage - Memcache(d)Facebook, Twitter, YouTube, … → need we say more ?Distributed memory caching systemMultiple machines ↔ 1 big memory-based hash-tableKey-value storage system Keys - max. 250bytes Values - max. 1Mbyte

19.
Caching storage - Memcache(d)Facebook, Twitter, YouTube, … → need we say more ?Distributed memory caching systemKey-value storage system Keys - max. 250bytes Values - max. 1MbyteExtremely fast... non-blocking, UDP (!)

20.
Memcache - where to install

21.
Memcache - where to install

22.
Memcache - installation & running itInstallation Distribution package PECL Windows : binariesRunning No config-files memcached -d -m <mem> -l <ip> -p <port> ex. : memcached -d -m 2048 -l 172.16.1.91 -p 11211

23.
Caching storage - Memcache - some notesNot fault-tolerant Its a cache ! Lose session data Lose shopping cart data ...

24.
Caching storage - Memcache - some notesNot fault-tolerant Its a cache ! Lose session data Lose shopping cart data …Firewall your Memcache port !

25.
Memcache in code<?php$memcache = new Memcache();$memcache->addServer(172.16.0.1, 11211);$memcache->addServer(172.16.0.2, 11211);$myData = $memcache->get(myKey);if ($myData === false) { $myData = GetMyDataFromDB(); // Put it in Memcache as myKey, without compression, with no expiration $memcache->set(myKey, $myData, false, 0);}echo $myData;

26.
Benchmark with Memcache Single webserver Proxy Static PHP Static PHPApache + PHP 3900 17.5 6700 17.5Apache + PHP + MC 3900 55 6700 108

27.
Memcache slabs (or why Memcache says its full when its not)Multiple slabs of different sizes : Slab 1 : 400 bytes Slab 2 : 480 bytes (400 * 1.2) Slab 3 : 576 bytes (480 * 1.2) (and so on...)Multiplier (1.2 here) can be configuredStore a lot of very large objects→ Large slabs : full→ Rest : free→ Eviction of data !

28.
Memcache - Is it working ?Connect to it using telnet STAT pid 2941 STAT uptime 10878 STAT time 1296074240 "stats" command → STAT version 1.4.5 STAT pointer_size 64 STAT rusage_user 20.089945 Use Cacti or other monitoring tools STAT rusage_system 58.499106 STAT curr_connections 16 STAT total_connections 276950 STAT connection_structures 96 STAT cmd_get 276931 STAT cmd_set 584148 STAT cmd_flush 0 STAT get_hits 211106 STAT get_misses 65825 STAT delete_misses 101 STAT delete_hits 276829 STAT incr_misses 0 STAT incr_hits 0 STAT decr_misses 0 STAT decr_hits 0 STAT cas_misses 0 STAT cas_hits 0 STAT cas_badval 0 STAT auth_cmds 0 STAT auth_errors 0 STAT bytes_read 613193860 STAT bytes_written 553991373 STAT limit_maxbytes 268435456 STAT accepting_conns 1 STAT listen_disabled_num 0 STAT threads 4 STAT conn_yields 0 STAT bytes 20418140 STAT curr_items 65826 STAT total_items 553856 STAT evictions 0 STAT reclaimed 0

29.
Memcache - backing up

30.
Memcache - tipPage with multiple blocks ?→ use Memcached::getMulti() Hashing getMulti($array) algorithmBut : what if you get some hits and some misses ?

31.
Updating data

32.
Updating data LCD_Popular_Product_List

33.
Adding/updating data$memcache->delete(LCD_Popular_Product_List);

34.
Adding/updating data

35.
Adding/updating data - Why it crashed

36.
Adding/updating data - Why it crashed

37.
Adding/updating data - Why it crashed

38.
Cache stampeding

39.
Cache stampeding

40.
Memcache code ? Memcache code Visitor interface Admin interface DB

41.
Cache warmup scriptsUsed to fill your cache when its emptyRun it before starting Webserver !2 ways : Visit all URLs Error-prone Hard to maintain Call all cache-updating methods Make sure you have a warmup script !

42.
Cache stampeding - what about locking ?Seems like a nice idea, but...While lock in placeWhat if the process that created the lock fails ?

43.
LAMP... → LAMMP → LNMMP

44.
NginxWeb serverReverse proxyLightweight, fast12.2% of all Websites

45.
NginxNo threads, event-drivenUses epoll / kqueueLow memory footprint10000 active connections = normal

46.
Nginx - Configuration server { listen 80; server_name www.domain.ext *.domain.ext; index index.html; root /home/domain.ext/www; } server { listen 80; server_name photo.domain.ext; index index.html; root /home/domain.ext/photo; }

47.
Nginx with PHP-FPM Since PHP 5.3.3 Runs on port 9000 Nginx connects using fastcgi method location / { fastcgi_pass 127.0.0.1:9000; fastcgi_index index.php; include fastcgi_params; fastcgi_param SCRIPT_NAME $fastcgi_script_name; fastcgi_param SCRIPT_FILENAME /home/www.4developers.pl/$fastcgi_script_name; fastcgi_param SERVER_NAME $host; fastcgi_intercept_errors on; }

48.
Nginx + PHP-FPM featuresGraceful upgradeSpawn new processes under high loadChrootSlow request log !

49.
Nginx + PHP-FPM featuresGraceful upgradeSpawn new processes under high loadChrootSlow request log !fastcgi_finish_request() → offline processing

50.
Nginx + PHP-FPM - performance ? Single webserver Proxy Static PHP Static PHPApache + PHP 3900 17.5 6700 17.5Apache + PHP + MC 3900 55 6700 108Nginx + PHP-FPM + MC 11700 57 11200 112 Limit : single-threaded Apachebench

51.
Reverse proxy time...

52.
VarnishNot just a load balancerReverse proxy cache / http accelerator / …Caches (parts of) pages in memoryCareful : uses threads (like Apache) Nginx usually scales better (but doesnt have VCL)

53.
Varnish - backends + load balancing backend server1 { .host = "192.168.0.10"; } backend server2 { .host = "192.168.0.11"; } director example_director round-robin { { .backend = server1; } { .backend = server2; } }

54.
Varnish - VCLVarnish Configuration LanguageDSL (Domain Specific Language) → compiled to CHooks into each requestDefines : Backends (web servers) ACLs Load balancing strategyCan be reloaded while running

55.
Varnish - whatever you wantReal-time statistics (varnishtop, varnishhist, ...)ESI

56.
Varnish - ESIPerfect for caching pages Header (TTL : 60 min) In your article page output : /top <esi:include src="/top"/> <esi:include src="/nav"/> Latest news (TTL : 2 min) /news <esi:include src="/news"/> <esi:include src="/article/732"/>Navigation Article content page In your Varnish config : (TTL : sub vcl_fetch { 60 min) Article content (TTL : 15 min) if (req.url == "/news") { /nav /article/732 esi; /* Do ESI processing */ set obj.ttl = 2m; } elseif (req.url == "/nav") { esi; set obj.ttl = 1m; } elseif …. …. }

57.
Varnish with ESI - hold on tight ! Single webserver Proxy Static PHP Static PHPApache + PHP 3900 17.5 6700 17.5Apache + PHP + MC 3900 55 6700 108Nginx + PHP-FPM + MC 11700 57 11200 112Varnish - - 11200 4200

58.
Varnish - what can/cant be cached ?Can : Static pages Images, js, css Pages or parts of pages that dont change often (ESI)Cant : POST requests Very large files (its not a file server !) Requests with Set-Cookie User-specific content

59.
ESI → no caching on user-specific content ? Logged in as : Wim Godden TTL = 0s ? 5 messages TTL=1h TTL = 5min

60.
Coming soon...Based on NginxReduces load by 50 – 95% Requires code changes ! Well-built project → few changes Effect on webservers and database servers

61.
Whats the result ?

62.
Whats the result ?

63.
FiguresFirst customer : No. of web servers : 18 → 4 No. of db servers : 6 → 2 Total : 24 → 6 (75% reduction !)Second customer (already using Nginx + Memcache) : No. of web servers : 72 → 8 No. of db servers : 15 → 4 Total : 87 → 12 (86% reduction !)

64.
AvailabilityStable at 2 customersStill under heavy developmentBeta : July 2012Final : Sep 2012

65.
PHP speed - some tipsUpgrade PHP - every minor release has 5-15% speed gain !Use an opcode cache (APC, eAccelerator, XCache)

66.
DB speed - some tipsUse same types for joins i.e. dont join decimal with intRAND() is evil !count(*) is evil in InnoDB without a where clause !Persistent connect is sort-of evil

67.
Caching & Tuning @ frontend http://www.websiteoptimization.com/speed/tweak/average-web-page/

68.
Frontend tuning1. You optimize backend2. Frontend engineers messes up → havoc on backend3. Dont forget : frontend sends requests to backend ! SO...Care about frontendTest frontendCheck what requests frontend sends to backend

69.
Tuning frontendMinimize requests Combine CSS/JavaScript files

70.
Tuning frontendMinimize requests Combine CSS/JavaScript files Use CSS Sprites

71.
CSS Sprites

72.
Tuning content - CSS sprites

73.
Tuning content - CSS sprites 11 images 1 image 11 HTTP requests 1 HTTP requests 24KByte 14KByte

74.
Tuning frontendMinimize requests Combine CSS/JavaScript files Use CSS Sprites (horizontally if possible)Put CSS at topPut JavaScript at bottom Max. no connections Especially if JavaScript does Ajax (advertising-scripts, …) !Avoid iFrames Again : max no. of connectionsDont scale images in HTMLHave a favicon.ico (dont 404 it !) → see my blog

75.
What else can kill your site ?Redirect loops Multiple requests More load on Webserver More PHP to process Additional latency for visitor Try to avoid redirects anyway → In ZF : use $this->_forward instead of $this->_redirectWatch your logs, but equally important...Watch the logging process →Logging = disk I/O → can kill your server !

76.
Above all else... be prepared !Have a monitoring systemUse a cache abstraction layer (disk → Memcache)Dont install for the worst → prepare for the worstHave a test-setupHave fallbacks → Turn off non-critical functionality

77.
So...Cache But : never delete, always push ! Have a warmup script Monitor your cache Have an abstraction layerApache = fine, Nginx = betterStatic pages ? Use VarnishTune your frontend → impact on backend !

78.
Questions ?

79.
Questions ?

80.
Were hiring !Lots of challengesWork with cutting-edge technologyVaried : PHP development System / network architecture Scalability services Build our own services Work on Open Source → mail us : info@cu.be

81.
ContactTwitter @wimgtrWeb http://techblog.wimgodden.beSlides http://www.slideshare.net/wimgE-mail wim.godden@cu.be Please... Rate my talk : http://joind.in/6327

82.
Thanks ! Please...Rate my talk : http://joind.in/6327