1. Caching and tuning fun for high scalability Wim Godden Cu.be Solutions

3. Owner of Cu.be Solutions (http://cu.be)

4. Open Source developer since 1997

5. Developer of OpenX

6. Zend Certified Engineer

7. Zend Framework Certified Engineer

8. MySQL Certified Developer

10. System/network engineers ?

11. Managers ?

12. Caching experience ?

13. Caching and tuning fun for high scalability Wim Godden Cu.be Solutions

17. 5 visitors/day -> 5 million visitors/day

18. (Don't expect miracle cure !)

19. LAMP

20. Architecture

22. Result ? Single webserver Proxy Static PHP Static PHP Apache + PHP 3900 17.5 6700 17.5 Limit : CPU, network or disk Limit : database

23. Caching

24. What is caching ?

25. What is caching ? select * from article join user on article.user_id = user.id order by created desc limit 10

26. Theory of caching if ($data == false) DB

27. Theory of caching DB

29. Modification frequency ?

30. Retrieval frequency ?

33. Make a complete query log (don't forget to turn it off !)

34. Check page loading times

36. Caching SQL queries : preferably not

38. locking issues !

39. high latency

41. -> Hard to scale

44. Distributed memory caching system

45. Multiple machines ↔ 1 big memory-based hash-table

47. Values - max. 1Mbyte

49. Distributed memory caching system

50. Multiple machines ↔ 1 big memory-based hash-table

53. Memcache - where to install

54. Memcache - where to install

56. PECL

58. memcached -d -m <mem> -l <ip> -p <port>

59. ex. : memcached -d -m 2048 -l 172.16.1.91 -p 11211

61. Lose session data

62. Lose shopping cart data

63. ...

65. Lose session data

66. Lose shopping cart data

68. Memcache in code <?php $memcache = new Memcache(); $memcache->addServer( '172.16.0.1' , 11211); $memcache->addServer( '172.16.0.2' , 11211); $myData = $memcache->get( 'myKey' ); if ($myData === false ) { $myData = GetMyDataFromDB(); // Put it in Memcache as 'myKey', without compression, with no expiration $memcache->set( 'myKey' , $myData, false , 0); } echo $myData;

71. Benchmark with Memcache Single webserver Proxy Static PHP Static PHP Apache + PHP 3900 17.5 6700 17.5 Apache + PHP + MC 3900 55 6700 108

73. Slab 2 : 480 bytes (400 * 1.2)

75. Store a lot of very large objects

76. -> Large slabs : full

77. -> Rest : free

78. -> Eviction of data !

80. Use Cacti or other monitoring tools STAT pid 2941 STAT uptime 10878 STAT time 1296074240 STAT version 1.4.5 STAT pointer_size 64 STAT rusage_user 20.089945 STAT rusage_system 58.499106 STAT curr_connections 16 STAT total_connections 276950 STAT connection_structures 96 STAT cmd_get 276931 STAT cmd_set 584148 STAT cmd_flush 0 STAT get_hits 211106 STAT get_misses 65825 STAT delete_misses 101 STAT delete_hits 276829 STAT incr_misses 0 STAT incr_hits 0 STAT decr_misses 0 STAT decr_hits 0 STAT cas_misses 0 STAT cas_hits 0 STAT cas_badval 0 STAT auth_cmds 0 STAT auth_errors 0 STAT bytes_read 613193860 STAT bytes_written 553991373 STAT limit_maxbytes 268435456 STAT accepting_conns 1 STAT listen_disabled_num 0 STAT threads 4 STAT conn_yields 0 STAT bytes 20418140 STAT curr_items 65826 STAT total_items 553856 STAT evictions 0 STAT reclaimed 0

81. Memcache - backing up

83. Updating data

84. Updating data LCD_Popular_Product_List

85. Adding/updating data $memcache->delete( 'LCD_Popular_Product_List' );

86. Adding/updating data

87. Adding/updating data - Why it crashed

88. Adding/updating data - Why it crashed

89. Adding/updating data - Why it crashed

90. Cache stampeding

91. Cache stampeding

92. Memcache code ? DB

94. Run it before starting Webserver !

98. What if the process that created the lock fails ?

101. Reverse proxy

102. Lightweight, fast

103. 9.6% of all Websites

105. Uses epoll / kqueue

106. Low memory footprint

107. 10000 active connections = normal

109. mod_dav*

111. Some 3 rd party modules (needs recompilation !)

112. Nginx - Configuration server { listen 80; server_name www.domain.ext *.domain.ext; index index.html; root /home/domain.ext/www; } server { listen 80; server_name photo.domain.ext; index index.html; root /home/domain.ext/photo; }

113. Nginx + PHP-FPM - performance ? Single webserver Proxy Static PHP Static PHP Apache + PHP 3900 17.5 6700 17.5 Apache + PHP + MC 3900 55 6700 108 Nginx + PHP-FPM + MC 11700 57 11200 112

114. Reverse proxy time...

116. Reverse proxy cache / http accelerator / …

117. Caches (parts of) pages in memory

119. Nginx usually scales better (but doesn't have VCL)

120. Varnish - backends + load balancing backend server1 { .host = &quot;192.168.0.10&quot;; } backend server2{ .host = &quot;192.168.0.11&quot;; } director example_director round-robin { { .backend = server1; } { .backend = server2; } }

124. ACLs

127. ESI

129. Varnish with ESI - hold on tight ! Single webserver Proxy Static PHP Static PHP Apache + PHP 3900 17.5 6700 17.5 Apache + PHP + MC 3900 55 6700 108 Nginx + PHP-FPM + MC 11700 57 11200 112 Varnish - - 11200 4200

131. Images, js, css

133. Very large files (it's not a file server !)

134. Requests with Set-Cookie

135. User-specific content

136. ESI -> no caching on user-specific content ? Logged in as : Wim Godden 5 messages TTL = 5min TTL=1h TTL = 0s ?

139. Well-built project -> few changes

140. Effect on webservers and database servers

141. What's the result ?

142. What's the result ?

144. No. of db servers : 6 -> 2

146. No. of db servers : 15 -> 4

147. Total : 87 -> 12 (86% reduction !)

149. Still under heavy development

150. Final release : July 2012

151. Released as ????

152. Tuning

154. Use an opcode cache

155. Caching storage - Opcode caching

157. count(*) is evil in InnoDB without a where clause !

158. Persistent connect is sort-of evil

159. Caching & Tuning @ frontend http://www.websiteoptimization.com/speed/tweak/average-web-page/

161. Test frontend

162. Check what requests frontend sends to backend

165. Use CSS Sprites

166. CSS Sprites

167. Tuning content - CSS sprites

168. Tuning content - CSS sprites 11 images 11 HTTP requests 24KByte 1 image 1 HTTP requests 14KByte

177. Watch the logging process ->

178. Logging = disk I/O -> can kill your server !

179. Slashdot effect

181. Use a cache abstraction layer (disk -> Memcache)

182. Don't install for the worst -> prepare for the worst

183. Have a test-setup

188. Web http://techblog.wimgodden.be

189. Slides http://www.slideshare.net/wimg

190. E-mail [email_address]