This document provides strategies for optimizing MySQL performance as databases grow in complexity and size. It discusses ways to optimize queries, schemas, hardware, software configuration, caching, and monitoring. The key lessons are to optimize queries, choose appropriate data types and storage engines, configure MySQL and cache settings properly, benchmark and monitor performance, and scale reads and writes separately.

1. Strategic MySQL Planning for Complexity & Growth Or MySQL Scaling for Dummies Thomas Falgout Sr. Developer Yahoo!

3. Schema Definition

4. Hardware

5. Software

6. MySQL Config

7. Cache

8. Monitoring

10. Developers usually don't write proper SQL

11. SQL Statements will directly affect your performance

13. 1000 very quick unnecessary queries vs 1 slow query

15. Your coworkers might surprise you.

16. In a good way.

18. Quickly show you how much data your queries are processing

19. Could spend an entire session on just this.

20. But won't.

21. Read for yourself.

22. http://dev.mysql.com/doc/refman/5.1/en/explain.html

23. http://dev.mysql.com/doc/refman/5.1/en/using-explain.html

24. http://www.percona.com/files//presentations/EXPLAIN_demystified.pdf

25. LUUUUUCCCYYYYYYY!!!!!

28. Faster on MyISAM (stored in table info)

30. INSERT INTO a VALUES (1,23),(2,34),(4,33);

31. To speed up INSERT operations that require multiple statements, LOCK your tables

32. Can you use LOAD DATA? (Import via CSV)

34. Not much to say.

37. DROP table

38. RENAME tmp table to table

40. Aggregate data hourly/daily/per update/etc.

42. Send updates through daemon to load data

44. UPDATE counts SET count = count + 42 WHERE id=1

46. Does not handle transactions

47. Faster to Write, Slightly faster to Read

48. Uses separate Data (.MYD), Index (.MYI) and Frame (.frm)

53. Writes Binary logs to disk

54. Doesn't store data

55. Useful for offloading replication from the master

56. Blackhole Sun. Won't you come. Wash away my data.

59. http://forge.mysql.com/wiki/Falcon

61. http://dev.mysql.com/doc/refman/5.1/en/data-size.html

62. http://dev.mysql.com/doc/refman/5.0/en/storage-requirements.html

64. SMALLINT – 2 Bytes

65. MEDIUMINT – 3 Bytes

66. INT – 4 Bytes

67. BIGINT – 8 Bytes

68. http://dev.mysql.com/doc/refman/5.0/en/numeric-types.html

70. VARCHAR(N) = X + 1 byte. X = Number of actual characters in string.

71. N & X = 1 Byte. Unless using utf8 or a multi-byte format

72. Same rules apply for BINARY & VARBINARY.

73. Extra data is TRUNCATED!!!!!!

74. That's bad.

75. http://dev.mysql.com/doc/refman/5.0/en/string-types.html

77. VARCHAR(255) is 4.x limitation

78. When creating a temp table, it allocates the greatest size of data (255)

83. TIME = 3 bytes

86. 192.168.101.201 = 11000000.10101000.01100101.11001001

87. INT UNSIGNED = 4 Bytes!

88. INET_NTOA => NumberTOAddress

89. INET_ATON => AddressTONumber

94. Stores first as 768 Bytes, and then increments into 16k data pages per row per TEXT/BLOB field

97. Network latency affects experience

99. Send all READS to Slave

100. Send all WRITES to Master

102. Provides HA w/ no Single Point of Failure

103. Requires 3 servers (2 Data, 1 Mgmt Nodes)

104. Inserts on one DB are replicated to others

105. Selects can be performed on any DB

106. http://dev.mysql.com/tech-resources/articles/mysql-cluster-for-two-servers.html

107. http://www.lod.com/whitepapers/mysql-cluster-howto.html

110. Failover

111. Query Analysis

114. Table Sync (Resolve Differences)

115. Visual Explain (Explain! Visually!)

116. Heartbeat (Shows Slave Replication Delay)

119. Point Slaves to Master

120. Put Slaves Behind Load Balancer

122. Batch Insert / Update

123. Don't Delete

124. Partition your Data

127. table_cache

128. innodb_buffer_pool_size

131. Size of cache for MyISAM Indexes

133. key_reads

134. key_write_requests

135. key_writes

136. key_reads/key_read_requests < 0.01 == GOOD

138. This dictates the number of open file descriptors

140. Size in bytes of the memory buffer InnoDB uses to cache data and indexes.

141. Default 8MB

143. Great for tables that don't change often

144. Doesn't return stale data

145. Cache is invalidated if returned entry is modified

146. Another reason why SELECT(*) is bad

148. MyISAM – Leave more memory for OS page caches

150. Use long_query_time to specify x

151. ~1 sec is good starting point

152. http://dev.mysql.com/doc/refman/5.1/en/slow-query-log.html

154. Originally created by LiveJournal to handle scale

155. EASY Button

156. $memcache_obj = memcache_connect(&quot;localhost&quot;, 11211);

157. $memcache_obj->add('var_key', 'test variable');

158. $var = $memcache_obj->get('var_key');

161. mysqlslap -u root -concurrency=10000 -iterations=5 -number-of-queries=10000 -query=”SELECT name FROM city WHERE city_id=12”

163. Like mysqlslap

167. SHOW GLOBAL STATUS

168. SHOW GLOBAL VARIABLES

170. MUNIN – Similar to Cacti, but easier to maintain

171. Nagios – More of a notification tool

172. MySQL Enterprise Dashboard – Notification & Monitoring

174. Be descriptive

175. Avoid reserve words (date, time, timestamp)

178. Break your software.

180. The Most Important Lesson This part is absolutely important that you get. It's time to wake up.

181. The Most Important Lesson When you need to, Ignore everything I've suggested.

182. Thanks Thanks for listening.

183. Links Flickr - Scaling MySQL http://www.scribd.com/doc/21593/Scaling-PHPMySQL Very Basic PHP & MySQL: http://www.cs.gsu.edu/~stefan/teaching/csc-6370/notes/phpmysql.pdf JPIPES! http://jpipes.com/presentations/joinfu/joinfu_part_one.pdf http://jpipes.com/presentations/joinfu/joinfu_part_two.pdf http://jpipes.com/presentations/dqm/legend_drunken_query_master.pdf EXPLAIN: http://www.percona.com/files//presentations/EXPLAIN_demystified.pdf Query Optimization: http://www.percona.com/files//presentations/Zurich2007-QueryOptimization.pdf Server Settings Tuning: http://www.percona.com/files//presentations/UC2007-MySQL-Server-Settings-Tuning.pdf http://www.flupps.org/mysql/MySQL_Microscope.ppt http://dev.mysql.com/doc/refman/5.0/en/working-with-null.html http://www.devshed.com/c/a/MySQL/MySQL-Server-Tuning-Tips-and-Tricks/ http://www.planetmysql.org/