Copyright © 2013, Oracle and/or its affiliates. All rights reserved.1Dave StokesMySQL Community Managerslideshare.net/dave...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.2The following is intended to outline our general prod...
Radiologists & Gorillas83 Percent of Radiologists Didn’t Spot the GorillaHiding in This CT Scan– Picture of gorilla hidden...
Gorillas responded
Databases Do Not Play WellWith OthersDatabases Systems are the colicky, nastychildren of the software world with bad tempe...
So how do you gethappy MySQL Servers?This session for thosewho are not MySQLDBAs but are tasked totake care of MySQLInstan...
How does a Database server workClientSELECT phoneFROM friendsWHERE name = ‘Joe’;Server
How does a Database server workClientSELECT phoneFROM friendsWHERE name = ‘Joe’;ServerPARSEfind Joe in friendstable in mem...
How does a Database server workClientSELECT phoneFROM friendsWHERE name = ‘Joe’;. . .Process phone dataServerPARSEfind Joe...
How does a Database server workClientSELECT phoneFROM friendsWHERE name = ‘Joe’;. . .Process phone dataServerPARSEfind Joe...
Thing to do #1 – Lots of memory•Databases love data in memory
What if it is not in memory?MySQLPlease give me thedata from the citytableOS
What if it is not in memory?MySQLPlease give me thedata from the citytableOSGet inode
What if it is not in memory?MySQLPlease give me thedata from the citytableOSGet inodeAsk disk for data
What if it is not in memory?MySQLPlease give me thedata from the citytableOSGet inodeAsk disk for dataGet data into buffer
What if it is not in memory?MySQLPlease give me thedata from the citytableLoad data into memoryOSGet inodeAsk disk for dat...
What if it is not in memory?MySQLPlease give me thedata from the citytableLoad data into memoryOSGet inodeAsk disk for dat...
Rule #2 – Use Proper HardwareDatabases have to do unpredictable queries,random I/O, and sequential scans so slow I/Okills ...
BuffersWhat happens when you read a file into memoryDISKDiskControllerPageCacheUserBuffer
BuffersMemory Lookup – 100 nanoseconds, 12GB/secDISKDiskControllerPageCacheUserBuffer
BuffersMemory Lookup – 100 nanoseconds, 12GB/secDISK seek – 10 milliseconds, 760MB/secDISKDiskControllerPageCacheUserBuffer
Disk ReadsA disk read is 100,000 slower than reading memory●For comparison– 100,000 minutes is about 69.5 days– 100,000 fe...
SSDUse standard RAID controller●SSD as writeback cache●Battery backed●$2.5/GB verses .075/GB●Very fast seek time●Density
Hardware recommendations1. Memory – lots of it, ecc2. DISKs – more spindles, high speed, fast controllers, RAID10, write b...
Thing to do #3 –Understand Login/PrivsThe default MySQL Login and Privilege system is alittle … primitive.Be Stingy with p...
Quick Security Warning!MySQL login security is primitive.●Database mysql has users table●jsmith@co.com or fred@10.10.%– Ma...
MySQL Workbench – set up roles
Thing to do #4 – Use Current ReleaseSpeed– MySQL 5.5 is 20% faster than 5.1– MySQL 5.6 is ~15% faster than 5.5Features– Be...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.31MySQL 5.6: Scalability●Users can fully utilize lates...
Thing to do #5 – MonitorYou need somethingmonitoring yourinstances as you need aquick way to checkstatus & record events– ...
Turn on logging1. Slow query log -- not all long running queriesare bad2. Log queries not using indexes3. READ the logs!!!...
Just because you can not see all of the problemdoes not mean the problem is not there!
Things to do #6 – BackupsBackups are usually some sort of disk snap shotor serializing data to a fileThe more the better b...
Thing to do #7 -- ReplicationReplication for MySQL is the binary log for themaster being copied to a slave. The slavethen ...
Replication – was messy!MySQL 5.6 is multi threaded and has GTIDs
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.38 Simple to track & compare replication across the c...
Replication -- networkNetwork latency will affect MySQL replication. So plannetwork topology to minimize bandwidth competi...
Writes & Reads Reads Reads• Write to one master• Read from many slaves, easily add more as needed• Perfect for read/write ...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.41 New option: binlog-row-image=minimal Increases th...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.42Before:– Transaction Data: in tables– Replication In...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.43Master#Slave#MySQL 5.6: Replication Event ChecksumsE...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.44Thing to do #8 – Use InnoDB● Transactional, speedy, ...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.45●Enables export/import of tables between running MyS...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.46Same app can leverage: Key-value access toInnoDB vi...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.47●Subquery Optimizations●File sort optimizations for ...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.48Things to do #9 – IndexesRough rule – look atcolumns...
Indexes are goodWithout IndexDB needs to scanentire table or tablescanWith IndexDB can go right torecord
Indexes, the bad•Overhead -- space, speed, maintenance (youcan have too many)•Not a panacea – does not cure all problems•W...
Slide to check if audience is still awake
Thing to do #10 – TuningASet innodb_buffer_pool_size to 70-80% of memoryTurn off query cache
Where to get helpLots of Help Available●High Performance MySQL – Schwartz et al●MySQL Administrators Bible – Cabral●Effect...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.54 Optimized for Web, Cloud-based, Embedded use cases...
MySQL Connect ConferenceMySQL Connect ConferenceSept 21st- 23rdinSan FranciscoThe Call for Proposals (CFP)for MySQL Connec...
MySQL Marinate!Virtual self-study of MySQL through theBoston MySQL Users Group (http://www.meetup.com/mysqlbos/)http://www...
Q&AQ&ADavid.Stokes@oracle.com - slideshare.net/davestokes
Upcoming SlideShare
Loading in...5
×

PhpTek Ten Things to do to make your MySQL servers Happier and Healthier

2,204

Published on

Ten recommendations to make your MySQL servers better

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
2,204
On Slideshare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
6
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Transcript of "PhpTek Ten Things to do to make your MySQL servers Happier and Healthier"

  1. 1. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.1Dave StokesMySQL Community Managerslideshare.net/davestokesDavid.Stokes@Oracle.Com@stokerTen Things toTen Things toMake Your MySQLMake Your MySQLServers Faster and HappierServers Faster and Happier
  2. 2. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.2The following is intended to outline our general product direction. It isintended for information purposes only, and may not be incorporatedinto any contract. It is not a commitment to deliver any material, code,or functionality, and should not be relied upon in making purchasingdecision. The development, release, and timing of any features orfunctionality described for Oracle’s products remains at the solediscretion of Oracle.Safe Harbor Statement
  3. 3. Radiologists & Gorillas83 Percent of Radiologists Didn’t Spot the GorillaHiding in This CT Scan– Picture of gorilla hidden in last slide of ten lungscans.– They wanted to see if the radiologists, focused onthe telltale nodules, would be blind to the easilydetectable and highly anomalous gorilla– The gorilla was miniscule, but huge compared tothe nodules. It was about the size of a box ofmatches-or 48 times the size of a typical nodule.
  4. 4. Gorillas responded
  5. 5. Databases Do Not Play WellWith OthersDatabases Systems are the colicky, nastychildren of the software world with bad tempersand leaky noses that stubbornly insist on theirway and will become disruptive at the mostinopportune time, ruining man-years of work inan instant before demanding more--Said by just about every software developer ever
  6. 6. So how do you gethappy MySQL Servers?This session for thosewho are not MySQLDBAs but are tasked totake care of MySQLInstances among othertasks. This will notmake you a DBA anymore than a 1 hoursession onprogramming will makeyou a programmer.
  7. 7. How does a Database server workClientSELECT phoneFROM friendsWHERE name = ‘Joe’;Server
  8. 8. How does a Database server workClientSELECT phoneFROM friendsWHERE name = ‘Joe’;ServerPARSEfind Joe in friendstable in memoryreturn phone
  9. 9. How does a Database server workClientSELECT phoneFROM friendsWHERE name = ‘Joe’;. . .Process phone dataServerPARSEfind Joe in friendstable in memoryreturn phone
  10. 10. How does a Database server workClientSELECT phoneFROM friendsWHERE name = ‘Joe’;. . .Process phone dataServerPARSEfind Joe in friendstable in memoryreturn phoneWhat was that aboutmemory???
  11. 11. Thing to do #1 – Lots of memory•Databases love data in memory
  12. 12. What if it is not in memory?MySQLPlease give me thedata from the citytableOS
  13. 13. What if it is not in memory?MySQLPlease give me thedata from the citytableOSGet inode
  14. 14. What if it is not in memory?MySQLPlease give me thedata from the citytableOSGet inodeAsk disk for data
  15. 15. What if it is not in memory?MySQLPlease give me thedata from the citytableOSGet inodeAsk disk for dataGet data into buffer
  16. 16. What if it is not in memory?MySQLPlease give me thedata from the citytableLoad data into memoryOSGet inodeAsk disk for dataGet data into bufferHand buffer off
  17. 17. What if it is not in memory?MySQLPlease give me thedata from the citytableLoad data into memoryOSGet inodeAsk disk for dataGet data into bufferHand buffer offMuch longer thanjust reading frommemory
  18. 18. Rule #2 – Use Proper HardwareDatabases have to do unpredictable queries,random I/O, and sequential scans so slow I/Okills performance
  19. 19. BuffersWhat happens when you read a file into memoryDISKDiskControllerPageCacheUserBuffer
  20. 20. BuffersMemory Lookup – 100 nanoseconds, 12GB/secDISKDiskControllerPageCacheUserBuffer
  21. 21. BuffersMemory Lookup – 100 nanoseconds, 12GB/secDISK seek – 10 milliseconds, 760MB/secDISKDiskControllerPageCacheUserBuffer
  22. 22. Disk ReadsA disk read is 100,000 slower than reading memory●For comparison– 100,000 minutes is about 69.5 days– 100,000 feet is about 19 miles– 100,000 kilometers is 15.7 times aroundEarths equator
  23. 23. SSDUse standard RAID controller●SSD as writeback cache●Battery backed●$2.5/GB verses .075/GB●Very fast seek time●Density
  24. 24. Hardware recommendations1. Memory – lots of it, ecc2. DISKs – more spindles, high speed, fast controllers, RAID10, write back cache, and XFS/ZFS/ext4 not ext2/33. Write-through caches with battery backup units fordisks must be monitored, and have life span muchlonger than planned outages. Set learning cycles to offhours4. CPUs, Core less important (spend money on Memoryand IO)
  25. 25. Thing to do #3 –Understand Login/PrivsThe default MySQL Login and Privilege system is alittle … primitive.Be Stingy with privs as they are hard to get backSome tools may not give the privs you want – DROPPRIV is given by default by one popular admin tool
  26. 26. Quick Security Warning!MySQL login security is primitive.●Database mysql has users table●jsmith@co.com or fred@10.10.%– Matches host, then user, then password●Be explicit●Proxy and Pluggable logins in MySQL 5.6●joe@10.10.% may not be the same asjoe@yourco.com or joe@home.netMySQL privilege security●GRANT functions or access●Can get Byzantine quickly●Use a GUI
  27. 27. MySQL Workbench – set up roles
  28. 28. Thing to do #4 – Use Current ReleaseSpeed– MySQL 5.5 is 20% faster than 5.1– MySQL 5.6 is ~15% faster than 5.5Features– Better optimizer– Sub-queries– NoSQL/memcached– Online DDL
  29. 29. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.31MySQL 5.6: Scalability●Users can fully utilize latest generations of hardware and OS●Scales as data volumes and users grow
  30. 30. Thing to do #5 – MonitorYou need somethingmonitoring yourinstances as you need aquick way to checkstatus & record events– MySQL EnterpriseMonitor– MySQLWorkbench– Nagios, Cacti,phpMyAdmin, etc
  31. 31. Turn on logging1. Slow query log -- not all long running queriesare bad2. Log queries not using indexes3. READ the logs!!!4. Use monitoring software – MEM, Innotop,Cacti, Munin, Nagios, etc – and pay attentionto it
  32. 32. Just because you can not see all of the problemdoes not mean the problem is not there!
  33. 33. Things to do #6 – BackupsBackups are usually some sort of disk snap shotor serializing data to a fileThe more the better but you need to know stepsto recover dropped table, lost databases, ormangled data.Use data replication to a slave and then backupslave.Be paranoid!!!!!
  34. 34. Thing to do #7 -- ReplicationReplication for MySQL is the binary log for themaster being copied to a slave. The slavethen updates its copy of the dataTwo types:1. Asynchronous – server does not check changes sent toslave before proceeding2. Semi Synchronous – server checks that slave receivedchanges before proceeding
  35. 35. Replication – was messy!MySQL 5.6 is multi threaded and has GTIDs
  36. 36. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.38 Simple to track & compare replication across the cluster●Unique identifier for each transaction written to the Binlog Automatically identify the most up-to-date slave for failover Deploy n-tier replication hierarchiesMasterGTID=123456GTID=123456GTID=123456 GTID=123456MySQL 5.6: ReplicationGlobal Transaction IdsEliminates the need forcomplex 3rd partysolutions
  37. 37. Replication -- networkNetwork latency will affect MySQL replication. So plannetwork topology to minimize bandwidth competition withother systems/services.Slaves do not need to be as fast as the master but try tokeep things reasonably closeDo not have to replicate all tables/databases to all slaves.Cut down on traffic by replicating what is needed!
  38. 38. Writes & Reads Reads Reads• Write to one master• Read from many slaves, easily add more as needed• Perfect for read/write intensive appsApplicationMySQL ReplicationLoad BalancerMySQL DatabaseReplication Enables Scalability
  39. 39. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.41 New option: binlog-row-image=minimal Increases throughput for master and slave– Reduces Binlog size, memory & network bandwidth Only replicates elements of the Row image that have changedPrimary Key Changed ColumnsMySQL 5.6: ReplicationOptimized Row Base Replication
  40. 40. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.42Before:– Transaction Data: in tables– Replication Info: in filesMySQL 5.6– Transaction Data: in tables– Replication Info: in tablesDataPosition InfoCRASH!TimeDataPosition InfoTime Automatic recovery of a slave after a failure●Binlog and table data are transactionallyconsistent Resumes replication without Dev/Opintervention●Automatically rolling back replication tolast committed eventAtomicAtomicMySQL 5.6: Crash safe SlavesEliminates risk of data loss orcorruption
  41. 41. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.43Master#Slave#MySQL 5.6: Replication Event ChecksumsEliminates risk of dataloss or corruption Ensures replicated data is correct,consistent and accessible Detects corrupt replication events beforethey’re applied– Returns an error Protects entire replication path– Memory– Disk– Network– Bugs
  42. 42. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.44Thing to do #8 – Use InnoDB● Transactional, speedy, crash safe, and whereOracle is concentrating development.● Online DDL, SSD Optimizations, Dump/Restorewarm buffer pool (great for cloud)● NoSQL access via memcached API
  43. 43. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.45●Enables export/import of tables between running MySQL instancesMySQL 5.6: InnoDBTransportable TablespacesCREATE TABLE t(c1 INT) engine=InnoDB;FLUSH TABLE t FOR EXPORT; -- quiesce the table and create the meta data file$innodb_data_home_dir/test/t.cfgUNLOCK TABLES;Export:Import:CREATE TABLE t(c1 INT) engine=InnoDB; -- if it doesnt already existALTER TABLE t DISCARD TABLESPACE;-- The user must stop all updates on the tables, prior to the IMPORTALTER TABLE t IMPORT TABLESPACE;●Better Elasticity - Quickly spin up new instances to meet demand●Great for Cloud, Hosted, SaaS, On-premise deployments
  44. 44. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.46Same app can leverage: Key-value access toInnoDB via familiarMemcached API SQL for rich queries,JOINs, FKs, etc. Fully transactionalMySQL 5.6: InnoDBNoSQL Key Value Access to InnoDB●Up to 9x performanceboost for updates●Great for fast dataingestion in Big Datapipeline
  45. 45. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.47●Subquery Optimizations●File sort optimizations for most web use cases●4x better execution time – 40s to 10s Index Condition Pushdown●160x better execution time – 15s to 90ms Batched Key Access and Multi Range Read●280x better execution time – 2800s to 10sMySQL 5.6: OptimizerBetter complex query execution times ever growing data sets (Big DMEM + Query Analyzer key to utilizing full benefits of 5.6 OptimizerMySQL Consultative Support provides guidance on configuration
  46. 46. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.48Things to do #9 – IndexesRough rule – look atcolumns used injoins or after aWHERE
  47. 47. Indexes are goodWithout IndexDB needs to scanentire table or tablescanWith IndexDB can go right torecord
  48. 48. Indexes, the bad•Overhead -- space, speed, maintenance (youcan have too many)•Not a panacea – does not cure all problems•Will not help if you need to perform a tablescan•Composite indexes can be tricky – YearMonthDayusually better than DayMonthYear
  49. 49. Slide to check if audience is still awake
  50. 50. Thing to do #10 – TuningASet innodb_buffer_pool_size to 70-80% of memoryTurn off query cache
  51. 51. Where to get helpLots of Help Available●High Performance MySQL – Schwartz et al●MySQL Administrators Bible – Cabral●Effective MySQL series – Bradford●OurSQL podcast●Forums.MySQL.ComPlanet.MySQL.ComLocal MySQL User Group
  52. 52. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.54 Optimized for Web, Cloud-based, Embedded use cases Simplified, Pluggable architecture– Maintainability, more extensible– More NoSQL options (HTTP, JSON, JavaScript, etc.) Refactoring– Data Dictionary in InnoDB– Optimizer/Parser/Protocol InnoDB– Optimized for SSD– GIS Easy HA, Replication and ShardingMySQL Database Development Priorities
  53. 53. MySQL Connect ConferenceMySQL Connect ConferenceSept 21st- 23rdinSan FranciscoThe Call for Proposals (CFP)for MySQL Connect 2013 justclosed. Conferenceattendees will hear the bestideas from MySQL expertsfrom around the globe:developers, visionaries,customers, partners. Bookhotel early do to AmericasCup races in San Francisco.
  54. 54. MySQL Marinate!Virtual self-study of MySQL through theBoston MySQL Users Group (http://www.meetup.com/mysqlbos/)http://www.meetup.com/Virtual-Tech-Self-Study/events/84103332/
  55. 55. Q&AQ&ADavid.Stokes@oracle.com - slideshare.net/davestokes
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×