Copyright © 2013, Oracle and/or its affiliates. All rights reserved.2Dave StokesMySQL Community Managerslideshare.net/dave...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.3
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.4The following is intended to outline our general prod...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.5Radiologists & Gorillas● 83 Percent of Radiologists D...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.6Gorillas responded
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.7Databases Do Not Play WellWith OthersDatabases System...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.8So how do you gethappy MySQL Servers?● This session f...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.9How does a Database server workClientSELECT phoneFROM...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.10How does a Database server workClientSELECT phoneFRO...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.11How does a Database server workClientSELECT phoneFRO...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.12How does a Database server workClientSELECT phoneFRO...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.13Thing to do #1 – Lots of memory•Databases love data ...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.14What if it is not in memory?MySQLPlease give me thed...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.15What if it is not in memory?MySQLPlease give me thed...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.16What if it is not in memory?MySQLPlease give me thed...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.17What if it is not in memory?MySQLPlease give me thed...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.18What if it is not in memory?MySQLPlease give me thed...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.19What if it is not in memory?MySQLPlease give me thed...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.20Thing to do #2 – Use ProperHardwareDatabases have to...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.21BuffersWhat happens when you read a file into memory...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.22BuffersMemory Lookup – 100 nanoseconds, 12GB/secDISK...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.23BuffersMemory Lookup – 100 nanoseconds, 12GB/secDISK...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.24Disk Reads● A disk read is 100,000 slower than readi...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.27SSD● Use standard RAID controller● SSD as writeback ...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.28Hardware recommendations1. Memory – lots of it, ecc2...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.29Thing to do #3 –Understand Login/PrivsThe default My...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.30Quick Security Warning!● MySQL login security is pri...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.31MySQL Workbench – set uproles
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.32Things to do #4 – Use MySQL Current Release● Speed– ...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.33MySQL 5.6: Scalability●Users can fully utilize lates...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.34Thing to do #5 – Monitor● You need somethingmonitori...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.35Turn on logging1. Slow query log -- not all long run...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.36Just because you can not see all of the problem does...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.37Things to do #6 – BackupsBackups are usually some so...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.38Thing to do #7 -- ReplicationReplication for MySQL i...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.39Replication – was messy!MySQL 5.6 is multi threaded ...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.40 Simple to track & compare replication across the c...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.41Replication -- networkNetwork latency will affect My...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.42Writes & Reads Reads Reads• Write to one master• Rea...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.43 New option: binlog-row-image=minimal Increases th...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.44Before:– Transaction Data: in tables– Replication In...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.45Master#Slave#MySQL 5.6: Replication Event ChecksumsE...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.46Thing to do #8 – Use InnoDB● Transactional, speedy, ...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.47●Enables export/import of tables between running MyS...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.48Same app can leverage: Key-value access toInnoDB vi...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.49●Subquery Optimizations●File sort optimizations for ...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.50Things to do #9 – IndexesRough rule – look atcolumns...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.51Indexes are goodWithout IndexDB needs to scanentire ...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.52Indexes, the bad•Overhead -- space, speed, maintenan...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.53Slide to check if audience is still awake●●●
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.54Thing to do #10 – TuningSet innodb_buffer_pool_size ...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.55Where to get help●Lots of Help Available●High Perfor...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.56 Optimized for Web, Cloud-based, Embedded use cases...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.57MySQL Connect ConferenceMySQL Connect Conference● Se...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.58MySQL Marinate!Virtual self-study of MySQL through t...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.59Q&AQ&ADavid.Stokes@oracle.com - slideshare.net/daves...
Upcoming SlideShare
Loading in...5
×

Tx lf propercareandfeedmysql

360

Published on

TexaslinuxFest Proper Care and Feeding of a MySQl Database

Published in: Technology, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
360
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
0
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Transcript of "Tx lf propercareandfeedmysql"

  1. 1. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.2Dave StokesMySQL Community Managerslideshare.net/davestokesDavid.Stokes@Oracle.Com@stokerThe Proper CareThe Proper Careand Feeding of aand Feeding of aMySQL DatabaseMySQL Database
  2. 2. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.3
  3. 3. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.4The following is intended to outline our general product direction. It isintended for information purposes only, and may not be incorporatedinto any contract. It is not a commitment to deliver any material, code,or functionality, and should not be relied upon in making purchasingdecision. The development, release, and timing of any features orfunctionality described for Oracle’s products remains at the solediscretion of Oracle.Safe Harbor Statement
  4. 4. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.5Radiologists & Gorillas● 83 Percent of Radiologists Didn’t Spot the Gorilla Hiding in ThisCT Scan– Picture of gorilla hidden in last slide of ten lung scans.– They wanted to see if the radiologists, focused on thetelltale nodules, would be blind to the easily detectable andhighly anomalous gorilla– The gorilla was miniscule, but huge compared to thenodules. It was about the size of a box of matches-or 48times the size of a typical nodule.
  5. 5. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.6Gorillas responded
  6. 6. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.7Databases Do Not Play WellWith OthersDatabases Systems are the colicky, nastychildren of the software world with bad tempersand leaky noses that stubbornly insist on theirway and will become disruptive at the mostinopportune time, ruining man-years of work inan instant before demanding more--Said by just about every software developer ever
  7. 7. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.8So how do you gethappy MySQL Servers?● This session for thosewho are not MySQLDBAs but are tasked totake care of MySQLInstances among othertasks. This will notmake you a DBA anymore than a 1 hoursession onprogramming will makeyou a programmer.
  8. 8. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.9How does a Database server workClientSELECT phoneFROM friendsWHERE name = ‘Joe’;Server
  9. 9. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.10How does a Database server workClientSELECT phoneFROM friendsWHERE name = ‘Joe’;ServerPARSEfind Joe in friendstable in memoryreturn phone
  10. 10. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.11How does a Database server workClientSELECT phoneFROM friendsWHERE name = ‘Joe’;. . .Process phone dataServerPARSEfind Joe in friendstable in memoryreturn phone
  11. 11. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.12How does a Database server workClientSELECT phoneFROM friendsWHERE name = ‘Joe’;. . .Process phone dataServerPARSEfind Joe in friendstable in memoryreturn phoneWhat was that aboutmemory???
  12. 12. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.13Thing to do #1 – Lots of memory•Databases love data in memory
  13. 13. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.14What if it is not in memory?MySQLPlease give me thedata from the citytableOS
  14. 14. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.15What if it is not in memory?MySQLPlease give me thedata from the citytableOSGet inode
  15. 15. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.16What if it is not in memory?MySQLPlease give me thedata from the citytableOSGet inodeAsk disk for data
  16. 16. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.17What if it is not in memory?MySQLPlease give me thedata from the citytableOSGet inodeAsk disk for dataGet data into buffer
  17. 17. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.18What if it is not in memory?MySQLPlease give me thedata from the citytableLoad data into memoryOSGet inodeAsk disk for dataGet data into bufferHand buffer off
  18. 18. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.19What if it is not in memory?MySQLPlease give me thedata from the citytableLoad data into memoryOSGet inodeAsk disk for dataGet data into bufferHand buffer offMuch longer thanjust reading frommemory
  19. 19. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.20Thing to do #2 – Use ProperHardwareDatabases have to do unpredictable queries,random I/O, and sequential scans so slow I/Okills performance. The ol Hand-me-down DellOpteron that was great for Windows ME thatworked okay in development will provefrustrating in production
  20. 20. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.21BuffersWhat happens when you read a file into memoryDISKDiskControllerPageCacheUserBuffer
  21. 21. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.22BuffersMemory Lookup – 100 nanoseconds, 12GB/secDISKDiskControllerPageCacheUserBuffer
  22. 22. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.23BuffersMemory Lookup – 100 nanoseconds, 12GB/secDISK seek – 10 milliseconds, 760MB/secDISKDiskControllerPageCacheUserBuffer
  23. 23. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.24Disk Reads● A disk read is 100,000 slower than reading memory● For comparison– 100,000 minutes is about 69.5 days– 100,000 feet is about 19 miles– 100,000 kilometers is 15.7 times around Earthsequator
  24. 24. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.27SSD● Use standard RAID controller● SSD as writeback cache● Battery backed● $2.5/GB verses .075/GB● Very fast seek time● Density
  25. 25. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.28Hardware recommendations1. Memory – lots of it, ecc2. DISKs – more spindles, high speed, fast controllers, RAID10, write back cache, and XFS/ZFS/ext4 not ext2/33. Write-through caches with battery backup units fordisks must be monitored, and have life span muchlonger than planned outages. Set learning cycles to offhours4. CPUs, Core less important (spend money on Memoryand IO)
  26. 26. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.29Thing to do #3 –Understand Login/PrivsThe default MySQL Login and Privilegesystem is a little … primitive.Be Stingy with privs as they are hard to getbackSome tools may not give the privs you want –DROP PRIV is given by default by onepopular admin tool
  27. 27. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.30Quick Security Warning!● MySQL login security is primitive.● Database mysql has users table● jsmith@co.com or fred@10.10.%– Matches host, then user, then password● Be explicit● Proxy and Pluggable logins in MySQL 5.6● joe@10.10.% may not be the same as joe@yourco.com orjoe@home.net● MySQL privilege security● GRANT functions or access● Can get Byzantine quickly● Use a GUI
  28. 28. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.31MySQL Workbench – set uproles
  29. 29. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.32Things to do #4 – Use MySQL Current Release● Speed– MySQL 5.5 is 20% faster than 5.1– MySQL 5.6 is ~15% faster than 5.5● Features– Better optimizer– Sub-queries– NoSQL/memcached– Online DDL
  30. 30. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.33MySQL 5.6: Scalability●Users can fully utilize latest generations of hardware and OS●Scales as data volumes and users grow
  31. 31. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.34Thing to do #5 – Monitor● You need somethingmonitoring yourinstances as you needa quick way to checkstatus & record events– MySQL EnterpriseMonitor– MySQL Workbench– Nagios, Cacti,phpMyAdmin, etc
  32. 32. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.35Turn on logging1. Slow query log -- not all long running queriesare bad2. Log queries not using indexes3. READ the logs!!!4. Use monitoring software – MEM, Innotop,Cacti, Munin, Nagios, etc – and pay attentionto it
  33. 33. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.36Just because you can not see all of the problem doesnot mean the problem is not there!
  34. 34. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.37Things to do #6 – BackupsBackups are usually some sort of disk snap shotor serializing data to a fileThe more the better but you need to know stepsto recover dropped table, lost databases, ormangled data.Use data replication to a slave and then backupslave.Be paranoid!!!!!
  35. 35. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.38Thing to do #7 -- ReplicationReplication for MySQL is the binary log for themaster being copied to a slave. The slavethen updates its copy of the dataTwo types:1. Asynchronous – server does not check changes sent toslave before proceeding2. Semi Synchronous – server checks that slave receivedchanges before proceeding
  36. 36. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.39Replication – was messy!MySQL 5.6 is multi threaded and has GTIDs
  37. 37. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.40 Simple to track & compare replication across the cluster●Unique identifier for each transaction written to the Binlog Automatically identify the most up-to-date slave for failover Deploy n-tier replication hierarchiesMasterGTID=123456GTID=123456GTID=123456 GTID=123456MySQL 5.6: ReplicationGlobal Transaction IdsEliminates the need forcomplex 3rd partysolutions
  38. 38. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.41Replication -- networkNetwork latency will affect MySQL replication. So plannetwork topology to minimize bandwidth competition withother systems/services.Slaves do not need to be as fast as the master but try tokeep things reasonably closeDo not have to replicate all tables/databases to all slaves.Cut down on traffic by replicating what is needed!
  39. 39. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.42Writes & Reads Reads Reads• Write to one master• Read from many slaves, easily add more as needed• Perfect for read/write intensive appsApplicationMySQL ReplicationLoad BalancerMySQL DatabaseReplication Enables Scalability
  40. 40. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.43 New option: binlog-row-image=minimal Increases throughput for master and slave– Reduces Binlog size, memory & network bandwidth Only replicates elements of the Row image that have changedPrimary Key Changed ColumnsMySQL 5.6: ReplicationOptimized Row Base Replication
  41. 41. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.44Before:– Transaction Data: in tables– Replication Info: in filesMySQL 5.6– Transaction Data: in tables– Replication Info: in tablesDataPosition InfoCRASH!TimeDataPosition InfoTime Automatic recovery of a slave after a failure●Binlog and table data are transactionallyconsistent Resumes replication without Dev/Opintervention●Automatically rolling back replication tolast committed eventAtomicAtomicMySQL 5.6: Crash safe SlavesEliminates risk of data loss orcorruption
  42. 42. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.45Master#Slave#MySQL 5.6: Replication Event ChecksumsEliminates risk of dataloss or corruption Ensures replicated data is correct,consistent and accessible Detects corrupt replication events beforethey’re applied– Returns an error Protects entire replication path– Memory– Disk– Network– Bugs
  43. 43. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.46Thing to do #8 – Use InnoDB● Transactional, speedy, crash safe, and whereOracle is concentrating development.● Online DDL, SSD Optimizations, Dump/Restorewarm buffer pool (great for cloud)● NoSQL access via memcached API
  44. 44. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.47●Enables export/import of tables between running MySQL instancesMySQL 5.6: InnoDBTransportable TablespacesCREATE TABLE t(c1 INT) engine=InnoDB;FLUSH TABLE t FOR EXPORT; -- quiesce the table and create the meta data file$innodb_data_home_dir/test/t.cfgUNLOCK TABLES;Export:Import:CREATE TABLE t(c1 INT) engine=InnoDB; -- if it doesnt already existALTER TABLE t DISCARD TABLESPACE;-- The user must stop all updates on the tables, prior to the IMPORTALTER TABLE t IMPORT TABLESPACE;●Better Elasticity - Quickly spin up new instances to meet demand●Great for Cloud, Hosted, SaaS, On-premise deployments
  45. 45. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.48Same app can leverage: Key-value access toInnoDB via familiarMemcached API SQL for rich queries,JOINs, FKs, etc. Fully transactionalMySQL 5.6: InnoDBNoSQL Key Value Access to InnoDB●Up to 9x performanceboost for updates●Great for fast dataingestion in Big Datapipeline
  46. 46. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.49●Subquery Optimizations●File sort optimizations for most web use cases●4x better execution time – 40s to 10s Index Condition Pushdown●160x better execution time – 15s to 90ms Batched Key Access and Multi Range Read●280x better execution time – 2800s to 10sMySQL 5.6: Optimizer●Better complex query execution times ever growing data sets (Big Data!)●MEM + Query Analyzer key to utilizing full benefits of 5.6 Optimizer●MySQL Consultative Support provides guidance on configuration
  47. 47. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.50Things to do #9 – IndexesRough rule – look atcolumns used injoins or after aWHERE
  48. 48. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.51Indexes are goodWithout IndexDB needs to scanentire table or tablescanWith IndexDB can go right torecord
  49. 49. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.52Indexes, the bad•Overhead -- space, speed, maintenance (youcan have too many)•Not a panacea – does not cure all problems•Will not help if you need to perform a tablescan•Composite indexes can be tricky – YearMonthDayusually better than DayMonthYear
  50. 50. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.53Slide to check if audience is still awake●●●
  51. 51. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.54Thing to do #10 – TuningSet innodb_buffer_pool_size to 70-80%of memoryTurn off query cache
  52. 52. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.55Where to get help●Lots of Help Available●High Performance MySQL – Schwartz et al●MySQL Administrators Bible – Cabral●Effective MySQL series – Bradford●OurSQL podcast●Forums.MySQL.ComPlanet.MySQL.ComLocal MySQL User Group
  53. 53. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.56 Optimized for Web, Cloud-based, Embedded use cases Simplified, Pluggable architecture– Maintainability, more extensible– More NoSQL options (HTTP, JSON, JavaScript, etc.) Refactoring– Data Dictionary in InnoDB– Optimizer/Parser/Protocol InnoDB– Optimized for SSD– GIS Easy HA, Replication and ShardingMySQL Database Development Priorities
  54. 54. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.57MySQL Connect ConferenceMySQL Connect Conference● Sept 21st- 23rd inSanFrancisco●● The Call for Proposals(CFP) for MySQL Connect2013 just closed.Conference attendees willhear the best ideas fromMySQL experts from aroundthe globe: developers,visionaries, customers,partners. Book hotel earlydo to Americas Cup races inSan Francisco.
  55. 55. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.58MySQL Marinate!Virtual self-study of MySQL through theBoston MySQL Users Group (http://www.meetup.com/mysqlbos/)http://www.meetup.com/Virtual-Tech-Self-Study/events/84103332/
  56. 56. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.59Q&AQ&ADavid.Stokes@oracle.com - slideshare.net/davestokes

×