• Save
Tx lf propercareandfeedmysql
Upcoming SlideShare
Loading in...5
×
 

Tx lf propercareandfeedmysql

on

  • 470 views

TexaslinuxFest Proper Care and Feeding of a MySQl Database

TexaslinuxFest Proper Care and Feeding of a MySQl Database

Statistics

Views

Total Views
470
Views on SlideShare
470
Embed Views
0

Actions

Likes
0
Downloads
0
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Tx lf propercareandfeedmysql Presentation Transcript

  • 1. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.2Dave StokesMySQL Community Managerslideshare.net/davestokesDavid.Stokes@Oracle.Com@stokerThe Proper CareThe Proper Careand Feeding of aand Feeding of aMySQL DatabaseMySQL Database
  • 2. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.3
  • 3. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.4The following is intended to outline our general product direction. It isintended for information purposes only, and may not be incorporatedinto any contract. It is not a commitment to deliver any material, code,or functionality, and should not be relied upon in making purchasingdecision. The development, release, and timing of any features orfunctionality described for Oracle’s products remains at the solediscretion of Oracle.Safe Harbor Statement
  • 4. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.5Radiologists & Gorillas● 83 Percent of Radiologists Didn’t Spot the Gorilla Hiding in ThisCT Scan– Picture of gorilla hidden in last slide of ten lung scans.– They wanted to see if the radiologists, focused on thetelltale nodules, would be blind to the easily detectable andhighly anomalous gorilla– The gorilla was miniscule, but huge compared to thenodules. It was about the size of a box of matches-or 48times the size of a typical nodule.
  • 5. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.6Gorillas responded
  • 6. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.7Databases Do Not Play WellWith OthersDatabases Systems are the colicky, nastychildren of the software world with bad tempersand leaky noses that stubbornly insist on theirway and will become disruptive at the mostinopportune time, ruining man-years of work inan instant before demanding more--Said by just about every software developer ever
  • 7. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.8So how do you gethappy MySQL Servers?● This session for thosewho are not MySQLDBAs but are tasked totake care of MySQLInstances among othertasks. This will notmake you a DBA anymore than a 1 hoursession onprogramming will makeyou a programmer.
  • 8. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.9How does a Database server workClientSELECT phoneFROM friendsWHERE name = ‘Joe’;Server
  • 9. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.10How does a Database server workClientSELECT phoneFROM friendsWHERE name = ‘Joe’;ServerPARSEfind Joe in friendstable in memoryreturn phone
  • 10. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.11How does a Database server workClientSELECT phoneFROM friendsWHERE name = ‘Joe’;. . .Process phone dataServerPARSEfind Joe in friendstable in memoryreturn phone
  • 11. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.12How does a Database server workClientSELECT phoneFROM friendsWHERE name = ‘Joe’;. . .Process phone dataServerPARSEfind Joe in friendstable in memoryreturn phoneWhat was that aboutmemory???
  • 12. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.13Thing to do #1 – Lots of memory•Databases love data in memory
  • 13. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.14What if it is not in memory?MySQLPlease give me thedata from the citytableOS
  • 14. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.15What if it is not in memory?MySQLPlease give me thedata from the citytableOSGet inode
  • 15. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.16What if it is not in memory?MySQLPlease give me thedata from the citytableOSGet inodeAsk disk for data
  • 16. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.17What if it is not in memory?MySQLPlease give me thedata from the citytableOSGet inodeAsk disk for dataGet data into buffer
  • 17. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.18What if it is not in memory?MySQLPlease give me thedata from the citytableLoad data into memoryOSGet inodeAsk disk for dataGet data into bufferHand buffer off
  • 18. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.19What if it is not in memory?MySQLPlease give me thedata from the citytableLoad data into memoryOSGet inodeAsk disk for dataGet data into bufferHand buffer offMuch longer thanjust reading frommemory
  • 19. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.20Thing to do #2 – Use ProperHardwareDatabases have to do unpredictable queries,random I/O, and sequential scans so slow I/Okills performance. The ol Hand-me-down DellOpteron that was great for Windows ME thatworked okay in development will provefrustrating in production
  • 20. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.21BuffersWhat happens when you read a file into memoryDISKDiskControllerPageCacheUserBuffer
  • 21. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.22BuffersMemory Lookup – 100 nanoseconds, 12GB/secDISKDiskControllerPageCacheUserBuffer
  • 22. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.23BuffersMemory Lookup – 100 nanoseconds, 12GB/secDISK seek – 10 milliseconds, 760MB/secDISKDiskControllerPageCacheUserBuffer
  • 23. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.24Disk Reads● A disk read is 100,000 slower than reading memory● For comparison– 100,000 minutes is about 69.5 days– 100,000 feet is about 19 miles– 100,000 kilometers is 15.7 times around Earthsequator
  • 24. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.27SSD● Use standard RAID controller● SSD as writeback cache● Battery backed● $2.5/GB verses .075/GB● Very fast seek time● Density
  • 25. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.28Hardware recommendations1. Memory – lots of it, ecc2. DISKs – more spindles, high speed, fast controllers, RAID10, write back cache, and XFS/ZFS/ext4 not ext2/33. Write-through caches with battery backup units fordisks must be monitored, and have life span muchlonger than planned outages. Set learning cycles to offhours4. CPUs, Core less important (spend money on Memoryand IO)
  • 26. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.29Thing to do #3 –Understand Login/PrivsThe default MySQL Login and Privilegesystem is a little … primitive.Be Stingy with privs as they are hard to getbackSome tools may not give the privs you want –DROP PRIV is given by default by onepopular admin tool
  • 27. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.30Quick Security Warning!● MySQL login security is primitive.● Database mysql has users table● jsmith@co.com or fred@10.10.%– Matches host, then user, then password● Be explicit● Proxy and Pluggable logins in MySQL 5.6● joe@10.10.% may not be the same as joe@yourco.com orjoe@home.net● MySQL privilege security● GRANT functions or access● Can get Byzantine quickly● Use a GUI
  • 28. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.31MySQL Workbench – set uproles
  • 29. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.32Things to do #4 – Use MySQL Current Release● Speed– MySQL 5.5 is 20% faster than 5.1– MySQL 5.6 is ~15% faster than 5.5● Features– Better optimizer– Sub-queries– NoSQL/memcached– Online DDL
  • 30. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.33MySQL 5.6: Scalability●Users can fully utilize latest generations of hardware and OS●Scales as data volumes and users grow
  • 31. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.34Thing to do #5 – Monitor● You need somethingmonitoring yourinstances as you needa quick way to checkstatus & record events– MySQL EnterpriseMonitor– MySQL Workbench– Nagios, Cacti,phpMyAdmin, etc
  • 32. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.35Turn on logging1. Slow query log -- not all long running queriesare bad2. Log queries not using indexes3. READ the logs!!!4. Use monitoring software – MEM, Innotop,Cacti, Munin, Nagios, etc – and pay attentionto it
  • 33. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.36Just because you can not see all of the problem doesnot mean the problem is not there!
  • 34. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.37Things to do #6 – BackupsBackups are usually some sort of disk snap shotor serializing data to a fileThe more the better but you need to know stepsto recover dropped table, lost databases, ormangled data.Use data replication to a slave and then backupslave.Be paranoid!!!!!
  • 35. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.38Thing to do #7 -- ReplicationReplication for MySQL is the binary log for themaster being copied to a slave. The slavethen updates its copy of the dataTwo types:1. Asynchronous – server does not check changes sent toslave before proceeding2. Semi Synchronous – server checks that slave receivedchanges before proceeding
  • 36. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.39Replication – was messy!MySQL 5.6 is multi threaded and has GTIDs
  • 37. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.40 Simple to track & compare replication across the cluster●Unique identifier for each transaction written to the Binlog Automatically identify the most up-to-date slave for failover Deploy n-tier replication hierarchiesMasterGTID=123456GTID=123456GTID=123456 GTID=123456MySQL 5.6: ReplicationGlobal Transaction IdsEliminates the need forcomplex 3rd partysolutions
  • 38. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.41Replication -- networkNetwork latency will affect MySQL replication. So plannetwork topology to minimize bandwidth competition withother systems/services.Slaves do not need to be as fast as the master but try tokeep things reasonably closeDo not have to replicate all tables/databases to all slaves.Cut down on traffic by replicating what is needed!
  • 39. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.42Writes & Reads Reads Reads• Write to one master• Read from many slaves, easily add more as needed• Perfect for read/write intensive appsApplicationMySQL ReplicationLoad BalancerMySQL DatabaseReplication Enables Scalability
  • 40. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.43 New option: binlog-row-image=minimal Increases throughput for master and slave– Reduces Binlog size, memory & network bandwidth Only replicates elements of the Row image that have changedPrimary Key Changed ColumnsMySQL 5.6: ReplicationOptimized Row Base Replication
  • 41. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.44Before:– Transaction Data: in tables– Replication Info: in filesMySQL 5.6– Transaction Data: in tables– Replication Info: in tablesDataPosition InfoCRASH!TimeDataPosition InfoTime Automatic recovery of a slave after a failure●Binlog and table data are transactionallyconsistent Resumes replication without Dev/Opintervention●Automatically rolling back replication tolast committed eventAtomicAtomicMySQL 5.6: Crash safe SlavesEliminates risk of data loss orcorruption
  • 42. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.45Master#Slave#MySQL 5.6: Replication Event ChecksumsEliminates risk of dataloss or corruption Ensures replicated data is correct,consistent and accessible Detects corrupt replication events beforethey’re applied– Returns an error Protects entire replication path– Memory– Disk– Network– Bugs
  • 43. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.46Thing to do #8 – Use InnoDB● Transactional, speedy, crash safe, and whereOracle is concentrating development.● Online DDL, SSD Optimizations, Dump/Restorewarm buffer pool (great for cloud)● NoSQL access via memcached API
  • 44. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.47●Enables export/import of tables between running MySQL instancesMySQL 5.6: InnoDBTransportable TablespacesCREATE TABLE t(c1 INT) engine=InnoDB;FLUSH TABLE t FOR EXPORT; -- quiesce the table and create the meta data file$innodb_data_home_dir/test/t.cfgUNLOCK TABLES;Export:Import:CREATE TABLE t(c1 INT) engine=InnoDB; -- if it doesnt already existALTER TABLE t DISCARD TABLESPACE;-- The user must stop all updates on the tables, prior to the IMPORTALTER TABLE t IMPORT TABLESPACE;●Better Elasticity - Quickly spin up new instances to meet demand●Great for Cloud, Hosted, SaaS, On-premise deployments
  • 45. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.48Same app can leverage: Key-value access toInnoDB via familiarMemcached API SQL for rich queries,JOINs, FKs, etc. Fully transactionalMySQL 5.6: InnoDBNoSQL Key Value Access to InnoDB●Up to 9x performanceboost for updates●Great for fast dataingestion in Big Datapipeline
  • 46. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.49●Subquery Optimizations●File sort optimizations for most web use cases●4x better execution time – 40s to 10s Index Condition Pushdown●160x better execution time – 15s to 90ms Batched Key Access and Multi Range Read●280x better execution time – 2800s to 10sMySQL 5.6: Optimizer●Better complex query execution times ever growing data sets (Big Data!)●MEM + Query Analyzer key to utilizing full benefits of 5.6 Optimizer●MySQL Consultative Support provides guidance on configuration
  • 47. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.50Things to do #9 – IndexesRough rule – look atcolumns used injoins or after aWHERE
  • 48. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.51Indexes are goodWithout IndexDB needs to scanentire table or tablescanWith IndexDB can go right torecord
  • 49. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.52Indexes, the bad•Overhead -- space, speed, maintenance (youcan have too many)•Not a panacea – does not cure all problems•Will not help if you need to perform a tablescan•Composite indexes can be tricky – YearMonthDayusually better than DayMonthYear
  • 50. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.53Slide to check if audience is still awake●●●
  • 51. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.54Thing to do #10 – TuningSet innodb_buffer_pool_size to 70-80%of memoryTurn off query cache
  • 52. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.55Where to get help●Lots of Help Available●High Performance MySQL – Schwartz et al●MySQL Administrators Bible – Cabral●Effective MySQL series – Bradford●OurSQL podcast●Forums.MySQL.ComPlanet.MySQL.ComLocal MySQL User Group
  • 53. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.56 Optimized for Web, Cloud-based, Embedded use cases Simplified, Pluggable architecture– Maintainability, more extensible– More NoSQL options (HTTP, JSON, JavaScript, etc.) Refactoring– Data Dictionary in InnoDB– Optimizer/Parser/Protocol InnoDB– Optimized for SSD– GIS Easy HA, Replication and ShardingMySQL Database Development Priorities
  • 54. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.57MySQL Connect ConferenceMySQL Connect Conference● Sept 21st- 23rd inSanFrancisco●● The Call for Proposals(CFP) for MySQL Connect2013 just closed.Conference attendees willhear the best ideas fromMySQL experts from aroundthe globe: developers,visionaries, customers,partners. Book hotel earlydo to Americas Cup races inSan Francisco.
  • 55. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.58MySQL Marinate!Virtual self-study of MySQL through theBoston MySQL Users Group (http://www.meetup.com/mysqlbos/)http://www.meetup.com/Virtual-Tech-Self-Study/events/84103332/
  • 56. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.59Q&AQ&ADavid.Stokes@oracle.com - slideshare.net/davestokes