Congratsyourthedbatoo

412 views

Published on

Congratulations -- You're the new Linux Admin and the DBA: a guide for running MySQL database instances for Linux Admins

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
412
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
5
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Congratsyourthedbatoo

  1. 1. The Proper Care and Feeding of aMySQL Database for LinuxAdministratorsDave StokesMySQL Community ManagerDavid.Stokes@Oracle.Com
  2. 2. Congratulations! Youre the new LinuxAdmin AND the MySQL DBA too!Dave StokesMySQL Community ManagerDavid.Stokes@Oracle.Com @stoker
  3. 3. Safe Harbor StatementThe following is intended to outline our generalproduct direction. It is intended for informationpurposes only, and may not be incorporated into anycontract. It is not a commitment to deliver anymaterial, code, or functionality, and should not berelied upon in making purchasing decisions.The development, release, and timing of anyfeatures or functionality described for Oracle’sproducts remains at the sole discretion of Oracle.
  4. 4. Simple IntroductionThis is a general introduction to running aMySQL database server(s) for LinuxAdministrator
  5. 5. Simple IntroductionThis is a general introduction to running aMySQL database server(s) for LinuxAdministratorDatabase servers have needs different thanSMTP, HTTP, or other servers
  6. 6. Simple IntroductionThis is a general introduction to running aMySQL database server(s) for LinuxAdministratorDatabase servers have needs different thanSMTP, HTTP, or other serversHardware choices are critical! (do not go cheap)
  7. 7. Simple IntroductionThis is a general introduction to running aMySQL database server(s) for LinuxAdministratorDatabase servers have needs different thanSMTP, HTTP, or other serversHardware choices are critical! (do not go cheap)Tuning to 80% efficiency is relatively easy
  8. 8. Simple IntroductionThis is a general introduction to running aMySQL database server(s) for LinuxAdministratorDatabase servers have needs different thanSMTP, HTTP, or other serversHardware choices are critical! (do not go cheap)Tuning to 80% efficiency is relatively easy (last20% is tricky)
  9. 9. Session Overview1. Basics of a database server2. Hardware3. MySQL Configuration4. Monitoring Operations5. Backups6. Replication7. Indexes8. Tuning9. Q/A
  10. 10. How does a Database server workClient ServerSELECT phoneFROM friendsWHERE name = ‘Joe’;
  11. 11. How does a Database server workClient ServerSELECT phoneFROM friends PARSEWHERE name = ‘Joe’; find Joe in friends table in memory return phone
  12. 12. How does a Database server workClient ServerSELECT phoneFROM friends PARSEWHERE name = ‘Joe’; find Joe in friends... table in memoryProcess phone data return phone
  13. 13. How does a Database server workClient ServerSELECT phoneFROM friends PARSEWHERE name = ‘Joe’; find Joe in friends... table in memoryProcess phone data return phone What was that about memory???
  14. 14. Rule #1• Databases love data in memory
  15. 15. Rule #1• Databases love data in memoryCorollary #1 – getting data in/out of memory willcause you nightmares!
  16. 16. What if it is not in memory?MySQL OSPlease give me thedata from the citytable
  17. 17. What if it is not in memory?MySQL OSPlease give me the Get inodedata from the citytable
  18. 18. What if it is not in memory?MySQL OSPlease give me the Get inodedata from the city Ask disk for datatable
  19. 19. What if it is not in memory?MySQL OSPlease give me the Get inodedata from the city Ask disk for datatable Get data into buffer
  20. 20. What if it is not in memory?MySQL OSPlease give me the Get inodedata from the city Ask disk for datatable Get data into buffer Hand buffer offLoad data into memory
  21. 21. What if it is not in memory?MySQL OSPlease give me the Get inodedata from the city Ask disk for datatable Get data into buffer Hand buffer off Much longer thanLoad data into just reading from memory memory
  22. 22. Rule #2Databases have to do unpredictable queries,random I/O, and sequential scans so slow I/Okills performance
  23. 23. BuffersWhat happens when you read a file into memory Disk Page User DISK Controller Cache Buffer
  24. 24. BuffersMemory Lookup – 100 nanoseconds, 12GB/sec Disk Page User DISK Controller Cache Buffer
  25. 25. BuffersMemory Lookup – 100 nanoseconds, 12GB/secDISK seek – 10 milliseconds, 760MB/sec Disk Page User DISK Controller Cache Buffer
  26. 26. Disk ReadsA disk read is 100,000 slower than reading memory ● For comparison – 100,000 minutes is about 69.5 days – 100,000 feet is about 19 miles – 100,000 kilometers is 15.7 times around Earths equator
  27. 27. CacheWriteback and writethrough caches Disk DISK Controller Writethrough on disk Writethrough on controller
  28. 28. CacheRecommended setup Disk DISK Controller Writethrough on disk Writeback on controller
  29. 29. SSDUse standard RAID controller ● SSD as writeback cache ● Battery backed ● $2.5/GB verses .075/GB ● Very fast seek time ● Density
  30. 30. Hardware recommendations1. Memory – lots of it, ecc
  31. 31. Hardware recommendations1. Memory – lots of it, ecc2. DISKs – more spindles, high speed, fast controllers, RAID 10, write back cache, and XFS/ZFS/ext4 not ext2/3
  32. 32. Hardware recommendations1. Memory – lots of it, ecc2. DISKs – more spindles, high speed, fast controllers, RAID 10, write back cache, and XFS/ZFS/ext4 not ext2/33. Write-through caches with battery backup units for disks must be monitored, and have life span much longer than planned outages
  33. 33. Hardware recommendations1. Memory – lots of it, ecc2. DISKs – more spindles, high speed, fast controllers, RAID 10, write back cache, and XFS/ZFS/ext4 not ext2/33. Write-through caches with battery backup units for disks must be monitored, and have life span much longer than planned outages4. CPUs, Core less important (spend money on Memory and IO)
  34. 34. Quick Security Warning!MySQL login security is primitive. ● Database mysql has users table ● jsmith@co.com or fred@10.10.% – Matches host, then user, then password fields ● Be explicit ● Proxy and Pluggable logins on the wayMySQL privilege security ● GRANT functions or access ● Can get Byzantine quickly ● Use a GUI
  35. 35. When root is the root of your rootproblem OR stingy is your best friendLinux server has root MySQL daemon has ● Highly privileged root ● Dangerous ● Highly privileged ● Dangerous ● Understand MySQL priv system and be stingy ● Proxy and plug-in adapters soon ● Really limit Linux root is not the same as MySQL access, not just root but both can be dangerous to you!!! play at it
  36. 36. Installation1. Use pre built MySQL packages
  37. 37. Installation1. Use pre built MySQL packages2. Don’t double up with other services
  38. 38. Installation1. Use prebuilt MySQL packages2. Don’t double up with other services3. Supplied configuration files are OLD!
  39. 39. Installation1. Use prebuilt MySQL packages2. Don’t double up with other services3. Supplied configuration files are OLD!4. Move logs to different disk than data
  40. 40. Installation1. Use prebuilt packages2. Don’t double up with other services3. Supplied configuration files are OLD!4. Move logs to different disk than data5. Spread data over different drives
  41. 41. Installation1. Use prebuilt packages2. Don’t double up with other services3. Supplied configuration files are OLD!4. Move logs to different disk than data5. Spread data over different drives6. Backups are necessary – and practice recovery!
  42. 42. Monitoring Operations1. Slow query log -- not all long running queries are bad
  43. 43. Monitoring Operations1. Slow query log -- not all long running queries are bad2. Log queries not using indexes
  44. 44. Monitoring Operations1. Slow query log -- not all long running queries are bad2. Log queries not using indexes3. Use monitoring software – MEM, Innotop, Cacti, Munin, Nagios, etc – and pay attention to it
  45. 45. Monitor!!!
  46. 46. Monitoring Operations1. Slow query log -- not all long running queries are bad2. Log queries not using indexes3. Use monitoring software – MEM, Innotop, Cacti, Munin, Nagios, etc – and pay attention to it4. More in tuning ….
  47. 47. BackupsBackups are usually some sort of disk snap shot or serializing data to a file
  48. 48. BackupsBackups are usually some sort of disk snap shot or serializing data to a fileThe more the better but you need to know steps to recover dropped table, lost databases, or mangled data
  49. 49. BackupsBackups are usually some sort of disk snap shot or serializing data to a fileThe more the better but you need to know steps to recover dropped table, lost databases, or mangled data.Use data replication to a slave and then backup slave
  50. 50. BackupsBackups are usually some sort of disk snap shot or serializing data to a fileThe more the better but you need to know steps to recover dropped table, lost databases, or mangled data.Use data replication to a slave and then backup slaveBe paranoid!!!!!
  51. 51. ReplicationReplication for MySQL is the binary log for the master being copied to a slave. The slave then updates its copy of the data
  52. 52. ReplicationReplication for MySQL is the binary log for the master being copied to a slave. The slave then updates its copy of the dataTwo types:1. Asynchronous – server does not check changes sent to slave before proceeding
  53. 53. ReplicationReplication for MySQL is the binary log for the master being copied to a slave. The slave then updates its copy of the dataTwo types:1. Asynchronous – server does not check changes sent to slave before proceeding2. Semi Synchronous – server checks that slave received changes before proceeding
  54. 54. Replication -- threadsCurrently single threaded – 5.6 will fix that
  55. 55. Replication -- networkNetwork latency will affect MySQL replication. So plannetwork topology to minimize bandwidth competition withother systems/services.
  56. 56. Replication -- networkNetwork latency will affect MySQL replication. So plannetwork topology to minimize bandwidth competition withother systems/services.Slaves do not need to be as fast as the master but try tokeep things reasonably close
  57. 57. Replication -- networkNetwork latency will affect MySQL replication. So plannetwork topology to minimize bandwidth competition withother systems/services.Slaves do not need to be as fast as the master but try tokeep things reasonably closeDo not have to replicate all tables/databases to all slaves.Cut down on traffic by replicating what is needed!
  58. 58. MySQL DatabaseReplication Enables Scalability Application Load Balancer Writes & Reads Reads Reads MySQL Replication • Write to one master • Read from many slaves, easily add more as needed • Perfect for read/write intensive apps
  59. 59. Indexes are goodWithout Index With Index DB needs to scan DB can go right toentire table or table recordscan
  60. 60. Indexes, the bad• Overhead -- space, speed, maintenance
  61. 61. Indexes, the bad• Overhead -- space, speed, maintenance• Not a panacea – does not cure all problems
  62. 62. Indexes, the bad• Overhead -- space, speed, maintenance• Not a panacea – does not cure all problems• Will not help if you need to perform a table scan
  63. 63. Indexes, the bad• Overhead -- space, speed, maintenance• Not a panacea – does not cure all problems• Will not help if you need to perform a table scan• Composite indexes can be tricky – YearMonthDay usually better than DayMonthYear
  64. 64. Tuning to 80%• Use InnoDB
  65. 65. Tuning to 80%• Use InnoDB• Set innodb_buffer_pool_size to 70- 80% of memory
  66. 66. Tuning to 80%• Use InnoDB• Set innodb_buffer_pool_size to 70- 80% of memory• Use XFS/ZFS/ext4
  67. 67. Tuning to 80%• Use InnoDB• Set innodb_buffer_pool_size to 70- 80% of memory• Use XFS/ZFS/ext4• Partition data -- divide and conquer
  68. 68. Tuning to 80%• Use InnoDB• Set innodb_buffer_pool_size to 70- 80% of memory• Use XFS/ZFS/ext4• Partition data -- divide and conquer• Architect your data
  69. 69. Tuning to 80%• Use InnoDB• Set innodb_buffer_pool_size to 70- 80% of memory• Use XFS/ZFS/ext4• Partition data -- divide and conquer• Architect your data• Review your SQL statements
  70. 70. Dont hurt yourself Some RAID controllers have a battery learning cycle when BBU goes write-back. Schedule during off-time! Minimize time for most frequent queries Keep on top of upgrades - 5.5 20% faster than 5.1
  71. 71. Tuning Past 80%Will depend on your data
  72. 72. Tuning Past 80%Will depend on your dataLots of Tuning Help Available
  73. 73. Tuning Past 80%Will depend on your dataLots of Tuning Help Available ● High Performance MySQL – Schwartz et al ● MySQL Administrators Bible – Cabral
  74. 74. Tuning Past 80%Will depend on your dataLots of Tuning Help Available ● High Performance MySQL – Schwartz et al ● MySQL Administrators Bible – Cabral ● Effective MySQL series – Bradford ● OurSQL podcast
  75. 75. Tuning Past 80%Will depend on your dataLots of Tuning Help Available ● High Performance MySQL – Schwartz et al ● MySQL Administrators Bible – Cabral ● Effective MySQL series – Bradford ● OurSQL podcast ● Performance forum on Forums.MySQL.Com ● Planet.MySQL.com & MySQL.Com
  76. 76. Tuning Past 80%Will depend on your dataLots of Tuning Help Available ● High Performance MySQL – Schwartz et al ● MySQL Administrators Bible – Cabral – Effective MySQL series – Bradford ● OurSQL podcast ● Performance forum on Forums.MySQL.Org ● Planet.MySQL.com & MySQL.comSkilled DBA
  77. 77. Tuning Past 80%Will depend on your dataLots of Tuning Help Available ● High Performance MySQL – Schwartz et al ● MySQL Administrators Bible – Cabral ● OurSQL podcast ● Performance forum on Forums.MySQL.Org ● Planet.MySQL.COM & MySQL.ComSkilled DBA ● Defined as obsessive professional looking to save nanoseconds off queries, possess current backups, helps developers rewrite queries, plans for future, and watches buffer hits rates compulsively.
  78. 78. Big hintSeek to optimize the system as a whole. Often thedatabase is blamed for systemic slowness whenother components are at fault.
  79. 79. General Last hintsSQL Mysqld ● Fetch needed data ● Pay attention to only, no SELECT * new technologies, ● Use EXPLAIN updates ● Think in data sets, not ● Know which nested loops buffers are per- session, global ● Set benchmark times ● Do not ignore log ● Use smallest data files type possible ● Rewrite subqueries to joins
  80. 80. MySQL Connect Conference st The Call for Proposals (CFP)Sept 21 - for MySQL Connect 2013 is rd now open. Conference23 in San attendees want to hear the best ideas from MySQLFrancisco experts from around the globe: developers, visionaries, customers, partners. Proposals must be submitted by April 12, so don’t wait to develop the content that only you know— and that MySQL attendees want to hear.
  81. 81. MySQL Marinate! Virtual self-study of MySQL through the Boston MySQL Users Group ( http://www.meetup.com/mysqlbos/) http://www.meetup.com/Virtual-Tech-Self- Study/events/84103332/
  82. 82. Q&ADavid.Stokes@oracle.com - slideshare.net/davestokes

×