Mountain Moodle Moot -- MySQL Server & Query Tuing
Upcoming SlideShare
Loading in...5
×
 

Mountain Moodle Moot -- MySQL Server & Query Tuing

on

  • 237 views

Presentation from Mountain Moodle Moot, Helena MT

Presentation from Mountain Moodle Moot, Helena MT

Statistics

Views

Total Views
237
Views on SlideShare
211
Embed Views
26

Actions

Likes
2
Downloads
12
Comments
1

2 Embeds 26

https://twitter.com 25
https://www.linkedin.com 1

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
  • You know a presentation will be good when the 2nd slide contains:

    '
    Databases Can Be a Mystery, Misery,or Magnificent

    ● You do not need to be a DBA
    ● Nothing wrong with being a DBA
    ● You should think in sets
    ● 'Relational Calculus for $500, Alex.'
    ● Let the Database do the Heavy Lifting
    ● Select What you need for speed
    ● Yes, YOU do need to do
    '

    Thanks
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Mountain Moodle Moot -- MySQL Server & Query Tuing Mountain Moodle Moot -- MySQL Server & Query Tuing Presentation Transcript

  • Query Tuning, System Tuning, and Little Known Tricks Using MySQL Workbench David.Stokes@Oracle.com @stoker Slideshare.net/davestokes Mountain Moodle Moot
  • Databases Can Be a Mystery, Misery, or Magnificent ● You do not need to be a DBA ● Nothing wrong with being a DBA ● You should think in sets ● 'Relational Calculus for $500, Alex.' ● Let the Database do the Heavy Lifting ● Select What you need for speed ● Yes, YOU do need to do backups ● And know how to restore from them
  • Server Tuning  Run latest release of MySQL  Better performance  Bug Fixes  New features  Don't skimp on hardware  Memory is better ROI than disk  New SSDs are amazing  FusionIO cards OMG
  • Configuration  Configuration should match hardware  Remove Query Cache  Nasty Mutex  But what if you have 'repeatable data', i.e. sports score  Memcached  In Application  Know your parameters  Session, Global  Needs reboot to change or on the fly
  • I_S, P_S, SYS  Information_Schema  Performance_Schema  Similar to V$ variables for Oracle  2-5% performance overhead  SYS  Predefined for most common questions You will not need them (now) but know that they are there when you need to eek out last %
  • Intro to Query Tuning  Reading from disk to memory is 100,000 slower than memory alone  'SELECT *' on a wide table with many col- umns when you need only two columns MUL- TIPLED by N thousands access a minute adds up  Read what you need for speed, no SELECT *  Use joins to get desired sets
  • Our First Query SELECT City.Name, Country.name, City.Population FROM City JOIN Country ON (Country.code = City.CountryCode)
  • Results
  • Our First Query SELECT City.Name, Country.name, City.Population FROM City JOIN Country ON (Country.code = City.CountryCode) Columns Selected (note Name & Name)
  • Our First Query SELECT City.Name, Country.name, City.Population FROM City JOIN Country ON (Country.code = City.CountryCode) Tables used
  • Venn Diagram  City is A, Country is B
  • Make query match diagram SELECT a.Name, b.name, a.Population FROM City AS a JOIN Country AS b ON (b.code = a.CountryCode)
  • Make query match diagram SELECT a.Name, b.name, a.Population FROM City a JOIN Country b ON (b.code = a.CountryCode) Can also use AS to alias column names
  • Aliased Column Names SELECT a.Name as 'City', b.name as 'Country' , a.Population as 'Pop.' FROM City a JOIN Country b ON (b.code = a.CountryCode)
  • Updated Results
  • Query Plan
  • EXPLAIN  EXPLAIN is pre-pended to a SQL statement and produces output like:
  • EXPLAIN  We see how many roads to be read: Guess on rows needing to be read 4079 x 1 = 4079 rows to be read
  • EXPLAIN  In English – We read all the rows in the City table and find via an index to find the match in the Country table:
  • So what is an index?  http://en.wikipedia.org/wiki/Database_index A database index is a data structure that improves the speed of data retrieval operations on a database table at the cost of additional writes and the use of more storage space to main- tain the extra copy of data. Indexes are used to quickly locate data without having to search every row in a database table every time a database table is accessed. Indexes can be cre- ated using one or more columns of a database table, provid- ing the basis for both rapid random lookups and efficient ac- cess of ordered records.
  • Uh, in English  Indexes let you go directly to the record you want  Rolodex  Card Catalog  Overhead  Maintenance  Sometimes you DO have read all the table
  • B-Tree Index
  • A Peek at an Index  SHOW INDEX FROM <table>;
  • Statistics  Optimizer uses table statistics to generate cost data.  Out of data stats can make optimizer plan poorly  See 14.13.17.1 Con- figuring Persistent Optimizer Statistics Parameters in MySQL Manual  Run ANALYSE TA- BLE to update stats  Run ANALYSE TA- BLE after adding a new index  Store stat data at SHUTDOWN, Re- store at REBOOT
  • Lets look at the tables  EER Map of tables in World Database
  • Just City and Country  The two tables we have been using
  • Just City and Country JOIN Country b ON (b.code = a.CountryCode)
  • Without Indexes  Plural of Moose  List of definitions not alphabetized  Duplicates
  • Compound Index CREATE TABLE test ( id INT NOT NULL, last_name CHAR(30) NOT NULL, first_name CHAR(30) NOT NULL, PRIMARY KEY (id), INDEX name (last_name,first_name) ); This index can use be used for searching on last_name + first_name or just last_name
  • Multi Column Index  CREATE INDEX CityStateZip ON City (City, State, Zip);  This index can be used to search on City + State + Zip, City + State, or City  This index can not be used to search on Zip, State + Zip  Speeds up finding Zip when used in City + State
  • Why is it Important to Optimize?  MySQL, unlike other databases, has no way to lock a query plan. That means each time a query is submitted it goes through the opti- mizer.
  • Optimize Data  Ideally you want to allocate just enough stor- age for any possible record.  Have to use best judgment  Human age range 0-999  Postal codes  Address lines  Phone numbers 01-123-123-1234 and extension ?!  PROCEDURE ANALYSE will examine records, make suggestion  Not all Integers BIGINTs  Not all text fields VARCHAR(255)  Excess column widths are extra baggage
  • Integers Type Storage (bytes) Minimum Signed Maximum Signed Minimum Unsigned Maximum Unsigned TINYINT 1 -128 127 0 255 SMALLINT 2 -32768 32768 0 65535 MEDIUMIN T 3 -8388608 8388607 0 16777215 INT 4 -2147483648 2147483647 0 4294967295 BIGINT 8 - 9223372036854 775808 9223372036854 775807 0 1844674407370 9551615 Are you really going to have 18,446,744,073,709,551,615 student id numbers?
  • Making it up in volume  Consider the worst case  Data needs to be read off disk  Transferred into memory  Sent across network to application server  Application server loads data into memory  Application sends that data over network to end user  INT VS BIGINT – 4 extra bytes each times X number of records X number of users
  • Flexibility  Making changes to a schema is costly. It is a lot better for 5.6  Much better for 5.7  Must update replication masters and slaves (double check if done through replication)  Gender Example 2, 3, 17, or 58+
  • Replication  Changes made to master  Replicated to slave server(s)  Three threads  BINLOG dump thread  Slave I/O thread  Slave SQL thread
  • Read/Write Splits  Writes to Master  Reads from slave(s)
  • Types of Replication  Statement Based – SQL  Row Based – Binary changes after SQL exe- cuted  Mixed – Uses Statement Based but switches when advantageous
  • Smarter Row Based Replication  MySQL 5.6 and later  No need to copy unmodified parts of a row
  • Global Transaction ID  Previously had to match Master's log position to when starting replication  Each transaction unique  Point slave at master and ask it to catch up
  • MySQL Utilities  Free, open sources Python scripts to auto- mate functions  mysqldbcompare  Compare databases on two servers or the same server  Compare definitions and data  Generate a difference report  Generate SQL transformation statements  mysqldbcopy  Copy databases between servers  Clone databases on the same server  Supports rename  mysqldbexport  Export metadata and/or data from one or more databases  Formats: SQL, CSV, TAB, Grid, Vertical  mysqldbimport  Import metadata and data from one or more files  Reads all formats from mysqldbexport  mysqldiff  Compare object definitions  Generate a difference report
  •  These utilities are those designed to perform general operations such as reporting and searching.  mysqldiskusage  Show disk usage for databases  Generate reports in SQL, CSV, TAB, Grid, Vertical  mysqlfrm  Reads .frm files, optionally in byte-by-byte diagnostic mode  Generates CREATE statements from table definition data  mysqlindexcheck  Read indexes for one or more tables  Check for redundant and duplicate indexes  Generate reports in SQL, CSV, TAB, Grid, Vertical  mysqlmetagrep  Search metadata  Regexp, database search  Generate SQL statement for search query  mysqlprocgrep  Search process information  Generate SQL statement for search  Kill processes that match query  mysqluserclone  Clone a user account, to the same or different server  Show user grants  mysqluc  Command line client for running MySQL Utilities  Allows a persistent connection to a MySQL Server  Tab completion for utility names and options  Allows calling the commands with shorter names, such as using "serverinfo" instead of mysqlserverinfo
  •  These utilities are those designed to support replication and high availability operations for MySQL servers.  mysqlfailover  Provides automatic failover on a replication topology  Uses Global Transaction Identifiers (GTID, MySQL Server 5.6.5+)  mysqlreplicate  Setup replication  Start from beginning, current, specific binlog, pos  mysqlrplms  Provides round-robin multi-source replication (a slave server continually cycles through multiple masters in order to store a consolidated data set)  Uses Global Transaction Identifiers (GTID, MySQL Server 5.6.9+)  mysqlrpladmin  Administers the replication topology  Allows recovery of the master  Commands include elect, failover, gtid, health, start, stop, and switchover  mysqlrplcheck  Check replication configuration  Tests binary logging on master  mysqlrplshow  Show slaves attached to master  Can search recursively  Show the replication topology as a graph or list  mysqlrplsync  Check data consistency between servers in a replicated setup  Uses Global Transaction Identifiers (GTID)  Requires MySQL Server 5.6.14 and higher
  • MySQL Fabric  Also written in Python and free  Sharding  High Availability
  • Exercise  Mysqldbcopy –source=root:pass@localhost --destination=root:pass@localhost world:xworld  Use MySQL Utility mysqldbcopy to make a copy of world database
  • Exercise  Using world database, run query from earlier to see query plan. SELECT a.Name as 'City', b.name as 'Country' , a.Population as 'Pop.' FROM City a JOIN Country b ON (b.code = a.CountryCode);
  • Remove Indexes  Use worldx schema which redefines the World database without indexes
  • Exercise  Using worldx database again, run query from earlier to see query plan to see cost. SELECT a.Name as 'City', b.name as 'Country' , a.Population as 'Pop.' FROM City a JOIN Country b ON (b.code = a.CountryCode)
  • Back to World How does a 'qualifier' effect query plan?  SELECT a.Name as 'City', b.name as 'Country' , a.Population as 'Pop.' FROM City a JOIN Country b ON (b.code = a.CountryCode) WHERE a.Population > 500000;
  • Back to World Again How does a second 'qualifier' effect query plan?  SELECT a.Name as 'City', b.name as 'Country' , a.Population as 'Pop.' FROM City a JOIN Country b ON (b.code = a.CountryCode) WHERE a.Population > 500000 ORDER By b.name;
  • Run those last two queries on worldx  Did indexes help/hurt?  Are the qualifiers run before/after join?
  • Moodle 2.7  A quick look at the schema
  • More Info on Tuning  High Performance MySQL - 3rd Edition  Schwartz, Zaitsev, & Tkachenko  Effective MySql Optimizing Sql Statements  Bradford
  • MySQL Marinate  Free on-line Virtual class  Boston MySQL Users Group  One Chapter a week  Homework Checked by Mozilla's senior DBA
  • MySQL Training from Oracle  Introduction to MySQL  MySQL Quickstart Fundamentals  MySQL for Beginners  MySQL for Database Administrators  MySQL for Developers  MySQL Performance Tuning  MySQL High Availability  MySQL Cluster  MySQL Developer Techniques  MySQL and PHP - Developing Dynamic Web Applica- tions  MySQL Advanced Stored Procedures
  • MySQL Central @ Oracle Open World  Keynotes from Booking.com, Twitter & Dropbox  50+ session, tutorials, BOFs and reception
  • Q/A  David.Stokes@oracle.com  @stoker  MySQL Central at Oracle Open World San Francisco Septermber 29th to October 3rd