Performance Tuning Best Practices

                                     Jay Pipes
                     Community Relations...
MySQL: The World’s Most Popular
                 Open Source Database

     Founded in 1995; operations in 23 countries
  ...
Second Generation Open Source
      •    MySQL AB is a profitable company
            – Develops the software in-house; co...
Overview
   ●    Profiling and Benchmarking Concepts
   ●    Sources of Problems
   ●    Indexing Guidelines
   ●    Schem...
Benchmarking Concepts
   ●    Provides a track record of changes
         ➢ Baseline is the starting point

         ➢ Tes...
Benchmarking Tips
      ●     Always give yourself a target
      ●     Record everything
          ✔   Schema dump
      ...
Benchmarking Toolbox
   ●    SysBench
         ➢ http://sysbench.sourceforge.net/


   ●    mysqlslap (5.1+)
         ➢ ht...
Profiling Concepts
   ●    Diagnose a running system
   ●    Low hanging fruit
         ➢ Diminishing returns

         ➢ ...
Profiling Toolbox
   ●    SHOW Commands
         ➢   SHOW PROCESSLIST | STATUS | INNODB STATUS
         ➢http://dev.mysql....
Slow Query Log
   ●    Slow Query Log
          ●   log_slow_queries=/var/lib/mysql/slow­queries.log
          ●   long_qu...
Profiling Tips
   ●    Get very familiar with EXPLAIN
         ➢ Access types

         ➢ Learn the type, key, ref, rows, ...
Sources of Problems
   ●    Poor or nonexistent indexing
   ●    Inefficient or bloated schema design
   ●    Bad SQL Codi...
Indexing Guidelines
      ●   Poor or missing index fastest way to kill a system
      ●   Ensure good selectivity on fiel...
Common Index Problem

                                               // This top query uses the index
    CREATE TABLE Tag...
Common Index Problem Solved


    CREATE TABLE Tags (
      tag_id INT NOT NULL AUTO_INCREMENT
    , tag_text VARCHAR(50) ...
Schema Guidelines
      ●   Inefficient schema another great way to kill
          performance
      ●   Use the smallest ...
Schema Tips
      ●   Consider horizontally splitting many-columned tables
          (example ahead)
      ●   Consider ve...
Horizontal Partitioning Example
                                              CREATE TABLE Users (
                       ...
Horizontal Partitioning Benefits
      ●   Main table has narrow rows, so...
          ✔ More records fit into a single da...
Counter Table Example

                                               CREATE TABLE Products (
                            ...
Counter Table Benefits
      ●   Critical for InnoDB because of complications of MVCC
      ●   Allows query cache to cach...
Schema Tips (cont'd)
      ●   Ensure small clustering key (InnoDB)
      ●   Don't use surrogate keys when a naturally oc...
Coding Guidelines
      ●   Use “chunky” coding habits (KISS)
      ●   Use stored procedures for a performance boost (5.0...
Isolating Indexed Fields Example
      ✔   Task: get the Order ID, date of order, and Customer ID
          for all orders...
Calculated Fields Example
      ✔   Task: search for top-level domain in email addresses

                                ...
Correlated Subquery Conversion Example
      ✔   Task: convert a correlated subquery in the SELECT
          clause to a s...
Derived Table Example
      ✔   Task: convert a correlated subquery in the WHERE
          clause to a standard join on a ...
Server Parameters
      ●   Be aware of what is global vs per thread
      ●   Make small changes, then test
      ●   Oft...
Additional Resources
      ✔   http://www.mysqlperformanceblog.com/
          ➢ Peter Zaitsev's blog – Excellent material
...
THANK YOU!

      ●   Please email me:
          ➢ Success stories

          ➢ War stories

          ➢ Inventive uses of...
Upcoming SlideShare
Loading in...5
×

Perf Tuning Best Practices

2,482

Published on

Best way for tunning

Published in: Technology
0 Comments
8 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
2,482
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
158
Comments
0
Likes
8
Embeds 0
No embeds

No notes for slide

Transcript of "Perf Tuning Best Practices"

  1. 1. Performance Tuning Best Practices Jay Pipes Community Relations Manager, North America (jay@mysql.com) TELECONFERENCE: please dial a number to hear the audio portion of this presentation. Toll-free US/Canada: 866-469-3239 Direct US/Canada: 650-429-3300 : 0800-295-240 Austria 0800-71083 Belgium 80-884912 Denmark 0-800-1-12585 Finland 0800-90-5571 France 0800-101-6943 Germany 00800-12-6759 Greece 1-800-882019 Ireland 800-780-632 Italy 0-800-9214652 Israel 800-2498 Luxembourg 0800-022-6826 Netherlands 800-15888 Norway 900-97-1417 Spain 020-79-7251 Sweden 0800-561-201 Switzerland 0800-028-8023 UK 1800-093-897 Australia 800-90-3575 Hong Kong 00531-12-1688 Japan EVENT NUMBER/ACCESS CODE: 921155876 Copyright MySQL AB The World’s Most Popular Open Source Database 1
  2. 2. MySQL: The World’s Most Popular Open Source Database Founded in 1995; operations in 23 countries Fastest growing relational database Over 8,000,000 installations; 40,000 downloads / day Dramatically reduces Total Cost of Ownership (TCO) Used by leading IT organizations and ISVs Copyright 2005 MySQL AB The World’s Most Popular Open Source Database 2
  3. 3. Second Generation Open Source • MySQL AB is a profitable company – Develops the software in-house; community helps test it – Owns source code, copyrights and trademarks – Targets the “commoditized” market for databases • “Quid Pro Quo” dual licensing for OEM market – Open source GPL license for open source projects – Cost-effective commercial licenses for commercial use • Annual MySQL Network subscription for Enterprise and Web – Per server annual subscription – Includes support, alert and update advisors, Knowledge Base, Certified/Optimized Binaries • MySQL supports it users – Worldwide 24 x 7 support – Training and certification – Consulting “Reasoning's inspection study shows that the code quality of MySQL was six times better than that of comparable proprietary code. ” Reasoning Inc. Copyright MySQL AB The World’s Most Popular Open Source Database 3
  4. 4. Overview ● Profiling and Benchmarking Concepts ● Sources of Problems ● Indexing Guidelines ● Schema Guidelines ● Coding Guidelines ● Server Parameters Copyright MySQL AB The World’s Most Popular Open Source Database 4
  5. 5. Benchmarking Concepts ● Provides a track record of changes ➢ Baseline is the starting point ➢ Testing done iteratively ➢ Deltas between tests show difference that the change(s) made ● Stress/Load testing of application and/or database ● Harness or framework useful to automate many benchmark tasks Copyright MySQL AB The World’s Most Popular Open Source Database 5
  6. 6. Benchmarking Tips ● Always give yourself a target ● Record everything ✔ Schema dump ✔ my.cnf files ✔ hardware/os configuration files as needed ● Isolate the problem ✔ Shut down unnecessary programs ✔ Stop network traffic to machine ✔ Disable the query cache ✔ Change one thing at a time Copyright MySQL AB The World’s Most Popular Open Source Database 6
  7. 7. Benchmarking Toolbox ● SysBench ➢ http://sysbench.sourceforge.net/ ● mysqlslap (5.1+) ➢ http://dev.mysql.com/doc/refman/5.1/en/mysqlslap.html ● Apache Bench (ab) ● supersmack ➢ http://www.vegan.net/tony/supersmack/ ● MyBench ● http://jeremy.zawodny.com/mysql/mybench/ Copyright MySQL AB The World’s Most Popular Open Source Database 7
  8. 8. Profiling Concepts ● Diagnose a running system ● Low hanging fruit ➢ Diminishing returns ➢ Be careful not to over-optimize ● Identify performance bottlenecks in ➢ Memory ➢ CPU ➢ I/O (Disk) ➢ Network and OS Copyright MySQL AB The World’s Most Popular Open Source Database 8
  9. 9. Profiling Toolbox ● SHOW Commands ➢ SHOW PROCESSLIST | STATUS | INNODB STATUS ➢http://dev.mysql.com/show ● EXPLAIN ➢ http://dev.mysql.com/explain ● MyTop ➢ http://jeremy.zawodny.com/mysql/mytop/ ● Whole host of Linux power tools ➢ gprof / oprofile ➢ vmstat / ps / top / mpstat / procinfo ● apd for PHP developers ➢ http://pecl.php.net/package/apd Copyright MySQL AB The World’s Most Popular Open Source Database 9
  10. 10. Slow Query Log ● Slow Query Log ● log_slow_queries=/var/lib/mysql/slow­queries.log ● long_query_time=2 ● Use mysqldumpslow ● (5.1+) Can log directly to a table, plus does not require restart of server ● SET GLOBAL SLOW_QUERY_LOG = { ON | OFF } ● http://dev.mysql.com/doc/refman/5.1/en/log- tables.html Copyright MySQL AB The World’s Most Popular Open Source Database 10
  11. 11. Profiling Tips ● Get very familiar with EXPLAIN ➢ Access types ➢ Learn the type, key, ref, rows, Extra columns ● Low hanging fruit (diminishing returns) ● Use MyTop to catch locking and long-running queries in real-time Copyright MySQL AB The World’s Most Popular Open Source Database 11
  12. 12. Sources of Problems ● Poor or nonexistent indexing ● Inefficient or bloated schema design ● Bad SQL Coding Practices ● Server variables not tuned properly ● Hardware and/or network bottlenecks Copyright MySQL AB The World’s Most Popular Open Source Database 12
  13. 13. Indexing Guidelines ● Poor or missing index fastest way to kill a system ● Ensure good selectivity on field ● Look for covering index opportunities ● On multi-column indexes, pay attention to the order of the fields in the index (example ahead) ● As database grows, examine distribution of values within indexed field ● Remove redundant indexes for faster write performance Copyright MySQL AB The World’s Most Popular Open Source Database 13
  14. 14. Common Index Problem // This top query uses the index CREATE TABLE Tags ( // on Products2Tags   tag_id INT NOT NULL AUTO_INCREMENT , tag_text VARCHAR(50) NOT NULL SELECT p.name , PRIMARY KEY (tag_id) , COUNT(*) as tags ) ENGINE=MyISAM; FROM Products2Tags p2t INNER JOIN Products p CREATE TABLE Products ( ON p2t.product_id = p.product_id   product_id INT NOT NULL AUTO_INCREMENT GROUP BY p.name; , name VARCHAR(100) NOT NULL // many more fields... // This one does not because , PRIMARY KEY (product_id) // index order prohibits it ) ENGINE=MyISAM;  SELECT t.tag_text CREATE TABLE Products2Tags ( , COUNT(*) as products   product_id INT NOT NULL FROM Products2Tags p2t , tag_id INT NOT NULL INNER JOIN Tags t , PRIMARY KEY (product_id, tag_id) ON p2t.tag_id = t.tag_id ) ENGINE=MyISAM; GROUP BY t.tag_text; Copyright MySQL AB The World’s Most Popular Open Source Database 14
  15. 15. Common Index Problem Solved CREATE TABLE Tags (   tag_id INT NOT NULL AUTO_INCREMENT , tag_text VARCHAR(50) NOT NULL , PRIMARY KEY (tag_id) ) ENGINE=MyISAM; CREATE INDEX ix_tag  ON Products2Tags (tag_id); CREATE TABLE Products (   product_id INT NOT NULL AUTO_INCREMENT // or... create a covering index: , name VARCHAR(100) NOT NULL // many more fields... CREATE INDEX ix_tag_prod , PRIMARY KEY (product_id) ON Products2Tags (tag_id, product_id); ) ENGINE=MyISAM;  // But, only if not InnoDB... why? CREATE TABLE Products2Tags (   product_id INT NOT NULL , tag_id INT NOT NULL , PRIMARY KEY (product_id, tag_id) ) ENGINE=MyISAM; Copyright MySQL AB The World’s Most Popular Open Source Database 15
  16. 16. Schema Guidelines ● Inefficient schema another great way to kill performance ● Use the smallest data types necessary ➢ Do you really need that BIGINT? ● Normalize first, denormalize only in extreme cases Copyright MySQL AB The World’s Most Popular Open Source Database 16
  17. 17. Schema Tips ● Consider horizontally splitting many-columned tables (example ahead) ● Consider vertically partitioning many-rowed tables ➢ Merge tables (MyISAM only) ➢ Homegrown ➢ Partitioning (5.1+) ● Fewer fields = Narrow rows = More records per block ● Use “counter” tables to mitigate query cache issues (example ahead) ➢ Essential for InnoDB Copyright MySQL AB The World’s Most Popular Open Source Database 17
  18. 18. Horizontal Partitioning Example CREATE TABLE Users (   user_id INT NOT NULL AUTO_INCREMENT , email VARCHAR(80) NOT NULL CREATE TABLE Users ( , display_name VARCHAR(50) NOT NULL   user_id INT NOT NULL AUTO_INCREMENT , password CHAR(41) NOT NULL , email VARCHAR(80) NOT NULL , PRIMARY KEY (user_id) , display_name VARCHAR(50) NOT NULL , UNIQUE INDEX (email) , password CHAR(41) NOT NULL ) ENGINE=InnoDB; , first_name VARCHAR(25) NOT NULL , last_name VARCHAR(25) NOT NULL CREATE TABLE UserExtra ( , address VARCHAR(80) NOT NULL   user_id INT NOT NULL , city VARCHAR(30) NOT NULL , first_name VARCHAR(25) NOT NULL , province CHAR(2) NOT NULL , last_name VARCHAR(25) NOT NULL , postcode CHAR(7) NOT NULL , address VARCHAR(80) NOT NULL , interests TEXT NULL , city VARCHAR(30) NOT NULL , bio TEXT NULL , province CHAR(2) NOT NULL , signature TEXT NULL , postcode CHAR(7) NOT NULL , skills TEXT NULL , interests TEXT NULL , company TEXT NULL , bio TEXT NULL , PRIMARY KEY (user_id) , signature TEXT NULL , UNIQUE INDEX (email) , skills TEXT NULL ) ENGINE=InnoDB; , company TEXT NULL , PRIMARY KEY (user_id) ) ENGINE=InnoDB; Copyright MySQL AB The World’s Most Popular Open Source Database 18
  19. 19. Horizontal Partitioning Benefits ● Main table has narrow rows, so... ✔ More records fit into a single data page ✔ Fewer reads from memory/disk to get same number of records ● Less frequently queried data doesn't take up memory ● More possibilities for indexing and different storage engines ➢ Allows targeted multiple MyISAM key caches for hot and cold data Copyright MySQL AB The World’s Most Popular Open Source Database 19
  20. 20. Counter Table Example CREATE TABLE Products (   product_id INT NOT NULL AUTO_INCREMENT , name VARCHAR(80) NOT NULL CREATE TABLE Products ( , unit_cost DECIMAL(7,2) NOT NULL   product_id INT NOT NULL AUTO_INCREMENT , description TEXT NULL , name VARCHAR(80) NOT NULL , image_path TEXT NULL , unit_cost DECIMAL(7,2) NOT NULL , PRIMARY KEY (product_id) , description TEXT NULL , INDEX (name(20)) , image_path TEXT NULL ) ENGINE=InnoDB; // Or MyISAM , num_views INT UNSIGNED NOT NULL , num_in_stock INT UNSIGNED NOT NULL CREATE TABLE ProductCounts ( , num_on_order INT UNSIGNED NOT NULL   product_id INT NOT NULL , PRIMARY KEY (product_id) , num_views INT UNSIGNED NOT NULL , INDEX (name(20)) , num_in_stock INT UNSIGNED NOT NULL ) ENGINE=InnoDB; // Or MyISAM , num_on_order INT UNSIGNED NOT NULL , PRIMARY KEY (product_id) // Getting a simple COUNT of products ) ENGINE=InnoDB; // easy on MyISAM, terrible on InnoDB SELECT COUNT(*) CREATE TABLE ProductCountSummary ( FROM Products;   total_products INT UNSIGNED NOT NULL ) ENGINE=MEMORY; Copyright MySQL AB The World’s Most Popular Open Source Database 20
  21. 21. Counter Table Benefits ● Critical for InnoDB because of complications of MVCC ● Allows query cache to cache specific data set which will be invalidated only infrequently ● Allows you to target SQL_NO_CACHE for SELECTs against counter tables, freeing query cache ● Allows MEMORY storage engine for summary counters, since stats can be rebuilt Copyright MySQL AB The World’s Most Popular Open Source Database 21
  22. 22. Schema Tips (cont'd) ● Ensure small clustering key (InnoDB) ● Don't use surrogate keys when a naturally occurring primary key exists ● Example (of what not to do): CREATE TABLE Products2Tags (   record_id INT UNSIGNED NOT NULL AUTO_INCREMENT , product_id INT UNSIGNED NOT NULL , tag_id INT UNSIGNED NOT NULL , PRIMARY KEY (record_id) , UNIQUE INDEX (product_id, tag_id) ) ENGINE=InnoDB; Copyright MySQL AB The World’s Most Popular Open Source Database 22
  23. 23. Coding Guidelines ● Use “chunky” coding habits (KISS) ● Use stored procedures for a performance boost (5.0+) ● Isolate indexed fields on one side of equation (example ahead) ● Use calculated fields if necessary (example ahead) ● Learn to use joins (!) ➢ Eliminate correlated subqueries using standard joins (examples ahead) ● Don't try to outthink the optimizer ➢ Sergey, Timour and Igor are really, really smart... Copyright MySQL AB The World’s Most Popular Open Source Database 23
  24. 24. Isolating Indexed Fields Example ✔ Task: get the Order ID, date of order, and Customer ID for all orders in the last 7 days // Better idea SELECT * FROM Orders WHERE  // Bad idea order_created >= CURRENT_DATE() – INTERVAL 7 DAY; SELECT * FROM Orders // Best idea is to factor out the CURRENT_DATE WHERE  // non­deterministic function in your application TO_DAYS(order_created) –  // code and replace the function with a constant. TO_DAYS(CURRENT_DATE()) >= 7; // Now, query cache can actually cache the query! SELECT order_id, order_created, customer_id FROM Orders WHERE order_created >= '2006­05­24' – INTERVAL 7 DAY; Copyright MySQL AB The World’s Most Popular Open Source Database 24
  25. 25. Calculated Fields Example ✔ Task: search for top-level domain in email addresses // So, we enable fast searching on a reversed field // value by inserting a calculated field ALTER TABLE Customers // Initial schema ADD COLUMN rv_email VARCHAR(80) NOT NULL; CREATE TABLE Customers (   customer_id INT NOT NULL // Now, we update the existing table values , email VARCHAR(80) NOT NULL UPDATE Customers SET rv_email = REVERSE(email); // more fields , PRIMARY KEY (customer_id) // Then, we make a trigger to keep our data in sync , INDEX (email(40)) DELIMITER ;; ) ENGINE=InnoDB; CREATE TRIGGER trg_bi_cust  BEFORE INSERT ON Customers // Bad idea, can't use index FOR EACH ROW BEGIN // on email field  SET NEW.rv_email = REVERSE(NEW.email); SELECT * END ;; FROM Customers WHERE email LIKE '%.com'; // same trigger for BEFORE UPDATE... // Then SELECT on the new field... WHERE rv_email LIKE CONCAT(REVERSE('.com'), '%'); Copyright MySQL AB The World’s Most Popular Open Source Database 25
  26. 26. Correlated Subquery Conversion Example ✔ Task: convert a correlated subquery in the SELECT clause to a standard join // Good practice // Bad practice SELECT p.name SELECT p.name , MAX(oi.price) AS max_sold_price ,  (SELECT MAX(price) FROM Products p     FROM OrderItems  INNER JOIN OrderItems oi     WHERE product_id = p.product_id)   ON p.product_id = oi.product_id AS max_sold_price GROUP BY p.name; FROM Products p; Copyright MySQL AB The World’s Most Popular Open Source Database 26
  27. 27. Derived Table Example ✔ Task: convert a correlated subquery in the WHERE clause to a standard join on a derived table // Good performance SELECT c.company // Bad performance , o.* SELECT FROM Customers c c.company  INNER JOIN ( , o.*   SELECT  FROM Customers c      customer_id  INNER JOIN Orders o    , MAX(order_date) as max_order   ON c.customer_id = o.customer_id      FROM Orders WHERE order_date = (   GROUP BY customer_id  SELECT MAX(order_date)  ) AS m   FROM Orders    ON c.customer_id = m.customer_id  WHERE customer = o.customer  INNER JOIN Orders o )   ON c.customer_id = o.customer_id GROUP BY c.company;   AND o.order_date = m.max_order GROUP BY c.company; Copyright MySQL AB The World’s Most Popular Open Source Database 27
  28. 28. Server Parameters ● Be aware of what is global vs per thread ● Make small changes, then test ● Often provide a quick solution, but temporary ● Query Cache is not a panacea ● key_buffer_size != innodb_buffer_size ➢Also, remember mysql system database is MyISAM ● Memory is cheapest, fastest, easiest way to increase performance Copyright MySQL AB The World’s Most Popular Open Source Database 28
  29. 29. Additional Resources ✔ http://www.mysqlperformanceblog.com/ ➢ Peter Zaitsev's blog – Excellent material ✔ Optimizing Linux Performance ➢ Philip Ezolt (HP Press) ✔ http://dev.mysql.com/tech-resources/articles/pro- mysql-ch6.pdf ➢ Pro MySQL (Apress) chapter on profiling (EXPLAIN) ✔ Advanced PHP Programming ➢ George Schlossnagle (Developer's Library) Copyright MySQL AB The World’s Most Popular Open Source Database 29
  30. 30. THANK YOU! ● Please email me: ➢ Success stories ➢ War stories ➢ Inventive uses of MySQL ● Feedback on webinar ● Other webinar or article topics you'd like to hear about ● Gripes :) ● Anything else you feel like talking about! Jay Pipes (jay@mysql.com) MySQL, Inc. Copyright MySQL AB The World’s Most Popular Open Source Database 30
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×