ith
                                  ew
                               nc .5
                           ma d 5
          ...
Who's this guy?




   about me
                             2


Wednesday, 13 January 2010
Giuseppe Maxia
     •   a.k.a. The Data Charmer
     •   MySQL Community Team Lead
     •   Long time hacking with MySQL f...
MySQL 5.1 GA




                             4

Wednesday, 13 January 2010
MySQL 5.1 GA




                             5

Wednesday, 13 January 2010
Defining the problem




               YOUR
               NEEDS         6


Wednesday, 13 January 2010
The problem(s)
     •   Too much data
     •   Not enough RAM
     •   Historical data
     •   Growing data
     •   Rota...
Too much data




                             data




                                    8


Wednesday, 13 January 2010
Not enough RAM



                                              RAM


                                              INDEXE...
Not enough RAM




                                              INDEXES
                                              RAM...
MySQL 5.1 partitions




                     WHAT
                             11


Wednesday, 13 January 2010
What is it?
     • Logical splitting of tables
     • Transparent to user




                                     12


We...
Logical splitting
     • No need to create separate tables
     • No need to move chunks of data across files




        ...
MySQL 5.1 partitions




       TRANSPARENT           SQL
                                   14


Wednesday, 13 January 20...
COMPARED TO MERGE TABLES

                  MERGE TABLE   •   separate tables
                                •   risk of ...
COMPARED TO MERGE TABLES

       PARTITIONED TABLE     •   One table
                             •   No risk of duplicate...
Wait a minute ...
     • WHAT THE HELL DOES "LOGICAL SPLIT"
       REALLY MEANS?
     • LET ME EXPLAIN ...




           ...
Physical partitioning (1)
     • Take a map




                               18


Wednesday, 13 January 2010
Physical partitioning (2)
     • cut it into pieces




                               19


Wednesday, 13 January 2010
Physical partitioning (3)
     • What you have, is several different pieces




                                          ...
Physical partitioning (4)
     • If you want the map back, you need some
       application (adhesive tape) and you may ge...
Logical partitioning (1)
     • Take a map




                              22


Wednesday, 13 January 2010
Logical partitioning (2)
     • fold it to show the piece you need




                                            23


We...
Logical partitioning (3)
     • what you have is still a map, even if you see only
       one part.




                  ...
Logical partitioning (4)
     • if you unfold the map, you still have the whole thing




                                ...
What partitions can do
     •   logical split
     •   but you can also split the data physically
     •   granular partit...
Partition pruning
    1a - unpartitioned table - SINGLE RECORD



                             select *
                  ...
Partition pruning
    1b - unpartitioned table - SINGLE RECORD



                             select *
                  ...
Partition pruning
    1c - unpartitioned table - RANGE


                               select *
                         ...
Partition pruning
    2a - table partitioned by colx - SINGLE REC
                 1-99


             100-199
           ...
Partition pruning
    2b - table partitioned by colx - SINGLE REC
                 1-99


             100-199
           ...
Partition pruning
    2c - table partitioned by colx - RANGE
                 1-99

                               select ...
MySQL 5.1 partitions




                             WHY
                                   33


Wednesday, 13 January 20...
4 main reasons to use partitions
     •   To make single inserts and selects faster
     •   To make range selects faster
...
MySQL 5.1 partitions




                             HOW
                                   35


Wednesday, 13 January 20...
HOW TO MAKE PARTITIONS

     •RTFM ...
     • No, seriously, the manual has everything
     • But if you absolutely insist...
HOW TO MAKE PARTITIONS
CREATE TABLE t1 (
   id int
) ENGINE=InnoDB
   # or MyISAM, ARCHIVE
PARTITION BY RANGE (id)
(
  PAR...
HOW TO MAKE PARTITIONS
CREATE TABLE t1 (
   id int
) ENGINE=InnoDB
PARTITION BY LIST (id)
(
  PARTITION P1 VALUES IN (1,2,...
HOW TO MAKE PARTITIONS
CREATE TABLE t1 (
  id int not null primary key
) ENGINE=InnoDB
PARTITION BY HASH (id)
PARTITIONS 1...
HOW TO MAKE PARTITIONS
CREATE TABLE t1 (
  id int not null primary key
) ENGINE=InnoDB
PARTITION BY KEY ()
PARTITIONS 10;
...
Limitations
     • Can partition only by INTEGER columns
     • OR you can partition by an expression, which must
       r...
PARTITIONING BY DATE
CREATE TABLE t1 (
  d date
) ENGINE=InnoDB
PARTITION BY RANGE (year(d))
(
  PARTITION P1 VALUES
     ...
PARTITIONING BY DATE
CREATE TABLE t1 (
  d date
) ENGINE=InnoDB
PARTITION BY RANGE (to_days(d))
(
  PARTITION P1 VALUES
  ...
MySQL 5.1 partitions




                   WHEN
                             44


Wednesday, 13 January 2010
When to use partitions?
     • if you have large tables
     • if you know that you will always query for the
       parti...
MySQL 5.1 partitions




              HANDS
               ON            46


Wednesday, 13 January 2010
components used for testing
     • MySQL Sandbox
         > created MySQL instances in seconds
         > http://launchpad...
employees test database




                              48


Wednesday, 13 January 2010
How many partitions
   from information_schema.partitions
   +-------+-----------------+----------+
   | pname | expr     ...
How many records
   select count(*) from salaries;
    +----------+
    | count(*) |
    +----------+
    | 2844047 |
    ...
How many records in 1998
   not partitioned
   select count(*) from salaries
   where from_date between
   '1998-01-01' an...
How many records in 1998
   PARTITIONED
   select count(*) from salaries
   where from_date between
   '1998-01-01' and '1...
How many records in 1999
   not partitioned
  select count(*) from salaries
  where from_date between
  '1999-01-01' and '...
How many records in 1999
   PARTITIONED
   select count(*) from salaries
   where from_date between
   '1999-01-01' and '1...
Deleting 1998 records
   not partitioned
   delete from salaries where
   from_date between '1998-01-01'
   and '1998-12-3...
Deleting 1998 records
   partitioned
   alter table salaries drop
   partition p15;
   Query OK, 0 rows affected (1.35
   ...
MySQL 5.1 partitions




        LEVERAGING
        REPLICATION
                             57


Wednesday, 13 January 20...
Replication schemes
                             INNODB                   INNODB
                             NOT         ...
Replication schemes
                                             concurrent insert
                               INNODB
 ...
MySQL 5.1 partitions




     PITFALLS
                             60


Wednesday, 13 January 2010
Expressions
     • Partition by function
     • Search by column




                               61


Wednesday, 13 Jan...
WRONG
   PARTITION BY RANGE(year(from_date))

   select count(*) from salaries where
   year(from_date) = 1998;
   +------...
RIGHT
   PARTITION BY RANGE(year(from_date))

   select count(*) from salaries where
   from_date between '1998-01-01' and...
EXPLAIN
   explain partitions select count(*)
   from salaries where year(from_date)
   = 1998G
   ***** 1. row ****
     ...
EXPLAIN
   explain partitions select count(*)
   from salaries where from_date
   between '1998-01-01' and
   '1998-12-31'...
MySQL 5.1 partitions




             BENCHMARKS

                             66


Wednesday, 13 January 2010
Partitions benchmarks (laptop)


     engine                                storage
                                      ...
Benchmarking results (laptop)
          engine                query year   query year
                                2000...
Partitioning storage (huge server)


     engine                            storage
                                      ...
Benchmarking results (huge server)
   engine                          6 month range
    InnoDB                            ...
MySQL 5.1 partitions




               TOOLS
                             71


Wednesday, 13 January 2010
The partition helper
     • The INFORMATION_SCHEMA.PARTITIONS table
     • The partition helper
         > http://datachar...
MySQL 5.1 partitions




                             TIPS
                                    73


Wednesday, 13 January ...
TIPS
     • For large table ( indexes > available RAM)
         > DO NOT USE INDEXES
         > PARTITIONS ARE MORE EFFICI...
TIPS
     • Before adopting partitions in production, benchmark!
     • you can create partitions of different sizes
     ...
TIPS
     • For large historical tables, consider using ARCHIVE
       + partitions
         > You see benefits for very l...
MySQL 5.5 partitions




          NEW
        FEATURES
                             77


Wednesday, 13 January 2010
MySQL 5.5 enhancements
     • PARTITION BY RANGE COLUMNS
     • PARTITION BY LIST COLUMNS
     • TO_SECONDS




          ...
R E
   MySQL 5.5 enhancements
   CREATE TABLE t (          FO
     dt date
                         B E     5.1
   )
   PA...
R E
   MySQL 5.5 enhancements
                              FO
   SHOW CREATE TABLE t G B E     5.1
          Table: t
   ...
MySQL 5.5 enhancements
                                 E R
   CREATE TABLE t (
                               FT
     dt ...
MySQL 5.5 enhancements
                                 E R
                               FT
 SHOW CREATE TABLE t
       ...
MySQL 5.5 - Multiple columns
   CREATE TABLE t (
     a int,
     b int
   )PARTITION BY RANGE COLUMNS   (a,b)
   (
     P...
partition definition
                                         by range (a,b)
          records            partition       ...
partition definition
                                                 by range (a,b)
        records
                     ...
partition definition
                                                 by range (a,b)
      records                        ...
partition definition
                                             by range (a,b)
      records                    partitio...
partition definition
                                             by range (a,b)
      records                    partitio...
CREATE TABLE employees (
  emp_no int(11) NOT NULL,
  birth_date date NOT NULL,
  first_name varchar(14) NOT NULL,
  last_...
MySQL 5.5 enhancements
     • TRUNCATE PARTITION
     • TO_SECONDS()




                             90


Wednesday, 13 J...
More new features




                  COMING
                   SOON
                             91


Wednesday, 13 Jan...
PLANNED FEATURES - DISCUSS
     • ALTER TABLE t1 EXCHANGE PARTITION p WITH
       TABLE t2

     • SELECT * FROM T (PARTIT...
Read more
   • articles
     http://dev.mysql.com

   • official docs
     http://dev.mysql.com/doc

   • my blog
     htt...
Thank you!



     Giuseppe Maxia
     http://datacharmer.blogspot.com
     datacharmer@sun.com
Wednesday, 13 January 2010
MySQL 5.1 partitions




                             BONUS
                             SLIDES
                          ...
MySQL 5.1 partitions




  ANNOYANCES
                             96


Wednesday, 13 January 2010
Annoyances with partitions
     • CREATE TABLE STATEMENT hard to read




                                             97
...
Annoyances with partitions
     • hard to read (FIXED!! in 5.1.31)




                                          98


Wedn...
Annoyances with partitions
     • CREATE TABLE only keeps the result of the
       expression for each partition.
     • (...
Annoyances with partitions
CREATE TABLE t1 (
  d date
) ENGINE=InnoDB
PARTITION BY RANGE (to_days(d))
(
  PARTITION P1 VAL...
Annoyances with partitions
CREATE TABLE `t1` (
  `d` date DEFAULT NULL
) ENGINE=InnoDB DEFAULT
CHARSET=latin1
/*!50100 PAR...
Fixing annoyances with partitions
  select partition_name part,
  partition_expression expr,
  partition_description val
 ...
fixing annoyances with partitions
  select partition_name part ,
  partition_expression expr, from_days
  (partition_descr...
Upcoming SlideShare
Loading in...5
×

Partitions Performance with MySQL 5.1 and 5.5

24,358

Published on

learn the basics of MySQL 5.1 partitions and the MySQL 5.5. enhancements

2 Comments
39 Likes
Statistics
Notes
No Downloads
Views
Total Views
24,358
On Slideshare
0
From Embeds
0
Number of Embeds
31
Actions
Shares
0
Downloads
670
Comments
2
Likes
39
Embeds 0
No embeds

No notes for slide

Partitions Performance with MySQL 5.1 and 5.5

  1. 1. ith ew nc .5 ma d 5 for an st per 5.1 B oo SQL ing M y rtit ion pa Giuseppe Maxia MySQL Community Team Lead Sun Microsystems Wednesday, 13 January 2010
  2. 2. Who's this guy? about me 2 Wednesday, 13 January 2010
  3. 3. Giuseppe Maxia • a.k.a. The Data Charmer • MySQL Community Team Lead • Long time hacking with MySQL features • Formerly, database consultant, designer, coder. • A passion for QA • An even greater passion for open source • ... and community • Passionate blogger • http://datacharmer.blogspot.com 3 Wednesday, 13 January 2010
  4. 4. MySQL 5.1 GA 4 Wednesday, 13 January 2010
  5. 5. MySQL 5.1 GA 5 Wednesday, 13 January 2010
  6. 6. Defining the problem YOUR NEEDS 6 Wednesday, 13 January 2010
  7. 7. The problem(s) • Too much data • Not enough RAM • Historical data • Growing data • Rotating data 7 Wednesday, 13 January 2010
  8. 8. Too much data data 8 Wednesday, 13 January 2010
  9. 9. Not enough RAM RAM INDEXES data INDEXES 9 Wednesday, 13 January 2010
  10. 10. Not enough RAM INDEXES RAM data INDEXES 10 Wednesday, 13 January 2010
  11. 11. MySQL 5.1 partitions WHAT 11 Wednesday, 13 January 2010
  12. 12. What is it? • Logical splitting of tables • Transparent to user 12 Wednesday, 13 January 2010
  13. 13. Logical splitting • No need to create separate tables • No need to move chunks of data across files 13 Wednesday, 13 January 2010
  14. 14. MySQL 5.1 partitions TRANSPARENT SQL 14 Wednesday, 13 January 2010
  15. 15. COMPARED TO MERGE TABLES MERGE TABLE • separate tables • risk of duplicates • insert in each table • no constraints 15 Wednesday, 13 January 2010
  16. 16. COMPARED TO MERGE TABLES PARTITIONED TABLE • One table • No risk of duplicates • insert in one table • constraints enforced 16 Wednesday, 13 January 2010
  17. 17. Wait a minute ... • WHAT THE HELL DOES "LOGICAL SPLIT" REALLY MEANS? • LET ME EXPLAIN ... 17 Wednesday, 13 January 2010
  18. 18. Physical partitioning (1) • Take a map 18 Wednesday, 13 January 2010
  19. 19. Physical partitioning (2) • cut it into pieces 19 Wednesday, 13 January 2010
  20. 20. Physical partitioning (3) • What you have, is several different pieces 20 Wednesday, 13 January 2010
  21. 21. Physical partitioning (4) • If you want the map back, you need some application (adhesive tape) and you may get it wrong 21 Wednesday, 13 January 2010
  22. 22. Logical partitioning (1) • Take a map 22 Wednesday, 13 January 2010
  23. 23. Logical partitioning (2) • fold it to show the piece you need 23 Wednesday, 13 January 2010
  24. 24. Logical partitioning (3) • what you have is still a map, even if you see only one part. 24 Wednesday, 13 January 2010
  25. 25. Logical partitioning (4) • if you unfold the map, you still have the whole thing 25 Wednesday, 13 January 2010
  26. 26. What partitions can do • logical split • but you can also split the data physically • granular partition (subpartitioning) • different methods (range, list, hash, key) 26 Wednesday, 13 January 2010
  27. 27. Partition pruning 1a - unpartitioned table - SINGLE RECORD select * from table_name where colx = 120 Wednesday, 13 January 2010
  28. 28. Partition pruning 1b - unpartitioned table - SINGLE RECORD select * from table_name where colx = 350 Wednesday, 13 January 2010
  29. 29. Partition pruning 1c - unpartitioned table - RANGE select * from table_name where colx between 120 and 230 Wednesday, 13 January 2010
  30. 30. Partition pruning 2a - table partitioned by colx - SINGLE REC 1-99 100-199 select * from 200-299 table_name where colx = 300-399 120 400-499 500-599 Wednesday, 13 January 2010
  31. 31. Partition pruning 2b - table partitioned by colx - SINGLE REC 1-99 100-199 select * from 200-299 table_name where colx = 300-399 350 400-499 500-599 Wednesday, 13 January 2010
  32. 32. Partition pruning 2c - table partitioned by colx - RANGE 1-99 select * 100-199 from table_name 200-299 where colx 300-399 between 120 and 230 400-499 500-599 Wednesday, 13 January 2010
  33. 33. MySQL 5.1 partitions WHY 33 Wednesday, 13 January 2010
  34. 34. 4 main reasons to use partitions • To make single inserts and selects faster • To make range selects faster • To help split the data across different paths • To store historical data efficiently • If you need to delete large chunks of data INSTANTLY 34 Wednesday, 13 January 2010
  35. 35. MySQL 5.1 partitions HOW 35 Wednesday, 13 January 2010
  36. 36. HOW TO MAKE PARTITIONS •RTFM ... • No, seriously, the manual has everything • But if you absolutely insist ... 36 Wednesday, 13 January 2010
  37. 37. HOW TO MAKE PARTITIONS CREATE TABLE t1 ( id int ) ENGINE=InnoDB # or MyISAM, ARCHIVE PARTITION BY RANGE (id) ( PARTITION P1 VALUES LESS THAN (10), PARTITION P2 VALUES LESS THAN (20) ) 37 Wednesday, 13 January 2010
  38. 38. HOW TO MAKE PARTITIONS CREATE TABLE t1 ( id int ) ENGINE=InnoDB PARTITION BY LIST (id) ( PARTITION P1 VALUES IN (1,2,4), PARTITION P2 VALUES IN (3,5,9) ) 38 Wednesday, 13 January 2010
  39. 39. HOW TO MAKE PARTITIONS CREATE TABLE t1 ( id int not null primary key ) ENGINE=InnoDB PARTITION BY HASH (id) PARTITIONS 10; 39 Wednesday, 13 January 2010
  40. 40. HOW TO MAKE PARTITIONS CREATE TABLE t1 ( id int not null primary key ) ENGINE=InnoDB PARTITION BY KEY () PARTITIONS 10; 40 Wednesday, 13 January 2010
  41. 41. Limitations • Can partition only by INTEGER columns • OR you can partition by an expression, which must return an integer • Maximum 1024 partitions • If you have a Unique Key or PK, the partition column must be part of that key • No Foreign Key support • No Fulltext or GIS support 41 Wednesday, 13 January 2010
  42. 42. PARTITIONING BY DATE CREATE TABLE t1 ( d date ) ENGINE=InnoDB PARTITION BY RANGE (year(d)) ( PARTITION P1 VALUES LESS THAN (1999), PARTITION P2 VALUES LESS THAN (2000) 42 ) Wednesday, 13 January 2010
  43. 43. PARTITIONING BY DATE CREATE TABLE t1 ( d date ) ENGINE=InnoDB PARTITION BY RANGE (to_days(d)) ( PARTITION P1 VALUES LESS THAN (to_days('1999-01-01')), PARTITION P2 VALUES LESS THAN (to_days('2000-01-01')) 43 ) Wednesday, 13 January 2010
  44. 44. MySQL 5.1 partitions WHEN 44 Wednesday, 13 January 2010
  45. 45. When to use partitions? • if you have large tables • if you know that you will always query for the partitioning column • if you have historical tables that you want to purge quickly • if your indexes are larger than the available RAM 45 Wednesday, 13 January 2010
  46. 46. MySQL 5.1 partitions HANDS ON 46 Wednesday, 13 January 2010
  47. 47. components used for testing • MySQL Sandbox > created MySQL instances in seconds > http://launchpad.net/mysql-sandbox • MySQL Employees Test Database > has ~ 4 mil records in 6 tables > http://launchpad.net/test-db • Command line ability > fingers > ingenuity 47 Wednesday, 13 January 2010
  48. 48. employees test database 48 Wednesday, 13 January 2010
  49. 49. How many partitions from information_schema.partitions +-------+-----------------+----------+ | pname | expr | descr | +-------+-----------------+----------+ | p01 | year(from_date) | 1985 | | p02 | year(from_date) | 1986 | ... | p13 | year(from_date) | 1997 | | p14 | year(from_date) | 1998 | | p15 | year(from_date) | 1999 | | p16 | year(from_date) | 2000 | ... | p19 | year(from_date) | MAXVALUE | +-------+-----------------+----------+ 49 Wednesday, 13 January 2010
  50. 50. How many records select count(*) from salaries; +----------+ | count(*) | +----------+ | 2844047 | +----------+ 1 row in set (0.01 sec) 50 Wednesday, 13 January 2010
  51. 51. How many records in 1998 not partitioned select count(*) from salaries where from_date between '1998-01-01' and '1998-12-31'; +----------+ | count(*) | +----------+ | 247489 | +----------+ 1 row in set (1.52 sec) # NOT PARTITIONED 51 Wednesday, 13 January 2010
  52. 52. How many records in 1998 PARTITIONED select count(*) from salaries where from_date between '1998-01-01' and '1998-12-31'; +----------+ | count(*) | +----------+ | 247489 | +----------+ 1 row in set (0.41 sec) # partition p15 52 Wednesday, 13 January 2010
  53. 53. How many records in 1999 not partitioned select count(*) from salaries where from_date between '1999-01-01' and '1999-12-31'; +----------+ | count(*) | +----------+ | 260957 | +----------+ 1 row in set (1.54 sec) # NOT PARTITIONED 53 Wednesday, 13 January 2010
  54. 54. How many records in 1999 PARTITIONED select count(*) from salaries where from_date between '1999-01-01' and '1999-12-31'; +----------+ | count(*) | +----------+ | 260957 | +----------+ 1 row in set (0.17 sec) # partition p16 54 Wednesday, 13 January 2010
  55. 55. Deleting 1998 records not partitioned delete from salaries where from_date between '1998-01-01' and '1998-12-31'; Query OK, 247489 rows affected (19.13 sec) # NOT PARTITIONED 55 Wednesday, 13 January 2010
  56. 56. Deleting 1998 records partitioned alter table salaries drop partition p15; Query OK, 0 rows affected (1.35 sec) 56 Wednesday, 13 January 2010
  57. 57. MySQL 5.1 partitions LEVERAGING REPLICATION 57 Wednesday, 13 January 2010
  58. 58. Replication schemes INNODB INNODB NOT NOT MASTER PARTITIONED PARTITIONED concurrent insert SLAVE concurrent read INNODB MyISAM PARTITIONED PARTITIONED BY RANGE BY RANGE SLAVE SLAVE 58 concurrent batch processing large batch processing Wednesday, 13 January 2010
  59. 59. Replication schemes concurrent insert INNODB PARTITIONED MASTER INNODB NON MyISAM PARTITIONED SLAVE PARTITIONED SLAVE 59 batch processing concurrent reads Wednesday, 13 January 2010
  60. 60. MySQL 5.1 partitions PITFALLS 60 Wednesday, 13 January 2010
  61. 61. Expressions • Partition by function • Search by column 61 Wednesday, 13 January 2010
  62. 62. WRONG PARTITION BY RANGE(year(from_date)) select count(*) from salaries where year(from_date) = 1998; +----------+ | count(*) | +----------+ | 247489 | +----------+ 1 row in set (2.25 sec) 62 Wednesday, 13 January 2010
  63. 63. RIGHT PARTITION BY RANGE(year(from_date)) select count(*) from salaries where from_date between '1998-01-01' and '1998-12-31'; +----------+ | count(*) | +----------+ | 247489 | +----------+ 1 row in set (0.46 sec) 63 Wednesday, 13 January 2010
  64. 64. EXPLAIN explain partitions select count(*) from salaries where year(from_date) = 1998G ***** 1. row **** id: 1 select_type: SIMPLE table: salaries partitions: p01,p02,p03,p04,p05,p06,p07,p08,p09 ,p10,p11,p12,p13,p14,p15,p16,p17,p1 8,p19 type: index 64 ... Wednesday, 13 January 2010
  65. 65. EXPLAIN explain partitions select count(*) from salaries where from_date between '1998-01-01' and '1998-12-31'G ***** 1. row **** id: 1 select_type: SIMPLE table: salaries partitions: p14,p15 ... 65 Wednesday, 13 January 2010
  66. 66. MySQL 5.1 partitions BENCHMARKS 66 Wednesday, 13 January 2010
  67. 67. Partitions benchmarks (laptop) engine storage (MB) innodb 221 myisam 181 archive 74 innodb partitioned (whole) 289 innodb partitioned (file per table) 676 myisam partitioned 182 archive partitioned 72 Wednesday, 13 January 2010
  68. 68. Benchmarking results (laptop) engine query year query year 2000 2002 InnoDB 1.25 1.25 MyISAM 1.72 1.73 Archive 2.47 2.45 InnoDB partitioned 0.24 0.10 whole InnoDB Partitioned 0.45 0.10 (file per table) MyISAM partitioned 0.18 0.12 Archive partitioned 0.22 0.12 Wednesday, 13 January 2010
  69. 69. Partitioning storage (huge server) engine storage (GB) innodb (with PK) 330 myisam (with PK) 141 archive 13 innodb partitioned (no PK) 237 myisam partitioned (no PK) 107 archive partitioned 13 Wednesday, 13 January 2010
  70. 70. Benchmarking results (huge server) engine 6 month range InnoDB 4 min 30s MyISAM 25.03s Archive 22 min 25s InnoDB partitioned by month 13.19 MyISAM partitioned by year 6.31 MyISAM partitioned by month 4.45 Archive partitioned by year 16.67 Archive partitioned by month 8.97 Wednesday, 13 January 2010
  71. 71. MySQL 5.1 partitions TOOLS 71 Wednesday, 13 January 2010
  72. 72. The partition helper • The INFORMATION_SCHEMA.PARTITIONS table • The partition helper > http://datacharmer.blogspot.com/2008/12/partition- helper-improving-usability.html > A Perl script that creates partitioning statements • ... anything you are creating right now :) 72 Wednesday, 13 January 2010
  73. 73. MySQL 5.1 partitions TIPS 73 Wednesday, 13 January 2010
  74. 74. TIPS • For large table ( indexes > available RAM) > DO NOT USE INDEXES > PARTITIONS ARE MORE EFFICIENT 74 Wednesday, 13 January 2010
  75. 75. TIPS • Before adopting partitions in production, benchmark! • you can create partitions of different sizes > in historical tables, you can partition – current data by day – recent data by month – distant past data by year > ALL IN THE SAME TABLE! • If you want to enforce a constraint on a integer column, you may set a partition by list, with only the values you want to admit. 75 Wednesday, 13 January 2010
  76. 76. TIPS • For large historical tables, consider using ARCHIVE + partitions > You see benefits for very large amounts of data ( > 500 MB) > Very effective when you run statistics > Almost the same performance of MyISAM > but 1/10 of storage 76 Wednesday, 13 January 2010
  77. 77. MySQL 5.5 partitions NEW FEATURES 77 Wednesday, 13 January 2010
  78. 78. MySQL 5.5 enhancements • PARTITION BY RANGE COLUMNS • PARTITION BY LIST COLUMNS • TO_SECONDS 78 Wednesday, 13 January 2010
  79. 79. R E MySQL 5.5 enhancements CREATE TABLE t ( FO dt date B E 5.1 ) PARTITION BY RANGE (TO_DAYS(dt)) ( PARTITION p01 VALUES LESS THAN (TO_DAYS('2007-01-01')), PARTITION p02 VALUES LESS THAN (TO_DAYS('2008-01-01')), PARTITION p03 VALUES LESS THAN (TO_DAYS('2009-01-01')), PARTITION p04 VALUES LESS THAN (MAXVALUE)); 79 Wednesday, 13 January 2010
  80. 80. R E MySQL 5.5 enhancements FO SHOW CREATE TABLE t G B E 5.1 Table: t Create Table: CREATE TABLE `t` ( `dt` date DEFAULT NULL ) ENGINE=MyISAM DEFAULT CHARSET=latin1 /*!50100 PARTITION BY RANGE (TO_DAYS (dt)) (PARTITION p01 VALUES LESS THAN (733042) ENGINE = MyISAM, […] 80 Wednesday, 13 January 2010
  81. 81. MySQL 5.5 enhancements E R CREATE TABLE t ( FT dt date A 5.5 ) PARTITION BY RANGE COLUMNS (dt) ( PARTITION p01 VALUES LESS THAN ('2007-01-01'), PARTITION p02 VALUES LESS THAN ('2008-01-01'), PARTITION p03 VALUES LESS THAN ('2009-01-01'), PARTITION p04 VALUES LESS THAN (MAXVALUE)); 81 Wednesday, 13 January 2010
  82. 82. MySQL 5.5 enhancements E R FT SHOW CREATE TABLE t Table: t A 5.5 Create Table: CREATE TABLE `t` ( `dt` date DEFAULT NULL ) ENGINE=MyISAM DEFAULT CHARSET=latin1 /*!50500 PARTITION BY RANGE COLUMNS(dt) (PARTITION p01 VALUES LESS THAN ('2007-01-01') ENGINE = MyISAM, […] 82 Wednesday, 13 January 2010
  83. 83. MySQL 5.5 - Multiple columns CREATE TABLE t ( a int, b int )PARTITION BY RANGE COLUMNS (a,b) ( PARTITION p01 VALUES LESS THAN (10,1), PARTITION p02 VALUES LESS THAN (10,10), PARTITION p03 VALUES LESS THAN (10,20), PARTITION p04 VALUES LESS THAN (MAXVALUE, MAXVALUE)); 83 Wednesday, 13 January 2010
  84. 84. partition definition by range (a,b) records partition LESS THAN a b p01 10 10 1 10 p02 10 20 10 9 p03 10 30 10 10 p04 10 MAXVALUE 10 11 p05 MAXVALUE MAXVALUE Wednesday, 13 January 2010
  85. 85. partition definition by range (a,b) records partition LESS THAN a b p01 10 10 1 10 p02 10 20 10 9 p03 10 30 10 10 p04 10 MAXVALUE 10 11 p05 MAXVALUE MAXVALUE (1,10) < (10,10) ? (a < 10) OR ((a = 10) AND (b < 10)) (1 < 10) OR TRUE ((1 = 10) AND (10 < 10)) Wednesday, 13 January 2010
  86. 86. partition definition by range (a,b) records partition LESS THAN a b p01 10 10 1 10 p02 10 20 10 9 p03 10 30 10 10 p04 10 MAXVALUE 10 11 p05 MAXVALUE MAXVALUE (10,9) < (10,10) ? (a < 10) OR ((a = 10) AND (b < 10)) (10 < 10) OR TRUE ((10 = 10) AND (9 < 10)) Wednesday, 13 January 2010
  87. 87. partition definition by range (a,b) records partition LESS THAN a b p01 10 10 1 10 p02 10 20 10 9 p03 10 30 10 10 p04 10 MAXVALUE 10 11 p05 MAXVALUE MAXVALUE (10,10) < (10,10) ? (a < 10) OR ((a = 10) AND (b < 10)) (10 < 10) OR FALSE ((10 = 10) AND (10 < 10)) Wednesday, 13 January 2010
  88. 88. partition definition by range (a,b) records partition LESS THAN a b p01 10 10 1 10 p02 10 20 10 9 p03 10 30 10 10 p04 10 MAXVALUE 10 11 p05 MAXVALUE MAXVALUE (10,10) < (10,20) ? (a < 10) OR ((a = 10) AND (b < 20)) (10 < 10) OR TRUE ((10 = 10) AND (10 < 20)) Wednesday, 13 January 2010
  89. 89. CREATE TABLE employees ( emp_no int(11) NOT NULL, birth_date date NOT NULL, first_name varchar(14) NOT NULL, last_name varchar(16) NOT NULL, gender char(1) DEFAULT NULL, hire_date date NOT NULL ) ENGINE=MyISAM PARTITION BY RANGE COLUMNS(gender,hire_date) (PARTITION p01 VALUES LESS THAN ('F','1990-01-01'), PARTITION p02 VALUES LESS THAN ('F','2000-01-01'), PARTITION p03 VALUES LESS THAN ('F',MAXVALUE), PARTITION p04 VALUES LESS THAN ('M','1990-01-01'), PARTITION p05 VALUES LESS THAN ('M','2000-01-01'), PARTITION p06 VALUES LESS THAN ('M',MAXVALUE), PARTITION p07 VALUES LESS THAN (MAXVALUE,MAXVALUE) 89 Wednesday, 13 January 2010
  90. 90. MySQL 5.5 enhancements • TRUNCATE PARTITION • TO_SECONDS() 90 Wednesday, 13 January 2010
  91. 91. More new features COMING SOON 91 Wednesday, 13 January 2010
  92. 92. PLANNED FEATURES - DISCUSS • ALTER TABLE t1 EXCHANGE PARTITION p WITH TABLE t2 • SELECT * FROM T (PARTITION p) WHERE ... 92 Wednesday, 13 January 2010
  93. 93. Read more • articles http://dev.mysql.com • official docs http://dev.mysql.com/doc • my blog http://datacharmer.blogspot.com 93 Wednesday, 13 January 2010
  94. 94. Thank you! Giuseppe Maxia http://datacharmer.blogspot.com datacharmer@sun.com Wednesday, 13 January 2010
  95. 95. MySQL 5.1 partitions BONUS SLIDES 95 Wednesday, 13 January 2010
  96. 96. MySQL 5.1 partitions ANNOYANCES 96 Wednesday, 13 January 2010
  97. 97. Annoyances with partitions • CREATE TABLE STATEMENT hard to read 97 Wednesday, 13 January 2010
  98. 98. Annoyances with partitions • hard to read (FIXED!! in 5.1.31) 98 Wednesday, 13 January 2010
  99. 99. Annoyances with partitions • CREATE TABLE only keeps the result of the expression for each partition. • (you can use the information_schema to ease the pain in this case) 99 Wednesday, 13 January 2010
  100. 100. Annoyances with partitions CREATE TABLE t1 ( d date ) ENGINE=InnoDB PARTITION BY RANGE (to_days(d)) ( PARTITION P1 VALUES LESS THAN (to_days('1999-01-01')), PARTITION P2 VALUES LESS THAN (to_days('2000-01-01')) 100 ) Wednesday, 13 January 2010
  101. 101. Annoyances with partitions CREATE TABLE `t1` ( `d` date DEFAULT NULL ) ENGINE=InnoDB DEFAULT CHARSET=latin1 /*!50100 PARTITION BY RANGE (to_days (d)) (PARTITION P1 VALUES LESS THAN (730120) ENGINE = InnoDB, PARTITION P2 VALUES LESS THAN (730485) ENGINE = InnoDB) */ 101 Wednesday, 13 January 2010
  102. 102. Fixing annoyances with partitions select partition_name part, partition_expression expr, partition_description val from information_schema.partitions where table_name='t1'; +------+------------+--------+ | part | expr | val | +------+------------+--------+ | P1 | to_days(d) | 730120 | | P2 | to_days(d) | 730485 | 102 +------+------------+--------+ Wednesday, 13 January 2010
  103. 103. fixing annoyances with partitions select partition_name part , partition_expression expr, from_days (partition_description) val from information_schema.partitions where table_name='t1'; +------+------------+------------+ | part | expr | val | +------+------------+------------+ | P1 | to_days(d) | 1999-01-01 | | P2 | to_days(d) | 2000-01-01 | +------+------------+------------+ 103 Wednesday, 13 January 2010
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×