SlideShare a Scribd company logo
Boost performance
with
MySQL 5.1 partitions

Giuseppe Maxia
MySQL Community Team Lead
Sun Microsystems
Who's this guy?




about me
                  2
Giuseppe Maxia
•   a.k.a. The Data Charmer
•   MySQL Community Team Lead
•   Long time hacking with MySQL features
•   Formerly, database consultant, designer, coder.
•   A passion for QA
•   An even greater passion for open source
•   ... and community
•   Passionate blogger
•   http://datacharmer.blogspot.com              3
MySQL 5.1 GA




               4
MySQL 5.1 GA




               5
MySQL 5.1 GA




               5
Defining the problem




   YOUR
   NEEDS               6
The problem(s)
•   Too much data
•   Not enough RAM
•   Historical data
•   Growing data
•   Rotating data




                      7
Too much data




         data




                8
Not enough RAM



                       RAM


                       INDEXES
      data   INDEXES




                                 9
Not enough RAM




                       INDEXES
                       RAM
      data   INDEXES




                                 10
MySQL 5.1 partitions




     WHAT
                       11
What is it?
• Logical splitting of tables
• Transparent to user




                                12
Logical splitting
• No need to create separate tables
• No need to move chunks of data across files




                                                13
MySQL 5.1 partitions




          SQL
 TRANSPARENT
                       14
COMPARED TO MERGE TABLES

   MERGE TABLE   •   separate tables
                 •   risk of duplicates
                 •   insert in each table
                 •   no constraints




                                      15
COMPARED TO MERGE TABLES

PARTITIONED TABLE   •   One table
                    •   No risk of duplicates
                    •   insert in one table
                    •   constraints enforced




                                        16
Wait a minute ...
• WHAT THE HELL DOES quot;LOGICAL SPLITquot;
  REALLY MEANS?
• LET ME EXPLAIN ...




                                       17
Physical partitioning (1)
• Take a map




                            18
Physical partitioning (2)
• cut it into pieces




                            19
Physical partitioning (3)
• What you have, is several different pieces




                                               20
Physical partitioning (4)
• If you want the map back, you need some
  application (adhesive tape) and you may get it
  wrong




                                               21
Logical partitioning (1)
• Take a map




                           22
Logical partitioning (2)
• fold it to show the piece you need




                                       23
Logical partitioning (3)
• what you have is still a map, even if you see only
  one part.




                                                24
Logical partitioning (4)
• if you unfold the map, you still have the whole thing




                                                 25
What partitions can do
•   logical split
•   but you can also split the data physically
•   granular partition (subpartitioning)
•   different methods (range, list, hash, key)




                                                 26
MySQL 5.1 partitions




        WHY
                       27
4 main reasons to use partitions
•   To make single inserts and selects faster
•   To make range selects faster
•   To help split the data across different paths
•   To store historical data efficiently
•   If you need to delete large chunks of data
    INSTANTLY



                                                    28
MySQL 5.1 partitions




       HOW
                       29
HOW TO MAKE PARTITIONS




                         30
HOW TO MAKE PARTITIONS

•RTFM ...



                         30
HOW TO MAKE PARTITIONS

•RTFM ...
• No, seriously, the manual has everything




                                             30
HOW TO MAKE PARTITIONS

•RTFM ...
• No, seriously, the manual has everything
• But if you absolutely insist ...




                                             30
HOW TO MAKE PARTITIONS
CREATE TABLE t1 (
   id int
) ENGINE=InnoDB
   # or MyISAM, ARCHIVE
PARTITION BY RANGE (id)
(
  PARTITION P1 VALUES LESS THAN (10),
  PARTITION P2 VALUES LESS THAN (20)
)                              31
HOW TO MAKE PARTITIONS
CREATE TABLE t1 (
   id int
) ENGINE=InnoDB
PARTITION BY LIST (id)
(
  PARTITION P1 VALUES IN (1,2,4),
  PARTITION P2 VALUES IN (3,5,9)
)
                               32
HOW TO MAKE PARTITIONS
CREATE TABLE t1 (
  id int not null primary key
) ENGINE=InnoDB
PARTITION BY HASH (id)
PARTITIONS 10;




                                33
HOW TO MAKE PARTITIONS
CREATE TABLE t1 (
  id int not null primary key
) ENGINE=InnoDB
PARTITION BY KEY ()
PARTITIONS 10;




                                34
Limitations
• Can partition only by INTEGER columns
• OR you can partition by an expression, which must
  return an integer
• Maximum 1024 partitions

• If you have a Unique Key or PK, the partition
  column must be part of that key
• No Foreign Key support
• No Fulltext or GIS support
                                                  35
PARTITIONING BY DATE
CREATE TABLE t1 (
  d date
) ENGINE=InnoDB
PARTITION BY RANGE (year(d))
(
  PARTITION P1 VALUES
     LESS THAN (1999),
  PARTITION P2 VALUES
    LESS THAN (2000)           36

)
PARTITIONING BY DATE
CREATE TABLE t1 (
  d date
) ENGINE=InnoDB
PARTITION BY RANGE (to_days(d))
(
  PARTITION P1 VALUES
   LESS THAN (to_days('1999-01-01')),
  PARTITION P2 VALUES
  LESS THAN (to_days('2000-01-01'))
                               37

)
MySQL 5.1 partitions




 Partition
 pruning               38
Partition pruning
1a - unpartitioned table - SINGLE RECORD



                         select *
                         from
                         table_name
                         where colx =
                         120
Partition pruning
1a - unpartitioned table - SINGLE RECORD



            I            select *
D                        from
            N
A                        table_name
            D
T                        where colx =
            E
A                        120
            X
Partition pruning
1b - unpartitioned table - SINGLE RECORD



                         select *
                         from
                         table_name
                         where colx =
                         350
Partition pruning
1c - unpartitioned table - RANGE


                           select *
                           from
                           table_name
                           where colx
                           between 120
                           and 230
Partition pruning
2a - table partitioned by colx - SINGLE REC
     1-99


    100-199
                           select *
                           from
    200-299                table_name
                           where colx =
    300-399                120
    400-499


    500-599
Partition pruning
2a - table partitioned by colx - SINGLE REC
     1-99


    100-199
                           select *
D                          from
A   200-299                table_name
T                          where colx =
A              I
    300-399                120
              N
              D
    400-499
              E
              X
    500-599
Partition pruning
2b - table partitioned by colx - SINGLE REC
     1-99


    100-199
                           select *
                           from
    200-299                table_name
                           where colx =
    300-399                350
    400-499


    500-599
Partition pruning
2c - table partitioned by colx - RANGE
     1-99

                           select *
    100-199                from
                           table_name
    200-299
                           where colx
    300-399                between 120
                           and 230
    400-499


    500-599
Partition pruning

       EXPLAIN
       select *
before from table_name
       where colx = 120

         EXPLAIN PARTITIONS
in 5.1   select *
         from table_name
         where colx = 120
Partition pruning - not using partition column
explain partitions select count(*)
from table_name where colx=120G
***** 1. row ****
            id: 1
  select_type: SIMPLE
         table: table_name
    partitions:
p01,p02,p03,p04,p05,p06,p07,p08,p09,
p10,p11,p12,p13,p14,p15,p16,p17,p18,
p19
          type: index
...
Partition pruning - not using partition column
explain partitions select count(*)
from table_name where colx between
120 and 230G
***** 1. row ****
            id: 1
  select_type: SIMPLE
         table: table_name
    partitions:
p01,p02,p03,p04,p05,p06,p07,p08,p09,
p10,p11,p12,p13,p14,p15,p16,p17,p18,
p19
          type: index
...
Partition pruning - table partitioned by colx
explain partitions select count(*)
from table_name where colx between
120 and 230G
***** 1. row ****
            id: 1
  select_type: SIMPLE
         table: table_name
    partitions: p02,p03
          type: index
...
MySQL 5.1 partitions




     WHEN
                       51
When to use partitions?
• if you have large tables
• if you know that you will always query for the
  partitioning column
• if you have historical tables that you want to purge
  quickly
• if your indexes are larger than the available RAM




                                                 52
MySQL 5.1 partitions




   HANDS
    ON                 53
components used for testing




                              54
components used for testing
• MySQL Sandbox
 > created MySQL instances in seconds
 > http://launchpad.net/mysql-sandbox




                                        54
components used for testing
• MySQL Sandbox
  > created MySQL instances in seconds
  > http://launchpad.net/mysql-sandbox

• MySQL Employees Test Database
  > has ~ 4 mil records in 6 tables
  > http://launchpad.net/test-db




                                         54
components used for testing
• MySQL Sandbox
  > created MySQL instances in seconds
  > http://launchpad.net/mysql-sandbox

• MySQL Employees Test Database
  > has ~ 4 mil records in 6 tables
  > http://launchpad.net/test-db

• Command line ability
  > fingers
  > ingenuity
                                         54
employees test database




                          55
How many partitions
from information_schema.partitions
+-------+-----------------+----------+
| pname | expr            | descr    |
+-------+-----------------+----------+
| p01   | year(from_date) | 1985     |
| p02   | year(from_date) | 1986     |

...
| p13   | year(from_date) | 1997     |
| p14   | year(from_date) | 1998     |
| p15   | year(from_date) | 1999     |
| p16   | year(from_date) | 2000     |
  ...
| p19   | year(from_date) | MAXVALUE |
+-------+-----------------+----------+   56
How many records
select count(*) from salaries;
 +----------+
 | count(*) |
 +----------+
 | 2844047 |
 +----------+
1 row in set (0.01 sec)



                             57
How many records in 1998
not partitioned
select count(*) from salaries
where from_date between
'1998-01-01' and '1998-12-31';
+----------+
| count(*) |
+----------+
|   247489 |
+----------+
1 row in set (1.52 sec)

# NOT PARTITIONED            58
How many records in 1998
PARTITIONED
select count(*) from salaries
where from_date between
'1998-01-01' and '1998-12-31';
+----------+
| count(*) |
+----------+
|   247489 |
+----------+
1 row in set (0.41 sec)

# partition p15
                             59
How many records in 1999
not partitioned
select count(*) from salaries
where from_date between
'1999-01-01' and '1999-12-31';
+----------+
| count(*) |
+----------+
|   260957 |
+----------+
1 row in set (1.54 sec)

# NOT PARTITIONED
                             60
How many records in 1999
PARTITIONED
select count(*) from salaries
where from_date between
'1999-01-01' and '1999-12-31';
+----------+
| count(*) |
+----------+
|   260957 |
+----------+
1 row in set (0.17 sec)

# partition p16
                             61
Deleting 1998 records
not partitioned
delete from salaries where
from_date between '1998-01-01'
and '1998-12-31';
Query OK, 247489 rows affected
(19.13 sec)



# NOT PARTITIONED

                             62
Deleting 1998 records
partitioned
alter table salaries drop
partition p15;
Query OK, 0 rows affected (1.35
sec)




                             63
MySQL 5.1 partitions




 LEVERAGING
 REPLICATION
                       64
Replication schemes
                     INNODB                    INNODB
                     NOT                       NOT
   MASTER            PARTITIONED               PARTITIONED

 concurrent insert                                           SLAVE
                                                       concurrent read

           INNODB                   MyISAM
           PARTITIONED              PARTITIONED
           BY RANGE                 BY RANGE
SLAVE                                             SLAVE
                                                            65
concurrent batch processing        large batch processing
Replication schemes
                              concurrent insert
                INNODB
                PARTITIONED
 MASTER


                                        INNODB
                                        NON
            MyISAM                      PARTITIONED
 SLAVE      PARTITIONED
                                                     SLAVE
                                                     66
  batch processing                       concurrent reads
Replication schemes - dimensions
                     INNODB                          ARCHIVE
                     NOT                             PARTITIONED
                     PARTITIONED               SLAVE BY RANGE
  MASTER                                             (locations)

concurrent insert                               dimensional processing

ARCHIVE                                    ARCHIVE
PARTITIONED         SLAVE          SLAVE   PARTITIONED
BY RANGE                                   BY RANGE
(date)                                     (product)
                                                          67
dimensional processing                     dimensional processing
MySQL 5.1 partitions




PITFALLS
                       68
Expressions
• Partition by function
• Search by column




                          69
WRONG
PARTITION BY RANGE(year(from_date))

select count(*) from salaries where
year(from_date) = 1998;
+----------+
| count(*) |
+----------+
|   247489 |
+----------+
1 row in set (2.25 sec)
                             70
RIGHT
PARTITION BY RANGE(year(from_date))

select count(*) from salaries where
from_date between '1998-01-01' and
'1998-12-31';
+----------+
| count(*) |
+----------+
|   247489 |
+----------+
1 row in set (0.46 sec)
                             71
EXPLAIN
explain partitions select count(*)
from salaries where year(from_date)
= 1998G
***** 1. row ****
            id: 1
  select_type: SIMPLE
         table: salaries
   partitions:
p01,p02,p03,p04,p05,p06,p07,p08,p09
,p10,p11,p12,p13,p14,p15,p16,p17,p1
8,p19
          type: index        72

...
EXPLAIN
explain partitions select count(*)
from salaries where from_date
between '1998-01-01' and
'1998-12-31'G
***** 1. row ****
           id: 1
  select_type: SIMPLE
        table: salaries
   partitions: p14,p15
...

                             73
MySQL 5.1 partitions




   BENCHMARKS

                       74
Partitions benchmarks (laptop)


 engine                                storage
                                        (MB)
 innodb                                     221
 myisam                                     181
 archive                                     74
 innodb partitioned (whole)                 289
 innodb partitioned (file per table)        676
 myisam partitioned                         182
 archive partitioned                         72
Benchmarking results (laptop)
   engine                query year   query year
                         2000         2002
   InnoDB                    1.25         1.25

   MyISAM                    1.72         1.73

   Archive                   2.47         2.45

   InnoDB partitioned        0.24         0.10
   whole
   InnoDB Partitioned        0.45         0.10
   (file per table)
   MyISAM partitioned        0.18         0.12

   Archive partitioned       0.22         0.12
Partitioning storage (huge server)


 engine                              storage
                                       (GB)
 innodb (with PK)                         330
 myisam (with PK)                         141
 archive                                    13
 innodb partitioned (no PK)               237
 myisam partitioned (no PK)               107
 archive partitioned                        13
Benchmarking results (huge server)
engine                         6 month range
InnoDB                              4 min 30s
MyISAM                               25.03s
Archive                            22 min 25s
InnoDB partitioned by month          13.19
MyISAM partitioned by year            6.31
MyISAM partitioned by month           4.45
Archive partitioned by year          16.67
Archive partitioned by month          8.97
MySQL 5.1 partitions




   TOOLS
                       79
The partition helper
• The INFORMATION_SCHEMA.PARTITIONS table
• The partition helper
  > http://datacharmer.blogspot.com/2008/12/partition-
    helper-improving-usability.html
  > A Perl script that creates partitioning statements


• ... anything you are creating right now :)



                                                         80
the partition helper
./partitions_helper 
   --table=mytable --column=prod_id 
   --interval=1000 --start=1 --end=10000
ALTER TABLE mytable
PARTITION by range (prod_id)
(
   partition p001 VALUES LESS   THAN   (1000)
, partition p002 VALUES LESS    THAN   (2000)
, partition p003 VALUES LESS    THAN   (3000)
, partition p004 VALUES LESS    THAN   (4000)
, partition p005 VALUES LESS    THAN   (5000)
, partition p006 VALUES LESS    THAN   (6000)
, partition p007 VALUES LESS    THAN   (7000)
, partition p008 VALUES LESS    THAN   (8000)
, partition p009 VALUES LESS    THAN   (9000)
, partition p010 VALUES LESS    THAN   (10000)
                                            81
);
the partition helper
./partitions_helper
   --table=mytable --column=d --interval=year 
    --start=2004-01-01 --end=2009-01-01
ALTER TABLE mytable
PARTITION by range (to_date(d))
(
   partition p001 VALUES LESS THAN   (to_days('2004-01-01'))
, partition p002 VALUES LESS THAN    (to_days('2005-01-01'))
, partition p003 VALUES LESS THAN    (to_days('2006-01-01'))
, partition p004 VALUES LESS THAN    (to_days('2007-01-01'))
, partition p005 VALUES LESS THAN    (to_days('2008-01-01'))
, partition p006 VALUES LESS THAN    (to_days('2009-01-01'))
);




                                                    82
the partition helper
./partitions_helper --table=mytable 
   --column=d --interval=month 
   --start=2008-01-01 --end=2009-01-01
ALTER TABLE mytable
PARTITION by range (to_date(d))
(
   partition p001 VALUES LESS THAN   (to_days('2008-01-01'))
, partition p002 VALUES LESS THAN    (to_days('2008-02-01'))
, partition p003 VALUES LESS THAN    (to_days('2008-03-01'))
, partition p004 VALUES LESS THAN    (to_days('2008-04-01'))
, partition p005 VALUES LESS THAN    (to_days('2008-05-01'))
, partition p006 VALUES LESS THAN    (to_days('2008-06-01'))
, partition p007 VALUES LESS THAN    (to_days('2008-07-01'))
, partition p008 VALUES LESS THAN    (to_days('2008-08-01'))
, partition p009 VALUES LESS THAN    (to_days('2008-09-01'))
, partition p010 VALUES LESS THAN    (to_days('2008-10-01'))
, partition p011 VALUES LESS THAN    (to_days('2008-11-01'))
, partition p012 VALUES LESS THAN    (to_days('2008-12-01'))
, partition p013 VALUES LESS THAN    (to_days('2009-01-01'))
                                                    83
);
MySQL 5.1 partitions




        TIPS
                       84
TIPS
• For large table ( indexes > available RAM)
  > DO NOT USE INDEXES
  > PARTITIONS ARE MORE EFFICIENT




                                               85
TIPS
• Before adopting partitions in production, benchmark!
• you can create partitions of different sizes
  > in historical tables, you can partition
     – current data by day
     – recent data by month
     – distant past data by year
  > ALL IN THE SAME TABLE!
• If you want to enforce a constraint on a integer
  column, you may set a partition by list, with only the
  values you want to admit.
                                                 86
TIPS
• For large historical tables, consider using ARCHIVE
  + partitions
  > You see benefits for very large amounts of data ( > 500
    MB)
  > Very effective when you run statistics
  > Almost the same performance of MyISAM
  > but 1/10 of storage




                                                     87
Thank you!



Giuseppe Maxia
http://datacharmer@blogspot.com
datacharmer@sun.com
MySQL 5.1 partitions




        BONUS
        SLIDES
                       89
MySQL 5.1 partitions




ANNOYANCES
                       90
Annoyances with partitions
• CREATE TABLE STATEMENT hard to read




                                        91
Annoyances with partitions
• CREATE TABLE STATEMENT hard to read




                                        91
Annoyances with partitions
• hard to read (FIXED!! in 5.1.31)




                                     92
Annoyances with partitions
• hard to read (FIXED!! in 5.1.31)




                                     92
Annoyances with partitions
• CREATE TABLE only keeps the result of the
  expression for each partition.
• (you can use the information_schema to ease the
  pain in this case)




                                             93
Annoyances with partitions
CREATE TABLE t1 (
  d date
) ENGINE=InnoDB
PARTITION BY RANGE (to_days(d))
(
  PARTITION P1 VALUES
   LESS THAN (to_days('1999-01-01')),
  PARTITION P2 VALUES
  LESS THAN (to_days('2000-01-01'))
                               94

)
Annoyances with partitions
CREATE TABLE `t1` (
  `d` date DEFAULT NULL
) ENGINE=InnoDB DEFAULT
CHARSET=latin1
/*!50100 PARTITION BY RANGE
(to_days(d))
(PARTITION P1 VALUES LESS THAN
(730120) ENGINE = InnoDB,
 PARTITION P2 VALUES LESS THAN
(730485) ENGINE = InnoDB) */
                                 95
Fixing annoyances with partitions
select partition_name part,
partition_expression expr,
partition_description val
from information_schema.partitions
where table_name='t1';
+------+------------+--------+
| part | expr       | val    |
+------+------------+--------+
| P1   | to_days(d) | 730120 |
| P2   | to_days(d) | 730485 |
                              96
+------+------------+--------+
fixing annoyances with partitions
select partition_name part ,
partition_expression expr,
from_days(partition_description) val
from information_schema.partitions
where table_name='t1';
+------+------------+------------+
| part | expr       | val        |
+------+------------+------------+
| P1   | to_days(d) | 1999-01-01 |
| P2   | to_days(d) | 2000-01-01 |
+------+------------+------------+
                              97

More Related Content

What's hot

Oracle naveen Sql
Oracle naveen   SqlOracle naveen   Sql
Oracle naveen Sql
naveen
 
MySQL constraints
MySQL constraintsMySQL constraints
MySQL constraints
Harish Gyanani
 
mysqlHiep.ppt
mysqlHiep.pptmysqlHiep.ppt
mysqlHiep.ppt
webhostingguy
 
Les01
Les01Les01
Les11
Les11Les11
Les11 Including Constraints
Les11 Including ConstraintsLes11 Including Constraints
DBMS lab manual
DBMS lab manualDBMS lab manual
DBMS lab manual
maha tce
 
Les02
Les02Les02
DBMS information in detail || Dbms (lab) ppt
DBMS information in detail || Dbms (lab) pptDBMS information in detail || Dbms (lab) ppt
DBMS information in detail || Dbms (lab) ppt
gourav kottawar
 
Les10
Les10Les10
MySQL Pro
MySQL ProMySQL Pro
Select To Order By
Select  To  Order BySelect  To  Order By
Select To Order By
Krizia Capacio
 
Sql
SqlSql
Efficient Pagination Using MySQL
Efficient Pagination Using MySQLEfficient Pagination Using MySQL
Efficient Pagination Using MySQL
Evan Weaver
 
dbms lab manual
dbms lab manualdbms lab manual
dbms lab manual
stalinjothi
 
Les09 Manipulating Data
Les09 Manipulating DataLes09 Manipulating Data
SQL Sort Notes
SQL Sort NotesSQL Sort Notes
SQL Sort Notes
ShivaAdasule
 
My sql.ppt
My sql.pptMy sql.ppt
MY SQL
MY SQLMY SQL
MY SQL
sundar
 
Best sql plsql_material for B.TECH
Best sql plsql_material for B.TECH Best sql plsql_material for B.TECH
Best sql plsql_material for B.TECH
AmIt Prasad
 

What's hot (20)

Oracle naveen Sql
Oracle naveen   SqlOracle naveen   Sql
Oracle naveen Sql
 
MySQL constraints
MySQL constraintsMySQL constraints
MySQL constraints
 
mysqlHiep.ppt
mysqlHiep.pptmysqlHiep.ppt
mysqlHiep.ppt
 
Les01
Les01Les01
Les01
 
Les11
Les11Les11
Les11
 
Les11 Including Constraints
Les11 Including ConstraintsLes11 Including Constraints
Les11 Including Constraints
 
DBMS lab manual
DBMS lab manualDBMS lab manual
DBMS lab manual
 
Les02
Les02Les02
Les02
 
DBMS information in detail || Dbms (lab) ppt
DBMS information in detail || Dbms (lab) pptDBMS information in detail || Dbms (lab) ppt
DBMS information in detail || Dbms (lab) ppt
 
Les10
Les10Les10
Les10
 
MySQL Pro
MySQL ProMySQL Pro
MySQL Pro
 
Select To Order By
Select  To  Order BySelect  To  Order By
Select To Order By
 
Sql
SqlSql
Sql
 
Efficient Pagination Using MySQL
Efficient Pagination Using MySQLEfficient Pagination Using MySQL
Efficient Pagination Using MySQL
 
dbms lab manual
dbms lab manualdbms lab manual
dbms lab manual
 
Les09 Manipulating Data
Les09 Manipulating DataLes09 Manipulating Data
Les09 Manipulating Data
 
SQL Sort Notes
SQL Sort NotesSQL Sort Notes
SQL Sort Notes
 
My sql.ppt
My sql.pptMy sql.ppt
My sql.ppt
 
MY SQL
MY SQLMY SQL
MY SQL
 
Best sql plsql_material for B.TECH
Best sql plsql_material for B.TECH Best sql plsql_material for B.TECH
Best sql plsql_material for B.TECH
 

Similar to Boost performance with MySQL 5.1 partitions

MySQL 5.1 and beyond
MySQL 5.1 and beyondMySQL 5.1 and beyond
MySQL 5.1 and beyond
Giuseppe Maxia
 
Boost Performance With My S Q L 51 Partitions
Boost Performance With  My S Q L 51 PartitionsBoost Performance With  My S Q L 51 Partitions
Boost Performance With My S Q L 51 Partitions
PerconaPerformance
 
Partitioning Under The Hood
Partitioning Under The HoodPartitioning Under The Hood
Partitioning Under The Hood
MySQLConference
 
MySQL partitioning performance
MySQL partitioning performanceMySQL partitioning performance
MySQL partitioning performance
Tommy Đột
 
Things you should know about Oracle truncate
Things you should know about Oracle truncateThings you should know about Oracle truncate
Things you should know about Oracle truncate
Kazuhiro Takahashi
 
Data Purge Algorithm
Data Purge AlgorithmData Purge Algorithm
Data Purge Algorithm
Saurabh S Agrawal
 
"Mobage DBA Fight against Big Data" - NHN TE
"Mobage DBA Fight against Big Data" - NHN TE"Mobage DBA Fight against Big Data" - NHN TE
"Mobage DBA Fight against Big Data" - NHN TE
Ryosuke IWANAGA
 
Modern SQL in Open Source and Commercial Databases
Modern SQL in Open Source and Commercial DatabasesModern SQL in Open Source and Commercial Databases
Modern SQL in Open Source and Commercial Databases
Markus Winand
 
Oracle SQL AND PL/SQL
Oracle SQL AND PL/SQLOracle SQL AND PL/SQL
Oracle SQL AND PL/SQL
suriyae1
 
PPC2009_yahoo_mysql_pagination
PPC2009_yahoo_mysql_paginationPPC2009_yahoo_mysql_pagination
PPC2009_yahoo_mysql_pagination
mysqlops
 
Efficient Pagination Using MySQL
Efficient Pagination Using MySQLEfficient Pagination Using MySQL
Efficient Pagination Using MySQL
Surat Singh Bhati
 
Discard inport exchange table & tablespace
Discard inport exchange table & tablespaceDiscard inport exchange table & tablespace
Discard inport exchange table & tablespace
Marco Tusa
 
Practical Recipes for Daily DBA Activities using DB2 9 and 10 for z/OS
Practical Recipes for Daily DBA Activities using DB2 9 and 10 for z/OSPractical Recipes for Daily DBA Activities using DB2 9 and 10 for z/OS
Practical Recipes for Daily DBA Activities using DB2 9 and 10 for z/OS
Cuneyt Goksu
 
SQL WORKSHOP::Lecture 9
SQL WORKSHOP::Lecture 9SQL WORKSHOP::Lecture 9
SQL WORKSHOP::Lecture 9
Umair Amjad
 
Histograms in 12c era
Histograms in 12c eraHistograms in 12c era
Histograms in 12c era
Mauro Pagano
 
COIS 420 - Practice01
COIS 420 - Practice01COIS 420 - Practice01
COIS 420 - Practice01
Angel G Diaz
 
Most useful queries
Most useful queriesMost useful queries
Most useful queries
Sam Depp
 
Discover the power of Recursive SQL and query transformation with Informix da...
Discover the power of Recursive SQL and query transformation with Informix da...Discover the power of Recursive SQL and query transformation with Informix da...
Discover the power of Recursive SQL and query transformation with Informix da...
Ajay Gupte
 
SQLQueries
SQLQueriesSQLQueries
SQLQueries
karunakar81987
 
Les09[1]Manipulating Data
Les09[1]Manipulating DataLes09[1]Manipulating Data
Les09[1]Manipulating Data
siavosh kaviani
 

Similar to Boost performance with MySQL 5.1 partitions (20)

MySQL 5.1 and beyond
MySQL 5.1 and beyondMySQL 5.1 and beyond
MySQL 5.1 and beyond
 
Boost Performance With My S Q L 51 Partitions
Boost Performance With  My S Q L 51 PartitionsBoost Performance With  My S Q L 51 Partitions
Boost Performance With My S Q L 51 Partitions
 
Partitioning Under The Hood
Partitioning Under The HoodPartitioning Under The Hood
Partitioning Under The Hood
 
MySQL partitioning performance
MySQL partitioning performanceMySQL partitioning performance
MySQL partitioning performance
 
Things you should know about Oracle truncate
Things you should know about Oracle truncateThings you should know about Oracle truncate
Things you should know about Oracle truncate
 
Data Purge Algorithm
Data Purge AlgorithmData Purge Algorithm
Data Purge Algorithm
 
"Mobage DBA Fight against Big Data" - NHN TE
"Mobage DBA Fight against Big Data" - NHN TE"Mobage DBA Fight against Big Data" - NHN TE
"Mobage DBA Fight against Big Data" - NHN TE
 
Modern SQL in Open Source and Commercial Databases
Modern SQL in Open Source and Commercial DatabasesModern SQL in Open Source and Commercial Databases
Modern SQL in Open Source and Commercial Databases
 
Oracle SQL AND PL/SQL
Oracle SQL AND PL/SQLOracle SQL AND PL/SQL
Oracle SQL AND PL/SQL
 
PPC2009_yahoo_mysql_pagination
PPC2009_yahoo_mysql_paginationPPC2009_yahoo_mysql_pagination
PPC2009_yahoo_mysql_pagination
 
Efficient Pagination Using MySQL
Efficient Pagination Using MySQLEfficient Pagination Using MySQL
Efficient Pagination Using MySQL
 
Discard inport exchange table & tablespace
Discard inport exchange table & tablespaceDiscard inport exchange table & tablespace
Discard inport exchange table & tablespace
 
Practical Recipes for Daily DBA Activities using DB2 9 and 10 for z/OS
Practical Recipes for Daily DBA Activities using DB2 9 and 10 for z/OSPractical Recipes for Daily DBA Activities using DB2 9 and 10 for z/OS
Practical Recipes for Daily DBA Activities using DB2 9 and 10 for z/OS
 
SQL WORKSHOP::Lecture 9
SQL WORKSHOP::Lecture 9SQL WORKSHOP::Lecture 9
SQL WORKSHOP::Lecture 9
 
Histograms in 12c era
Histograms in 12c eraHistograms in 12c era
Histograms in 12c era
 
COIS 420 - Practice01
COIS 420 - Practice01COIS 420 - Practice01
COIS 420 - Practice01
 
Most useful queries
Most useful queriesMost useful queries
Most useful queries
 
Discover the power of Recursive SQL and query transformation with Informix da...
Discover the power of Recursive SQL and query transformation with Informix da...Discover the power of Recursive SQL and query transformation with Informix da...
Discover the power of Recursive SQL and query transformation with Informix da...
 
SQLQueries
SQLQueriesSQLQueries
SQLQueries
 
Les09[1]Manipulating Data
Les09[1]Manipulating DataLes09[1]Manipulating Data
Les09[1]Manipulating Data
 

More from Giuseppe Maxia

MySQL NDB 8.0 clusters in your laptop with dbdeployer
MySQL NDB 8.0 clusters in your laptop with dbdeployerMySQL NDB 8.0 clusters in your laptop with dbdeployer
MySQL NDB 8.0 clusters in your laptop with dbdeployer
Giuseppe Maxia
 
Test like a_boss
Test like a_bossTest like a_boss
Test like a_boss
Giuseppe Maxia
 
Dbdeployer, the universal installer
Dbdeployer, the universal installerDbdeployer, the universal installer
Dbdeployer, the universal installer
Giuseppe Maxia
 
Test complex database systems in your laptop with dbdeployer
Test complex database systems in your laptop with dbdeployerTest complex database systems in your laptop with dbdeployer
Test complex database systems in your laptop with dbdeployer
Giuseppe Maxia
 
Dbdeployer
DbdeployerDbdeployer
Dbdeployer
Giuseppe Maxia
 
Dbdeployer
DbdeployerDbdeployer
Dbdeployer
Giuseppe Maxia
 
A quick tour of Mysql 8 roles
A quick tour of Mysql 8 rolesA quick tour of Mysql 8 roles
A quick tour of Mysql 8 roles
Giuseppe Maxia
 
MySQL document_store
MySQL document_storeMySQL document_store
MySQL document_store
Giuseppe Maxia
 
Replication skeptic
Replication skepticReplication skeptic
Replication skeptic
Giuseppe Maxia
 
Synchronise your data between MySQL and MongoDB
Synchronise your data between MySQL and MongoDBSynchronise your data between MySQL and MongoDB
Synchronise your data between MySQL and MongoDB
Giuseppe Maxia
 
Juggle your data with Tungsten Replicator
Juggle your data with Tungsten ReplicatorJuggle your data with Tungsten Replicator
Juggle your data with Tungsten Replicator
Giuseppe Maxia
 
MySQL in your laptop
MySQL in your laptopMySQL in your laptop
MySQL in your laptop
Giuseppe Maxia
 
Script it
Script itScript it
Script it
Giuseppe Maxia
 
Tungsten Replicator tutorial
Tungsten Replicator tutorialTungsten Replicator tutorial
Tungsten Replicator tutorial
Giuseppe Maxia
 
Preventing multi master conflicts with tungsten
Preventing multi master conflicts with tungstenPreventing multi master conflicts with tungsten
Preventing multi master conflicts with tungsten
Giuseppe Maxia
 
MySQL high availability power and usability
MySQL high availability power and usabilityMySQL high availability power and usability
MySQL high availability power and usability
Giuseppe Maxia
 
Solving MySQL replication problems with Tungsten
Solving MySQL replication problems with TungstenSolving MySQL replication problems with Tungsten
Solving MySQL replication problems with Tungsten
Giuseppe Maxia
 
State of the art of MySQL replication and clustering
State of the art of MySQL replication and clusteringState of the art of MySQL replication and clustering
State of the art of MySQL replication and clustering
Giuseppe Maxia
 
Testing mysql creatively in a sandbox
Testing mysql creatively in a sandboxTesting mysql creatively in a sandbox
Testing mysql creatively in a sandbox
Giuseppe Maxia
 
Mysql 5.5 and 5.6 replication
Mysql 5.5 and 5.6 replicationMysql 5.5 and 5.6 replication
Mysql 5.5 and 5.6 replication
Giuseppe Maxia
 

More from Giuseppe Maxia (20)

MySQL NDB 8.0 clusters in your laptop with dbdeployer
MySQL NDB 8.0 clusters in your laptop with dbdeployerMySQL NDB 8.0 clusters in your laptop with dbdeployer
MySQL NDB 8.0 clusters in your laptop with dbdeployer
 
Test like a_boss
Test like a_bossTest like a_boss
Test like a_boss
 
Dbdeployer, the universal installer
Dbdeployer, the universal installerDbdeployer, the universal installer
Dbdeployer, the universal installer
 
Test complex database systems in your laptop with dbdeployer
Test complex database systems in your laptop with dbdeployerTest complex database systems in your laptop with dbdeployer
Test complex database systems in your laptop with dbdeployer
 
Dbdeployer
DbdeployerDbdeployer
Dbdeployer
 
Dbdeployer
DbdeployerDbdeployer
Dbdeployer
 
A quick tour of Mysql 8 roles
A quick tour of Mysql 8 rolesA quick tour of Mysql 8 roles
A quick tour of Mysql 8 roles
 
MySQL document_store
MySQL document_storeMySQL document_store
MySQL document_store
 
Replication skeptic
Replication skepticReplication skeptic
Replication skeptic
 
Synchronise your data between MySQL and MongoDB
Synchronise your data between MySQL and MongoDBSynchronise your data between MySQL and MongoDB
Synchronise your data between MySQL and MongoDB
 
Juggle your data with Tungsten Replicator
Juggle your data with Tungsten ReplicatorJuggle your data with Tungsten Replicator
Juggle your data with Tungsten Replicator
 
MySQL in your laptop
MySQL in your laptopMySQL in your laptop
MySQL in your laptop
 
Script it
Script itScript it
Script it
 
Tungsten Replicator tutorial
Tungsten Replicator tutorialTungsten Replicator tutorial
Tungsten Replicator tutorial
 
Preventing multi master conflicts with tungsten
Preventing multi master conflicts with tungstenPreventing multi master conflicts with tungsten
Preventing multi master conflicts with tungsten
 
MySQL high availability power and usability
MySQL high availability power and usabilityMySQL high availability power and usability
MySQL high availability power and usability
 
Solving MySQL replication problems with Tungsten
Solving MySQL replication problems with TungstenSolving MySQL replication problems with Tungsten
Solving MySQL replication problems with Tungsten
 
State of the art of MySQL replication and clustering
State of the art of MySQL replication and clusteringState of the art of MySQL replication and clustering
State of the art of MySQL replication and clustering
 
Testing mysql creatively in a sandbox
Testing mysql creatively in a sandboxTesting mysql creatively in a sandbox
Testing mysql creatively in a sandbox
 
Mysql 5.5 and 5.6 replication
Mysql 5.5 and 5.6 replicationMysql 5.5 and 5.6 replication
Mysql 5.5 and 5.6 replication
 

Recently uploaded

BLOCKCHAIN TECHNOLOGY - Advantages and Disadvantages
BLOCKCHAIN TECHNOLOGY - Advantages and DisadvantagesBLOCKCHAIN TECHNOLOGY - Advantages and Disadvantages
BLOCKCHAIN TECHNOLOGY - Advantages and Disadvantages
SAI KAILASH R
 
Best Practices for Effectively Running dbt in Airflow.pdf
Best Practices for Effectively Running dbt in Airflow.pdfBest Practices for Effectively Running dbt in Airflow.pdf
Best Practices for Effectively Running dbt in Airflow.pdf
Tatiana Al-Chueyr
 
RPA In Healthcare Benefits, Use Case, Trend And Challenges 2024.pptx
RPA In Healthcare Benefits, Use Case, Trend And Challenges 2024.pptxRPA In Healthcare Benefits, Use Case, Trend And Challenges 2024.pptx
RPA In Healthcare Benefits, Use Case, Trend And Challenges 2024.pptx
SynapseIndia
 
High Profile Girls call Service Pune 000XX00000 Provide Best And Top Girl Ser...
High Profile Girls call Service Pune 000XX00000 Provide Best And Top Girl Ser...High Profile Girls call Service Pune 000XX00000 Provide Best And Top Girl Ser...
High Profile Girls call Service Pune 000XX00000 Provide Best And Top Girl Ser...
bhumivarma35300
 
Litestack talk at Brighton 2024 (Unleashing the power of SQLite for Ruby apps)
Litestack talk at Brighton 2024 (Unleashing the power of SQLite for Ruby apps)Litestack talk at Brighton 2024 (Unleashing the power of SQLite for Ruby apps)
Litestack talk at Brighton 2024 (Unleashing the power of SQLite for Ruby apps)
Muhammad Ali
 
Google I/O Extended Harare Merged Slides
Google I/O Extended Harare Merged SlidesGoogle I/O Extended Harare Merged Slides
Google I/O Extended Harare Merged Slides
Google Developer Group - Harare
 
July Patch Tuesday
July Patch TuesdayJuly Patch Tuesday
July Patch Tuesday
Ivanti
 
Pigging Unit Lubricant Oil Blending Plant
Pigging Unit Lubricant Oil Blending PlantPigging Unit Lubricant Oil Blending Plant
Pigging Unit Lubricant Oil Blending Plant
LINUS PROJECTS (INDIA)
 
CiscoIconsLibrary cours de réseau VLAN.ppt
CiscoIconsLibrary cours de réseau VLAN.pptCiscoIconsLibrary cours de réseau VLAN.ppt
CiscoIconsLibrary cours de réseau VLAN.ppt
moinahousna
 
Three New Criminal Laws in India 1 July 2024
Three New Criminal Laws in India 1 July 2024Three New Criminal Laws in India 1 July 2024
Three New Criminal Laws in India 1 July 2024
aakash malhotra
 
Dublin_mulesoft_meetup_Mulesoft_Salesforce_Integration (1).pptx
Dublin_mulesoft_meetup_Mulesoft_Salesforce_Integration (1).pptxDublin_mulesoft_meetup_Mulesoft_Salesforce_Integration (1).pptx
Dublin_mulesoft_meetup_Mulesoft_Salesforce_Integration (1).pptx
Kunal Gupta
 
Integrating Kafka with MuleSoft 4 and usecase
Integrating Kafka with MuleSoft 4 and usecaseIntegrating Kafka with MuleSoft 4 and usecase
Integrating Kafka with MuleSoft 4 and usecase
shyamraj55
 
"Mastering Graphic Design: Essential Tips and Tricks for Beginners and Profes...
"Mastering Graphic Design: Essential Tips and Tricks for Beginners and Profes..."Mastering Graphic Design: Essential Tips and Tricks for Beginners and Profes...
"Mastering Graphic Design: Essential Tips and Tricks for Beginners and Profes...
Anant Gupta
 
“Deploying Large Language Models on a Raspberry Pi,” a Presentation from Usef...
“Deploying Large Language Models on a Raspberry Pi,” a Presentation from Usef...“Deploying Large Language Models on a Raspberry Pi,” a Presentation from Usef...
“Deploying Large Language Models on a Raspberry Pi,” a Presentation from Usef...
Edge AI and Vision Alliance
 
The Role of IoT in Australian Mobile App Development - PDF Guide
The Role of IoT in Australian Mobile App Development - PDF GuideThe Role of IoT in Australian Mobile App Development - PDF Guide
The Role of IoT in Australian Mobile App Development - PDF Guide
Shiv Technolabs
 
[Talk] Moving Beyond Spaghetti Infrastructure [AOTB] 2024-07-04.pdf
[Talk] Moving Beyond Spaghetti Infrastructure [AOTB] 2024-07-04.pdf[Talk] Moving Beyond Spaghetti Infrastructure [AOTB] 2024-07-04.pdf
[Talk] Moving Beyond Spaghetti Infrastructure [AOTB] 2024-07-04.pdf
Kief Morris
 
Uncharted Together- Navigating AI's New Frontiers in Libraries
Uncharted Together- Navigating AI's New Frontiers in LibrariesUncharted Together- Navigating AI's New Frontiers in Libraries
Uncharted Together- Navigating AI's New Frontiers in Libraries
Brian Pichman
 
The Rise of AI in Cybersecurity How Machine Learning Will Shape Threat Detect...
The Rise of AI in Cybersecurity How Machine Learning Will Shape Threat Detect...The Rise of AI in Cybersecurity How Machine Learning Will Shape Threat Detect...
The Rise of AI in Cybersecurity How Machine Learning Will Shape Threat Detect...
digitalxplive
 
BT & Neo4j: Knowledge Graphs for Critical Enterprise Systems.pptx.pdf
BT & Neo4j: Knowledge Graphs for Critical Enterprise Systems.pptx.pdfBT & Neo4j: Knowledge Graphs for Critical Enterprise Systems.pptx.pdf
BT & Neo4j: Knowledge Graphs for Critical Enterprise Systems.pptx.pdf
Neo4j
 
Figma AI Design Generator_ In-Depth Review.pdf
Figma AI Design Generator_ In-Depth Review.pdfFigma AI Design Generator_ In-Depth Review.pdf
Figma AI Design Generator_ In-Depth Review.pdf
Management Institute of Skills Development
 

Recently uploaded (20)

BLOCKCHAIN TECHNOLOGY - Advantages and Disadvantages
BLOCKCHAIN TECHNOLOGY - Advantages and DisadvantagesBLOCKCHAIN TECHNOLOGY - Advantages and Disadvantages
BLOCKCHAIN TECHNOLOGY - Advantages and Disadvantages
 
Best Practices for Effectively Running dbt in Airflow.pdf
Best Practices for Effectively Running dbt in Airflow.pdfBest Practices for Effectively Running dbt in Airflow.pdf
Best Practices for Effectively Running dbt in Airflow.pdf
 
RPA In Healthcare Benefits, Use Case, Trend And Challenges 2024.pptx
RPA In Healthcare Benefits, Use Case, Trend And Challenges 2024.pptxRPA In Healthcare Benefits, Use Case, Trend And Challenges 2024.pptx
RPA In Healthcare Benefits, Use Case, Trend And Challenges 2024.pptx
 
High Profile Girls call Service Pune 000XX00000 Provide Best And Top Girl Ser...
High Profile Girls call Service Pune 000XX00000 Provide Best And Top Girl Ser...High Profile Girls call Service Pune 000XX00000 Provide Best And Top Girl Ser...
High Profile Girls call Service Pune 000XX00000 Provide Best And Top Girl Ser...
 
Litestack talk at Brighton 2024 (Unleashing the power of SQLite for Ruby apps)
Litestack talk at Brighton 2024 (Unleashing the power of SQLite for Ruby apps)Litestack talk at Brighton 2024 (Unleashing the power of SQLite for Ruby apps)
Litestack talk at Brighton 2024 (Unleashing the power of SQLite for Ruby apps)
 
Google I/O Extended Harare Merged Slides
Google I/O Extended Harare Merged SlidesGoogle I/O Extended Harare Merged Slides
Google I/O Extended Harare Merged Slides
 
July Patch Tuesday
July Patch TuesdayJuly Patch Tuesday
July Patch Tuesday
 
Pigging Unit Lubricant Oil Blending Plant
Pigging Unit Lubricant Oil Blending PlantPigging Unit Lubricant Oil Blending Plant
Pigging Unit Lubricant Oil Blending Plant
 
CiscoIconsLibrary cours de réseau VLAN.ppt
CiscoIconsLibrary cours de réseau VLAN.pptCiscoIconsLibrary cours de réseau VLAN.ppt
CiscoIconsLibrary cours de réseau VLAN.ppt
 
Three New Criminal Laws in India 1 July 2024
Three New Criminal Laws in India 1 July 2024Three New Criminal Laws in India 1 July 2024
Three New Criminal Laws in India 1 July 2024
 
Dublin_mulesoft_meetup_Mulesoft_Salesforce_Integration (1).pptx
Dublin_mulesoft_meetup_Mulesoft_Salesforce_Integration (1).pptxDublin_mulesoft_meetup_Mulesoft_Salesforce_Integration (1).pptx
Dublin_mulesoft_meetup_Mulesoft_Salesforce_Integration (1).pptx
 
Integrating Kafka with MuleSoft 4 and usecase
Integrating Kafka with MuleSoft 4 and usecaseIntegrating Kafka with MuleSoft 4 and usecase
Integrating Kafka with MuleSoft 4 and usecase
 
"Mastering Graphic Design: Essential Tips and Tricks for Beginners and Profes...
"Mastering Graphic Design: Essential Tips and Tricks for Beginners and Profes..."Mastering Graphic Design: Essential Tips and Tricks for Beginners and Profes...
"Mastering Graphic Design: Essential Tips and Tricks for Beginners and Profes...
 
“Deploying Large Language Models on a Raspberry Pi,” a Presentation from Usef...
“Deploying Large Language Models on a Raspberry Pi,” a Presentation from Usef...“Deploying Large Language Models on a Raspberry Pi,” a Presentation from Usef...
“Deploying Large Language Models on a Raspberry Pi,” a Presentation from Usef...
 
The Role of IoT in Australian Mobile App Development - PDF Guide
The Role of IoT in Australian Mobile App Development - PDF GuideThe Role of IoT in Australian Mobile App Development - PDF Guide
The Role of IoT in Australian Mobile App Development - PDF Guide
 
[Talk] Moving Beyond Spaghetti Infrastructure [AOTB] 2024-07-04.pdf
[Talk] Moving Beyond Spaghetti Infrastructure [AOTB] 2024-07-04.pdf[Talk] Moving Beyond Spaghetti Infrastructure [AOTB] 2024-07-04.pdf
[Talk] Moving Beyond Spaghetti Infrastructure [AOTB] 2024-07-04.pdf
 
Uncharted Together- Navigating AI's New Frontiers in Libraries
Uncharted Together- Navigating AI's New Frontiers in LibrariesUncharted Together- Navigating AI's New Frontiers in Libraries
Uncharted Together- Navigating AI's New Frontiers in Libraries
 
The Rise of AI in Cybersecurity How Machine Learning Will Shape Threat Detect...
The Rise of AI in Cybersecurity How Machine Learning Will Shape Threat Detect...The Rise of AI in Cybersecurity How Machine Learning Will Shape Threat Detect...
The Rise of AI in Cybersecurity How Machine Learning Will Shape Threat Detect...
 
BT & Neo4j: Knowledge Graphs for Critical Enterprise Systems.pptx.pdf
BT & Neo4j: Knowledge Graphs for Critical Enterprise Systems.pptx.pdfBT & Neo4j: Knowledge Graphs for Critical Enterprise Systems.pptx.pdf
BT & Neo4j: Knowledge Graphs for Critical Enterprise Systems.pptx.pdf
 
Figma AI Design Generator_ In-Depth Review.pdf
Figma AI Design Generator_ In-Depth Review.pdfFigma AI Design Generator_ In-Depth Review.pdf
Figma AI Design Generator_ In-Depth Review.pdf
 

Boost performance with MySQL 5.1 partitions

  • 1. Boost performance with MySQL 5.1 partitions Giuseppe Maxia MySQL Community Team Lead Sun Microsystems
  • 3. Giuseppe Maxia • a.k.a. The Data Charmer • MySQL Community Team Lead • Long time hacking with MySQL features • Formerly, database consultant, designer, coder. • A passion for QA • An even greater passion for open source • ... and community • Passionate blogger • http://datacharmer.blogspot.com 3
  • 7. Defining the problem YOUR NEEDS 6
  • 8. The problem(s) • Too much data • Not enough RAM • Historical data • Growing data • Rotating data 7
  • 9. Too much data data 8
  • 10. Not enough RAM RAM INDEXES data INDEXES 9
  • 11. Not enough RAM INDEXES RAM data INDEXES 10
  • 13. What is it? • Logical splitting of tables • Transparent to user 12
  • 14. Logical splitting • No need to create separate tables • No need to move chunks of data across files 13
  • 15. MySQL 5.1 partitions SQL TRANSPARENT 14
  • 16. COMPARED TO MERGE TABLES MERGE TABLE • separate tables • risk of duplicates • insert in each table • no constraints 15
  • 17. COMPARED TO MERGE TABLES PARTITIONED TABLE • One table • No risk of duplicates • insert in one table • constraints enforced 16
  • 18. Wait a minute ... • WHAT THE HELL DOES quot;LOGICAL SPLITquot; REALLY MEANS? • LET ME EXPLAIN ... 17
  • 20. Physical partitioning (2) • cut it into pieces 19
  • 21. Physical partitioning (3) • What you have, is several different pieces 20
  • 22. Physical partitioning (4) • If you want the map back, you need some application (adhesive tape) and you may get it wrong 21
  • 24. Logical partitioning (2) • fold it to show the piece you need 23
  • 25. Logical partitioning (3) • what you have is still a map, even if you see only one part. 24
  • 26. Logical partitioning (4) • if you unfold the map, you still have the whole thing 25
  • 27. What partitions can do • logical split • but you can also split the data physically • granular partition (subpartitioning) • different methods (range, list, hash, key) 26
  • 29. 4 main reasons to use partitions • To make single inserts and selects faster • To make range selects faster • To help split the data across different paths • To store historical data efficiently • If you need to delete large chunks of data INSTANTLY 28
  • 31. HOW TO MAKE PARTITIONS 30
  • 32. HOW TO MAKE PARTITIONS •RTFM ... 30
  • 33. HOW TO MAKE PARTITIONS •RTFM ... • No, seriously, the manual has everything 30
  • 34. HOW TO MAKE PARTITIONS •RTFM ... • No, seriously, the manual has everything • But if you absolutely insist ... 30
  • 35. HOW TO MAKE PARTITIONS CREATE TABLE t1 ( id int ) ENGINE=InnoDB # or MyISAM, ARCHIVE PARTITION BY RANGE (id) ( PARTITION P1 VALUES LESS THAN (10), PARTITION P2 VALUES LESS THAN (20) ) 31
  • 36. HOW TO MAKE PARTITIONS CREATE TABLE t1 ( id int ) ENGINE=InnoDB PARTITION BY LIST (id) ( PARTITION P1 VALUES IN (1,2,4), PARTITION P2 VALUES IN (3,5,9) ) 32
  • 37. HOW TO MAKE PARTITIONS CREATE TABLE t1 ( id int not null primary key ) ENGINE=InnoDB PARTITION BY HASH (id) PARTITIONS 10; 33
  • 38. HOW TO MAKE PARTITIONS CREATE TABLE t1 ( id int not null primary key ) ENGINE=InnoDB PARTITION BY KEY () PARTITIONS 10; 34
  • 39. Limitations • Can partition only by INTEGER columns • OR you can partition by an expression, which must return an integer • Maximum 1024 partitions • If you have a Unique Key or PK, the partition column must be part of that key • No Foreign Key support • No Fulltext or GIS support 35
  • 40. PARTITIONING BY DATE CREATE TABLE t1 ( d date ) ENGINE=InnoDB PARTITION BY RANGE (year(d)) ( PARTITION P1 VALUES LESS THAN (1999), PARTITION P2 VALUES LESS THAN (2000) 36 )
  • 41. PARTITIONING BY DATE CREATE TABLE t1 ( d date ) ENGINE=InnoDB PARTITION BY RANGE (to_days(d)) ( PARTITION P1 VALUES LESS THAN (to_days('1999-01-01')), PARTITION P2 VALUES LESS THAN (to_days('2000-01-01')) 37 )
  • 42. MySQL 5.1 partitions Partition pruning 38
  • 43. Partition pruning 1a - unpartitioned table - SINGLE RECORD select * from table_name where colx = 120
  • 44. Partition pruning 1a - unpartitioned table - SINGLE RECORD I select * D from N A table_name D T where colx = E A 120 X
  • 45. Partition pruning 1b - unpartitioned table - SINGLE RECORD select * from table_name where colx = 350
  • 46. Partition pruning 1c - unpartitioned table - RANGE select * from table_name where colx between 120 and 230
  • 47. Partition pruning 2a - table partitioned by colx - SINGLE REC 1-99 100-199 select * from 200-299 table_name where colx = 300-399 120 400-499 500-599
  • 48. Partition pruning 2a - table partitioned by colx - SINGLE REC 1-99 100-199 select * D from A 200-299 table_name T where colx = A I 300-399 120 N D 400-499 E X 500-599
  • 49. Partition pruning 2b - table partitioned by colx - SINGLE REC 1-99 100-199 select * from 200-299 table_name where colx = 300-399 350 400-499 500-599
  • 50. Partition pruning 2c - table partitioned by colx - RANGE 1-99 select * 100-199 from table_name 200-299 where colx 300-399 between 120 and 230 400-499 500-599
  • 51. Partition pruning EXPLAIN select * before from table_name where colx = 120 EXPLAIN PARTITIONS in 5.1 select * from table_name where colx = 120
  • 52. Partition pruning - not using partition column explain partitions select count(*) from table_name where colx=120G ***** 1. row **** id: 1 select_type: SIMPLE table: table_name partitions: p01,p02,p03,p04,p05,p06,p07,p08,p09, p10,p11,p12,p13,p14,p15,p16,p17,p18, p19 type: index ...
  • 53. Partition pruning - not using partition column explain partitions select count(*) from table_name where colx between 120 and 230G ***** 1. row **** id: 1 select_type: SIMPLE table: table_name partitions: p01,p02,p03,p04,p05,p06,p07,p08,p09, p10,p11,p12,p13,p14,p15,p16,p17,p18, p19 type: index ...
  • 54. Partition pruning - table partitioned by colx explain partitions select count(*) from table_name where colx between 120 and 230G ***** 1. row **** id: 1 select_type: SIMPLE table: table_name partitions: p02,p03 type: index ...
  • 56. When to use partitions? • if you have large tables • if you know that you will always query for the partitioning column • if you have historical tables that you want to purge quickly • if your indexes are larger than the available RAM 52
  • 57. MySQL 5.1 partitions HANDS ON 53
  • 58. components used for testing 54
  • 59. components used for testing • MySQL Sandbox > created MySQL instances in seconds > http://launchpad.net/mysql-sandbox 54
  • 60. components used for testing • MySQL Sandbox > created MySQL instances in seconds > http://launchpad.net/mysql-sandbox • MySQL Employees Test Database > has ~ 4 mil records in 6 tables > http://launchpad.net/test-db 54
  • 61. components used for testing • MySQL Sandbox > created MySQL instances in seconds > http://launchpad.net/mysql-sandbox • MySQL Employees Test Database > has ~ 4 mil records in 6 tables > http://launchpad.net/test-db • Command line ability > fingers > ingenuity 54
  • 63. How many partitions from information_schema.partitions +-------+-----------------+----------+ | pname | expr | descr | +-------+-----------------+----------+ | p01 | year(from_date) | 1985 | | p02 | year(from_date) | 1986 | ... | p13 | year(from_date) | 1997 | | p14 | year(from_date) | 1998 | | p15 | year(from_date) | 1999 | | p16 | year(from_date) | 2000 | ... | p19 | year(from_date) | MAXVALUE | +-------+-----------------+----------+ 56
  • 64. How many records select count(*) from salaries; +----------+ | count(*) | +----------+ | 2844047 | +----------+ 1 row in set (0.01 sec) 57
  • 65. How many records in 1998 not partitioned select count(*) from salaries where from_date between '1998-01-01' and '1998-12-31'; +----------+ | count(*) | +----------+ | 247489 | +----------+ 1 row in set (1.52 sec) # NOT PARTITIONED 58
  • 66. How many records in 1998 PARTITIONED select count(*) from salaries where from_date between '1998-01-01' and '1998-12-31'; +----------+ | count(*) | +----------+ | 247489 | +----------+ 1 row in set (0.41 sec) # partition p15 59
  • 67. How many records in 1999 not partitioned select count(*) from salaries where from_date between '1999-01-01' and '1999-12-31'; +----------+ | count(*) | +----------+ | 260957 | +----------+ 1 row in set (1.54 sec) # NOT PARTITIONED 60
  • 68. How many records in 1999 PARTITIONED select count(*) from salaries where from_date between '1999-01-01' and '1999-12-31'; +----------+ | count(*) | +----------+ | 260957 | +----------+ 1 row in set (0.17 sec) # partition p16 61
  • 69. Deleting 1998 records not partitioned delete from salaries where from_date between '1998-01-01' and '1998-12-31'; Query OK, 247489 rows affected (19.13 sec) # NOT PARTITIONED 62
  • 70. Deleting 1998 records partitioned alter table salaries drop partition p15; Query OK, 0 rows affected (1.35 sec) 63
  • 71. MySQL 5.1 partitions LEVERAGING REPLICATION 64
  • 72. Replication schemes INNODB INNODB NOT NOT MASTER PARTITIONED PARTITIONED concurrent insert SLAVE concurrent read INNODB MyISAM PARTITIONED PARTITIONED BY RANGE BY RANGE SLAVE SLAVE 65 concurrent batch processing large batch processing
  • 73. Replication schemes concurrent insert INNODB PARTITIONED MASTER INNODB NON MyISAM PARTITIONED SLAVE PARTITIONED SLAVE 66 batch processing concurrent reads
  • 74. Replication schemes - dimensions INNODB ARCHIVE NOT PARTITIONED PARTITIONED SLAVE BY RANGE MASTER (locations) concurrent insert dimensional processing ARCHIVE ARCHIVE PARTITIONED SLAVE SLAVE PARTITIONED BY RANGE BY RANGE (date) (product) 67 dimensional processing dimensional processing
  • 76. Expressions • Partition by function • Search by column 69
  • 77. WRONG PARTITION BY RANGE(year(from_date)) select count(*) from salaries where year(from_date) = 1998; +----------+ | count(*) | +----------+ | 247489 | +----------+ 1 row in set (2.25 sec) 70
  • 78. RIGHT PARTITION BY RANGE(year(from_date)) select count(*) from salaries where from_date between '1998-01-01' and '1998-12-31'; +----------+ | count(*) | +----------+ | 247489 | +----------+ 1 row in set (0.46 sec) 71
  • 79. EXPLAIN explain partitions select count(*) from salaries where year(from_date) = 1998G ***** 1. row **** id: 1 select_type: SIMPLE table: salaries partitions: p01,p02,p03,p04,p05,p06,p07,p08,p09 ,p10,p11,p12,p13,p14,p15,p16,p17,p1 8,p19 type: index 72 ...
  • 80. EXPLAIN explain partitions select count(*) from salaries where from_date between '1998-01-01' and '1998-12-31'G ***** 1. row **** id: 1 select_type: SIMPLE table: salaries partitions: p14,p15 ... 73
  • 81. MySQL 5.1 partitions BENCHMARKS 74
  • 82. Partitions benchmarks (laptop) engine storage (MB) innodb 221 myisam 181 archive 74 innodb partitioned (whole) 289 innodb partitioned (file per table) 676 myisam partitioned 182 archive partitioned 72
  • 83. Benchmarking results (laptop) engine query year query year 2000 2002 InnoDB 1.25 1.25 MyISAM 1.72 1.73 Archive 2.47 2.45 InnoDB partitioned 0.24 0.10 whole InnoDB Partitioned 0.45 0.10 (file per table) MyISAM partitioned 0.18 0.12 Archive partitioned 0.22 0.12
  • 84. Partitioning storage (huge server) engine storage (GB) innodb (with PK) 330 myisam (with PK) 141 archive 13 innodb partitioned (no PK) 237 myisam partitioned (no PK) 107 archive partitioned 13
  • 85. Benchmarking results (huge server) engine 6 month range InnoDB 4 min 30s MyISAM 25.03s Archive 22 min 25s InnoDB partitioned by month 13.19 MyISAM partitioned by year 6.31 MyISAM partitioned by month 4.45 Archive partitioned by year 16.67 Archive partitioned by month 8.97
  • 87. The partition helper • The INFORMATION_SCHEMA.PARTITIONS table • The partition helper > http://datacharmer.blogspot.com/2008/12/partition- helper-improving-usability.html > A Perl script that creates partitioning statements • ... anything you are creating right now :) 80
  • 88. the partition helper ./partitions_helper --table=mytable --column=prod_id --interval=1000 --start=1 --end=10000 ALTER TABLE mytable PARTITION by range (prod_id) ( partition p001 VALUES LESS THAN (1000) , partition p002 VALUES LESS THAN (2000) , partition p003 VALUES LESS THAN (3000) , partition p004 VALUES LESS THAN (4000) , partition p005 VALUES LESS THAN (5000) , partition p006 VALUES LESS THAN (6000) , partition p007 VALUES LESS THAN (7000) , partition p008 VALUES LESS THAN (8000) , partition p009 VALUES LESS THAN (9000) , partition p010 VALUES LESS THAN (10000) 81 );
  • 89. the partition helper ./partitions_helper --table=mytable --column=d --interval=year --start=2004-01-01 --end=2009-01-01 ALTER TABLE mytable PARTITION by range (to_date(d)) ( partition p001 VALUES LESS THAN (to_days('2004-01-01')) , partition p002 VALUES LESS THAN (to_days('2005-01-01')) , partition p003 VALUES LESS THAN (to_days('2006-01-01')) , partition p004 VALUES LESS THAN (to_days('2007-01-01')) , partition p005 VALUES LESS THAN (to_days('2008-01-01')) , partition p006 VALUES LESS THAN (to_days('2009-01-01')) ); 82
  • 90. the partition helper ./partitions_helper --table=mytable --column=d --interval=month --start=2008-01-01 --end=2009-01-01 ALTER TABLE mytable PARTITION by range (to_date(d)) ( partition p001 VALUES LESS THAN (to_days('2008-01-01')) , partition p002 VALUES LESS THAN (to_days('2008-02-01')) , partition p003 VALUES LESS THAN (to_days('2008-03-01')) , partition p004 VALUES LESS THAN (to_days('2008-04-01')) , partition p005 VALUES LESS THAN (to_days('2008-05-01')) , partition p006 VALUES LESS THAN (to_days('2008-06-01')) , partition p007 VALUES LESS THAN (to_days('2008-07-01')) , partition p008 VALUES LESS THAN (to_days('2008-08-01')) , partition p009 VALUES LESS THAN (to_days('2008-09-01')) , partition p010 VALUES LESS THAN (to_days('2008-10-01')) , partition p011 VALUES LESS THAN (to_days('2008-11-01')) , partition p012 VALUES LESS THAN (to_days('2008-12-01')) , partition p013 VALUES LESS THAN (to_days('2009-01-01')) 83 );
  • 92. TIPS • For large table ( indexes > available RAM) > DO NOT USE INDEXES > PARTITIONS ARE MORE EFFICIENT 85
  • 93. TIPS • Before adopting partitions in production, benchmark! • you can create partitions of different sizes > in historical tables, you can partition – current data by day – recent data by month – distant past data by year > ALL IN THE SAME TABLE! • If you want to enforce a constraint on a integer column, you may set a partition by list, with only the values you want to admit. 86
  • 94. TIPS • For large historical tables, consider using ARCHIVE + partitions > You see benefits for very large amounts of data ( > 500 MB) > Very effective when you run statistics > Almost the same performance of MyISAM > but 1/10 of storage 87
  • 96. MySQL 5.1 partitions BONUS SLIDES 89
  • 98. Annoyances with partitions • CREATE TABLE STATEMENT hard to read 91
  • 99. Annoyances with partitions • CREATE TABLE STATEMENT hard to read 91
  • 100. Annoyances with partitions • hard to read (FIXED!! in 5.1.31) 92
  • 101. Annoyances with partitions • hard to read (FIXED!! in 5.1.31) 92
  • 102. Annoyances with partitions • CREATE TABLE only keeps the result of the expression for each partition. • (you can use the information_schema to ease the pain in this case) 93
  • 103. Annoyances with partitions CREATE TABLE t1 ( d date ) ENGINE=InnoDB PARTITION BY RANGE (to_days(d)) ( PARTITION P1 VALUES LESS THAN (to_days('1999-01-01')), PARTITION P2 VALUES LESS THAN (to_days('2000-01-01')) 94 )
  • 104. Annoyances with partitions CREATE TABLE `t1` ( `d` date DEFAULT NULL ) ENGINE=InnoDB DEFAULT CHARSET=latin1 /*!50100 PARTITION BY RANGE (to_days(d)) (PARTITION P1 VALUES LESS THAN (730120) ENGINE = InnoDB, PARTITION P2 VALUES LESS THAN (730485) ENGINE = InnoDB) */ 95
  • 105. Fixing annoyances with partitions select partition_name part, partition_expression expr, partition_description val from information_schema.partitions where table_name='t1'; +------+------------+--------+ | part | expr | val | +------+------------+--------+ | P1 | to_days(d) | 730120 | | P2 | to_days(d) | 730485 | 96 +------+------------+--------+
  • 106. fixing annoyances with partitions select partition_name part , partition_expression expr, from_days(partition_description) val from information_schema.partitions where table_name='t1'; +------+------------+------------+ | part | expr | val | +------+------------+------------+ | P1 | to_days(d) | 1999-01-01 | | P2 | to_days(d) | 2000-01-01 | +------+------------+------------+ 97